2024-04-26T21:35:32.696 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T21:35:32.696 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231 -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/orig.config.yaml 2024-04-26T21:35:32.750 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:35:32.837 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} email: null first_in_suite: false job_id: '7675231' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi055.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNHfiHQS3I6dJ/WKWqtAsSAT9nRcTILNLOnD6K1apilLYOYmbpnRRXf4UeH6aatsiRUTfrpI1fJyqC9OCd2TYzs= smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPR52kOYXVRc++48vWLWaJdMwOVhhuFoCYGtUuvxmig3vlX4qnZkFipYunP9pQsGGHdhA6tD5UtcmTKTgvBtspo= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJG7YMoZoGS5PR8OAdy72bO3isLjhgdYKa8FoLF16DUpxZCAvyw8Ruq6QrvaWsksvxxeUQMmpflsmDDyp5Wyx7M= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T21:35:32.838 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T21:35:32.838 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T21:35:32.838 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T21:35:32.840 INFO:teuthology.task.internal:Checking packages... 2024-04-26T21:35:32.864 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T21:35:32.864 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T21:35:32.864 INFO:teuthology.packaging:ref: None 2024-04-26T21:35:32.864 INFO:teuthology.packaging:tag: None 2024-04-26T21:35:32.864 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T21:35:32.864 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:35:32.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T21:35:33.013 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T21:35:33.014 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T21:35:33.021 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T21:35:33.021 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T21:35:33.027 INFO:teuthology.task.internal:Saving configuration 2024-04-26T21:35:33.047 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T21:35:33.053 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T21:35:33.082 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:30:15.400739', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNHfiHQS3I6dJ/WKWqtAsSAT9nRcTILNLOnD6K1apilLYOYmbpnRRXf4UeH6aatsiRUTfrpI1fJyqC9OCd2TYzs='} 2024-04-26T21:35:33.107 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:30:15.398736', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPR52kOYXVRc++48vWLWaJdMwOVhhuFoCYGtUuvxmig3vlX4qnZkFipYunP9pQsGGHdhA6tD5UtcmTKTgvBtspo='} 2024-04-26T21:35:33.127 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:30:15.402510', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJG7YMoZoGS5PR8OAdy72bO3isLjhgdYKa8FoLF16DUpxZCAvyw8Ruq6QrvaWsksvxxeUQMmpflsmDDyp5Wyx7M='} 2024-04-26T21:35:33.127 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T21:35:33.133 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T21:35:33.134 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T21:35:33.134 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T21:35:33.134 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T21:35:33.232 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2e21e0a160>, signals=[15]) 2024-04-26T21:35:33.233 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T21:35:33.240 INFO:teuthology.task.internal:Opening connections... 2024-04-26T21:35:33.240 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2024-04-26T21:35:33.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:35:33.320 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-04-26T21:35:33.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:35:33.397 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-04-26T21:35:33.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:35:33.473 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T21:35:33.480 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2024-04-26T21:35:33.499 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2024-04-26T21:35:33.499 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-04-26T21:35:33.556 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:35:33.557 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:35:33.558 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-04-26T21:35:33.582 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-04-26T21:35:33.601 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-04-26T21:35:33.601 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-04-26T21:35:33.659 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-04-26T21:35:33.659 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:35:33.660 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:35:33.660 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-04-26T21:35:33.682 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-04-26T21:35:33.700 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-04-26T21:35:33.700 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-04-26T21:35:33.757 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T21:35:33.758 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T21:35:33.759 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-04-26T21:35:33.782 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T21:35:33.792 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T21:35:33.797 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T21:35:33.797 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:35:33.800 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:35:33.802 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-04-26T21:35:33.819 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T21:35:33.824 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T21:35:33.824 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:35:33.857 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:35:33.860 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-04-26T21:35:33.950 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T21:35:34.030 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T21:35:34.045 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T21:35:34.045 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-26T21:35:34.046 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-26T21:35:34.046 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:35:34.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:35:34.064 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-04-26T21:35:34.122 INFO:teuthology.orchestra.run.smithi055.stdout:5.14.0-437.el9.x86_64 2024-04-26T21:35:34.123 INFO:teuthology.task.kernel:Running kernel on smithi055: 5.14.0-437.el9.x86_64 2024-04-26T21:35:34.123 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum install -y kernel 2024-04-26T21:35:36.647 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.0 MB 00:00 2024-04-26T21:35:40.565 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-04-26T21:35:45.057 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-26T21:35:46.884 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-26T21:35:48.087 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-26T21:35:53.359 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T21:35:55.315 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T21:35:55.315 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T21:35:55.412 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-04-26T21:35:55.413 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2024-04-26T21:35:55.413 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-04-26T21:35:55.543 DEBUG:teuthology.orchestra.run.smithi055:> echo no | sudo yum reinstall kernel || true 2024-04-26T21:35:56.064 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:35:53 PM UTC. 2024-04-26T21:35:56.199 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-04-26T21:35:56.204 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout:Reinstalling: 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 255 k 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 0 2024-04-26T21:35:56.205 INFO:teuthology.orchestra.run.smithi055.stderr:Operation aborted. 2024-04-26T21:35:56.263 INFO:teuthology.orchestra.run.smithi055.stdout:Is this ok [y/N]: 2024-04-26T21:35:56.264 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum reinstall -y kernel || true 2024-04-26T21:35:56.779 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 09:35:53 PM UTC. 2024-04-26T21:35:56.899 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout:Reinstalling: 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 255 k 2024-04-26T21:35:56.900 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 0 2024-04-26T21:35:56.901 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-04-26T21:35:57.203 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-26T21:35:57.203 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:35:57.203 INFO:teuthology.orchestra.run.smithi055.stdout:Total 843 kB/s | 255 kB 00:00 2024-04-26T21:35:57.224 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-04-26T21:35:57.419 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-04-26T21:35:57.419 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-04-26T21:35:57.424 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-04-26T21:35:57.425 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-04-26T21:35:57.556 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-04-26T21:35:57.637 INFO:teuthology.orchestra.run.smithi055.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:35:57.637 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:35:58.343 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:35:58.344 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T21:35:59.046 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T21:35:59.046 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:35:59.047 INFO:teuthology.orchestra.run.smithi055.stdout:Reinstalled: 2024-04-26T21:35:59.047 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T21:35:59.047 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:35:59.047 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-04-26T21:35:59.224 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T21:35:59.271 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T21:35:59.272 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T21:35:59.272 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T21:35:59.272 DEBUG:teuthology.parallel:result is None 2024-04-26T21:35:59.273 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T21:35:59.280 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T21:35:59.280 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:35:59.283 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:35:59.285 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T21:35:59.302 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T21:35:59.309 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T21:35:59.315 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T21:35:59.315 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:35:59.338 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:35:59.343 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T21:35:59.387 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T21:35:59.394 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T21:35:59.395 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:35:59.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:35:59.423 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T21:35:59.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:35:59.440 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:35:59.442 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:35:59.466 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T21:35:59.478 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:35:59.494 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:35:59.506 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:35:59.522 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:35:59.524 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:35:59.549 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T21:35:59.550 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T21:35:59.558 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T21:35:59.558 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:35:59.560 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:35:59.566 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T21:35:59.592 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T21:35:59.601 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T21:35:59.601 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:35:59.625 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:35:59.634 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T21:35:59.650 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:35:59.724 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:35:59.777 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:35:59.857 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:35:59.898 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:35:59.898 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:35:59.962 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:36:00.007 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:36:00.057 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:36:00.137 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:36:00.184 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:36:00.185 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:36:00.251 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:36:00.294 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T21:36:00.348 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:36:00.432 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T21:36:00.472 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:36:00.472 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T21:36:00.539 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2024-04-26T21:36:00.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-04-26T21:36:00.544 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-04-26T21:36:00.573 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:36:00.576 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:36:00.631 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:36:01.115 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T21:36:01.123 INFO:teuthology.task.internal:Starting timer... 2024-04-26T21:36:01.123 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T21:36:01.133 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T21:36:01.142 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T21:36:01.142 DEBUG:teuthology.orchestra.run.smithi055:> sudo service auditd rotate 2024-04-26T21:36:01.196 INFO:teuthology.orchestra.run.smithi055.stdout:Rotating logs: 2024-04-26T21:36:01.198 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-04-26T21:36:01.244 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-04-26T21:36:01.247 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-04-26T21:36:01.301 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-04-26T21:36:01.303 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T21:36:01.303 DEBUG:teuthology.orchestra.run.smithi055:> /usr/sbin/getenforce 2024-04-26T21:36:01.329 INFO:teuthology.orchestra.run.smithi055.stdout:Permissive 2024-04-26T21:36:01.329 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-04-26T21:36:01.358 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-04-26T21:36:01.358 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-04-26T21:36:01.385 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-04-26T21:36:01.385 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi055.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-04-26T21:36:01.386 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:36:01.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:36:01.414 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:36:01.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:36:01.444 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:36:01.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:36:01.472 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T21:36:01.472 DEBUG:teuthology.orchestra.run.smithi055:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:36:01.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:36:01.535 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-04-26T21:36:01.562 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T21:36:01.572 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T21:36:01.587 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T21:36:01.588 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi055.front.sepia.ceph.com,smithi064.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T21:40:00.415 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-04-26T21:40:00.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-04-26T21:40:00.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:40:00.494 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-04-26T21:40:00.582 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-04-26T21:40:00.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-04-26T21:40:00.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:40:00.658 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-04-26T21:40:00.738 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-04-26T21:40:00.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-26T21:40:00.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T21:40:00.814 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-04-26T21:40:00.910 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-26T21:40:00.910 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T21:40:00.921 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T21:40:00.921 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:40:00.921 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:40:00.924 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:40:00.924 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:40:00.926 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T21:40:00.926 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:40:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:40:00.955 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:40:00.967 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:40:00.969 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:40:00.979 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T21:40:00.991 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: ntpd: command not found 2024-04-26T21:40:00.993 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T21:40:00.993 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-04-26T21:40:01.002 INFO:teuthology.orchestra.run.smithi055.stdout:506 Cannot talk to daemon 2024-04-26T21:40:01.004 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-04-26T21:40:01.016 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:40:01.016 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-04-26T21:40:01.018 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:40:01.027 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-04-26T21:40:01.030 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:40:01.032 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:40:01.040 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T21:40:01.053 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T21:40:01.087 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:40:01.091 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:40:01.091 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-04-26T21:40:01.091 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.091 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.091 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.092 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.093 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:40:01.096 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:40:01.096 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-04-26T21:40:01.097 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.097 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.097 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.097 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.109 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:40:01.112 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:40:01.112 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-26T21:40:01.112 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.112 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.112 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.113 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:40:01.113 INFO:teuthology.run_tasks:Running task install... 2024-04-26T21:40:01.122 DEBUG:teuthology.task.install:project ceph 2024-04-26T21:40:01.122 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T21:40:01.122 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T21:40:01.122 INFO:teuthology.task.install:Using flavor: default 2024-04-26T21:40:01.128 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T21:40:01.128 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T21:40:01.129 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:40:01.129 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:01.131 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:40:01.131 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:01.133 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T21:40:01.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:40:01.351 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:40:01.351 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:40:01.362 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:40:01.362 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:40:01.383 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T21:40:01.383 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T21:40:01.476 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:40:01.476 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:40:01.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:40:01.479 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:40:01.479 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:40:01.479 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:40:01.494 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T21:40:01.494 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:40:01.494 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T21:40:01.503 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:40:01.503 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:40:01.504 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:40:01.504 DEBUG:teuthology.orchestra.run.smithi055:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:40:01.520 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T21:40:01.521 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T21:40:01.568 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:40:01.569 DEBUG:teuthology.orchestra.run.smithi055:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:40:01.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T21:40:01.640 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:40:01.643 DEBUG:teuthology.orchestra.run.smithi055:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:40:01.661 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T21:40:01.703 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-04-26T21:40:01.705 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-04-26T21:40:01.707 INFO:teuthology.orchestra.run.smithi055.stdout:check_obsoletes = 1 2024-04-26T21:40:01.709 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum clean all 2024-04-26T21:40:01.725 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-04-26T21:40:01.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-04-26T21:40:01.933 INFO:teuthology.orchestra.run.smithi165.stdout:48 files removed 2024-04-26T21:40:01.942 INFO:teuthology.orchestra.run.smithi055.stdout:48 files removed 2024-04-26T21:40:01.960 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:40:01.960 INFO:teuthology.orchestra.run.smithi064.stdout:48 files removed 2024-04-26T21:40:01.969 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:40:01.988 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T21:40:02.602 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 219 kB/s | 84 kB 00:00 2024-04-26T21:40:02.638 INFO:teuthology.orchestra.run.smithi055.stdout:ceph packages for x86_64 211 kB/s | 84 kB 00:00 2024-04-26T21:40:02.669 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 206 kB/s | 84 kB 00:00 2024-04-26T21:40:03.036 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-26T21:40:03.061 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-26T21:40:03.073 INFO:teuthology.orchestra.run.smithi055.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-26T21:40:03.387 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-26T21:40:03.454 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-26T21:40:03.515 INFO:teuthology.orchestra.run.smithi055.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-26T21:40:04.079 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T21:40:05.337 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-04-26T21:40:05.406 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-26T21:40:05.980 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-26T21:40:07.130 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-26T21:40:07.282 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-26T21:40:11.150 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-26T21:40:11.280 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T21:40:11.844 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-04-26T21:40:12.864 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-26T21:40:12.949 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-26T21:40:13.226 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 16 kB 00:00 2024-04-26T21:40:13.840 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-26T21:40:15.233 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-04-26T21:40:15.680 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-04-26T21:40:18.957 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-26T21:40:20.558 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-26T21:40:20.952 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-04-26T21:40:21.075 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:40:21.075 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:40:21.086 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:40:21.087 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:40:21.088 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:40:21.089 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:40:21.090 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:40:21.092 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:40:21.093 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:40:21.153 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-26T21:40:21.180 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-26T21:40:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-26T21:40:21.180 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-26T21:40:21.180 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-26T21:40:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:40:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:40:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:40:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:40:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:40:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:40:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:40:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:40:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:40:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:40:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:40:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:40:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-04-26T21:40:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:40:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:40:21.192 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-26T21:40:21.192 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-26T21:40:21.192 INFO:teuthology.orchestra.run.smithi165.stdout:Install 148 Packages 2024-04-26T21:40:21.192 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 4 Packages 2024-04-26T21:40:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:40:21.192 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 222 M 2024-04-26T21:40:21.192 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-26T21:40:21.800 INFO:teuthology.orchestra.run.smithi165.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-26T21:40:22.125 INFO:teuthology.orchestra.run.smithi165.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.7 MB/s | 889 kB 00:00 2024-04-26T21:40:22.208 INFO:teuthology.orchestra.run.smithi165.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.7 MB/s | 145 kB 00:00 2024-04-26T21:40:22.309 INFO:teuthology.orchestra.run.smithi165.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 7.4 MB/s | 5.4 MB 00:00 2024-04-26T21:40:22.517 INFO:teuthology.orchestra.run.smithi165.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 7.3 MB/s | 1.5 MB 00:00 2024-04-26T21:40:22.567 INFO:teuthology.orchestra.run.smithi165.stdout:(6/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 6.6 MB/s | 2.4 MB 00:00 2024-04-26T21:40:22.759 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:40:22.760 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:40:22.771 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:40:22.772 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:40:22.773 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:40:22.773 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:40:22.775 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:40:22.777 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:40:22.778 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:40:22.843 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:40:22.857 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:40:22.858 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:40:22.859 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:40:22.860 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:40:22.861 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:40:22.862 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:40:22.863 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:40:22.864 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:40:22.865 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:40:22.866 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:40:22.867 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Install 148 Packages 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 4 Packages 2024-04-26T21:40:22.868 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:40:22.869 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 222 M 2024-04-26T21:40:22.869 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-04-26T21:40:22.884 INFO:teuthology.orchestra.run.smithi165.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-26T21:40:23.179 INFO:teuthology.orchestra.run.smithi055.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:40:23.180 INFO:teuthology.orchestra.run.smithi055.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T21:40:23.191 INFO:teuthology.orchestra.run.smithi055.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T21:40:23.191 INFO:teuthology.orchestra.run.smithi055.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T21:40:23.192 INFO:teuthology.orchestra.run.smithi055.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T21:40:23.193 INFO:teuthology.orchestra.run.smithi055.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T21:40:23.195 INFO:teuthology.orchestra.run.smithi055.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T21:40:23.197 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T21:40:23.198 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T21:40:23.261 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-04-26T21:40:23.270 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T21:40:23.271 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T21:40:23.272 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T21:40:23.273 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrading: 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T21:40:23.274 INFO:teuthology.orchestra.run.smithi055.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T21:40:23.275 INFO:teuthology.orchestra.run.smithi055.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T21:40:23.276 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T21:40:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T21:40:23.278 INFO:teuthology.orchestra.run.smithi055.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T21:40:23.279 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T21:40:23.280 INFO:teuthology.orchestra.run.smithi055.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T21:40:23.281 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T21:40:23.282 INFO:teuthology.orchestra.run.smithi055.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout:Installing weak dependencies: 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout:======================================================================================= 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout:Install 148 Packages 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrade 4 Packages 2024-04-26T21:40:23.283 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:40:23.284 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 222 M 2024-04-26T21:40:23.284 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-04-26T21:40:23.653 INFO:teuthology.orchestra.run.smithi165.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 14 MB/s | 11 MB 00:00 2024-04-26T21:40:23.745 INFO:teuthology.orchestra.run.smithi165.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 275 kB/s | 25 kB 00:00 2024-04-26T21:40:24.109 INFO:teuthology.orchestra.run.smithi064.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-26T21:40:24.195 INFO:teuthology.orchestra.run.smithi055.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-04-26T21:40:24.359 INFO:teuthology.orchestra.run.smithi064.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.5 MB/s | 889 kB 00:00 2024-04-26T21:40:24.443 INFO:teuthology.orchestra.run.smithi064.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.7 MB/s | 145 kB 00:00 2024-04-26T21:40:24.851 INFO:teuthology.orchestra.run.smithi064.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 5.8 MB/s | 2.4 MB 00:00 2024-04-26T21:40:24.887 INFO:teuthology.orchestra.run.smithi055.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 889 kB 00:00 2024-04-26T21:40:25.029 INFO:teuthology.orchestra.run.smithi055.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.0 MB/s | 145 kB 00:00 2024-04-26T21:40:25.085 INFO:teuthology.orchestra.run.smithi064.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-04-26T21:40:25.537 INFO:teuthology.orchestra.run.smithi165.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 5.7 MB/s | 17 MB 00:02 2024-04-26T21:40:25.653 INFO:teuthology.orchestra.run.smithi165.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 225 kB 00:00 2024-04-26T21:40:25.760 INFO:teuthology.orchestra.run.smithi064.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 7.0 MB/s | 4.7 MB 00:00 2024-04-26T21:40:25.879 INFO:teuthology.orchestra.run.smithi165.stdout:(12/152): ceph-common-19.0.0-3306.gbc612ffb.el9 4.9 MB/s | 21 MB 00:04 2024-04-26T21:40:25.887 INFO:teuthology.orchestra.run.smithi055.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 2.8 MB/s | 2.4 MB 00:00 2024-04-26T21:40:25.904 INFO:teuthology.orchestra.run.smithi165.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 108 kB/s | 27 kB 00:00 2024-04-26T21:40:25.971 INFO:teuthology.orchestra.run.smithi165.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 346 kB/s | 32 kB 00:00 2024-04-26T21:40:26.079 INFO:teuthology.orchestra.run.smithi165.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.5 MB/s | 163 kB 00:00 2024-04-26T21:40:26.121 INFO:teuthology.orchestra.run.smithi055.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-26T21:40:26.171 INFO:teuthology.orchestra.run.smithi165.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 2.7 MB/s | 747 kB 00:00 2024-04-26T21:40:26.197 INFO:teuthology.orchestra.run.smithi165.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 1.1 MB/s | 127 kB 00:00 2024-04-26T21:40:26.380 INFO:teuthology.orchestra.run.smithi165.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 2.3 MB/s | 501 kB 00:00 2024-04-26T21:40:26.463 INFO:teuthology.orchestra.run.smithi055.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-04-26T21:40:26.464 INFO:teuthology.orchestra.run.smithi165.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 542 kB/s | 45 kB 00:00 2024-04-26T21:40:26.573 INFO:teuthology.orchestra.run.smithi165.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T21:40:26.681 INFO:teuthology.orchestra.run.smithi165.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.5 MB/s | 161 kB 00:00 2024-04-26T21:40:26.881 INFO:teuthology.orchestra.run.smithi165.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 321 kB 00:00 2024-04-26T21:40:26.896 INFO:teuthology.orchestra.run.smithi055.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 6.1 MB/s | 4.7 MB 00:00 2024-04-26T21:40:27.065 INFO:teuthology.orchestra.run.smithi165.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 1.6 MB/s | 298 kB 00:00 2024-04-26T21:40:27.190 INFO:teuthology.orchestra.run.smithi165.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 796 kB/s | 99 kB 00:00 2024-04-26T21:40:27.307 INFO:teuthology.orchestra.run.smithi165.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 730 kB/s | 85 kB 00:00 2024-04-26T21:40:28.205 INFO:teuthology.orchestra.run.smithi055.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 8.1 MB/s | 11 MB 00:01 2024-04-26T21:40:28.272 INFO:teuthology.orchestra.run.smithi055.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 379 kB/s | 25 kB 00:00 2024-04-26T21:40:28.691 INFO:teuthology.orchestra.run.smithi165.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 2.2 MB/s | 3.1 MB 00:01 2024-04-26T21:40:28.760 INFO:teuthology.orchestra.run.smithi064.stdout:(7/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 5.6 MB/s | 17 MB 00:02 2024-04-26T21:40:28.841 INFO:teuthology.orchestra.run.smithi165.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.1 MB/s | 171 kB 00:00 2024-04-26T21:40:28.941 INFO:teuthology.orchestra.run.smithi165.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 342 kB/s | 34 kB 00:00 2024-04-26T21:40:29.091 INFO:teuthology.orchestra.run.smithi165.stdout:(29/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 1.9 MB/s | 5.4 MB 00:02 2024-04-26T21:40:29.116 INFO:teuthology.orchestra.run.smithi165.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 812 kB/s | 142 kB 00:00 2024-04-26T21:40:31.208 INFO:teuthology.orchestra.run.smithi165.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 1.7 MB/s | 3.5 MB 00:02 2024-04-26T21:40:31.252 INFO:teuthology.orchestra.run.smithi064.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 4.3 MB/s | 11 MB 00:02 2024-04-26T21:40:31.344 INFO:teuthology.orchestra.run.smithi064.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 275 kB/s | 25 kB 00:00 2024-04-26T21:40:31.391 INFO:teuthology.orchestra.run.smithi165.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.3 MB/s | 249 kB 00:00 2024-04-26T21:40:31.410 INFO:teuthology.orchestra.run.smithi064.stdout:(10/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x 719 kB/s | 5.4 MB 00:07 2024-04-26T21:40:31.508 INFO:teuthology.orchestra.run.smithi165.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 422 kB/s | 49 kB 00:00 2024-04-26T21:40:31.625 INFO:teuthology.orchestra.run.smithi165.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 141 kB/s | 16 kB 00:00 2024-04-26T21:40:31.694 INFO:teuthology.orchestra.run.smithi064.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 794 kB/s | 225 kB 00:00 2024-04-26T21:40:31.794 INFO:teuthology.orchestra.run.smithi064.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 270 kB/s | 27 kB 00:00 2024-04-26T21:40:31.817 INFO:teuthology.orchestra.run.smithi165.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.4 MB/s | 273 kB 00:00 2024-04-26T21:40:31.903 INFO:teuthology.orchestra.run.smithi064.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 293 kB/s | 32 kB 00:00 2024-04-26T21:40:32.172 INFO:teuthology.orchestra.run.smithi055.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 2.9 MB/s | 17 MB 00:05 2024-04-26T21:40:32.297 INFO:teuthology.orchestra.run.smithi055.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 1.8 MB/s | 225 kB 00:00 2024-04-26T21:40:32.334 INFO:teuthology.orchestra.run.smithi165.stdout:(36/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 5.7 MB/s | 49 MB 00:08 2024-04-26T21:40:32.364 INFO:teuthology.orchestra.run.smithi055.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 406 kB/s | 27 kB 00:00 2024-04-26T21:40:32.418 INFO:teuthology.orchestra.run.smithi165.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 2.2 MB/s | 7.4 MB 00:03 2024-04-26T21:40:32.446 INFO:teuthology.orchestra.run.smithi064.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 1.3 MB/s | 747 kB 00:00 2024-04-26T21:40:32.446 INFO:teuthology.orchestra.run.smithi055.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 476 kB/s | 32 kB 00:00 2024-04-26T21:40:32.446 INFO:teuthology.orchestra.run.smithi165.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 167 kB/s | 18 kB 00:00 2024-04-26T21:40:32.477 INFO:teuthology.orchestra.run.smithi165.stdout:(39/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 1.1 MB/s | 762 kB 00:00 2024-04-26T21:40:32.528 INFO:teuthology.orchestra.run.smithi165.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-04-26T21:40:32.586 INFO:teuthology.orchestra.run.smithi165.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-26T21:40:32.603 INFO:teuthology.orchestra.run.smithi064.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.0 MB/s | 163 kB 00:00 2024-04-26T21:40:32.628 INFO:teuthology.orchestra.run.smithi165.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-26T21:40:32.656 INFO:teuthology.orchestra.run.smithi055.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.2 MB/s | 747 kB 00:00 2024-04-26T21:40:32.678 INFO:teuthology.orchestra.run.smithi165.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-26T21:40:32.720 INFO:teuthology.orchestra.run.smithi165.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-26T21:40:32.737 INFO:teuthology.orchestra.run.smithi064.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 953 kB/s | 127 kB 00:00 2024-04-26T21:40:32.779 INFO:teuthology.orchestra.run.smithi165.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-26T21:40:32.804 INFO:teuthology.orchestra.run.smithi165.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-26T21:40:32.838 INFO:teuthology.orchestra.run.smithi165.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-04-26T21:40:32.888 INFO:teuthology.orchestra.run.smithi165.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-26T21:40:32.996 INFO:teuthology.orchestra.run.smithi064.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 1.9 MB/s | 501 kB 00:00 2024-04-26T21:40:33.121 INFO:teuthology.orchestra.run.smithi165.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 518 kB/s | 107 kB 00:00 2024-04-26T21:40:33.150 INFO:teuthology.orchestra.run.smithi055.stdout:(15/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 10 MB/s | 49 MB 00:04 2024-04-26T21:40:33.155 INFO:teuthology.orchestra.run.smithi165.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-04-26T21:40:33.176 INFO:teuthology.orchestra.run.smithi055.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 315 kB/s | 163 kB 00:00 2024-04-26T21:40:33.188 INFO:teuthology.orchestra.run.smithi165.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-26T21:40:33.201 INFO:teuthology.orchestra.run.smithi055.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 2.5 MB/s | 127 kB 00:00 2024-04-26T21:40:33.214 INFO:teuthology.orchestra.run.smithi165.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 233 kB/s | 21 kB 00:00 2024-04-26T21:40:33.239 INFO:teuthology.orchestra.run.smithi165.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-26T21:40:33.256 INFO:teuthology.orchestra.run.smithi165.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 486 kB/s | 33 kB 00:00 2024-04-26T21:40:33.298 INFO:teuthology.orchestra.run.smithi165.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-26T21:40:33.323 INFO:teuthology.orchestra.run.smithi165.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-26T21:40:33.326 INFO:teuthology.orchestra.run.smithi055.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 3.3 MB/s | 501 kB 00:00 2024-04-26T21:40:33.382 INFO:teuthology.orchestra.run.smithi165.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-04-26T21:40:33.401 INFO:teuthology.orchestra.run.smithi055.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 603 kB/s | 45 kB 00:00 2024-04-26T21:40:33.407 INFO:teuthology.orchestra.run.smithi165.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-26T21:40:33.432 INFO:teuthology.orchestra.run.smithi165.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-04-26T21:40:33.466 INFO:teuthology.orchestra.run.smithi165.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-04-26T21:40:33.477 INFO:teuthology.orchestra.run.smithi055.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.8 MB/s | 135 kB 00:00 2024-04-26T21:40:33.499 INFO:teuthology.orchestra.run.smithi165.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-04-26T21:40:33.524 INFO:teuthology.orchestra.run.smithi165.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-04-26T21:40:33.550 INFO:teuthology.orchestra.run.smithi165.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-04-26T21:40:33.552 INFO:teuthology.orchestra.run.smithi055.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T21:40:33.575 INFO:teuthology.orchestra.run.smithi165.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-26T21:40:33.609 INFO:teuthology.orchestra.run.smithi165.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T21:40:33.634 INFO:teuthology.orchestra.run.smithi165.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 421 kB/s | 35 kB 00:00 2024-04-26T21:40:33.736 INFO:teuthology.orchestra.run.smithi055.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 10 MB/s | 5.4 MB 00:00 2024-04-26T21:40:33.761 INFO:teuthology.orchestra.run.smithi055.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.5 MB/s | 321 kB 00:00 2024-04-26T21:40:33.776 INFO:teuthology.orchestra.run.smithi165.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-04-26T21:40:33.795 INFO:teuthology.orchestra.run.smithi055.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 4.9 MB/s | 298 kB 00:00 2024-04-26T21:40:33.820 INFO:teuthology.orchestra.run.smithi055.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 99 kB 00:00 2024-04-26T21:40:33.837 INFO:teuthology.orchestra.run.smithi055.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.0 MB/s | 85 kB 00:00 2024-04-26T21:40:33.851 INFO:teuthology.orchestra.run.smithi165.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-26T21:40:33.896 INFO:teuthology.orchestra.run.smithi055.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-26T21:40:33.940 INFO:teuthology.orchestra.run.smithi165.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.4 MB/s | 911 kB 00:00 2024-04-26T21:40:33.946 INFO:teuthology.orchestra.run.smithi055.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 687 kB/s | 34 kB 00:00 2024-04-26T21:40:33.969 INFO:teuthology.orchestra.run.smithi165.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-26T21:40:33.996 INFO:teuthology.orchestra.run.smithi055.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.8 MB/s | 142 kB 00:00 2024-04-26T21:40:34.052 INFO:teuthology.orchestra.run.smithi165.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-26T21:40:34.086 INFO:teuthology.orchestra.run.smithi165.stdout:(72/152): postgresql-13.14-1.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-04-26T21:40:34.136 INFO:teuthology.orchestra.run.smithi165.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-04-26T21:40:34.211 INFO:teuthology.orchestra.run.smithi165.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-04-26T21:40:34.261 INFO:teuthology.orchestra.run.smithi165.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-04-26T21:40:34.295 INFO:teuthology.orchestra.run.smithi165.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-26T21:40:34.337 INFO:teuthology.orchestra.run.smithi165.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-04-26T21:40:34.362 INFO:teuthology.orchestra.run.smithi165.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-04-26T21:40:34.372 INFO:teuthology.orchestra.run.smithi055.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 9.4 MB/s | 3.5 MB 00:00 2024-04-26T21:40:34.412 INFO:teuthology.orchestra.run.smithi165.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-26T21:40:34.438 INFO:teuthology.orchestra.run.smithi165.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-04-26T21:40:34.497 INFO:teuthology.orchestra.run.smithi055.stdout:(31/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 4.6 MB/s | 3.1 MB 00:00 2024-04-26T21:40:34.504 INFO:teuthology.orchestra.run.smithi165.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-26T21:40:34.617 INFO:teuthology.orchestra.run.smithi165.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-26T21:40:34.622 INFO:teuthology.orchestra.run.smithi055.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.9 MB/s | 249 kB 00:00 2024-04-26T21:40:34.647 INFO:teuthology.orchestra.run.smithi165.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-26T21:40:34.697 INFO:teuthology.orchestra.run.smithi165.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 965 kB/s | 77 kB 00:00 2024-04-26T21:40:34.697 INFO:teuthology.orchestra.run.smithi055.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 657 kB/s | 49 kB 00:00 2024-04-26T21:40:34.722 INFO:teuthology.orchestra.run.smithi165.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-26T21:40:34.764 INFO:teuthology.orchestra.run.smithi055.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 247 kB/s | 16 kB 00:00 2024-04-26T21:40:34.798 INFO:teuthology.orchestra.run.smithi165.stdout:(86/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.4 MB/s | 6.0 MB 00:00 2024-04-26T21:40:34.823 INFO:teuthology.orchestra.run.smithi165.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-26T21:40:34.848 INFO:teuthology.orchestra.run.smithi165.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-26T21:40:34.873 INFO:teuthology.orchestra.run.smithi165.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-26T21:40:34.898 INFO:teuthology.orchestra.run.smithi055.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.0 MB/s | 273 kB 00:00 2024-04-26T21:40:34.899 INFO:teuthology.orchestra.run.smithi165.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-04-26T21:40:34.924 INFO:teuthology.orchestra.run.smithi165.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 714 kB/s | 54 kB 00:00 2024-04-26T21:40:34.974 INFO:teuthology.orchestra.run.smithi165.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-04-26T21:40:34.999 INFO:teuthology.orchestra.run.smithi165.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-04-26T21:40:35.050 INFO:teuthology.orchestra.run.smithi165.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-26T21:40:35.065 INFO:teuthology.orchestra.run.smithi055.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-26T21:40:35.074 INFO:teuthology.orchestra.run.smithi165.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 478 kB/s | 36 kB 00:00 2024-04-26T21:40:35.125 INFO:teuthology.orchestra.run.smithi165.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-04-26T21:40:35.175 INFO:teuthology.orchestra.run.smithi165.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-26T21:40:35.287 INFO:teuthology.orchestra.run.smithi055.stdout:(37/152): ceph-common-19.0.0-3306.gbc612ffb.el9 1.9 MB/s | 21 MB 00:11 2024-04-26T21:40:35.316 INFO:teuthology.orchestra.run.smithi055.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 73 kB/s | 18 kB 00:00 2024-04-26T21:40:35.349 INFO:teuthology.orchestra.run.smithi055.stdout:(39/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 1.7 MB/s | 762 kB 00:00 2024-04-26T21:40:35.367 INFO:teuthology.orchestra.run.smithi165.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-04-26T21:40:35.384 INFO:teuthology.orchestra.run.smithi165.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 223 kB/s | 58 kB 00:00 2024-04-26T21:40:35.393 INFO:teuthology.orchestra.run.smithi055.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 779 kB/s | 80 kB 00:00 2024-04-26T21:40:35.409 INFO:teuthology.orchestra.run.smithi165.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 401 kB/s | 94 kB 00:00 2024-04-26T21:40:35.424 INFO:teuthology.orchestra.run.smithi055.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-26T21:40:35.443 INFO:teuthology.orchestra.run.smithi165.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 762 kB/s | 45 kB 00:00 2024-04-26T21:40:35.458 INFO:teuthology.orchestra.run.smithi055.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.5 MB/s | 803 kB 00:00 2024-04-26T21:40:35.460 INFO:teuthology.orchestra.run.smithi165.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 689 kB/s | 64 kB 00:00 2024-04-26T21:40:35.477 INFO:teuthology.orchestra.run.smithi165.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 331 kB/s | 22 kB 00:00 2024-04-26T21:40:35.501 INFO:teuthology.orchestra.run.smithi055.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-26T21:40:35.534 INFO:teuthology.orchestra.run.smithi055.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-26T21:40:35.554 INFO:teuthology.orchestra.run.smithi064.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 2.1 MB/s | 5.4 MB 00:02 2024-04-26T21:40:35.601 INFO:teuthology.orchestra.run.smithi055.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-04-26T21:40:35.622 INFO:teuthology.orchestra.run.smithi064.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 681 kB/s | 45 kB 00:00 2024-04-26T21:40:35.634 INFO:teuthology.orchestra.run.smithi055.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-04-26T21:40:35.660 INFO:teuthology.orchestra.run.smithi055.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 995 kB/s | 125 kB 00:00 2024-04-26T21:40:35.661 INFO:teuthology.orchestra.run.smithi165.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-04-26T21:40:35.710 INFO:teuthology.orchestra.run.smithi055.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-26T21:40:35.729 INFO:teuthology.orchestra.run.smithi064.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.2 MB/s | 135 kB 00:00 2024-04-26T21:40:35.786 INFO:teuthology.orchestra.run.smithi165.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-26T21:40:35.811 INFO:teuthology.orchestra.run.smithi165.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-04-26T21:40:35.836 INFO:teuthology.orchestra.run.smithi165.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 861 kB/s | 309 kB 00:00 2024-04-26T21:40:35.838 INFO:teuthology.orchestra.run.smithi064.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.5 MB/s | 161 kB 00:00 2024-04-26T21:40:35.852 INFO:teuthology.orchestra.run.smithi055.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-04-26T21:40:35.877 INFO:teuthology.orchestra.run.smithi055.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-04-26T21:40:35.886 INFO:teuthology.orchestra.run.smithi165.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 349 kB/s | 26 kB 00:00 2024-04-26T21:40:35.903 INFO:teuthology.orchestra.run.smithi055.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-04-26T21:40:35.912 INFO:teuthology.orchestra.run.smithi165.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-04-26T21:40:35.936 INFO:teuthology.orchestra.run.smithi055.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-04-26T21:40:35.961 INFO:teuthology.orchestra.run.smithi055.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-04-26T21:40:35.978 INFO:teuthology.orchestra.run.smithi165.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-04-26T21:40:35.987 INFO:teuthology.orchestra.run.smithi055.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-26T21:40:35.995 INFO:teuthology.orchestra.run.smithi165.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 630 kB/s | 67 kB 00:00 2024-04-26T21:40:36.012 INFO:teuthology.orchestra.run.smithi055.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-04-26T21:40:36.021 INFO:teuthology.orchestra.run.smithi165.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T21:40:36.037 INFO:teuthology.orchestra.run.smithi055.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-26T21:40:36.063 INFO:teuthology.orchestra.run.smithi055.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T21:40:36.088 INFO:teuthology.orchestra.run.smithi055.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-04-26T21:40:36.121 INFO:teuthology.orchestra.run.smithi055.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-26T21:40:36.163 INFO:teuthology.orchestra.run.smithi165.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-04-26T21:40:36.163 INFO:teuthology.orchestra.run.smithi055.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-04-26T21:40:36.180 INFO:teuthology.orchestra.run.smithi055.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-26T21:40:36.188 INFO:teuthology.orchestra.run.smithi165.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-26T21:40:36.213 INFO:teuthology.orchestra.run.smithi165.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 226 kB/s | 43 kB 00:00 2024-04-26T21:40:36.231 INFO:teuthology.orchestra.run.smithi055.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-04-26T21:40:36.238 INFO:teuthology.orchestra.run.smithi165.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 434 kB/s | 32 kB 00:00 2024-04-26T21:40:36.248 INFO:teuthology.orchestra.run.smithi055.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-04-26T21:40:36.255 INFO:teuthology.orchestra.run.smithi165.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-04-26T21:40:36.273 INFO:teuthology.orchestra.run.smithi055.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-26T21:40:36.280 INFO:teuthology.orchestra.run.smithi165.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-26T21:40:36.290 INFO:teuthology.orchestra.run.smithi055.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-04-26T21:40:36.314 INFO:teuthology.orchestra.run.smithi165.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-26T21:40:36.315 INFO:teuthology.orchestra.run.smithi055.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T21:40:36.340 INFO:teuthology.orchestra.run.smithi165.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-04-26T21:40:36.356 INFO:teuthology.orchestra.run.smithi165.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 476 kB/s | 35 kB 00:00 2024-04-26T21:40:36.363 INFO:teuthology.orchestra.run.smithi064.stdout:(22/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 9.8 MB/s | 49 MB 00:05 2024-04-26T21:40:36.388 INFO:teuthology.orchestra.run.smithi064.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 584 kB/s | 321 kB 00:00 2024-04-26T21:40:36.389 INFO:teuthology.orchestra.run.smithi165.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-26T21:40:36.407 INFO:teuthology.orchestra.run.smithi055.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-26T21:40:36.430 INFO:teuthology.orchestra.run.smithi064.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 4.4 MB/s | 298 kB 00:00 2024-04-26T21:40:36.440 INFO:teuthology.orchestra.run.smithi165.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-04-26T21:40:36.465 INFO:teuthology.orchestra.run.smithi165.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-26T21:40:36.472 INFO:teuthology.orchestra.run.smithi064.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.0 MB/s | 85 kB 00:00 2024-04-26T21:40:36.474 INFO:teuthology.orchestra.run.smithi055.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-04-26T21:40:36.490 INFO:teuthology.orchestra.run.smithi165.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 109 kB/s | 11 kB 00:00 2024-04-26T21:40:36.497 INFO:teuthology.orchestra.run.smithi064.stdout:(26/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 914 kB/s | 99 kB 00:00 2024-04-26T21:40:36.507 INFO:teuthology.orchestra.run.smithi055.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-26T21:40:36.515 INFO:teuthology.orchestra.run.smithi165.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 355 kB/s | 18 kB 00:00 2024-04-26T21:40:36.532 INFO:teuthology.orchestra.run.smithi055.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-26T21:40:36.541 INFO:teuthology.orchestra.run.smithi165.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 309 kB/s | 23 kB 00:00 2024-04-26T21:40:36.566 INFO:teuthology.orchestra.run.smithi165.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-26T21:40:36.591 INFO:teuthology.orchestra.run.smithi165.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-04-26T21:40:36.592 INFO:teuthology.orchestra.run.smithi055.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T21:40:36.623 INFO:teuthology.orchestra.run.smithi064.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-26T21:40:36.625 INFO:teuthology.orchestra.run.smithi055.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-26T21:40:36.650 INFO:teuthology.orchestra.run.smithi055.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T21:40:36.716 INFO:teuthology.orchestra.run.smithi064.stdout:(28/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 13 MB/s | 3.1 MB 00:00 2024-04-26T21:40:36.717 INFO:teuthology.orchestra.run.smithi055.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-26T21:40:36.729 INFO:teuthology.orchestra.run.smithi165.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.4 MB/s | 1.0 MB 00:00 2024-04-26T21:40:36.741 INFO:teuthology.orchestra.run.smithi064.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3306.g 291 kB/s | 34 kB 00:00 2024-04-26T21:40:36.750 INFO:teuthology.orchestra.run.smithi165.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 252 kB/s | 46 kB 00:00 2024-04-26T21:40:36.766 INFO:teuthology.orchestra.run.smithi064.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.8 MB/s | 142 kB 00:00 2024-04-26T21:40:36.775 INFO:teuthology.orchestra.run.smithi165.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 430 kB/s | 79 kB 00:00 2024-04-26T21:40:36.792 INFO:teuthology.orchestra.run.smithi055.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-26T21:40:36.800 INFO:teuthology.orchestra.run.smithi165.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 816 kB/s | 58 kB 00:00 2024-04-26T21:40:36.826 INFO:teuthology.orchestra.run.smithi165.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-26T21:40:36.851 INFO:teuthology.orchestra.run.smithi165.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-04-26T21:40:36.860 INFO:teuthology.orchestra.run.smithi055.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-26T21:40:36.876 INFO:teuthology.orchestra.run.smithi165.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-26T21:40:36.885 INFO:teuthology.orchestra.run.smithi055.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 278 kB/s | 26 kB 00:00 2024-04-26T21:40:36.893 INFO:teuthology.orchestra.run.smithi165.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 424 kB/s | 28 kB 00:00 2024-04-26T21:40:36.910 INFO:teuthology.orchestra.run.smithi055.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-26T21:40:36.935 INFO:teuthology.orchestra.run.smithi165.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-04-26T21:40:36.935 INFO:teuthology.orchestra.run.smithi055.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-26T21:40:36.961 INFO:teuthology.orchestra.run.smithi165.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T21:40:36.961 INFO:teuthology.orchestra.run.smithi055.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-04-26T21:40:36.985 INFO:teuthology.orchestra.run.smithi165.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 640 kB/s | 59 kB 00:00 2024-04-26T21:40:37.011 INFO:teuthology.orchestra.run.smithi165.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-04-26T21:40:37.036 INFO:teuthology.orchestra.run.smithi165.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-26T21:40:37.037 INFO:teuthology.orchestra.run.smithi055.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-26T21:40:37.061 INFO:teuthology.orchestra.run.smithi055.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-26T21:40:37.062 INFO:teuthology.orchestra.run.smithi165.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T21:40:37.086 INFO:teuthology.orchestra.run.smithi165.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T21:40:37.087 INFO:teuthology.orchestra.run.smithi055.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-26T21:40:37.112 INFO:teuthology.orchestra.run.smithi165.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-26T21:40:37.112 INFO:teuthology.orchestra.run.smithi055.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T21:40:37.137 INFO:teuthology.orchestra.run.smithi055.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-26T21:40:37.153 INFO:teuthology.orchestra.run.smithi165.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 304 kB/s | 20 kB 00:00 2024-04-26T21:40:37.162 INFO:teuthology.orchestra.run.smithi055.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-26T21:40:37.179 INFO:teuthology.orchestra.run.smithi165.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-04-26T21:40:37.188 INFO:teuthology.orchestra.run.smithi055.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-26T21:40:37.213 INFO:teuthology.orchestra.run.smithi055.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-26T21:40:37.220 INFO:teuthology.orchestra.run.smithi165.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-26T21:40:37.238 INFO:teuthology.orchestra.run.smithi055.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T21:40:37.264 INFO:teuthology.orchestra.run.smithi055.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T21:40:37.329 INFO:teuthology.orchestra.run.smithi165.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-26T21:40:37.347 INFO:teuthology.orchestra.run.smithi055.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-26T21:40:37.372 INFO:teuthology.orchestra.run.smithi055.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-04-26T21:40:37.383 INFO:teuthology.orchestra.run.smithi064.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-26T21:40:37.398 INFO:teuthology.orchestra.run.smithi055.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-04-26T21:40:37.423 INFO:teuthology.orchestra.run.smithi055.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-26T21:40:37.448 INFO:teuthology.orchestra.run.smithi055.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-04-26T21:40:37.454 INFO:teuthology.orchestra.run.smithi165.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-26T21:40:37.473 INFO:teuthology.orchestra.run.smithi055.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-26T21:40:37.499 INFO:teuthology.orchestra.run.smithi055.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-26T21:40:37.524 INFO:teuthology.orchestra.run.smithi055.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-26T21:40:37.541 INFO:teuthology.orchestra.run.smithi055.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-26T21:40:37.575 INFO:teuthology.orchestra.run.smithi055.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-26T21:40:37.583 INFO:teuthology.orchestra.run.smithi064.stdout:(32/152): ceph-common-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 21 MB 00:13 2024-04-26T21:40:37.608 INFO:teuthology.orchestra.run.smithi055.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 667 kB/s | 45 kB 00:00 2024-04-26T21:40:37.609 INFO:teuthology.orchestra.run.smithi064.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3306.gbc 1.1 MB/s | 249 kB 00:00 2024-04-26T21:40:37.804 INFO:teuthology.orchestra.run.smithi064.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 84 kB/s | 16 kB 00:00 2024-04-26T21:40:37.834 INFO:teuthology.orchestra.run.smithi064.stdout:(35/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 197 kB/s | 49 kB 00:00 2024-04-26T21:40:37.892 INFO:teuthology.orchestra.run.smithi055.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-26T21:40:37.909 INFO:teuthology.orchestra.run.smithi064.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.5 MB/s | 273 kB 00:00 2024-04-26T21:40:38.163 INFO:teuthology.orchestra.run.smithi165.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-04-26T21:40:38.201 INFO:teuthology.orchestra.run.smithi055.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-26T21:40:38.243 INFO:teuthology.orchestra.run.smithi064.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 1.8 MB/s | 762 kB 00:00 2024-04-26T21:40:38.259 INFO:teuthology.orchestra.run.smithi055.stdout:(104/152): lua-devel-5.4.4-4.el9.x86_64.rpm 33 kB/s | 22 kB 00:00 2024-04-26T21:40:38.268 INFO:teuthology.orchestra.run.smithi064.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 51 kB/s | 18 kB 00:00 2024-04-26T21:40:38.284 INFO:teuthology.orchestra.run.smithi055.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-04-26T21:40:38.385 INFO:teuthology.orchestra.run.smithi055.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-04-26T21:40:38.402 INFO:teuthology.orchestra.run.smithi064.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 507 kB/s | 80 kB 00:00 2024-04-26T21:40:38.446 INFO:teuthology.orchestra.run.smithi165.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 2.6 MB/s | 3.4 MB 00:01 2024-04-26T21:40:38.454 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:40:38.455 INFO:teuthology.orchestra.run.smithi165.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-26T21:40:38.460 INFO:teuthology.orchestra.run.smithi064.stdout:(40/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 2.1 MB/s | 3.5 MB 00:01 2024-04-26T21:40:38.477 INFO:teuthology.orchestra.run.smithi064.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-26T21:40:38.493 INFO:teuthology.orchestra.run.smithi055.stdout:(107/152): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-26T21:40:38.519 INFO:teuthology.orchestra.run.smithi055.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 194 kB/s | 26 kB 00:00 2024-04-26T21:40:38.577 INFO:teuthology.orchestra.run.smithi055.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-04-26T21:40:38.578 INFO:teuthology.orchestra.run.smithi064.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-26T21:40:38.628 INFO:teuthology.orchestra.run.smithi055.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-26T21:40:38.653 INFO:teuthology.orchestra.run.smithi055.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 503 kB/s | 67 kB 00:00 2024-04-26T21:40:38.686 INFO:teuthology.orchestra.run.smithi064.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-26T21:40:38.703 INFO:teuthology.orchestra.run.smithi055.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-04-26T21:40:38.736 INFO:teuthology.orchestra.run.smithi064.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-04-26T21:40:38.761 INFO:teuthology.orchestra.run.smithi064.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-26T21:40:38.762 INFO:teuthology.orchestra.run.smithi055.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 747 kB/s | 43 kB 00:00 2024-04-26T21:40:38.787 INFO:teuthology.orchestra.run.smithi064.stdout:(46/152): python3-ply-3.11-14.el9.noarch.rpm 510 kB/s | 106 kB 00:00 2024-04-26T21:40:38.812 INFO:teuthology.orchestra.run.smithi064.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-26T21:40:38.820 INFO:teuthology.orchestra.run.smithi055.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-04-26T21:40:38.846 INFO:teuthology.orchestra.run.smithi055.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-26T21:40:38.867 INFO:teuthology.orchestra.run.smithi064.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-26T21:40:38.879 INFO:teuthology.orchestra.run.smithi055.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-04-26T21:40:38.913 INFO:teuthology.orchestra.run.smithi055.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-26T21:40:38.938 INFO:teuthology.orchestra.run.smithi055.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-26T21:40:38.988 INFO:teuthology.orchestra.run.smithi055.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-26T21:40:39.014 INFO:teuthology.orchestra.run.smithi055.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-04-26T21:40:39.046 INFO:teuthology.orchestra.run.smithi064.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 583 kB/s | 107 kB 00:00 2024-04-26T21:40:39.055 INFO:teuthology.orchestra.run.smithi055.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-26T21:40:39.105 INFO:teuthology.orchestra.run.smithi064.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 369 kB/s | 21 kB 00:00 2024-04-26T21:40:39.106 INFO:teuthology.orchestra.run.smithi055.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-04-26T21:40:39.131 INFO:teuthology.orchestra.run.smithi055.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-26T21:40:39.138 INFO:teuthology.orchestra.run.smithi064.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-04-26T21:40:39.156 INFO:teuthology.orchestra.run.smithi055.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-26T21:40:39.163 INFO:teuthology.orchestra.run.smithi064.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-26T21:40:39.182 INFO:teuthology.orchestra.run.smithi055.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-26T21:40:39.188 INFO:teuthology.orchestra.run.smithi064.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-04-26T21:40:39.207 INFO:teuthology.orchestra.run.smithi055.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-04-26T21:40:39.214 INFO:teuthology.orchestra.run.smithi064.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-04-26T21:40:39.232 INFO:teuthology.orchestra.run.smithi055.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-26T21:40:39.247 INFO:teuthology.orchestra.run.smithi064.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-04-26T21:40:39.266 INFO:teuthology.orchestra.run.smithi055.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-26T21:40:39.273 INFO:teuthology.orchestra.run.smithi064.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-26T21:40:39.306 INFO:teuthology.orchestra.run.smithi064.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T21:40:39.325 INFO:teuthology.orchestra.run.smithi055.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-04-26T21:40:39.332 INFO:teuthology.orchestra.run.smithi064.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-04-26T21:40:39.358 INFO:teuthology.orchestra.run.smithi055.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-26T21:40:39.392 INFO:teuthology.orchestra.run.smithi055.stdout:(131/152): protobuf-compiler-3.14.0-13.el9.x86_ 484 kB/s | 863 kB 00:01 2024-04-26T21:40:39.407 INFO:teuthology.orchestra.run.smithi064.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-04-26T21:40:39.417 INFO:teuthology.orchestra.run.smithi055.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 853 kB/s | 79 kB 00:00 2024-04-26T21:40:39.442 INFO:teuthology.orchestra.run.smithi055.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 687 kB/s | 58 kB 00:00 2024-04-26T21:40:39.474 INFO:teuthology.orchestra.run.smithi064.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.6 MB/s | 3.0 MB 00:00 2024-04-26T21:40:39.478 INFO:teuthology.orchestra.run.smithi055.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-04-26T21:40:39.499 INFO:teuthology.orchestra.run.smithi064.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-04-26T21:40:39.501 INFO:teuthology.orchestra.run.smithi055.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-26T21:40:39.524 INFO:teuthology.orchestra.run.smithi064.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-26T21:40:39.535 INFO:teuthology.orchestra.run.smithi055.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-04-26T21:40:39.550 INFO:teuthology.orchestra.run.smithi064.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-26T21:40:39.560 INFO:teuthology.orchestra.run.smithi055.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-26T21:40:39.568 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-26T21:40:39.575 INFO:teuthology.orchestra.run.smithi064.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-26T21:40:39.594 INFO:teuthology.orchestra.run.smithi055.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-26T21:40:39.600 INFO:teuthology.orchestra.run.smithi064.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-26T21:40:39.619 INFO:teuthology.orchestra.run.smithi055.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-26T21:40:39.625 INFO:teuthology.orchestra.run.smithi064.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-04-26T21:40:39.654 INFO:teuthology.orchestra.run.smithi055.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-04-26T21:40:39.678 INFO:teuthology.orchestra.run.smithi055.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-04-26T21:40:39.703 INFO:teuthology.orchestra.run.smithi055.stdout:(142/152): python3-pecan-1.4.2-3.el9.noarch.rpm 877 kB/s | 272 kB 00:00 2024-04-26T21:40:39.716 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-26T21:40:39.716 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-26T21:40:39.729 INFO:teuthology.orchestra.run.smithi055.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T21:40:39.754 INFO:teuthology.orchestra.run.smithi055.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T21:40:39.784 INFO:teuthology.orchestra.run.smithi064.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-26T21:40:39.787 INFO:teuthology.orchestra.run.smithi055.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-26T21:40:39.813 INFO:teuthology.orchestra.run.smithi055.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-26T21:40:39.838 INFO:teuthology.orchestra.run.smithi055.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-26T21:40:39.893 INFO:teuthology.orchestra.run.smithi064.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-26T21:40:39.899 INFO:teuthology.orchestra.run.smithi055.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-26T21:40:39.985 INFO:teuthology.orchestra.run.smithi064.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-26T21:40:40.005 INFO:teuthology.orchestra.run.smithi055.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 13 MB/s | 1.3 MB 00:00 2024-04-26T21:40:40.010 INFO:teuthology.orchestra.run.smithi064.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-26T21:40:40.068 INFO:teuthology.orchestra.run.smithi064.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.5 MB/s | 4.4 MB 00:00 2024-04-26T21:40:40.147 INFO:teuthology.orchestra.run.smithi055.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-04-26T21:40:40.197 INFO:teuthology.orchestra.run.smithi055.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 8.7 MB/s | 3.1 MB 00:00 2024-04-26T21:40:40.219 INFO:teuthology.orchestra.run.smithi064.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-04-26T21:40:40.244 INFO:teuthology.orchestra.run.smithi064.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 499 kB/s | 87 kB 00:00 2024-04-26T21:40:40.303 INFO:teuthology.orchestra.run.smithi064.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T21:40:40.356 INFO:teuthology.orchestra.run.smithi055.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 6.3 MB/s | 3.4 MB 00:00 2024-04-26T21:40:40.358 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:40:40.359 INFO:teuthology.orchestra.run.smithi055.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-26T21:40:40.369 INFO:teuthology.orchestra.run.smithi064.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-26T21:40:40.445 INFO:teuthology.orchestra.run.smithi064.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-26T21:40:40.470 INFO:teuthology.orchestra.run.smithi064.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-04-26T21:40:40.504 INFO:teuthology.orchestra.run.smithi064.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-26T21:40:40.529 INFO:teuthology.orchestra.run.smithi064.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T21:40:40.613 INFO:teuthology.orchestra.run.smithi064.stdout:(80/152): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-26T21:40:40.637 INFO:teuthology.orchestra.run.smithi064.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 260 kB/s | 35 kB 00:00 2024-04-26T21:40:40.696 INFO:teuthology.orchestra.run.smithi064.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-26T21:40:40.721 INFO:teuthology.orchestra.run.smithi064.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-26T21:40:40.755 INFO:teuthology.orchestra.run.smithi064.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T21:40:40.780 INFO:teuthology.orchestra.run.smithi064.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 721 kB/s | 42 kB 00:00 2024-04-26T21:40:40.831 INFO:teuthology.orchestra.run.smithi064.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-26T21:40:40.855 INFO:teuthology.orchestra.run.smithi064.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T21:40:40.889 INFO:teuthology.orchestra.run.smithi064.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-26T21:40:40.915 INFO:teuthology.orchestra.run.smithi064.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-04-26T21:40:40.990 INFO:teuthology.orchestra.run.smithi064.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-26T21:40:41.015 INFO:teuthology.orchestra.run.smithi064.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 426 kB/s | 54 kB 00:00 2024-04-26T21:40:41.066 INFO:teuthology.orchestra.run.smithi064.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-26T21:40:41.099 INFO:teuthology.orchestra.run.smithi064.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 528 kB/s | 44 kB 00:00 2024-04-26T21:40:41.141 INFO:teuthology.orchestra.run.smithi064.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-26T21:40:41.183 INFO:teuthology.orchestra.run.smithi064.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 429 kB/s | 36 kB 00:00 2024-04-26T21:40:41.217 INFO:teuthology.orchestra.run.smithi064.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-26T21:40:41.267 INFO:teuthology.orchestra.run.smithi064.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-26T21:40:41.300 INFO:teuthology.orchestra.run.smithi064.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 693 kB/s | 58 kB 00:00 2024-04-26T21:40:41.342 INFO:teuthology.orchestra.run.smithi064.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-26T21:40:41.384 INFO:teuthology.orchestra.run.smithi064.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-04-26T21:40:41.418 INFO:teuthology.orchestra.run.smithi064.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-04-26T21:40:41.443 INFO:teuthology.orchestra.run.smithi064.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-04-26T21:40:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-04-26T21:40:41.481 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-26T21:40:41.482 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-26T21:40:41.493 INFO:teuthology.orchestra.run.smithi064.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-26T21:40:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-04-26T21:40:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-04-26T21:40:41.802 INFO:teuthology.orchestra.run.smithi064.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-26T21:40:42.044 INFO:teuthology.orchestra.run.smithi064.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-04-26T21:40:42.086 INFO:teuthology.orchestra.run.smithi064.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 548 kB/s | 309 kB 00:00 2024-04-26T21:40:42.154 INFO:teuthology.orchestra.run.smithi064.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-04-26T21:40:42.178 INFO:teuthology.orchestra.run.smithi064.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 746 kB/s | 548 kB 00:00 2024-04-26T21:40:42.228 INFO:teuthology.orchestra.run.smithi064.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-04-26T21:40:42.279 INFO:teuthology.orchestra.run.smithi064.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-04-26T21:40:42.312 INFO:teuthology.orchestra.run.smithi064.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-26T21:40:42.588 INFO:teuthology.orchestra.run.smithi064.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-04-26T21:40:42.621 INFO:teuthology.orchestra.run.smithi064.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.4 MB/s | 839 kB 00:00 2024-04-26T21:40:42.688 INFO:teuthology.orchestra.run.smithi064.stdout:(114/152): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-04-26T21:40:42.713 INFO:teuthology.orchestra.run.smithi064.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 346 kB/s | 43 kB 00:00 2024-04-26T21:40:42.742 INFO:teuthology.orchestra.run.smithi064.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-04-26T21:40:42.764 INFO:teuthology.orchestra.run.smithi064.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-26T21:40:42.872 INFO:teuthology.orchestra.run.smithi064.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-26T21:40:43.006 INFO:teuthology.orchestra.run.smithi064.stdout:(119/152): python3-cheroot-8.6.0-4.el9.noarch.r 643 kB/s | 172 kB 00:00 2024-04-26T21:40:43.031 INFO:teuthology.orchestra.run.smithi064.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 222 kB/s | 35 kB 00:00 2024-04-26T21:40:43.084 INFO:teuthology.orchestra.run.smithi064.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 948 kB/s | 303 kB 00:00 2024-04-26T21:40:43.134 INFO:teuthology.orchestra.run.smithi064.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-26T21:40:43.159 INFO:teuthology.orchestra.run.smithi064.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-26T21:40:43.209 INFO:teuthology.orchestra.run.smithi064.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-26T21:40:43.234 INFO:teuthology.orchestra.run.smithi064.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-26T21:40:43.260 INFO:teuthology.orchestra.run.smithi064.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-26T21:40:43.285 INFO:teuthology.orchestra.run.smithi064.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-26T21:40:43.310 INFO:teuthology.orchestra.run.smithi064.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-26T21:40:43.335 INFO:teuthology.orchestra.run.smithi064.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-26T21:40:43.377 INFO:teuthology.orchestra.run.smithi064.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 694 kB/s | 46 kB 00:00 2024-04-26T21:40:43.403 INFO:teuthology.orchestra.run.smithi064.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-26T21:40:43.453 INFO:teuthology.orchestra.run.smithi064.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-04-26T21:40:43.478 INFO:teuthology.orchestra.run.smithi064.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-26T21:40:43.505 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-04-26T21:40:43.505 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-04-26T21:40:43.520 INFO:teuthology.orchestra.run.smithi064.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-26T21:40:43.562 INFO:teuthology.orchestra.run.smithi064.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-26T21:40:43.596 INFO:teuthology.orchestra.run.smithi064.stdout:(136/152): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-04-26T21:40:43.613 INFO:teuthology.orchestra.run.smithi064.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-26T21:40:43.630 INFO:teuthology.orchestra.run.smithi064.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-04-26T21:40:43.682 INFO:teuthology.orchestra.run.smithi064.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 883 kB/s | 59 kB 00:00 2024-04-26T21:40:43.705 INFO:teuthology.orchestra.run.smithi064.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-26T21:40:43.730 INFO:teuthology.orchestra.run.smithi064.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-26T21:40:43.756 INFO:teuthology.orchestra.run.smithi064.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-26T21:40:43.781 INFO:teuthology.orchestra.run.smithi064.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T21:40:43.814 INFO:teuthology.orchestra.run.smithi064.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-26T21:40:43.840 INFO:teuthology.orchestra.run.smithi064.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-26T21:40:43.865 INFO:teuthology.orchestra.run.smithi064.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-26T21:40:43.924 INFO:teuthology.orchestra.run.smithi064.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T21:40:43.966 INFO:teuthology.orchestra.run.smithi064.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-26T21:40:44.307 INFO:teuthology.orchestra.run.smithi064.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.0 MB/s | 1.3 MB 00:00 2024-04-26T21:40:44.524 INFO:teuthology.orchestra.run.smithi064.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 5.2 MB/s | 3.4 MB 00:00 2024-04-26T21:40:44.575 INFO:teuthology.orchestra.run.smithi064.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-04-26T21:40:46.016 INFO:teuthology.orchestra.run.smithi064.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 1.5 MB/s | 3.1 MB 00:02 2024-04-26T21:40:46.023 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:40:46.023 INFO:teuthology.orchestra.run.smithi064.stdout:Total 9.6 MB/s | 222 MB 00:23 2024-04-26T21:40:47.165 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-04-26T21:40:47.326 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-04-26T21:40:47.326 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-04-26T21:40:47.456 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-26T21:40:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:40:47.663 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:40:47.937 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:40:47.985 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:40:48.009 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:40:48.054 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:40:48.383 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:40:48.493 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:40:48.640 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:40:48.661 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:40:48.744 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:40:48.790 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:40:48.939 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:40:48.987 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:40:49.014 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:40:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:40:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:40:49.196 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-04-26T21:40:49.196 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-04-26T21:40:49.203 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:40:49.225 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:40:49.249 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:40:49.441 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:40:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:40:49.582 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:40:49.676 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:40:49.715 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:40:49.726 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-04-26T21:40:49.749 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:40:49.815 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:40:49.849 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:40:49.852 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:40:49.893 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:40:49.925 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:40:49.953 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:40:50.023 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:40:50.082 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:40:50.084 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:40:50.123 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:40:50.138 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:40:50.167 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:40:50.179 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:40:50.212 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:40:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:40:50.256 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:40:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:40:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:40:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:40:50.419 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:40:50.464 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:40:50.538 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:40:50.599 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:40:50.681 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:40:50.712 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:40:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:40:50.837 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:40:50.888 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:40:51.097 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:40:51.126 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:40:51.260 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:40:51.289 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:40:51.340 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:40:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:40:51.367 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:40:51.381 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:40:51.424 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:40:51.462 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:40:51.473 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:40:51.502 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:40:51.504 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:40:51.564 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:40:51.586 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:40:51.611 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:40:51.659 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:40:51.707 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:40:51.761 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:40:51.804 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:40:51.836 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:40:51.842 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:40:51.874 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:40:51.910 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:40:51.955 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:40:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:40:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:40:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:40:52.050 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:40:52.078 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:40:52.210 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:40:52.214 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:40:52.245 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:40:52.275 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:40:52.305 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:40:52.320 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:40:52.338 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:40:52.361 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:40:52.405 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:40:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:40:52.437 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:40:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:40:52.465 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:40:52.503 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:40:52.529 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:40:52.535 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:40:52.553 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:40:52.588 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:40:52.626 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:40:52.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:40:52.670 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:40:52.747 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:40:52.755 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:40:52.785 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:40:52.789 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:40:52.813 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:40:52.833 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:40:52.847 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:40:52.860 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:40:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:40:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:40:52.932 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:40:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:40:52.976 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:40:52.982 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:40:53.016 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:40:53.052 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:40:53.057 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:40:53.092 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:40:53.178 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:40:53.196 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:40:53.222 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:40:53.253 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:40:53.253 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:40:53.291 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:40:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:40:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:40:53.802 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:40:53.892 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:40:53.935 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:40:53.979 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:40:54.016 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:40:54.059 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:40:54.166 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:40:54.209 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:40:54.226 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:40:54.260 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:40:54.261 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:40:54.317 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:40:54.359 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:40:54.390 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:40:54.409 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:40:54.420 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:40:54.456 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:40:54.497 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:40:54.503 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:40:54.539 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:40:54.545 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:40:54.582 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:40:54.625 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:40:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:40:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:40:54.766 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:40:54.805 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:40:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:40:54.835 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:40:54.877 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:40:54.974 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:40:55.010 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:40:55.050 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:40:55.075 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:40:55.100 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:40:55.177 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:40:55.267 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-04-26T21:40:55.306 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:40:55.362 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T21:40:55.409 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T21:40:55.507 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:40:55.538 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T21:40:55.575 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:40:55.602 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:40:55.615 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:40:55.637 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:40:55.655 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T21:40:55.677 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:40:55.680 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T21:40:55.725 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T21:40:55.725 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:40:55.754 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:40:55.789 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:40:55.822 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:40:55.864 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:40:55.898 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:40:55.989 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:40:56.038 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:40:56.068 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:40:56.068 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T21:40:56.115 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:40:56.180 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:40:56.224 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:40:56.280 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:40:56.305 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:40:56.331 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:40:56.424 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:40:56.453 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:40:56.610 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T21:40:56.658 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T21:40:56.685 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:40:56.774 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:40:56.803 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:40:56.849 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:40:56.866 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T21:40:56.887 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:40:56.906 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T21:40:56.920 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:40:56.948 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:40:56.981 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:40:57.034 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:40:57.083 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:40:57.105 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:40:57.117 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:40:57.137 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T21:40:57.220 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:40:57.247 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T21:40:57.276 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:40:57.294 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:40:57.336 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:40:57.345 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T21:40:57.373 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:40:57.386 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T21:40:57.426 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:40:57.430 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T21:40:57.457 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:40:57.461 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:40:57.493 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:40:57.505 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T21:40:57.546 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T21:40:57.589 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T21:40:57.621 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T21:40:57.645 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:40:57.649 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T21:40:57.675 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:40:57.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T21:40:57.755 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:40:57.798 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T21:40:57.827 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T21:40:57.871 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T21:40:57.958 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T21:40:57.971 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:40:57.990 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T21:40:57.996 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:40:58.038 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T21:40:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:40:58.071 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T21:40:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:40:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T21:40:58.202 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T21:40:58.280 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T21:40:58.425 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T21:40:58.429 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:40:58.480 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T21:40:58.552 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:40:59.042 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T21:40:59.127 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T21:40:59.162 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T21:40:59.205 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T21:40:59.243 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T21:40:59.285 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T21:40:59.453 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T21:40:59.506 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T21:40:59.560 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T21:40:59.635 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T21:40:59.671 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T21:40:59.708 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T21:40:59.755 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T21:40:59.790 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T21:40:59.835 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T21:40:59.858 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:40:59.876 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T21:40:59.938 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:40:59.966 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:40:59.999 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:41:00.019 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T21:41:00.074 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T21:41:00.104 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T21:41:00.145 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T21:41:00.242 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T21:41:00.277 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T21:41:00.318 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T21:41:00.328 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:41:00.343 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T21:41:00.366 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:00.368 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T21:41:00.370 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:41:00.402 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:00.446 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T21:41:00.453 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:41:00.483 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:41:00.519 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:41:00.574 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T21:41:00.608 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T21:41:00.636 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T21:41:00.670 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T21:41:00.710 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T21:41:00.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T21:41:00.771 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T21:41:00.805 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T21:41:00.847 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T21:41:00.890 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T21:41:00.923 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T21:41:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:01.006 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:41:01.014 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T21:41:01.026 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:41:01.056 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:41:01.063 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T21:41:01.096 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:41:01.097 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:41:01.097 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:41:01.097 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:01.101 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T21:41:01.123 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:41:01.127 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:41:01.150 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T21:41:01.158 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:41:01.261 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T21:41:01.312 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T21:41:02.076 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T21:41:02.125 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T21:41:02.158 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T21:41:02.261 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T21:41:02.326 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T21:41:02.415 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T21:41:02.452 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T21:41:02.635 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T21:41:02.666 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T21:41:02.754 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T21:41:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T21:41:03.555 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T21:41:03.588 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:04.160 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:04.199 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:41:04.290 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:41:04.290 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:41:04.290 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:41:04.290 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:04.317 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:41:04.894 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T21:41:04.969 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T21:41:04.998 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T21:41:05.030 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T21:41:05.359 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T21:41:05.393 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:41:05.485 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T21:41:05.515 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T21:41:05.551 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T21:41:06.055 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T21:41:06.079 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:41:06.151 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T21:41:06.172 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T21:41:08.594 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:08.627 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:09.210 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T21:41:09.247 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:41:09.351 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T21:41:09.351 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T21:41:09.352 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T21:41:09.352 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:09.381 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-04-26T21:41:12.461 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-04-26T21:41:12.462 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:12.657 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:41:12.673 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:41:12.673 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:12.673 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:41:12.674 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:14.152 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:41:14.177 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:41:14.177 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:14.177 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:41:14.178 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:41:14.178 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:41:14.178 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:14.394 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:41:14.417 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:41:14.417 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:14.417 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:41:14.417 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:41:14.417 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:41:14.417 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:41:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:41:14.884 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:14.884 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:41:14.884 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:41:14.884 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:41:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:14.991 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:41:15.048 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:41:15.104 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:41:15.289 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:41:15.337 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:41:15.415 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:41:15.503 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:41:15.635 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:41:15.679 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:41:15.710 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:41:15.738 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:41:16.156 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:41:16.186 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:41:16.218 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:41:16.266 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:41:16.266 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /sys 2024-04-26T21:41:16.266 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /proc 2024-04-26T21:41:16.266 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /mnt 2024-04-26T21:41:16.266 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /var/tmp 2024-04-26T21:41:16.266 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /home 2024-04-26T21:41:16.266 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /root 2024-04-26T21:41:16.267 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /tmp 2024-04-26T21:41:16.267 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:16.288 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:41:16.326 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:41:16.439 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:41:16.459 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:41:16.459 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:16.459 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:41:16.459 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:17.362 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:41:17.402 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:41:17.993 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:41:18.019 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:41:18.019 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:18.020 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:41:18.020 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:41:18.020 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:41:18.020 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:18.251 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:41:18.275 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:41:18.275 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:18.275 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:41:18.275 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:41:18.276 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:41:18.276 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:18.367 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:41:18.402 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:41:18.505 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:41:18.672 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:41:18.704 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:41:18.728 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:41:18.728 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:18.728 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:41:18.728 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:41:18.728 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:41:18.728 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:18.744 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:41:18.768 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:41:18.768 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:18.768 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:41:18.768 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:41:18.768 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:41:18.769 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:18.780 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:41:18.791 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:41:18.883 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:41:18.889 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:41:18.938 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:41:18.941 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:41:19.005 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:41:19.197 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:41:19.253 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:41:19.334 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:41:19.424 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:41:19.566 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:41:19.612 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:41:19.644 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:41:19.671 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:41:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:41:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:41:19.930 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:41:19.931 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:19.931 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:41:19.931 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:41:19.931 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:41:19.931 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:19.975 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:41:19.997 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:41:19.997 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:19.997 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:41:19.997 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:20.042 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:41:20.059 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:41:20.065 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:41:20.065 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:20.065 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:41:20.065 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:41:20.065 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:41:20.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:20.086 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:41:20.119 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:41:20.183 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:41:20.210 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:41:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:41:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:41:20.391 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:20.391 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:41:20.391 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:41:20.391 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:41:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:21.303 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:41:21.336 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:41:21.478 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T21:41:21.478 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-04-26T21:41:21.478 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-04-26T21:41:21.478 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-04-26T21:41:21.478 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-04-26T21:41:21.478 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-04-26T21:41:21.478 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-04-26T21:41:21.478 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-04-26T21:41:21.479 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:21.658 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:41:21.674 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T21:41:21.674 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:21.675 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T21:41:21.675 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:22.350 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:41:22.370 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:41:22.477 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:41:22.647 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:41:22.681 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:41:22.705 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:41:22.705 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:22.705 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:41:22.705 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:41:22.705 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:41:22.705 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:22.756 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:41:22.767 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:41:22.869 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:41:22.922 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:41:23.229 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:41:23.256 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T21:41:23.256 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:23.257 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T21:41:23.257 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:41:23.257 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T21:41:23.257 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:23.487 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:41:23.511 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T21:41:23.512 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:23.512 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T21:41:23.512 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:41:23.512 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T21:41:23.512 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:23.898 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:41:23.934 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:41:23.958 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:41:23.959 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:23.959 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:41:23.959 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:41:23.959 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:41:23.959 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:23.971 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:41:23.996 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T21:41:23.996 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:23.996 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T21:41:23.996 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:41:23.996 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T21:41:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:24.002 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:41:24.025 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:41:24.025 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:24.025 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:41:24.025 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:24.077 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:41:24.101 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:41:24.101 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:24.101 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:41:24.101 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:41:24.102 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:41:24.102 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:24.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T21:41:24.168 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T21:41:24.232 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T21:41:24.419 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:41:24.426 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T21:41:24.443 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:41:24.443 INFO:teuthology.orchestra.run.smithi055.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:24.444 INFO:teuthology.orchestra.run.smithi055.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:41:24.444 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:41:24.444 INFO:teuthology.orchestra.run.smithi055.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:41:24.444 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:24.481 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T21:41:24.561 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T21:41:24.650 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:41:24.793 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:41:24.831 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T21:41:24.862 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T21:41:24.890 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T21:41:25.013 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:41:25.054 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:41:25.095 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:41:25.127 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:41:25.154 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:41:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:41:25.361 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:41:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:41:25.388 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T21:41:25.413 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:41:25.421 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T21:41:25.485 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T21:41:25.511 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:41:25.999 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:26.043 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:26.064 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:26.605 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T21:41:26.646 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:41:26.907 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:41:27.006 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:41:27.047 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:41:27.137 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:41:27.176 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:41:27.177 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:41:27.269 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:41:27.302 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:41:27.302 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:41:27.392 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:41:27.458 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:27.660 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T21:41:27.688 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:41:27.798 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T21:41:27.965 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T21:41:27.998 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:41:28.023 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T21:41:28.023 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:28.023 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T21:41:28.024 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:41:28.024 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T21:41:28.024 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:28.074 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:41:28.086 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T21:41:28.187 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T21:41:28.241 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T21:41:29.216 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T21:41:29.252 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:41:29.276 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T21:41:29.276 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:29.276 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T21:41:29.277 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:41:29.277 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T21:41:29.277 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:29.298 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:41:29.319 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:41:29.341 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:41:29.343 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T21:41:29.343 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:29.343 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T21:41:29.343 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:29.381 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:41:29.403 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:41:29.421 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:41:29.427 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T21:41:29.427 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:29.427 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T21:41:29.427 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:41:29.427 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T21:41:29.427 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:29.449 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:41:29.646 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:41:29.694 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:41:29.724 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:41:29.745 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:41:29.769 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T21:41:29.769 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T21:41:29.769 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T21:41:29.769 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:41:29.769 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T21:41:29.770 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:30.337 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:30.372 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:30.392 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:31.284 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:41:31.377 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:41:31.419 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:41:31.503 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:41:31.540 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:41:31.540 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:41:31.616 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:41:31.648 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:41:31.648 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:41:31.738 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:41:31.809 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:33.379 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:41:33.380 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:41:33.381 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:41:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:41:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:41:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:41:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:41:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:41:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:41:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:41:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:41:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:41:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:41:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:41:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:41:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:41:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:41:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:41:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:41:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:41:34.658 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T21:41:34.699 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T21:41:34.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T21:41:34.772 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T21:41:34.799 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T21:41:34.994 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T21:41:35.095 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T21:41:35.124 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T21:41:35.521 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:35.521 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:35.521 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-04-26T21:41:35.521 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.521 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.521 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:35.521 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.522 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:35.523 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:41:35.524 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:41:35.525 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:41:35.526 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:41:35.527 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:41:35.528 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:41:35.529 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:41:35.530 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:41:35.531 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:41:35.532 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:41:35.532 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:41:35.532 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-26T21:41:35.727 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:35.763 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:35.784 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T21:41:35.962 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:36.690 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T21:41:36.777 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T21:41:36.818 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T21:41:36.902 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T21:41:36.939 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T21:41:36.939 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:41:37.023 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T21:41:37.055 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:41:37.056 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:41:37.154 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T21:41:37.224 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:37.925 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:37.925 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:41:37.926 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:41:37.927 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:41:37.939 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:41:37.940 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:41:37.941 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:41:37.942 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:41:37.943 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:41:37.944 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:41:37.945 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:41:37.946 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:41:37.947 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:41:40.059 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:40.059 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:40.059 INFO:teuthology.orchestra.run.smithi055.stdout:Upgraded: 2024-04-26T21:41:40.059 INFO:teuthology.orchestra.run.smithi055.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.060 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.061 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:41:40.062 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:41:40.063 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:41:40.064 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.065 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:41:40.066 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:41:40.067 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:41:40.068 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:41:40.069 INFO:teuthology.orchestra.run.smithi055.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:41:40.070 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:41:40.071 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:41:40.071 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:41:40.071 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-04-26T21:41:40.476 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T21:41:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T21:41:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T21:41:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T21:41:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T21:41:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T21:41:42.859 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T21:41:42.860 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T21:41:42.861 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T21:41:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T21:41:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T21:41:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T21:41:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T21:41:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T21:41:42.876 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T21:41:42.877 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T21:41:42.878 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T21:41:42.879 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T21:41:42.880 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T21:41:42.881 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T21:41:42.882 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T21:41:42.883 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T21:41:42.884 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T21:41:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T21:41:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T21:41:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T21:41:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T21:41:42.885 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T21:41:44.958 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T21:41:44.959 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.960 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T21:41:44.961 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T21:41:44.962 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T21:41:44.963 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.964 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T21:41:44.965 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T21:41:44.966 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T21:41:44.967 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T21:41:44.968 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T21:41:44.969 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T21:41:44.970 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T21:41:44.970 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T21:41:44.970 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T21:41:44.970 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T21:41:44.970 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-04-26T21:41:44.970 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-04-26T21:41:45.346 DEBUG:teuthology.parallel:result is None 2024-04-26T21:41:45.346 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:41:45.704 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:41:45.730 INFO:teuthology.orchestra.run.smithi055.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:41:45.730 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:41:45.730 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:41:45.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:41:45.877 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:41:45.900 INFO:teuthology.orchestra.run.smithi064.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:41:45.900 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:41:45.900 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:41:45.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:41:46.271 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T21:41:46.297 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T21:41:46.298 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T21:41:46.298 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T21:41:46.300 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T21:41:46.300 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:41:46.300 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:41:46.332 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:41:46.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:41:46.360 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:41:46.360 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T21:41:46.390 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T21:41:46.390 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:41:46.390 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:41:46.417 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:41:46.484 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:41:46.484 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:41:46.508 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:41:46.575 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:41:46.575 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T21:41:46.603 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T21:41:46.671 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T21:41:46.671 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:41:46.671 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:41:46.698 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:41:46.765 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:41:46.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:41:46.791 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:41:46.856 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:41:46.856 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T21:41:46.884 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T21:41:46.951 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T21:41:46.951 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:41:46.952 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:41:46.979 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:41:47.046 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:41:47.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:41:47.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:41:47.134 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:41:47.134 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T21:41:47.162 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T21:41:47.229 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T21:41:47.321 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-26T21:41:47.321 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:41:47.322 INFO:tasks.cephadm:Cluster fsid is c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:41:47.322 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T21:41:47.322 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.55', 'mon.b': '172.21.15.64', 'mon.c': '172.21.15.165'} 2024-04-26T21:41:47.322 INFO:tasks.cephadm:First mon is mon.a on smithi055 2024-04-26T21:41:47.323 INFO:tasks.cephadm:First mgr is x 2024-04-26T21:41:47.323 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T21:41:47.323 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname $(hostname -s) 2024-04-26T21:41:47.352 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-04-26T21:41:47.377 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-04-26T21:41:47.404 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T21:41:47.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:41:47.545 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T21:41:47.682 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:41:47.684 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-26T21:41:47.684 DEBUG:teuthology.orchestra.run.smithi055:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:48.466 INFO:teuthology.orchestra.run.smithi055.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:41 /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:48.466 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:49.396 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:41 /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:49.396 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:50.149 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 26 21:41 /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:50.149 DEBUG:teuthology.orchestra.run.smithi055:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:50.169 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:50.188 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T21:41:50.225 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-26T21:41:50.225 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:41:50.228 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:41:50.231 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-26T21:41:51.853 INFO:teuthology.orchestra.run.smithi055.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:41:51.867 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:41:51.937 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:42:19.775 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:42:19.775 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:42:19.775 INFO:teuthology.orchestra.run.smithi055.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:42:19.775 INFO:teuthology.orchestra.run.smithi055.stdout: "repo_digests": [ 2024-04-26T21:42:19.776 INFO:teuthology.orchestra.run.smithi055.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:42:19.776 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T21:42:19.776 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:42:21.563 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T21:42:21.563 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:42:21.563 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:42:21.563 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-04-26T21:42:21.563 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:42:21.564 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-26T21:42:21.564 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T21:42:31.584 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-26T21:42:31.585 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T21:42:31.585 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-26T21:42:31.585 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-04-26T21:42:31.585 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-26T21:42:31.585 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-04-26T21:42:31.585 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-26T21:42:31.608 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph 2024-04-26T21:42:31.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-04-26T21:42:31.670 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-04-26T21:42:31.701 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /etc/ceph 2024-04-26T21:42:31.729 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-04-26T21:42:31.753 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-04-26T21:42:31.779 INFO:tasks.cephadm:Writing seed config... 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T21:42:31.780 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T21:42:31.781 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T21:42:31.782 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T21:42:31.783 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T21:42:31.783 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-26T21:42:31.783 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:42:31.783 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T21:42:31.801 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c84f1140-0415-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T21:42:31.801 DEBUG:teuthology.orchestra.run.smithi055:mon.a> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T21:42:31.844 DEBUG:teuthology.orchestra.run.smithi055:mgr.x> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T21:42:31.888 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T21:42:31.889 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid c84f1140-0415-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.55 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:42:32.090 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-04-26T21:42:32.090 INFO:teuthology.orchestra.run.smithi055.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', 'c84f1140-0415-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.55', '--skip-admin-label'] 2024-04-26T21:42:32.091 INFO:teuthology.orchestra.run.smithi055.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T21:42:32.091 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying podman|docker is present... 2024-04-26T21:42:32.118 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:42:32.118 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying lvm2 is present... 2024-04-26T21:42:32.118 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying time synchronization is in place... 2024-04-26T21:42:32.126 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:42:32.126 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:42:32.132 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:42:32.133 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-04-26T21:42:32.140 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout enabled 2024-04-26T21:42:32.146 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout active 2024-04-26T21:42:32.146 INFO:teuthology.orchestra.run.smithi055.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:42:32.146 INFO:teuthology.orchestra.run.smithi055.stdout:Repeating the final host check... 2024-04-26T21:42:32.171 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T21:42:32.171 INFO:teuthology.orchestra.run.smithi055.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T21:42:32.171 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl is present 2024-04-26T21:42:32.171 INFO:teuthology.orchestra.run.smithi055.stdout:lvcreate is present 2024-04-26T21:42:32.177 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T21:42:32.178 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T21:42:32.185 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T21:42:32.185 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-04-26T21:42:32.192 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout enabled 2024-04-26T21:42:32.199 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout active 2024-04-26T21:42:32.199 INFO:teuthology.orchestra.run.smithi055.stdout:Unit chronyd.service is enabled and running 2024-04-26T21:42:32.199 INFO:teuthology.orchestra.run.smithi055.stdout:Host looks OK 2024-04-26T21:42:32.199 INFO:teuthology.orchestra.run.smithi055.stdout:Cluster fsid: c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:32.199 INFO:teuthology.orchestra.run.smithi055.stdout:Acquiring lock 139787777666208 on /run/cephadm/c84f1140-0415-11ef-bc93-c7b262605968.lock 2024-04-26T21:42:32.199 INFO:teuthology.orchestra.run.smithi055.stdout:Lock 139787777666208 acquired on /run/cephadm/c84f1140-0415-11ef-bc93-c7b262605968.lock 2024-04-26T21:42:32.200 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying IP 172.21.15.55 port 3300 ... 2024-04-26T21:42:32.200 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying IP 172.21.15.55 port 6789 ... 2024-04-26T21:42:32.200 INFO:teuthology.orchestra.run.smithi055.stdout:Base mon IP(s) is [172.21.15.55:3300, 172.21.15.55:6789], mon addrv is [v2:172.21.15.55:3300,v1:172.21.15.55:6789] 2024-04-26T21:42:32.203 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.55 metric 100 2024-04-26T21:42:32.203 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.55 metric 100 2024-04-26T21:42:32.206 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T21:42:32.206 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-26T21:42:32.206 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-26T21:42:32.209 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T21:42:32.209 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T21:42:32.209 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:42:32.209 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-26T21:42:32.209 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-04-26T21:42:32.209 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T21:42:32.210 INFO:teuthology.orchestra.run.smithi055.stdout:Mon IP `172.21.15.55` is in CIDR network `172.21.0.0/20` 2024-04-26T21:42:32.210 INFO:teuthology.orchestra.run.smithi055.stdout:Mon IP `172.21.15.55` is in CIDR network `172.21.0.0/20` 2024-04-26T21:42:32.210 INFO:teuthology.orchestra.run.smithi055.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T21:42:32.210 INFO:teuthology.orchestra.run.smithi055.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T21:42:32.211 INFO:teuthology.orchestra.run.smithi055.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:42:32.751 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:42:32.752 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-26T21:42:32.752 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T21:42:32.752 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-26T21:42:32.752 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T21:42:32.752 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-26T21:42:32.752 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T21:42:33.672 INFO:teuthology.orchestra.run.smithi055.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:42:33.673 INFO:teuthology.orchestra.run.smithi055.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-26T21:42:33.673 INFO:teuthology.orchestra.run.smithi055.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T21:42:34.487 INFO:teuthology.orchestra.run.smithi055.stdout:stat: stdout 167 167 2024-04-26T21:42:34.488 INFO:teuthology.orchestra.run.smithi055.stdout:Creating initial keys... 2024-04-26T21:42:35.534 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph-authtool: stdout AQDKHyxmPb39MRAAbL2mYjnetczSXiWk0oz/Sw== 2024-04-26T21:42:44.923 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph-authtool: stdout AQDOHyxmR94NBBAAXypoDLVrByY8MiVf3pmqlQ== 2024-04-26T21:42:48.767 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph-authtool: stdout AQDXHyxmaGbEGRAAwVs6llVm7nvSpslCMXqZxA== 2024-04-26T21:42:48.768 INFO:teuthology.orchestra.run.smithi055.stdout:Creating initial monmap... 2024-04-26T21:42:49.757 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:42:49.757 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T21:42:49.758 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:49.758 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:42:49.758 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool for a [v2:172.21.15.55:3300,v1:172.21.15.55:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T21:42:49.758 INFO:teuthology.orchestra.run.smithi055.stdout:setting min_mon_release = quincy 2024-04-26T21:42:49.758 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: set fsid to c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:49.758 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T21:42:49.758 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:42:49.758 INFO:teuthology.orchestra.run.smithi055.stdout:Creating mon... 2024-04-26T21:42:51.288 INFO:teuthology.orchestra.run.smithi055.stdout:create mon.a on 2024-04-26T21:42:51.498 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T21:42:51.728 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T21:42:51.921 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c84f1140-0415-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-c84f1140-0415-11ef-bc93-c7b262605968.target. 2024-04-26T21:42:51.921 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c84f1140-0415-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-c84f1140-0415-11ef-bc93-c7b262605968.target. 2024-04-26T21:42:52.165 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.a 2024-04-26T21:42:52.165 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T21:42:52.332 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c84f1140-0415-11ef-bc93-c7b262605968.target.wants/ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c84f1140-0415-11ef-bc93-c7b262605968@.service. 2024-04-26T21:42:52.829 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:42:52.829 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout disabled 2024-04-26T21:42:52.836 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:42:52.836 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-04-26T21:42:52.836 INFO:teuthology.orchestra.run.smithi055.stdout:firewalld.service is not enabled 2024-04-26T21:42:52.836 INFO:teuthology.orchestra.run.smithi055.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:42:52.836 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mon to start... 2024-04-26T21:42:52.836 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mon... 2024-04-26T21:42:52.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:52 smithi055 systemd[1]: Started Ceph mon.a for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: monmap epoch 1 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: last_changed 2024-04-26T21:42:49.108371+0000 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: created 2024-04-26T21:42:49.108371+0000 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: min_mon_release 19 (squid) 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: election_strategy: 1 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: fsmap 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:42:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: mgrmap e1: no daemons active 2024-04-26T21:42:54.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:53 smithi055 ceph-mon[28848]: from='client.? 172.21.15.55:0/1812046355' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T21:42:55.266 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T21:42:55.266 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout id: c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout services: 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.594487s) 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout data: 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.302+0000 7f8d76db8700 1 Processor -- start 2024-04-26T21:42:55.267 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.302+0000 7f8d76db8700 1 -- start start 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.302+0000 7f8d76db8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d70109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.302+0000 7f8d76db8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70109740 con 0x7f8d70108d50 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.303+0000 7f8d74b54700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d70109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.303+0000 7f8d74b54700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d70109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52510/0 (socket says 172.21.15.55:52510) 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.303+0000 7f8d74b54700 1 -- 172.21.15.55:0/3867214431 learned_addr learned my addr 172.21.15.55:0/3867214431 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.303+0000 7f8d74b54700 1 -- 172.21.15.55:0/3867214431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d70109f50 con 0x7f8d70108d50 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.303+0000 7f8d74b54700 1 --2- 172.21.15.55:0/3867214431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d70109170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8d600098a0 tx=0x7f8d60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d841602ef72148e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.303+0000 7f8d6f7fe700 1 -- 172.21.15.55:0/3867214431 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d60004160 con 0x7f8d70108d50 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.303+0000 7f8d6f7fe700 1 -- 172.21.15.55:0/3867214431 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60004300 con 0x7f8d70108d50 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.303+0000 7f8d6f7fe700 1 -- 172.21.15.55:0/3867214431 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d600036a0 con 0x7f8d70108d50 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.304+0000 7f8d76db8700 1 -- 172.21.15.55:0/3867214431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 msgr2=0x7f8d70109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.304+0000 7f8d76db8700 1 --2- 172.21.15.55:0/3867214431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d70109170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8d600098a0 tx=0x7f8d60009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.268 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.304+0000 7f8d76db8700 1 -- 172.21.15.55:0/3867214431 shutdown_connections 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.304+0000 7f8d76db8700 1 --2- 172.21.15.55:0/3867214431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d70109170 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.304+0000 7f8d76db8700 1 -- 172.21.15.55:0/3867214431 >> 172.21.15.55:0/3867214431 conn(0x7f8d701042f0 msgr2=0x7f8d70106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.304+0000 7f8d76db8700 1 -- 172.21.15.55:0/3867214431 shutdown_connections 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.304+0000 7f8d76db8700 1 -- 172.21.15.55:0/3867214431 wait complete. 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d76db8700 1 Processor -- start 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d76db8700 1 -- start start 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d76db8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d7019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d76db8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7010a480 con 0x7f8d70108d50 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d74b54700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d7019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d74b54700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d7019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52516/0 (socket says 172.21.15.55:52516) 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d74b54700 1 -- 172.21.15.55:0/1812046355 learned_addr learned my addr 172.21.15.55:0/1812046355 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d74b54700 1 -- 172.21.15.55:0/1812046355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7019eb60 con 0x7f8d70108d50 2024-04-26T21:42:55.269 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.305+0000 7f8d74b54700 1 --2- 172.21.15.55:0/1812046355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d7019e620 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8d60000c00 tx=0x7f8d60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.306+0000 7f8d6dffb700 1 -- 172.21.15.55:0/1812046355 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d60003db0 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.306+0000 7f8d6dffb700 1 -- 172.21.15.55:0/1812046355 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60011630 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.306+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7019edf0 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.306+0000 7f8d6dffb700 1 -- 172.21.15.55:0/1812046355 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d60011950 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.306+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d701a1ae0 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.306+0000 7f8d6dffb700 1 -- 172.21.15.55:0/1812046355 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d60011af0 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.306+0000 7f8d6dffb700 1 -- 172.21.15.55:0/1812046355 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8d6001c450 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.307+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d70064a80 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.309+0000 7f8d6dffb700 1 -- 172.21.15.55:0/1812046355 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8d60032020 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.382+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f8d701a1cb0 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.383+0000 7f8d6dffb700 1 -- 172.21.15.55:0/1812046355 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f8d60004260 con 0x7f8d70108d50 2024-04-26T21:42:55.270 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.383+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 msgr2=0x7f8d7019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.383+0000 7f8d76db8700 1 --2- 172.21.15.55:0/1812046355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d7019e620 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8d60000c00 tx=0x7f8d60004750 comp rx=0 tx=0).stop 2024-04-26T21:42:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.384+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 shutdown_connections 2024-04-26T21:42:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.384+0000 7f8d76db8700 1 --2- 172.21.15.55:0/1812046355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d70108d50 0x7f8d7019e620 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.384+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 >> 172.21.15.55:0/1812046355 conn(0x7f8d701042f0 msgr2=0x7f8d7007e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.384+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 shutdown_connections 2024-04-26T21:42:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:53.384+0000 7f8d76db8700 1 -- 172.21.15.55:0/1812046355 wait complete. 2024-04-26T21:42:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:mon is available 2024-04-26T21:42:55.271 INFO:teuthology.orchestra.run.smithi055.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T21:42:55.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:55 smithi055 ceph-mon[28848]: from='client.? 172.21.15.55:0/1203452834' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:42:55.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:55 smithi055 ceph-mon[28848]: from='client.? 172.21.15.55:0/1203452834' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T21:42:56.332 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout fsid = c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.55:3300,v1:172.21.15.55:6789] 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:42:56.333 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.713+0000 7ff3e0c65700 1 Processor -- start 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.713+0000 7ff3e0c65700 1 -- start start 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.713+0000 7ff3e0c65700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.713+0000 7ff3e0c65700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc109740 con 0x7ff3dc108d50 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.713+0000 7ff3da59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.713+0000 7ff3da59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52526/0 (socket says 172.21.15.55:52526) 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.713+0000 7ff3da59c700 1 -- 172.21.15.55:0/3608456196 learned_addr learned my addr 172.21.15.55:0/3608456196 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.714+0000 7ff3da59c700 1 -- 172.21.15.55:0/3608456196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc109f50 con 0x7ff3dc108d50 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.714+0000 7ff3da59c700 1 --2- 172.21.15.55:0/3608456196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc109170 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40098a0 tx=0x7ff3c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460c18ff704b46ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.334 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.714+0000 7ff3d959a700 1 -- 172.21.15.55:0/3608456196 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3c4004160 con 0x7ff3dc108d50 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.714+0000 7ff3d959a700 1 -- 172.21.15.55:0/3608456196 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c4004300 con 0x7ff3dc108d50 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.714+0000 7ff3d959a700 1 -- 172.21.15.55:0/3608456196 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3c40036a0 con 0x7ff3dc108d50 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.715+0000 7ff3e0c65700 1 -- 172.21.15.55:0/3608456196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 msgr2=0x7ff3dc109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.715+0000 7ff3e0c65700 1 --2- 172.21.15.55:0/3608456196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc109170 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40098a0 tx=0x7ff3c4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.715+0000 7ff3e0c65700 1 -- 172.21.15.55:0/3608456196 shutdown_connections 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.715+0000 7ff3e0c65700 1 --2- 172.21.15.55:0/3608456196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc109170 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.715+0000 7ff3e0c65700 1 -- 172.21.15.55:0/3608456196 >> 172.21.15.55:0/3608456196 conn(0x7ff3dc1042f0 msgr2=0x7ff3dc106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.715+0000 7ff3e0c65700 1 -- 172.21.15.55:0/3608456196 shutdown_connections 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.715+0000 7ff3e0c65700 1 -- 172.21.15.55:0/3608456196 wait complete. 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3e0c65700 1 Processor -- start 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3e0c65700 1 -- start start 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3e0c65700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:56.335 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3e0c65700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc10a480 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3da59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3da59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc19c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52534/0 (socket says 172.21.15.55:52534) 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3da59c700 1 -- 172.21.15.55:0/1203452834 learned_addr learned my addr 172.21.15.55:0/1203452834 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3da59c700 1 -- 172.21.15.55:0/1203452834 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc19c920 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.716+0000 7ff3da59c700 1 --2- 172.21.15.55:0/1203452834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc19c3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000c00 tx=0x7ff3c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.717+0000 7ff3d37fe700 1 -- 172.21.15.55:0/1203452834 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3c4003db0 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.717+0000 7ff3d37fe700 1 -- 172.21.15.55:0/1203452834 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c4011630 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.717+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3dc19cbb0 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.717+0000 7ff3d37fe700 1 -- 172.21.15.55:0/1203452834 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3c4011950 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.717+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3dc19f8a0 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.717+0000 7ff3d37fe700 1 -- 172.21.15.55:0/1203452834 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff3c4011af0 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.717+0000 7ff3d37fe700 1 -- 172.21.15.55:0/1203452834 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff3c401c450 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.718+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3dc064a80 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.720+0000 7ff3d37fe700 1 -- 172.21.15.55:0/1203452834 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff3c4032020 con 0x7ff3dc108d50 2024-04-26T21:42:56.336 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.791+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff3dc02e610 con 0x7ff3dc108d50 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.794+0000 7ff3d37fe700 1 -- 172.21.15.55:0/1203452834 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7ff3c4004260 con 0x7ff3dc108d50 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.795+0000 7ff3d37fe700 1 -- 172.21.15.55:0/1203452834 <== mon.0 v2:172.21.15.55:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff3c401c650 con 0x7ff3dc108d50 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.796+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 msgr2=0x7ff3dc19c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.796+0000 7ff3e0c65700 1 --2- 172.21.15.55:0/1203452834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc19c3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4000c00 tx=0x7ff3c4004750 comp rx=0 tx=0).stop 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.796+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 shutdown_connections 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.796+0000 7ff3e0c65700 1 --2- 172.21.15.55:0/1203452834 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3dc108d50 0x7ff3dc19c3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.796+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 >> 172.21.15.55:0/1203452834 conn(0x7ff3dc1042f0 msgr2=0x7ff3dc193050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.796+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 shutdown_connections 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:55.796+0000 7ff3e0c65700 1 -- 172.21.15.55:0/1203452834 wait complete. 2024-04-26T21:42:56.337 INFO:teuthology.orchestra.run.smithi055.stdout:Generating new minimal ceph.conf... 2024-04-26T21:42:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:56 smithi055 ceph-mon[28848]: from='client.? 172.21.15.55:0/1328917945' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.781+0000 7fb768a11700 1 Processor -- start 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.781+0000 7fb768a11700 1 -- start start 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb768a11700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb7641079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb768a11700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb76407ce70 con 0x7fb7641075b0 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb76259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb7641079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb76259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb7641079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52542/0 (socket says 172.21.15.55:52542) 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb76259c700 1 -- 172.21.15.55:0/2725256179 learned_addr learned my addr 172.21.15.55:0/2725256179 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb76259c700 1 -- 172.21.15.55:0/2725256179 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb764107f10 con 0x7fb7641075b0 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb76259c700 1 --2- 172.21.15.55:0/2725256179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb7641079d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098a0 tx=0x7fb754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb339016ffa638dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb76159a700 1 -- 172.21.15.55:0/2725256179 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754004160 con 0x7fb7641075b0 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.782+0000 7fb76159a700 1 -- 172.21.15.55:0/2725256179 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb754004300 con 0x7fb7641075b0 2024-04-26T21:42:57.387 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.783+0000 7fb76159a700 1 -- 172.21.15.55:0/2725256179 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754011af0 con 0x7fb7641075b0 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.783+0000 7fb768a11700 1 -- 172.21.15.55:0/2725256179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 msgr2=0x7fb7641079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.783+0000 7fb768a11700 1 --2- 172.21.15.55:0/2725256179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb7641079d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098a0 tx=0x7fb754009bb0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.783+0000 7fb768a11700 1 -- 172.21.15.55:0/2725256179 shutdown_connections 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.783+0000 7fb768a11700 1 --2- 172.21.15.55:0/2725256179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb7641079d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.783+0000 7fb768a11700 1 -- 172.21.15.55:0/2725256179 >> 172.21.15.55:0/2725256179 conn(0x7fb764103190 msgr2=0x7fb7641055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.783+0000 7fb768a11700 1 -- 172.21.15.55:0/2725256179 shutdown_connections 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.783+0000 7fb768a11700 1 -- 172.21.15.55:0/2725256179 wait complete. 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.784+0000 7fb768a11700 1 Processor -- start 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.784+0000 7fb768a11700 1 -- start start 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.784+0000 7fb768a11700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb76419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.784+0000 7fb768a11700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb764108f40 con 0x7fb7641075b0 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.784+0000 7fb76259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb76419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.784+0000 7fb76259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb76419ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52552/0 (socket says 172.21.15.55:52552) 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.784+0000 7fb76259c700 1 -- 172.21.15.55:0/1328917945 learned_addr learned my addr 172.21.15.55:0/1328917945 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb76259c700 1 -- 172.21.15.55:0/1328917945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76419f390 con 0x7fb7641075b0 2024-04-26T21:42:57.388 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb76259c700 1 --2- 172.21.15.55:0/1328917945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb76419ee50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb754011ac0 tx=0x7fb754011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb7537fe700 1 -- 172.21.15.55:0/1328917945 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754003970 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb7537fe700 1 -- 172.21.15.55:0/1328917945 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb754003b10 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76419f620 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb7537fe700 1 -- 172.21.15.55:0/1328917945 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754024610 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7641a2310 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb7537fe700 1 -- 172.21.15.55:0/1328917945 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb754024c20 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.785+0000 7fb7537fe700 1 -- 172.21.15.55:0/1328917945 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb7540199c0 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.786+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb764064a80 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.789+0000 7fb7537fe700 1 -- 172.21.15.55:0/1328917945 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb754043ec0 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.860+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb7640008d0 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.860+0000 7fb7537fe700 1 -- 172.21.15.55:0/1328917945 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb754032030 con 0x7fb7641075b0 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.861+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 msgr2=0x7fb76419ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.861+0000 7fb768a11700 1 --2- 172.21.15.55:0/1328917945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb76419ee50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb754011ac0 tx=0x7fb754011ba0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.861+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 shutdown_connections 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.861+0000 7fb768a11700 1 --2- 172.21.15.55:0/1328917945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7641075b0 0x7fb76419ee50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:42:57.389 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.861+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 >> 172.21.15.55:0/1328917945 conn(0x7fb764103190 msgr2=0x7fb764195ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:42:57.390 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.861+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 shutdown_connections 2024-04-26T21:42:57.390 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:56.861+0000 7fb768a11700 1 -- 172.21.15.55:0/1328917945 wait complete. 2024-04-26T21:42:57.390 INFO:teuthology.orchestra.run.smithi055.stdout:Restarting the monitor... 2024-04-26T21:42:57.684 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:57 smithi055 systemd[1]: Stopping Ceph mon.a for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:42:57.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a[28844]: 2024-04-26T21:42:57.682+0000 7fe39dc25700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T21:42:57.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a[28844]: 2024-04-26T21:42:57.682+0000 7fe39dc25700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T21:42:57.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:57 smithi055 podman[29093]: 2024-04-26 21:42:57.847457067 +0000 UTC m=+0.284715081 container died d67d22a96a827d8d9c539682a859d621a0989897e3815ede0220b5aadf5ecf0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T21:42:58.569 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:58 smithi055 podman[29093]: 2024-04-26 21:42:58.341808857 +0000 UTC m=+0.779066871 container cleanup d67d22a96a827d8d9c539682a859d621a0989897e3815ede0220b5aadf5ecf0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T21:42:58.569 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:58 smithi055 bash[29093]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a 2024-04-26T21:42:58.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:58 smithi055 podman[29107]: 2024-04-26 21:42:58.567676508 +0000 UTC m=+0.718410343 container remove d67d22a96a827d8d9c539682a859d621a0989897e3815ede0220b5aadf5ecf0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-26T21:42:58.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:58 smithi055 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T21:42:58.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:58 smithi055 systemd[1]: Stopped Ceph mon.a for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:42:58.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:58 smithi055 systemd[1]: Starting Ceph mon.a for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:42:59.178 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:58 smithi055 podman[29203]: 2024-04-26 21:42:58.926936545 +0000 UTC m=+0.112161552 container create 8c4a4ee318a8d27bdd6b29be04c79001b7f625ffb9b2800b282376e26bf68907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:58 smithi055 podman[29203]: 2024-04-26 21:42:58.833169081 +0000 UTC m=+0.018394085 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 podman[29203]: 2024-04-26 21:42:59.137734701 +0000 UTC m=+0.322959705 container init 8c4a4ee318a8d27bdd6b29be04c79001b7f625ffb9b2800b282376e26bf68907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 podman[29203]: 2024-04-26 21:42:59.141414288 +0000 UTC m=+0.326639293 container start 8c4a4ee318a8d27bdd6b29be04c79001b7f625ffb9b2800b282376e26bf68907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: pidfile_write: ignore empty --pid-file 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: load: jerasure load: lrc 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Git sha 0 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: DB SUMMARY 2024-04-26T21:42:59.179 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: DB Session ID: YK4T3X0E1FNXYO1WFB81 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104064 ; 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.env: 0x55f8576bfcc0 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:42:59.180 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.info_log: 0x55f859685d80 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.statistics: (nil) 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.use_fsync: 0 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:42:59.181 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.db_log_dir: 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.wal_dir: 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.write_buffer_manager: 0x55f8596570e0 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:42:59.182 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:42:59.183 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:42:59.184 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:42:59.184 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:42:59.184 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:42:59.184 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.unordered_write: 0 2024-04-26T21:42:59.184 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:42:59.184 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:42:59.184 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:42:59.184 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.row_cache: None 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.wal_filter: None 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.wal_compression: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:42:59.185 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:42:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_open_files: -1 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Compression algorithms supported: 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: kZSTD supported: 0 2024-04-26T21:42:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.merge_operator: 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_filter: None 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:42:59.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f859685f40) 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: index_type: 0 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: data_block_index_type: 0 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: index_shortening: 1 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: checksum: 4 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: no_block_cache: 0 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_cache: 0x55f8596891f0 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:42:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_cache_options: 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: capacity : 536870912 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: num_shard_bits : 4 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: strict_capacity_limit : 0 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_cache_compressed: (nil) 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: persistent_cache: (nil) 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_size: 4096 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_size_deviation: 10 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_restart_interval: 16 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: index_block_restart_interval: 1 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: metadata_block_size: 4096 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: partition_filters: 0 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: use_delta_encoding: 1 2024-04-26T21:42:59.190 INFO:journalctl@ceph.mon.a.smithi055.stdout: filter_policy: bloomfilter 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: whole_key_filtering: 1 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: verify_compression: 0 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: format_version: 5 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: enable_index_compression: 1 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_align: 0 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: max_auto_readahead_size: 262144 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: prepopulate_block_cache: 0 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression: NoCompression 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:42:59.191 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.num_levels: 7 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:42:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:42:59.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:42:59.194 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:42:59.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.ttl: 2592000 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:42:59.196 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 20f036f1-453a-4492-9150-b0bff3e69720 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167779170680, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167779171786, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99292, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97238, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13917, "raw_average_key_size": 49, "raw_value_size": 89632, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714167779, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "20f036f1-453a-4492-9150-b0bff3e69720", "db_session_id": "YK4T3X0E1FNXYO1WFB81", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:42:59.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167779171873, "job": 1, "event": "recovery_finished"} 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f8597a6000 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: DB pointer 0x55f859792000 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout: ** DB Stats ** 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:42:59.198 INFO:journalctl@ceph.mon.a.smithi055.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: ** Compaction Stats [default] ** 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: L0 2/0 98.83 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: Sum 2/0 98.83 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: ** Compaction Stats [default] ** 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-04-26T21:42:59.199 INFO:journalctl@ceph.mon.a.smithi055.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: Cumulative compaction: 0.00 GB write, 11.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: Interval compaction: 0.00 GB write, 11.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: Block cache BinnedLRUCache@0x55f8596891f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: starting mon.a rank 0 at public addrs [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] at bind addrs [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a@-1(???) e1 preinit fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:59.200 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a@-1(???).mds e1 new map 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a@-1(???).mds e1 print_map 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout: e1 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout: legacy client fscid: -1 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout: No filesystems configured 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:42:59.201 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T21:42:59.226 INFO:teuthology.orchestra.run.smithi055.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T21:42:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 bash[29203]: 8c4a4ee318a8d27bdd6b29be04c79001b7f625ffb9b2800b282376e26bf68907 2024-04-26T21:42:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 systemd[1]: Started Ceph mon.a for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:42:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T21:42:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: monmap epoch 1 2024-04-26T21:42:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:42:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: last_changed 2024-04-26T21:42:49.108371+0000 2024-04-26T21:42:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: created 2024-04-26T21:42:49.108371+0000 2024-04-26T21:42:59.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: min_mon_release 19 (squid) 2024-04-26T21:42:59.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: election_strategy: 1 2024-04-26T21:42:59.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-04-26T21:42:59.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: fsmap 2024-04-26T21:42:59.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T21:42:59.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:42:59 smithi055 ceph-mon[29221]: mgrmap e1: no daemons active 2024-04-26T21:43:00.655 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.637+0000 7f1ae6713700 1 Processor -- start 2024-04-26T21:43:00.660 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.637+0000 7f1ae6713700 1 -- start start 2024-04-26T21:43:00.660 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.637+0000 7f1ae6713700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae0109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.660 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.637+0000 7f1ae6713700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0109740 con 0x7f1ae0108d50 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.638+0000 7f1adffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae0109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.638+0000 7f1adffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae0109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52568/0 (socket says 172.21.15.55:52568) 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.638+0000 7f1adffff700 1 -- 172.21.15.55:0/1998023211 learned_addr learned my addr 172.21.15.55:0/1998023211 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.638+0000 7f1adffff700 1 -- 172.21.15.55:0/1998023211 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae0109f50 con 0x7f1ae0108d50 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.638+0000 7f1adffff700 1 --2- 172.21.15.55:0/1998023211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae0109170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00098a0 tx=0x7f1ad0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e46e6efc8ac548e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1adeffd700 1 -- 172.21.15.55:0/1998023211 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad0004160 con 0x7f1ae0108d50 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1adeffd700 1 -- 172.21.15.55:0/1998023211 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ad0004300 con 0x7f1ae0108d50 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1adeffd700 1 -- 172.21.15.55:0/1998023211 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad0011af0 con 0x7f1ae0108d50 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1ae6713700 1 -- 172.21.15.55:0/1998023211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 msgr2=0x7f1ae0109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1ae6713700 1 --2- 172.21.15.55:0/1998023211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae0109170 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00098a0 tx=0x7f1ad0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1ae6713700 1 -- 172.21.15.55:0/1998023211 shutdown_connections 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1ae6713700 1 --2- 172.21.15.55:0/1998023211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae0109170 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1ae6713700 1 -- 172.21.15.55:0/1998023211 >> 172.21.15.55:0/1998023211 conn(0x7f1ae01042f0 msgr2=0x7f1ae0106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1ae6713700 1 -- 172.21.15.55:0/1998023211 shutdown_connections 2024-04-26T21:43:00.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.639+0000 7f1ae6713700 1 -- 172.21.15.55:0/1998023211 wait complete. 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.640+0000 7f1ae6713700 1 Processor -- start 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.640+0000 7f1ae6713700 1 -- start start 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.640+0000 7f1ae6713700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.640+0000 7f1ae6713700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010d180 con 0x7f1ae0108d50 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.640+0000 7f1adffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.640+0000 7f1adffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52582/0 (socket says 172.21.15.55:52582) 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.640+0000 7f1adffff700 1 -- 172.21.15.55:0/3269190268 learned_addr learned my addr 172.21.15.55:0/3269190268 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1adffff700 1 -- 172.21.15.55:0/3269190268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae019d140 con 0x7f1ae0108d50 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1adffff700 1 --2- 172.21.15.55:0/3269190268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae019cc00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0000c00 tx=0x7f1ad0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1add7fa700 1 -- 172.21.15.55:0/3269190268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad0003b20 con 0x7f1ae0108d50 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae019d3d0 con 0x7f1ae0108d50 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1add7fa700 1 -- 172.21.15.55:0/3269190268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1ad001a8c0 con 0x7f1ae0108d50 2024-04-26T21:43:00.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1add7fa700 1 -- 172.21.15.55:0/3269190268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ad0024940 con 0x7f1ae0108d50 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01a00c0 con 0x7f1ae0108d50 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1add7fa700 1 -- 172.21.15.55:0/3269190268 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ad0022020 con 0x7f1ae0108d50 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.641+0000 7f1add7fa700 1 -- 172.21.15.55:0/3269190268 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ad00193c0 con 0x7f1ae0108d50 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.642+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae010d300 con 0x7f1ae0108d50 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.645+0000 7f1add7fa700 1 -- 172.21.15.55:0/3269190268 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1ad002dde0 con 0x7f1ae0108d50 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.716+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f1ae01a0300 con 0x7f1ae0108d50 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.718+0000 7f1add7fa700 1 -- 172.21.15.55:0/3269190268 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f1ad001aa70 con 0x7f1ae0108d50 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.718+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 msgr2=0x7f1ae019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.718+0000 7f1ae6713700 1 --2- 172.21.15.55:0/3269190268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae019cc00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0000c00 tx=0x7f1ad0011b70 comp rx=0 tx=0).stop 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.719+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 shutdown_connections 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.719+0000 7f1ae6713700 1 --2- 172.21.15.55:0/3269190268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ae0108d50 0x7f1ae019cc00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.719+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 >> 172.21.15.55:0/3269190268 conn(0x7f1ae01042f0 msgr2=0x7f1ae0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.719+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 shutdown_connections 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:42:59.719+0000 7f1ae6713700 1 -- 172.21.15.55:0/3269190268 wait complete. 2024-04-26T21:43:00.663 INFO:teuthology.orchestra.run.smithi055.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T21:43:00.685 INFO:teuthology.orchestra.run.smithi055.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:43:00.685 INFO:teuthology.orchestra.run.smithi055.stdout:Creating mgr... 2024-04-26T21:43:00.685 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T21:43:00.686 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T21:43:00.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:00 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3269190268' entity='client.admin' 2024-04-26T21:43:00.953 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.x 2024-04-26T21:43:00.953 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T21:43:01.101 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c84f1140-0415-11ef-bc93-c7b262605968.target.wants/ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c84f1140-0415-11ef-bc93-c7b262605968@.service. 2024-04-26T21:43:01.321 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 systemd[1]: Starting Ceph mgr.x for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:43:01.599 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 podman[29436]: 2024-04-26 21:43:01.32096029 +0000 UTC m=+0.099149344 container create 60ea280c07805f4512b85e66ee4c46ef7cec8d36ed20e7e237d9dba51314ddae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-26T21:43:01.599 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 podman[29436]: 2024-04-26 21:43:01.240359546 +0000 UTC m=+0.018548628 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:43:01.599 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 podman[29436]: 2024-04-26 21:43:01.523701997 +0000 UTC m=+0.301891050 container init 60ea280c07805f4512b85e66ee4c46ef7cec8d36ed20e7e237d9dba51314ddae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T21:43:01.599 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 podman[29436]: 2024-04-26 21:43:01.526234601 +0000 UTC m=+0.304423656 container start 60ea280c07805f4512b85e66ee4c46ef7cec8d36ed20e7e237d9dba51314ddae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T21:43:01.599 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 bash[29436]: 60ea280c07805f4512b85e66ee4c46ef7cec8d36ed20e7e237d9dba51314ddae 2024-04-26T21:43:01.613 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:43:01.613 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout disabled 2024-04-26T21:43:01.620 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:43:01.621 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-04-26T21:43:01.621 INFO:teuthology.orchestra.run.smithi055.stdout:firewalld.service is not enabled 2024-04-26T21:43:01.621 INFO:teuthology.orchestra.run.smithi055.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T21:43:01.629 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:43:01.629 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout disabled 2024-04-26T21:43:01.637 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:43:01.637 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-04-26T21:43:01.637 INFO:teuthology.orchestra.run.smithi055.stdout:firewalld.service is not enabled 2024-04-26T21:43:01.637 INFO:teuthology.orchestra.run.smithi055.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T21:43:01.637 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mgr to start... 2024-04-26T21:43:01.637 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mgr... 2024-04-26T21:43:01.877 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 systemd[1]: Started Ceph mgr.x for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:43:01.877 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:01.683+0000 7fefb4078200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:43:01.877 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:01.876+0000 7fefb4078200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:43:02.187 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:01.983+0000 7fefb4078200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:43:02.538 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:02 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2436102845' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:43:02.539 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:02.327+0000 7fefb4078200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:43:02.539 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:02.414+0000 7fefb4078200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:43:02.834 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:43:02.834 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:02.834 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:43:02.834 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:43:02.834 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:43:02.835 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:43:02.836 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:43:02.837 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:42:52.793467+0000", 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:02.838 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.106+0000 7f52b4bb2700 1 Processor -- start 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.106+0000 7f52b4bb2700 1 -- start start 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.107+0000 7f52b4bb2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b0109100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.107+0000 7f52b4bb2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01096d0 con 0x7f52b0108ce0 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.107+0000 7f52ae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b0109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.107+0000 7f52ae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b0109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38850/0 (socket says 172.21.15.55:38850) 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.107+0000 7f52ae59c700 1 -- 172.21.15.55:0/1777157949 learned_addr learned my addr 172.21.15.55:0/1777157949 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.107+0000 7f52ae59c700 1 -- 172.21.15.55:0/1777157949 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b0109ee0 con 0x7f52b0108ce0 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.107+0000 7f52ae59c700 1 --2- 172.21.15.55:0/1777157949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b0109100 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f52980098a0 tx=0x7f5298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf8c112e0b4e6d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.108+0000 7f52ad59a700 1 -- 172.21.15.55:0/1777157949 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5298004160 con 0x7f52b0108ce0 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.108+0000 7f52ad59a700 1 -- 172.21.15.55:0/1777157949 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5298004300 con 0x7f52b0108ce0 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.108+0000 7f52ad59a700 1 -- 172.21.15.55:0/1777157949 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5298011af0 con 0x7f52b0108ce0 2024-04-26T21:43:02.839 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.108+0000 7f52b4bb2700 1 -- 172.21.15.55:0/1777157949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 msgr2=0x7f52b0109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.108+0000 7f52b4bb2700 1 --2- 172.21.15.55:0/1777157949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b0109100 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f52980098a0 tx=0x7f5298009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.109+0000 7f52b4bb2700 1 -- 172.21.15.55:0/1777157949 shutdown_connections 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.109+0000 7f52b4bb2700 1 --2- 172.21.15.55:0/1777157949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b0109100 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.109+0000 7f52b4bb2700 1 -- 172.21.15.55:0/1777157949 >> 172.21.15.55:0/1777157949 conn(0x7f52b01042e0 msgr2=0x7f52b0106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.109+0000 7f52b4bb2700 1 -- 172.21.15.55:0/1777157949 shutdown_connections 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.109+0000 7f52b4bb2700 1 -- 172.21.15.55:0/1777157949 wait complete. 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.109+0000 7f52b4bb2700 1 Processor -- start 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.109+0000 7f52b4bb2700 1 -- start start 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.110+0000 7f52b4bb2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b007eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.110+0000 7f52b4bb2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f52b010d290 con 0x7f52b0108ce0 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.110+0000 7f52ae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b007eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:02.840 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.110+0000 7f52ae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b007eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38862/0 (socket says 172.21.15.55:38862) 2024-04-26T21:43:02.842 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.110+0000 7f52ae59c700 1 -- 172.21.15.55:0/2436102845 learned_addr learned my addr 172.21.15.55:0/2436102845 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.110+0000 7f52ae59c700 1 -- 172.21.15.55:0/2436102845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b007f0d0 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.110+0000 7f52ae59c700 1 --2- 172.21.15.55:0/2436102845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b007eb90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5298000c00 tx=0x7f5298011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.111+0000 7f52a77fe700 1 -- 172.21.15.55:0/2436102845 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5298003b20 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.111+0000 7f52a77fe700 1 -- 172.21.15.55:0/2436102845 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f529801a8c0 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.111+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b007d2c0 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.111+0000 7f52a77fe700 1 -- 172.21.15.55:0/2436102845 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5298024940 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.111+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b007d800 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.111+0000 7f52a77fe700 1 -- 172.21.15.55:0/2436102845 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5298022020 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.111+0000 7f52a77fe700 1 -- 172.21.15.55:0/2436102845 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f529802ddd0 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.112+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b0064a80 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.114+0000 7f52a77fe700 1 -- 172.21.15.55:0/2436102845 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f5298042ec0 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.203+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f52b007dab0 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.203+0000 7f52a77fe700 1 -- 172.21.15.55:0/2436102845 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5298031020 con 0x7f52b0108ce0 2024-04-26T21:43:02.843 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.204+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 msgr2=0x7f52b007eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.204+0000 7f52b4bb2700 1 --2- 172.21.15.55:0/2436102845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b007eb90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5298000c00 tx=0x7f5298011b70 comp rx=0 tx=0).stop 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.204+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 shutdown_connections 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.204+0000 7f52b4bb2700 1 --2- 172.21.15.55:0/2436102845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52b0108ce0 0x7f52b007eb90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.204+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 >> 172.21.15.55:0/2436102845 conn(0x7f52b01042e0 msgr2=0x7f52b0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.204+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 shutdown_connections 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:02.204+0000 7f52b4bb2700 1 -- 172.21.15.55:0/2436102845 wait complete. 2024-04-26T21:43:02.844 INFO:teuthology.orchestra.run.smithi055.stdout:mgr not available, waiting (1/15)... 2024-04-26T21:43:03.923 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:03.674+0000 7fefb4078200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:43:03.924 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:03.853+0000 7fefb4078200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:43:04.186 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:04.034+0000 7fefb4078200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:43:04.658 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:04.346+0000 7fefb4078200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:43:04.658 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:04.446+0000 7fefb4078200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:43:04.658 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:04.657+0000 7fefb4078200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:43:04.911 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:04.861+0000 7fefb4078200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:43:05.505 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:05 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/594455642' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:43:05.936 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:05 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:05.503+0000 7fefb4078200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:43:06.059 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:06.060 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:43:06.061 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:43:06.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:43:06.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:42:52.793467+0000", 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:43:06.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.335+0000 7fc2c7bab700 1 Processor -- start 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.335+0000 7fc2c7bab700 1 -- start start 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.335+0000 7fc2c7bab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c0109110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.335+0000 7fc2c7bab700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01096e0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.335+0000 7fc2c5947700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c0109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.335+0000 7fc2c5947700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c0109110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38866/0 (socket says 172.21.15.55:38866) 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.335+0000 7fc2c5947700 1 -- 172.21.15.55:0/3049598669 learned_addr learned my addr 172.21.15.55:0/3049598669 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.336+0000 7fc2c5947700 1 -- 172.21.15.55:0/3049598669 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c0109ef0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.336+0000 7fc2c5947700 1 --2- 172.21.15.55:0/3049598669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c0109110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0098a0 tx=0x7fc2bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43635c92d3190591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.336+0000 7fc2c4945700 1 -- 172.21.15.55:0/3049598669 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2bc004160 con 0x7fc2c0108cf0 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.336+0000 7fc2c4945700 1 -- 172.21.15.55:0/3049598669 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc2bc004300 con 0x7fc2c0108cf0 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.336+0000 7fc2c4945700 1 -- 172.21.15.55:0/3049598669 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2bc011af0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.337+0000 7fc2c7bab700 1 -- 172.21.15.55:0/3049598669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 msgr2=0x7fc2c0109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.337+0000 7fc2c7bab700 1 --2- 172.21.15.55:0/3049598669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c0109110 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0098a0 tx=0x7fc2bc009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.337+0000 7fc2c7bab700 1 -- 172.21.15.55:0/3049598669 shutdown_connections 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.337+0000 7fc2c7bab700 1 --2- 172.21.15.55:0/3049598669 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c0109110 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.337+0000 7fc2c7bab700 1 -- 172.21.15.55:0/3049598669 >> 172.21.15.55:0/3049598669 conn(0x7fc2c01042b0 msgr2=0x7fc2c01066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.337+0000 7fc2c7bab700 1 -- 172.21.15.55:0/3049598669 shutdown_connections 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.337+0000 7fc2c7bab700 1 -- 172.21.15.55:0/3049598669 wait complete. 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.338+0000 7fc2c7bab700 1 Processor -- start 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.338+0000 7fc2c7bab700 1 -- start start 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.338+0000 7fc2c7bab700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.338+0000 7fc2c7bab700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c010d2a0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.338+0000 7fc2c5947700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.338+0000 7fc2c5947700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38882/0 (socket says 172.21.15.55:38882) 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.338+0000 7fc2c5947700 1 -- 172.21.15.55:0/594455642 learned_addr learned my addr 172.21.15.55:0/594455642 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2c5947700 1 -- 172.21.15.55:0/594455642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c019d160 con 0x7fc2c0108cf0 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2c5947700 1 --2- 172.21.15.55:0/594455642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c019cc20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc003f40 tx=0x7fc2bc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:06.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2b6ffd700 1 -- 172.21.15.55:0/594455642 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2bc0038b0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2b6ffd700 1 -- 172.21.15.55:0/594455642 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc2bc003a50 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c019d3f0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2b6ffd700 1 -- 172.21.15.55:0/594455642 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2bc01a5c0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c01a00e0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2b6ffd700 1 -- 172.21.15.55:0/594455642 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc2bc01ac50 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.339+0000 7fc2b6ffd700 1 -- 172.21.15.55:0/594455642 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2bc019700 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.340+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c0064a80 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.342+0000 7fc2b6ffd700 1 -- 172.21.15.55:0/594455642 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc2bc019900 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.417+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc2c01a0300 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.417+0000 7fc2b6ffd700 1 -- 172.21.15.55:0/594455642 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc2bc019ae0 con 0x7fc2c0108cf0 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.419+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 msgr2=0x7fc2c019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.419+0000 7fc2c7bab700 1 --2- 172.21.15.55:0/594455642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c019cc20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc003f40 tx=0x7fc2bc011890 comp rx=0 tx=0).stop 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.419+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 shutdown_connections 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.419+0000 7fc2c7bab700 1 --2- 172.21.15.55:0/594455642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2c0108cf0 0x7fc2c019cc20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:06.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.419+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 >> 172.21.15.55:0/594455642 conn(0x7fc2c01042b0 msgr2=0x7fc2c0104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:06.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.419+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 shutdown_connections 2024-04-26T21:43:06.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:05.419+0000 7fc2c7bab700 1 -- 172.21.15.55:0/594455642 wait complete. 2024-04-26T21:43:06.075 INFO:teuthology.orchestra.run.smithi055.stdout:mgr not available, waiting (2/15)... 2024-04-26T21:43:06.253 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:05 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:05.942+0000 7fefb4078200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:43:06.253 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:06.043+0000 7fefb4078200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:43:06.253 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:06.252+0000 7fefb4078200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:43:07.139 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:06.794+0000 7fefb4078200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:43:07.139 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:06.889+0000 7fefb4078200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:43:07.436 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:07.138+0000 7fefb4078200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:43:07.437 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:07.275+0000 7fefb4078200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:43:07.437 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:07.374+0000 7fefb4078200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:43:07.803 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:07.492+0000 7fefb4078200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:43:07.803 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:07.802+0000 7fefb4078200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:43:08.436 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:08.432+0000 7fefb4078200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:43:08.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:08.659+0000 7fefb4078200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:43:08.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: Activating manager daemon x 2024-04-26T21:43:08.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: mgrmap e2: x(active, starting, since 0.00348596s) 2024-04-26T21:43:08.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:43:08.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:43:08.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:43:08.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:43:08.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:43:08.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: Manager daemon x is now available 2024-04-26T21:43:08.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:43:08.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' 2024-04-26T21:43:08.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:43:08.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' 2024-04-26T21:43:08.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:08 smithi055 ceph-mon[29221]: from='mgr.14100 172.21.15.55:0/4029032797' entity='mgr.x' 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:09.366 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:43:09.367 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:09.368 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:43:09.368 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:43:09.368 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:43:09.375 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:43:09.375 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:43:09.375 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:43:09.375 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:43:09.376 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:42:52.793467+0000", 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:43:09.377 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:09.378 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:43:09.378 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:09.378 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.625+0000 7fe93a888700 1 Processor -- start 2024-04-26T21:43:09.378 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.625+0000 7fe93a888700 1 -- start start 2024-04-26T21:43:09.378 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.625+0000 7fe93a888700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe934108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:09.378 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.625+0000 7fe93a888700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341085e0 con 0x7fe934107bf0 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.625+0000 7fe933fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe934108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.625+0000 7fe933fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe934108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38896/0 (socket says 172.21.15.55:38896) 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.625+0000 7fe933fff700 1 -- 172.21.15.55:0/131746654 learned_addr learned my addr 172.21.15.55:0/131746654 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.625+0000 7fe933fff700 1 -- 172.21.15.55:0/131746654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe934108df0 con 0x7fe934107bf0 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe933fff700 1 --2- 172.21.15.55:0/131746654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe934108010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9240098a0 tx=0x7fe924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eca990e12e41186 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe932ffd700 1 -- 172.21.15.55:0/131746654 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924004160 con 0x7fe934107bf0 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe932ffd700 1 -- 172.21.15.55:0/131746654 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe924004300 con 0x7fe934107bf0 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe932ffd700 1 -- 172.21.15.55:0/131746654 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924011af0 con 0x7fe934107bf0 2024-04-26T21:43:09.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe93a888700 1 -- 172.21.15.55:0/131746654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 msgr2=0x7fe934108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe93a888700 1 --2- 172.21.15.55:0/131746654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe934108010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe9240098a0 tx=0x7fe924009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe93a888700 1 -- 172.21.15.55:0/131746654 shutdown_connections 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe93a888700 1 --2- 172.21.15.55:0/131746654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe934108010 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.626+0000 7fe93a888700 1 -- 172.21.15.55:0/131746654 >> 172.21.15.55:0/131746654 conn(0x7fe934103190 msgr2=0x7fe9341055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.627+0000 7fe93a888700 1 -- 172.21.15.55:0/131746654 shutdown_connections 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.627+0000 7fe93a888700 1 -- 172.21.15.55:0/131746654 wait complete. 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.627+0000 7fe93a888700 1 Processor -- start 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.627+0000 7fe93a888700 1 -- start start 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.627+0000 7fe93a888700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe93407ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.627+0000 7fe93a888700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410c1a0 con 0x7fe934107bf0 2024-04-26T21:43:09.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe933fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe93407ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe933fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe93407ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38902/0 (socket says 172.21.15.55:38902) 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe933fff700 1 -- 172.21.15.55:0/355968944 learned_addr learned my addr 172.21.15.55:0/355968944 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe933fff700 1 -- 172.21.15.55:0/355968944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93419f9d0 con 0x7fe934107bf0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe933fff700 1 --2- 172.21.15.55:0/355968944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe93407ca70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe9240117b0 tx=0x7fe924011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe9317fa700 1 -- 172.21.15.55:0/355968944 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924003af0 con 0x7fe934107bf0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93419fba0 con 0x7fe934107bf0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe9317fa700 1 -- 172.21.15.55:0/355968944 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe924020070 con 0x7fe934107bf0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe9317fa700 1 -- 172.21.15.55:0/355968944 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924019630 con 0x7fe934107bf0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.628+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93419ff40 con 0x7fe934107bf0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.629+0000 7fe9317fa700 1 -- 172.21.15.55:0/355968944 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe924019c80 con 0x7fe934107bf0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.629+0000 7fe9317fa700 1 -- 172.21.15.55:0/355968944 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe92400b720 con 0x7fe934107bf0 2024-04-26T21:43:09.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.629+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe934064a80 con 0x7fe934107bf0 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.631+0000 7fe9317fa700 1 -- 172.21.15.55:0/355968944 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe9240042a0 con 0x7fe934107bf0 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.663+0000 7fe9317fa700 1 -- 172.21.15.55:0/355968944 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7fe92401a9c0 con 0x7fe934107bf0 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.714+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe9341a0250 con 0x7fe934107bf0 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.714+0000 7fe9317fa700 1 -- 172.21.15.55:0/355968944 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe924024580 con 0x7fe934107bf0 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.715+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 msgr2=0x7fe93407ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.715+0000 7fe93a888700 1 --2- 172.21.15.55:0/355968944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe93407ca70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe9240117b0 tx=0x7fe924011b70 comp rx=0 tx=0).stop 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.715+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 shutdown_connections 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.715+0000 7fe93a888700 1 --2- 172.21.15.55:0/355968944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe934107bf0 0x7fe93407ca70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.715+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 >> 172.21.15.55:0/355968944 conn(0x7fe934103190 msgr2=0x7fe934103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.715+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 shutdown_connections 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:08.715+0000 7fe93a888700 1 -- 172.21.15.55:0/355968944 wait complete. 2024-04-26T21:43:09.382 INFO:teuthology.orchestra.run.smithi055.stdout:mgr not available, waiting (3/15)... 2024-04-26T21:43:10.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:09 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/355968944' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:43:10.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:09 smithi055 ceph-mon[29221]: mgrmap e3: x(active, since 1.00758s) 2024-04-26T21:43:11.763 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:11 smithi055 ceph-mon[29221]: mgrmap e4: x(active, since 2s) 2024-04-26T21:43:12.856 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:43:12.856 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:12.856 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:43:12.856 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T21:43:12.856 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T21:43:12.856 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T21:43:12.856 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 0 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T21:43:12.857 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:12.858 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T21:43:12.858 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T21:43:12.858 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T21:43:12.869 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T21:43:12.870 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T21:42:52.793467+0000", 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.799+0000 7f1848e98700 1 Processor -- start 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.799+0000 7f1848e98700 1 -- start start 2024-04-26T21:43:12.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.799+0000 7f1848e98700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f1844109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.799+0000 7f1848e98700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1844109740 con 0x7f1844108d50 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.800+0000 7f184259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f1844109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.800+0000 7f184259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f1844109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36378/0 (socket says 172.21.15.55:36378) 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.800+0000 7f184259c700 1 -- 172.21.15.55:0/2843057114 learned_addr learned my addr 172.21.15.55:0/2843057114 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.800+0000 7f184259c700 1 -- 172.21.15.55:0/2843057114 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1844109f50 con 0x7f1844108d50 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.800+0000 7f184259c700 1 --2- 172.21.15.55:0/2843057114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f1844109170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f182c0098a0 tx=0x7f182c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ba981379c34da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.800+0000 7f184159a700 1 -- 172.21.15.55:0/2843057114 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182c004160 con 0x7f1844108d50 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.800+0000 7f184159a700 1 -- 172.21.15.55:0/2843057114 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f182c004300 con 0x7f1844108d50 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.800+0000 7f184159a700 1 -- 172.21.15.55:0/2843057114 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182c011af0 con 0x7f1844108d50 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.801+0000 7f1848e98700 1 -- 172.21.15.55:0/2843057114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 msgr2=0x7f1844109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.801+0000 7f1848e98700 1 --2- 172.21.15.55:0/2843057114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f1844109170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f182c0098a0 tx=0x7f182c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.801+0000 7f1848e98700 1 -- 172.21.15.55:0/2843057114 shutdown_connections 2024-04-26T21:43:12.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.801+0000 7f1848e98700 1 --2- 172.21.15.55:0/2843057114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f1844109170 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.801+0000 7f1848e98700 1 -- 172.21.15.55:0/2843057114 >> 172.21.15.55:0/2843057114 conn(0x7f18441042f0 msgr2=0x7f1844106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.801+0000 7f1848e98700 1 -- 172.21.15.55:0/2843057114 shutdown_connections 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.801+0000 7f1848e98700 1 -- 172.21.15.55:0/2843057114 wait complete. 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.802+0000 7f1848e98700 1 Processor -- start 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.802+0000 7f1848e98700 1 -- start start 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.802+0000 7f1848e98700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f184419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.802+0000 7f1848e98700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f184410d180 con 0x7f1844108d50 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.802+0000 7f184259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f184419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.802+0000 7f184259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f184419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36384/0 (socket says 172.21.15.55:36384) 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.802+0000 7f184259c700 1 -- 172.21.15.55:0/1224828134 learned_addr learned my addr 172.21.15.55:0/1224828134 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.803+0000 7f184259c700 1 -- 172.21.15.55:0/1224828134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184419d140 con 0x7f1844108d50 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.803+0000 7f184259c700 1 --2- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f184419cc00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f182c003f40 tx=0x7f182c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.803+0000 7f183b7fe700 1 -- 172.21.15.55:0/1224828134 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182c0038b0 con 0x7f1844108d50 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.803+0000 7f183b7fe700 1 -- 172.21.15.55:0/1224828134 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f182c003a50 con 0x7f1844108d50 2024-04-26T21:43:12.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.803+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184419d3d0 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.803+0000 7f183b7fe700 1 -- 172.21.15.55:0/1224828134 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f182c01a5c0 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.803+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441a00c0 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.804+0000 7f183b7fe700 1 -- 172.21.15.55:0/1224828134 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f182c01a870 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.804+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184410d300 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.804+0000 7f183b7fe700 1 --2- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f1830039560 0x7f183003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.804+0000 7f183b7fe700 1 -- 172.21.15.55:0/1224828134 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f182c04dbf0 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.804+0000 7f1841d9b700 1 --2- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f1830039560 0x7f183003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.805+0000 7f1841d9b700 1 --2- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f1830039560 0x7f183003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1834006fd0 tx=0x7f1834006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:11.810+0000 7f183b7fe700 1 -- 172.21.15.55:0/1224828134 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f182c019c70 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.104+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f18441a0710 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.104+0000 7f183b7fe700 1 -- 172.21.15.55:0/1224828134 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f182c02c9f0 con 0x7f1844108d50 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f1830039560 msgr2=0x7f183003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 --2- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f1830039560 0x7f183003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1834006fd0 tx=0x7f1834006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:12.874 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 msgr2=0x7f184419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:12.875 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 --2- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f184419cc00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f182c003f40 tx=0x7f182c011890 comp rx=0 tx=0).stop 2024-04-26T21:43:12.875 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 shutdown_connections 2024-04-26T21:43:12.875 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 --2- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f1830039560 0x7f183003ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.875 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 --2- 172.21.15.55:0/1224828134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1844108d50 0x7f184419cc00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:12.875 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 >> 172.21.15.55:0/1224828134 conn(0x7f18441042f0 msgr2=0x7f1844104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:12.875 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 shutdown_connections 2024-04-26T21:43:12.875 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:12.107+0000 7f1848e98700 1 -- 172.21.15.55:0/1224828134 wait complete. 2024-04-26T21:43:12.875 INFO:teuthology.orchestra.run.smithi055.stdout:mgr is available 2024-04-26T21:43:12.987 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:12 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1224828134' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T21:43:13.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:13 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1845880139' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T21:43:14.081 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout fsid = c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-04-26T21:43:14.082 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.315+0000 7fc9375bf700 1 Processor -- start 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.315+0000 7fc9375bf700 1 -- start start 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.315+0000 7fc9375bf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc93010bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc9375bf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc93007ce70 con 0x7fc9301097a0 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc93535b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc93010bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc93535b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc93010bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36398/0 (socket says 172.21.15.55:36398) 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc93535b700 1 -- 172.21.15.55:0/2049184729 learned_addr learned my addr 172.21.15.55:0/2049184729 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc93535b700 1 -- 172.21.15.55:0/2049184729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93010c0d0 con 0x7fc9301097a0 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc93535b700 1 --2- 172.21.15.55:0/2049184729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc93010bb90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0098a0 tx=0x7fc92c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae1368ec406b8a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc927fff700 1 -- 172.21.15.55:0/2049184729 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92c004160 con 0x7fc9301097a0 2024-04-26T21:43:14.083 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc927fff700 1 -- 172.21.15.55:0/2049184729 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc92c004300 con 0x7fc9301097a0 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.316+0000 7fc927fff700 1 -- 172.21.15.55:0/2049184729 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92c011af0 con 0x7fc9301097a0 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.317+0000 7fc9375bf700 1 -- 172.21.15.55:0/2049184729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 msgr2=0x7fc93010bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.317+0000 7fc9375bf700 1 --2- 172.21.15.55:0/2049184729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc93010bb90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0098a0 tx=0x7fc92c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.317+0000 7fc9375bf700 1 -- 172.21.15.55:0/2049184729 shutdown_connections 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.317+0000 7fc9375bf700 1 --2- 172.21.15.55:0/2049184729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc93010bb90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.317+0000 7fc9375bf700 1 -- 172.21.15.55:0/2049184729 >> 172.21.15.55:0/2049184729 conn(0x7fc9301031b0 msgr2=0x7fc9301055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.317+0000 7fc9375bf700 1 -- 172.21.15.55:0/2049184729 shutdown_connections 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.317+0000 7fc9375bf700 1 -- 172.21.15.55:0/2049184729 wait complete. 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.318+0000 7fc9375bf700 1 Processor -- start 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.318+0000 7fc9375bf700 1 -- start start 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.318+0000 7fc9375bf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc930109020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.318+0000 7fc9375bf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc93010d040 con 0x7fc9301097a0 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.318+0000 7fc93535b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc930109020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.318+0000 7fc93535b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc930109020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36410/0 (socket says 172.21.15.55:36410) 2024-04-26T21:43:14.084 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.318+0000 7fc93535b700 1 -- 172.21.15.55:0/1845880139 learned_addr learned my addr 172.21.15.55:0/1845880139 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.318+0000 7fc93535b700 1 -- 172.21.15.55:0/1845880139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc930109560 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.319+0000 7fc93535b700 1 --2- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc930109020 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc92c011ac0 tx=0x7fc92c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.319+0000 7fc9267fc700 1 -- 172.21.15.55:0/1845880139 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92c003b20 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.319+0000 7fc9267fc700 1 -- 172.21.15.55:0/1845880139 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc92c01a8c0 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.319+0000 7fc9267fc700 1 -- 172.21.15.55:0/1845880139 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92c0248d0 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.319+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc930107750 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.319+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc930107c90 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.320+0000 7fc9267fc700 1 -- 172.21.15.55:0/1845880139 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fc92c022020 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.320+0000 7fc9267fc700 1 --2- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fc91c039560 0x7fc91c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.320+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc930064a80 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.320+0000 7fc9267fc700 1 -- 172.21.15.55:0/1845880139 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc92c04d530 con 0x7fc9301097a0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.320+0000 7fc934b5a700 1 --2- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fc91c039560 0x7fc91c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:14.085 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.321+0000 7fc934b5a700 1 --2- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fc91c039560 0x7fc91c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc920006fd0 tx=0x7fc920006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.325+0000 7fc9267fc700 1 -- 172.21.15.55:0/1845880139 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc92c04d290 con 0x7fc9301097a0 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.533+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc930107f10 con 0x7fc9301097a0 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.534+0000 7fc9267fc700 1 -- 172.21.15.55:0/1845880139 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fc92c01aa70 con 0x7fc9301097a0 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fc91c039560 msgr2=0x7fc91c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 --2- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fc91c039560 0x7fc91c03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc920006fd0 tx=0x7fc920006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 msgr2=0x7fc930109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 --2- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc930109020 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc92c011ac0 tx=0x7fc92c011ba0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 shutdown_connections 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 --2- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fc91c039560 0x7fc91c03ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 --2- 172.21.15.55:0/1845880139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9301097a0 0x7fc930109020 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 >> 172.21.15.55:0/1845880139 conn(0x7fc9301031b0 msgr2=0x7fc930103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 shutdown_connections 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:13.536+0000 7fc9375bf700 1 -- 172.21.15.55:0/1845880139 wait complete. 2024-04-26T21:43:14.086 INFO:teuthology.orchestra.run.smithi055.stdout:Enabling cephadm module... 2024-04-26T21:43:15.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:14 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1617210137' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T21:43:15.187 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:14 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: ignoring --setuser ceph since I am not root 2024-04-26T21:43:15.187 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:14 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: ignoring --setgroup ceph since I am not root 2024-04-26T21:43:15.187 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:15.044+0000 7fb1bbc7a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:43:15.654 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:15.236+0000 7fb1bbc7a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:43:15.654 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:15.341+0000 7fb1bbc7a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:43:15.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.523+0000 7fe7dbe72700 1 Processor -- start 2024-04-26T21:43:15.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.524+0000 7fe7dbe72700 1 -- start start 2024-04-26T21:43:15.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.524+0000 7fe7dbe72700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d4109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.524+0000 7fe7dbe72700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d4109740 con 0x7fe7d4108d50 2024-04-26T21:43:15.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.524+0000 7fe7d9c0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d4109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.524+0000 7fe7d9c0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d4109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36420/0 (socket says 172.21.15.55:36420) 2024-04-26T21:43:15.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.524+0000 7fe7d9c0e700 1 -- 172.21.15.55:0/3631157561 learned_addr learned my addr 172.21.15.55:0/3631157561 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:15.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.524+0000 7fe7d9c0e700 1 -- 172.21.15.55:0/3631157561 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d4109f50 con 0x7fe7d4108d50 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7d9c0e700 1 --2- 172.21.15.55:0/3631157561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d4109170 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40098a0 tx=0x7fe7c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff7d94511d6d4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7d8c0c700 1 -- 172.21.15.55:0/3631157561 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7c4004160 con 0x7fe7d4108d50 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7d8c0c700 1 -- 172.21.15.55:0/3631157561 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe7c4004300 con 0x7fe7d4108d50 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7d8c0c700 1 -- 172.21.15.55:0/3631157561 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7c4011af0 con 0x7fe7d4108d50 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7dbe72700 1 -- 172.21.15.55:0/3631157561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 msgr2=0x7fe7d4109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7dbe72700 1 --2- 172.21.15.55:0/3631157561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d4109170 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40098a0 tx=0x7fe7c4009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7dbe72700 1 -- 172.21.15.55:0/3631157561 shutdown_connections 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7dbe72700 1 --2- 172.21.15.55:0/3631157561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d4109170 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.525+0000 7fe7dbe72700 1 -- 172.21.15.55:0/3631157561 >> 172.21.15.55:0/3631157561 conn(0x7fe7d41042f0 msgr2=0x7fe7d4106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.526+0000 7fe7dbe72700 1 -- 172.21.15.55:0/3631157561 shutdown_connections 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.526+0000 7fe7dbe72700 1 -- 172.21.15.55:0/3631157561 wait complete. 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.526+0000 7fe7dbe72700 1 Processor -- start 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.526+0000 7fe7dbe72700 1 -- start start 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.526+0000 7fe7dbe72700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d407eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.526+0000 7fe7dbe72700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d410d180 con 0x7fe7d4108d50 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7d9c0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d407eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7d9c0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d407eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36428/0 (socket says 172.21.15.55:36428) 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7d9c0e700 1 -- 172.21.15.55:0/1617210137 learned_addr learned my addr 172.21.15.55:0/1617210137 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7d9c0e700 1 -- 172.21.15.55:0/1617210137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d407f0d0 con 0x7fe7d4108d50 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7d9c0e700 1 --2- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d407eb90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4000c00 tx=0x7fe7c4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7caffd700 1 -- 172.21.15.55:0/1617210137 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7c4003b20 con 0x7fe7d4108d50 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7caffd700 1 -- 172.21.15.55:0/1617210137 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe7c401a8c0 con 0x7fe7d4108d50 2024-04-26T21:43:15.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d407d2c0 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7caffd700 1 -- 172.21.15.55:0/1617210137 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7c4024940 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.527+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d407d800 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.528+0000 7fe7caffd700 1 -- 172.21.15.55:0/1617210137 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fe7c4022020 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.528+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d4064a80 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.528+0000 7fe7caffd700 1 --2- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fe7c0039600 0x7fe7c003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.528+0000 7fe7caffd700 1 -- 172.21.15.55:0/1617210137 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe7c404d570 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.528+0000 7fe7d940d700 1 --2- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fe7c0039600 0x7fe7c003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.529+0000 7fe7d940d700 1 --2- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fe7c0039600 0x7fe7c003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006fd0 tx=0x7fe7d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.533+0000 7fe7caffd700 1 -- 172.21.15.55:0/1617210137 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe7c400b3e0 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.805+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fe7d407da80 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.859+0000 7fe7caffd700 1 -- 172.21.15.55:0/1617210137 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fe7c40042c0 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.861+0000 7fe7caffd700 1 -- 172.21.15.55:0/1617210137 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fe7c401aae0 con 0x7fe7d4108d50 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.862+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fe7c0039600 msgr2=0x7fe7c003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.862+0000 7fe7dbe72700 1 --2- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fe7c0039600 0x7fe7c003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006fd0 tx=0x7fe7d0006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.862+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 msgr2=0x7fe7d407eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.862+0000 7fe7dbe72700 1 --2- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d407eb90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4000c00 tx=0x7fe7c4011b70 comp rx=0 tx=0).stop 2024-04-26T21:43:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.862+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 shutdown_connections 2024-04-26T21:43:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.862+0000 7fe7dbe72700 1 --2- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fe7c0039600 0x7fe7c003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.862+0000 7fe7dbe72700 1 --2- 172.21.15.55:0/1617210137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7d4108d50 0x7fe7d407eb90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.862+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 >> 172.21.15.55:0/1617210137 conn(0x7fe7d41042f0 msgr2=0x7fe7d4104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.863+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 shutdown_connections 2024-04-26T21:43:15.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:14.863+0000 7fe7dbe72700 1 -- 172.21.15.55:0/1617210137 wait complete. 2024-04-26T21:43:15.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:15 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1617210137' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T21:43:15.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:15 smithi055 ceph-mon[29221]: mgrmap e5: x(active, since 6s) 2024-04-26T21:43:15.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:15.664+0000 7fb1bbc7a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:43:15.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:15.761+0000 7fb1bbc7a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:43:16.968 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:16 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/983233710' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:43:16.977 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:16.977 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T21:43:16.977 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.083+0000 7fb2f762a700 1 Processor -- start 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.083+0000 7fb2f762a700 1 -- start start 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.083+0000 7fb2f762a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f0109110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.083+0000 7fb2f762a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01096e0 con 0x7fb2f0108cf0 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.083+0000 7fb2f53c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f0109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.083+0000 7fb2f53c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f0109110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36458/0 (socket says 172.21.15.55:36458) 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.083+0000 7fb2f53c6700 1 -- 172.21.15.55:0/3522784075 learned_addr learned my addr 172.21.15.55:0/3522784075 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.084+0000 7fb2f53c6700 1 -- 172.21.15.55:0/3522784075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f0109ef0 con 0x7fb2f0108cf0 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.084+0000 7fb2f53c6700 1 --2- 172.21.15.55:0/3522784075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f0109110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0098a0 tx=0x7fb2ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf5d9c19982a92b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:16.978 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.084+0000 7fb2e7fff700 1 -- 172.21.15.55:0/3522784075 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ec004160 con 0x7fb2f0108cf0 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.084+0000 7fb2e7fff700 1 -- 172.21.15.55:0/3522784075 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb2ec004300 con 0x7fb2f0108cf0 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.084+0000 7fb2e7fff700 1 -- 172.21.15.55:0/3522784075 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ec011af0 con 0x7fb2f0108cf0 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.085+0000 7fb2f762a700 1 -- 172.21.15.55:0/3522784075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 msgr2=0x7fb2f0109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.085+0000 7fb2f762a700 1 --2- 172.21.15.55:0/3522784075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f0109110 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0098a0 tx=0x7fb2ec009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.085+0000 7fb2f762a700 1 -- 172.21.15.55:0/3522784075 shutdown_connections 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.085+0000 7fb2f762a700 1 --2- 172.21.15.55:0/3522784075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f0109110 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.085+0000 7fb2f762a700 1 -- 172.21.15.55:0/3522784075 >> 172.21.15.55:0/3522784075 conn(0x7fb2f01042b0 msgr2=0x7fb2f01066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.085+0000 7fb2f762a700 1 -- 172.21.15.55:0/3522784075 shutdown_connections 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.085+0000 7fb2f762a700 1 -- 172.21.15.55:0/3522784075 wait complete. 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.086+0000 7fb2f762a700 1 Processor -- start 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.086+0000 7fb2f762a700 1 -- start start 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.086+0000 7fb2f762a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.086+0000 7fb2f762a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010d2a0 con 0x7fb2f0108cf0 2024-04-26T21:43:16.979 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.086+0000 7fb2f53c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.086+0000 7fb2f53c6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36460/0 (socket says 172.21.15.55:36460) 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.086+0000 7fb2f53c6700 1 -- 172.21.15.55:0/983233710 learned_addr learned my addr 172.21.15.55:0/983233710 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.086+0000 7fb2f53c6700 1 -- 172.21.15.55:0/983233710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f019d160 con 0x7fb2f0108cf0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.087+0000 7fb2f53c6700 1 --2- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f019cc20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000c00 tx=0x7fb2ec011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.087+0000 7fb2e67fc700 1 -- 172.21.15.55:0/983233710 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ec003b20 con 0x7fb2f0108cf0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.087+0000 7fb2e67fc700 1 -- 172.21.15.55:0/983233710 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb2ec01a8c0 con 0x7fb2f0108cf0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.087+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f019d3f0 con 0x7fb2f0108cf0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.087+0000 7fb2e67fc700 1 -- 172.21.15.55:0/983233710 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ec024940 con 0x7fb2f0108cf0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.087+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a00e0 con 0x7fb2f0108cf0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.087+0000 7fb2e67fc700 1 -- 172.21.15.55:0/983233710 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb2ec022020 con 0x7fb2f0108cf0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.088+0000 7fb2e67fc700 1 --2- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fb2dc039250 0x7fb2dc03b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.088+0000 7fb2e67fc700 1 -- 172.21.15.55:0/983233710 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb2ec053060 con 0x7fb2f0108cf0 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.088+0000 7fb2f4bc5700 1 -- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fb2dc039250 msgr2=0x7fb2dc03b710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3795671875 2024-04-26T21:43:16.980 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.088+0000 7fb2f4bc5700 1 --2- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fb2dc039250 0x7fb2dc03b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.088+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f0064a80 con 0x7fb2f0108cf0 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.094+0000 7fb2e67fc700 1 -- 172.21.15.55:0/983233710 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb2ec01aa70 con 0x7fb2f0108cf0 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.289+0000 7fb2f4bc5700 1 -- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fb2dc039250 msgr2=0x7fb2dc03b710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3795671875 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.289+0000 7fb2f4bc5700 1 --2- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fb2dc039250 0x7fb2dc03b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.406+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb2f01a05e0 con 0x7fb2f0108cf0 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.406+0000 7fb2e67fc700 1 -- 172.21.15.55:0/983233710 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb2ec010540 con 0x7fb2f0108cf0 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.408+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fb2dc039250 msgr2=0x7fb2dc03b710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.408+0000 7fb2f762a700 1 --2- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fb2dc039250 0x7fb2dc03b710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.408+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 msgr2=0x7fb2f019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.408+0000 7fb2f762a700 1 --2- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f019cc20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec000c00 tx=0x7fb2ec011b70 comp rx=0 tx=0).stop 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.408+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 shutdown_connections 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.408+0000 7fb2f762a700 1 --2- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7fb2dc039250 0x7fb2dc03b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.408+0000 7fb2f762a700 1 --2- 172.21.15.55:0/983233710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0108cf0 0x7fb2f019cc20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.408+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 >> 172.21.15.55:0/983233710 conn(0x7fb2f01042b0 msgr2=0x7fb2f0104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:16.981 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.409+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 shutdown_connections 2024-04-26T21:43:16.982 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:16.409+0000 7fb2f762a700 1 -- 172.21.15.55:0/983233710 wait complete. 2024-04-26T21:43:16.982 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for the mgr to restart... 2024-04-26T21:43:16.982 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mgr epoch 5... 2024-04-26T21:43:17.241 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:17.029+0000 7fb1bbc7a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:43:17.242 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:17.207+0000 7fb1bbc7a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:43:17.655 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:17.389+0000 7fb1bbc7a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:43:17.936 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:17.691+0000 7fb1bbc7a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:43:17.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:17.787+0000 7fb1bbc7a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:43:18.436 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:17.997+0000 7fb1bbc7a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:43:18.437 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:18 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:18.196+0000 7fb1bbc7a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:43:19.186 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:18 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:18.823+0000 7fb1bbc7a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:43:19.573 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:19.260+0000 7fb1bbc7a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:43:19.573 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:19.361+0000 7fb1bbc7a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:43:19.573 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:19.572+0000 7fb1bbc7a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:43:20.436 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:20.128+0000 7fb1bbc7a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:43:20.437 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:20.225+0000 7fb1bbc7a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:43:20.715 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:20.478+0000 7fb1bbc7a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:43:20.715 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:20.615+0000 7fb1bbc7a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:43:20.715 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:20.714+0000 7fb1bbc7a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:43:21.144 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:20.833+0000 7fb1bbc7a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:43:21.144 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:21.143+0000 7fb1bbc7a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:43:22.047 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:21.769+0000 7fb1bbc7a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:43:22.048 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:21.993+0000 7fb1bbc7a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: Active manager daemon x restarted 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: Activating manager daemon x 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: mgrmap e6: x(active, starting, since 0.0121469s) 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:43:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:43:22.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: Manager daemon x is now available 2024-04-26T21:43:22.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:22.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:22 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:23.316 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:23 smithi055 ceph-mon[29221]: Found migration_current of "None". Setting to last migration. 2024-04-26T21:43:23.316 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:23 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:23.316 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:23 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:23.316 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:23 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:43:23.317 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:23 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:43:23.317 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:23 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:23.317 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:23 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:23.317 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:23 smithi055 ceph-mon[29221]: mgrmap e7: x(active, since 1.01388s) 2024-04-26T21:43:23.619 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:23.619 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.448+0000 7f03af929700 1 Processor -- start 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.448+0000 7f03af929700 1 -- start start 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.448+0000 7f03af929700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a8109100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.448+0000 7f03af929700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f03a81096d0 con 0x7f03a8108ce0 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.449+0000 7f03ad6c5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a8109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.449+0000 7f03ad6c5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a8109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36476/0 (socket says 172.21.15.55:36476) 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.449+0000 7f03ad6c5700 1 -- 172.21.15.55:0/1307241900 learned_addr learned my addr 172.21.15.55:0/1307241900 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.449+0000 7f03ad6c5700 1 -- 172.21.15.55:0/1307241900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a8109ee0 con 0x7f03a8108ce0 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.449+0000 7f03ad6c5700 1 --2- 172.21.15.55:0/1307241900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a8109100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f03980098a0 tx=0x7f0398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267712d98d116a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.449+0000 7f039ffff700 1 -- 172.21.15.55:0/1307241900 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0398004160 con 0x7f03a8108ce0 2024-04-26T21:43:23.620 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.449+0000 7f039ffff700 1 -- 172.21.15.55:0/1307241900 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0398004300 con 0x7f03a8108ce0 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.449+0000 7f039ffff700 1 -- 172.21.15.55:0/1307241900 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0398011af0 con 0x7f03a8108ce0 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.450+0000 7f03af929700 1 -- 172.21.15.55:0/1307241900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 msgr2=0x7f03a8109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.450+0000 7f03af929700 1 --2- 172.21.15.55:0/1307241900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a8109100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f03980098a0 tx=0x7f0398009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.450+0000 7f03af929700 1 -- 172.21.15.55:0/1307241900 shutdown_connections 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.450+0000 7f03af929700 1 --2- 172.21.15.55:0/1307241900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a8109100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.450+0000 7f03af929700 1 -- 172.21.15.55:0/1307241900 >> 172.21.15.55:0/1307241900 conn(0x7f03a81042e0 msgr2=0x7f03a8106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.450+0000 7f03af929700 1 -- 172.21.15.55:0/1307241900 shutdown_connections 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.450+0000 7f03af929700 1 -- 172.21.15.55:0/1307241900 wait complete. 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.451+0000 7f03af929700 1 Processor -- start 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.451+0000 7f03af929700 1 -- start start 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.451+0000 7f03af929700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.621 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.451+0000 7f03af929700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f03a810d290 con 0x7f03a8108ce0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.451+0000 7f03ad6c5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.451+0000 7f03ad6c5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36490/0 (socket says 172.21.15.55:36490) 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.451+0000 7f03ad6c5700 1 -- 172.21.15.55:0/202150074 learned_addr learned my addr 172.21.15.55:0/202150074 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.452+0000 7f03ad6c5700 1 -- 172.21.15.55:0/202150074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a819d170 con 0x7f03a8108ce0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.452+0000 7f03ad6c5700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a819cc30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0398003f40 tx=0x7f0398011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.452+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03980038b0 con 0x7f03a8108ce0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.452+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0398003a50 con 0x7f03a8108ce0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.452+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a819d400 con 0x7f03a8108ce0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.452+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f039801a5c0 con 0x7f03a8108ce0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.452+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a81a00f0 con 0x7f03a8108ce0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.453+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f039801a870 con 0x7f03a8108ce0 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.453+0000 7f039e7fc700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 0x7f039403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.453+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 --> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f039403c180 con 0x7f03940395b0 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.453+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f039804da70 con 0x7f03a8108ce0 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.453+0000 7f03acec4700 1 -- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 msgr2=0x7f039403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3795671875 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.453+0000 7f03acec4700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 0x7f039403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.654+0000 7f03acec4700 1 -- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 msgr2=0x7f039403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3795671875 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:17.654+0000 7f03acec4700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 0x7f039403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.055+0000 7f03acec4700 1 -- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 msgr2=0x7f039403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3795671875 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.055+0000 7f03acec4700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 0x7f039403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.856+0000 7f03acec4700 1 -- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 msgr2=0x7f039403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3795671875 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:18.856+0000 7f03acec4700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 0x7f039403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.457+0000 7f03acec4700 1 -- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 msgr2=0x7f039403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/3795671875 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:20.457+0000 7f03acec4700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 0x7f039403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:22.006+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f039801abb0 con 0x7f03a8108ce0 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:22.006+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 msgr2=0x7f039403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:22.006+0000 7f039e7fc700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 0x7f039403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.006+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f039804f8c0 con 0x7f03a8108ce0 2024-04-26T21:43:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.006+0000 7f039e7fc700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f039403cea0 0x7f039403f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.006+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f039403c180 con 0x7f039403cea0 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.006+0000 7f03acec4700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f039403cea0 0x7f039403f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.007+0000 7f03acec4700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f039403cea0 0x7f039403f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f03a4003a10 tx=0x7f03a40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.008+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mgr.14120 v2:172.21.15.55:6800/2562126894 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f039403c180 con 0x7f039403cea0 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f03a806a510 con 0x7f039403cea0 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f039e7fc700 1 -- 172.21.15.55:0/202150074 <== mgr.14120 v2:172.21.15.55:6800/2562126894 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f03a806a510 con 0x7f039403cea0 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f039403cea0 msgr2=0x7f039403f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f039403cea0 0x7f039403f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f03a4003a10 tx=0x7f03a40092b0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 msgr2=0x7f03a819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a819cc30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0398003f40 tx=0x7f0398011890 comp rx=0 tx=0).stop 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 shutdown_connections 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f039403cea0 0x7f039403f290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03a8108ce0 0x7f03a819cc30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 --2- 172.21.15.55:0/202150074 >> [v2:172.21.15.55:6800/3795671875,v1:172.21.15.55:6801/3795671875] conn(0x7f03940395b0 0x7f039403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:23.624 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.016+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 >> 172.21.15.55:0/202150074 conn(0x7f03a81042e0 msgr2=0x7f03a8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:23.625 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.017+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 shutdown_connections 2024-04-26T21:43:23.625 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:23.017+0000 7f03af929700 1 -- 172.21.15.55:0/202150074 wait complete. 2024-04-26T21:43:23.625 INFO:teuthology.orchestra.run.smithi055.stdout:mgr epoch 5 is available 2024-04-26T21:43:23.625 INFO:teuthology.orchestra.run.smithi055.stdout:Setting orchestrator backend to cephadm... 2024-04-26T21:43:24.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:24 smithi055 ceph-mon[29221]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:43:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:24 smithi055 ceph-mon[29221]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:43:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:24 smithi055 ceph-mon[29221]: [26/Apr/2024:21:43:23] ENGINE Bus STARTING 2024-04-26T21:43:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:24 smithi055 ceph-mon[29221]: [26/Apr/2024:21:43:23] ENGINE Serving on https://172.21.15.55:7150 2024-04-26T21:43:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:24 smithi055 ceph-mon[29221]: [26/Apr/2024:21:43:23] ENGINE Serving on http://172.21.15.55:8765 2024-04-26T21:43:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:24 smithi055 ceph-mon[29221]: [26/Apr/2024:21:43:23] ENGINE Bus STARTED 2024-04-26T21:43:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:24 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.038+0000 7f148b258700 1 Processor -- start 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.038+0000 7f148b258700 1 -- start start 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.038+0000 7f148b258700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f148410a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.038+0000 7f148b258700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f148410acd0 con 0x7f14841082f0 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.038+0000 7f1488ff4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f148410a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.038+0000 7f1488ff4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f148410a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60536/0 (socket says 172.21.15.55:60536) 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.038+0000 7f1488ff4700 1 -- 172.21.15.55:0/1413509530 learned_addr learned my addr 172.21.15.55:0/1413509530 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.038+0000 7f1488ff4700 1 -- 172.21.15.55:0/1413509530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148410ae50 con 0x7f14841082f0 2024-04-26T21:43:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.039+0000 7f1488ff4700 1 --2- 172.21.15.55:0/1413509530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f148410a6e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14780098a0 tx=0x7f1478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2980e45ef6f06a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.039+0000 7f14837fe700 1 -- 172.21.15.55:0/1413509530 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1478004160 con 0x7f14841082f0 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.039+0000 7f14837fe700 1 -- 172.21.15.55:0/1413509530 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1478004300 con 0x7f14841082f0 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.039+0000 7f14837fe700 1 -- 172.21.15.55:0/1413509530 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1478011af0 con 0x7f14841082f0 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.039+0000 7f148b258700 1 -- 172.21.15.55:0/1413509530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 msgr2=0x7f148410a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.039+0000 7f148b258700 1 --2- 172.21.15.55:0/1413509530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f148410a6e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14780098a0 tx=0x7f1478009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 -- 172.21.15.55:0/1413509530 shutdown_connections 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 --2- 172.21.15.55:0/1413509530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f148410a6e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 -- 172.21.15.55:0/1413509530 >> 172.21.15.55:0/1413509530 conn(0x7f1484102480 msgr2=0x7f14841048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 -- 172.21.15.55:0/1413509530 shutdown_connections 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 -- 172.21.15.55:0/1413509530 wait complete. 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 Processor -- start 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 -- start start 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f1484192cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.941 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f148b258700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f148410bed0 con 0x7f14841082f0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f1488ff4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f1484192cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f1488ff4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f1484192cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60544/0 (socket says 172.21.15.55:60544) 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.040+0000 7f1488ff4700 1 -- 172.21.15.55:0/564628171 learned_addr learned my addr 172.21.15.55:0/564628171 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.041+0000 7f1488ff4700 1 -- 172.21.15.55:0/564628171 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1484193200 con 0x7f14841082f0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.041+0000 7f1488ff4700 1 --2- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f1484192cc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1478003f40 tx=0x7f1478011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.041+0000 7f1481ffb700 1 -- 172.21.15.55:0/564628171 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14780038b0 con 0x7f14841082f0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.041+0000 7f1481ffb700 1 -- 172.21.15.55:0/564628171 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1478003a50 con 0x7f14841082f0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.041+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14841913f0 con 0x7f14841082f0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.041+0000 7f1481ffb700 1 -- 172.21.15.55:0/564628171 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f147801a5c0 con 0x7f14841082f0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.041+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1484191930 con 0x7f14841082f0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.042+0000 7f1481ffb700 1 -- 172.21.15.55:0/564628171 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f147801a870 con 0x7f14841082f0 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.042+0000 7f1481ffb700 1 --2- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f14740394a0 0x7f147403b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:24.942 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.042+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1484064a80 con 0x7f14841082f0 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.042+0000 7f1481ffb700 1 -- 172.21.15.55:0/564628171 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f147804da20 con 0x7f14841082f0 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.042+0000 7f1483fff700 1 --2- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f14740394a0 0x7f147403b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.043+0000 7f1483fff700 1 --2- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f14740394a0 0x7f147403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1470006fd0 tx=0x7f1470006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.048+0000 7f1481ffb700 1 -- 172.21.15.55:0/564628171 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1478019c70 con 0x7f14841082f0 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.056+0000 7f1481ffb700 1 -- 172.21.15.55:0/564628171 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f147804c630 con 0x7f14841082f0 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.275+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1484196a80 con 0x7f14740394a0 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.282+0000 7f1481ffb700 1 -- 172.21.15.55:0/564628171 <== mgr.14120 v2:172.21.15.55:6800/2562126894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1484196a80 con 0x7f14740394a0 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f14740394a0 msgr2=0x7f147403b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 --2- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f14740394a0 0x7f147403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1470006fd0 tx=0x7f1470006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 msgr2=0x7f1484192cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 --2- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f1484192cc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1478003f40 tx=0x7f1478011890 comp rx=0 tx=0).stop 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 shutdown_connections 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 --2- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f14740394a0 0x7f147403b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.943 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 --2- 172.21.15.55:0/564628171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14841082f0 0x7f1484192cc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:24.944 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 >> 172.21.15.55:0/564628171 conn(0x7f1484102480 msgr2=0x7f1484103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:24.944 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 shutdown_connections 2024-04-26T21:43:24.944 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:24.285+0000 7f148b258700 1 -- 172.21.15.55:0/564628171 wait complete. 2024-04-26T21:43:25.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:25 smithi055 ceph-mon[29221]: mgrmap e8: x(active, since 2s) 2024-04-26T21:43:25.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:25 smithi055 ceph-mon[29221]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:25.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:25 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:25.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:25 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:26.146 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:26 smithi055 ceph-mon[29221]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:26.154 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T21:43:26.154 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.343+0000 7f5a02484700 1 Processor -- start 2024-04-26T21:43:26.154 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.343+0000 7f5a02484700 1 -- start start 2024-04-26T21:43:26.154 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.343+0000 7f5a02484700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.154 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.343+0000 7f5a02484700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc1085c0 con 0x7f59fc107bd0 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.343+0000 7f59fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.343+0000 7f59fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc107ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60554/0 (socket says 172.21.15.55:60554) 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.343+0000 7f59fbfff700 1 -- 172.21.15.55:0/3912776171 learned_addr learned my addr 172.21.15.55:0/3912776171 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.343+0000 7f59fbfff700 1 -- 172.21.15.55:0/3912776171 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc108dd0 con 0x7f59fc107bd0 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.344+0000 7f59fbfff700 1 --2- 172.21.15.55:0/3912776171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc107ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00bb40 tx=0x7f59ec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ed61c8b1aba8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.344+0000 7f59faffd700 1 -- 172.21.15.55:0/3912776171 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ec00dbe0 con 0x7f59fc107bd0 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.344+0000 7f59faffd700 1 -- 172.21.15.55:0/3912776171 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f59ec0044c0 con 0x7f59fc107bd0 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.344+0000 7f59faffd700 1 -- 172.21.15.55:0/3912776171 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ec010470 con 0x7f59fc107bd0 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.344+0000 7f5a02484700 1 -- 172.21.15.55:0/3912776171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 msgr2=0x7f59fc107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.344+0000 7f5a02484700 1 --2- 172.21.15.55:0/3912776171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc107ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00bb40 tx=0x7f59ec00be50 comp rx=0 tx=0).stop 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.344+0000 7f5a02484700 1 -- 172.21.15.55:0/3912776171 shutdown_connections 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.344+0000 7f5a02484700 1 --2- 172.21.15.55:0/3912776171 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc107ff0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.345+0000 7f5a02484700 1 -- 172.21.15.55:0/3912776171 >> 172.21.15.55:0/3912776171 conn(0x7f59fc1031b0 msgr2=0x7f59fc1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:26.155 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.345+0000 7f5a02484700 1 -- 172.21.15.55:0/3912776171 shutdown_connections 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.345+0000 7f5a02484700 1 -- 172.21.15.55:0/3912776171 wait complete. 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.345+0000 7f5a02484700 1 Processor -- start 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.345+0000 7f5a02484700 1 -- start start 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.345+0000 7f5a02484700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f5a02484700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc07bff0 con 0x7f59fc107bd0 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f59fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f59fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60564/0 (socket says 172.21.15.55:60564) 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f59fbfff700 1 -- 172.21.15.55:0/4238483191 learned_addr learned my addr 172.21.15.55:0/4238483191 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f59fbfff700 1 -- 172.21.15.55:0/4238483191 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc19af00 con 0x7f59fc107bd0 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f59fbfff700 1 --2- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc19a9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00bfd0 tx=0x7f59ec003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f59f97fa700 1 -- 172.21.15.55:0/4238483191 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ec027070 con 0x7f59fc107bd0 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f59f97fa700 1 -- 172.21.15.55:0/4238483191 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f59ec010c00 con 0x7f59fc107bd0 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc19b190 con 0x7f59fc107bd0 2024-04-26T21:43:26.156 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f59f97fa700 1 -- 172.21.15.55:0/4238483191 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59ec022c10 con 0x7f59fc107bd0 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.346+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc19de30 con 0x7f59fc107bd0 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.347+0000 7f59f97fa700 1 -- 172.21.15.55:0/4238483191 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f59ec02a070 con 0x7f59fc107bd0 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.347+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc06a510 con 0x7f59fc107bd0 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.347+0000 7f59f97fa700 1 --2- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f59e4039560 0x7f59e403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.347+0000 7f59f97fa700 1 -- 172.21.15.55:0/4238483191 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f59ec01f070 con 0x7f59fc107bd0 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.348+0000 7f59fb7fe700 1 --2- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f59e4039560 0x7f59e403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.348+0000 7f59fb7fe700 1 --2- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f59e4039560 0x7f59e403ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59f0006fd0 tx=0x7f59f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.353+0000 7f59f97fa700 1 -- 172.21.15.55:0/4238483191 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f59ec015070 con 0x7f59fc107bd0 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.562+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f59fc0008d0 con 0x7f59e4039560 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.562+0000 7f59f97fa700 1 -- 172.21.15.55:0/4238483191 <== mgr.14120 v2:172.21.15.55:6800/2562126894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f59fc0008d0 con 0x7f59e4039560 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f59e4039560 msgr2=0x7f59e403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 --2- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f59e4039560 0x7f59e403ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f59f0006fd0 tx=0x7f59f0006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 msgr2=0x7f59fc19a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:26.157 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 --2- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc19a9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00bfd0 tx=0x7f59ec003ff0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.158 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 shutdown_connections 2024-04-26T21:43:26.158 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 --2- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f59e4039560 0x7f59e403ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.158 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 --2- 172.21.15.55:0/4238483191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59fc107bd0 0x7f59fc19a9c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:26.158 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 >> 172.21.15.55:0/4238483191 conn(0x7f59fc1031b0 msgr2=0x7f59fc1915a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:26.158 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 shutdown_connections 2024-04-26T21:43:26.158 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:25.565+0000 7f5a02484700 1 -- 172.21.15.55:0/4238483191 wait complete. 2024-04-26T21:43:26.158 INFO:teuthology.orchestra.run.smithi055.stdout:Generating ssh key... 2024-04-26T21:43:27.410 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-mon[29221]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:27.410 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-mon[29221]: Generating ssh key... 2024-04-26T21:43:27.410 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: Generating public/private rsa key pair. 2024-04-26T21:43:27.410 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: Your identification has been saved in /tmp/tmpu1rzrim0/key. 2024-04-26T21:43:27.411 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: Your public key has been saved in /tmp/tmpu1rzrim0/key.pub. 2024-04-26T21:43:27.411 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: The key fingerprint is: 2024-04-26T21:43:27.411 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: SHA256:ICi1hPMhKQPmuTUOpTqZ9ZkUW2Z5N5P343gs6MhXWOg ceph-c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:27.411 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: The key's randomart image is: 2024-04-26T21:43:27.411 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: +---[RSA 3072]----+ 2024-04-26T21:43:27.411 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: |o+o.. +. . | 2024-04-26T21:43:27.411 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: |Xo=o *. . = . | 2024-04-26T21:43:27.411 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: |oX+++ .. . = . | 2024-04-26T21:43:27.412 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: |.=*o.+ . . . o | 2024-04-26T21:43:27.412 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: |=. .+ S. + + . | 2024-04-26T21:43:27.412 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: | . E + + | 2024-04-26T21:43:27.412 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: | . o . o | 2024-04-26T21:43:27.412 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: | o o | 2024-04-26T21:43:27.412 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: | . | 2024-04-26T21:43:27.412 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: +----[SHA256]-----+ 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.574+0000 7fe8c84f6700 1 Processor -- start 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.574+0000 7fe8c84f6700 1 -- start start 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.575+0000 7fe8c84f6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c0109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.575+0000 7fe8c84f6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0109740 con 0x7fe8c0108d50 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.575+0000 7fe8c6292700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c0109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.575+0000 7fe8c6292700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c0109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60580/0 (socket says 172.21.15.55:60580) 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.575+0000 7fe8c6292700 1 -- 172.21.15.55:0/2456776996 learned_addr learned my addr 172.21.15.55:0/2456776996 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.575+0000 7fe8c6292700 1 -- 172.21.15.55:0/2456776996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c0109f50 con 0x7fe8c0108d50 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.575+0000 7fe8c6292700 1 --2- 172.21.15.55:0/2456776996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c0109170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00098a0 tx=0x7fe8b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec71c37a901f322 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c5290700 1 -- 172.21.15.55:0/2456776996 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8b0004160 con 0x7fe8c0108d50 2024-04-26T21:43:27.686 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c5290700 1 -- 172.21.15.55:0/2456776996 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe8b0004300 con 0x7fe8c0108d50 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c5290700 1 -- 172.21.15.55:0/2456776996 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8b0011af0 con 0x7fe8c0108d50 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c84f6700 1 -- 172.21.15.55:0/2456776996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 msgr2=0x7fe8c0109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c84f6700 1 --2- 172.21.15.55:0/2456776996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c0109170 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00098a0 tx=0x7fe8b0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c84f6700 1 -- 172.21.15.55:0/2456776996 shutdown_connections 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c84f6700 1 --2- 172.21.15.55:0/2456776996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c0109170 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c84f6700 1 -- 172.21.15.55:0/2456776996 >> 172.21.15.55:0/2456776996 conn(0x7fe8c01042f0 msgr2=0x7fe8c0106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c84f6700 1 -- 172.21.15.55:0/2456776996 shutdown_connections 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.576+0000 7fe8c84f6700 1 -- 172.21.15.55:0/2456776996 wait complete. 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.577+0000 7fe8c84f6700 1 Processor -- start 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.577+0000 7fe8c84f6700 1 -- start start 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.577+0000 7fe8c84f6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.577+0000 7fe8c84f6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c010d180 con 0x7fe8c0108d50 2024-04-26T21:43:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.577+0000 7fe8c6292700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.577+0000 7fe8c6292700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60596/0 (socket says 172.21.15.55:60596) 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.577+0000 7fe8c6292700 1 -- 172.21.15.55:0/622439613 learned_addr learned my addr 172.21.15.55:0/622439613 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.578+0000 7fe8c6292700 1 -- 172.21.15.55:0/622439613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c019d140 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.578+0000 7fe8c6292700 1 --2- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c019cc00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0003f40 tx=0x7fe8b0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.578+0000 7fe8b77fe700 1 -- 172.21.15.55:0/622439613 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8b00038b0 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.578+0000 7fe8b77fe700 1 -- 172.21.15.55:0/622439613 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe8b0003a50 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.578+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c019d3d0 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.578+0000 7fe8b77fe700 1 -- 172.21.15.55:0/622439613 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8b001a5c0 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.578+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c01a00c0 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.579+0000 7fe8b77fe700 1 -- 172.21.15.55:0/622439613 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe8b001a870 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.579+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c010d300 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.579+0000 7fe8b77fe700 1 --2- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fe8ac0395b0 0x7fe8ac03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.579+0000 7fe8b77fe700 1 -- 172.21.15.55:0/622439613 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe8b004db80 con 0x7fe8c0108d50 2024-04-26T21:43:27.688 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.579+0000 7fe8c5a91700 1 --2- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fe8ac0395b0 0x7fe8ac03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.580+0000 7fe8c5a91700 1 --2- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fe8ac0395b0 0x7fe8ac03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc006fd0 tx=0x7fe8bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.584+0000 7fe8b77fe700 1 -- 172.21.15.55:0/622439613 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe8b0019c70 con 0x7fe8c0108d50 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:26.793+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe8c01054d0 con 0x7fe8ac0395b0 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.096+0000 7fe8b77fe700 1 -- 172.21.15.55:0/622439613 <== mgr.14120 v2:172.21.15.55:6800/2562126894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8c01054d0 con 0x7fe8ac0395b0 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fe8ac0395b0 msgr2=0x7fe8ac03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 --2- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fe8ac0395b0 0x7fe8ac03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc006fd0 tx=0x7fe8bc006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 msgr2=0x7fe8c019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 --2- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c019cc00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0003f40 tx=0x7fe8b0011890 comp rx=0 tx=0).stop 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 shutdown_connections 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 --2- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fe8ac0395b0 0x7fe8ac03ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 --2- 172.21.15.55:0/622439613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8c0108d50 0x7fe8c019cc00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 >> 172.21.15.55:0/622439613 conn(0x7fe8c01042f0 msgr2=0x7fe8c0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 shutdown_connections 2024-04-26T21:43:27.689 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:27.098+0000 7fe8c84f6700 1 -- 172.21.15.55:0/622439613 wait complete. 2024-04-26T21:43:28.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:28 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:28.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:28 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD0dx9JkRXk+yKQjZUFBN6ZdZHpT7lRv6tKyV53lc0IzWJIbkq04OASRwUFE3Uo6r4zkNdU0lugxSdjGFzDbqKQczIYs9g/zYMklFQVnjM+3MSNzrUf6hME/HCZciyG7rTgFVvxH9w7zqWY6wnsQkzJbGxhiNqGLlCXP7xKuPeVuSvqBHpDJP6H6uaJ1PjJWUuOaNDTyvlH6KpLyjtsb6kQdUAczawMZCNK0jwqA9oDWZfgGqcPZ743OqvYK4UN4PCSA4Vs4hQr8gl8p3uHv0YxCs7kkC+vMuwnzh9byNyU/pSA+nNVhmjY038WQv4RdjH1fkqoF6MzmFRl0GtPSYSVYIdF7fOU/cFoE6ZatpRux0M5o+Q435IyGiosizWT6zlsl9bvdb7UjNd7LmiZhqRhp4uyEs+7Vy+BgB3Yz+UXcxbBa6mdNg+7ZgkWZF14AQiP6YxMeyedxMk8uiOntJ4hHZtqJNYrBbQkPScaXHlxynFuIrq6gkEjoo35erHc6ac= ceph-c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.150+0000 7fb505f05700 1 Processor -- start 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.151+0000 7fb505f05700 1 -- start start 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.151+0000 7fb505f05700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb500108010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.151+0000 7fb505f05700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb5001085e0 con 0x7fb500107bf0 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.151+0000 7fb4ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb500108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.151+0000 7fb4ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb500108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60600/0 (socket says 172.21.15.55:60600) 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.151+0000 7fb4ff7fe700 1 -- 172.21.15.55:0/344029575 learned_addr learned my addr 172.21.15.55:0/344029575 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.151+0000 7fb4ff7fe700 1 -- 172.21.15.55:0/344029575 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb500108df0 con 0x7fb500107bf0 2024-04-26T21:43:28.951 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.151+0000 7fb4ff7fe700 1 --2- 172.21.15.55:0/344029575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb500108010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00098a0 tx=0x7fb4f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247be85f73bab2d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb4fe7fc700 1 -- 172.21.15.55:0/344029575 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0004160 con 0x7fb500107bf0 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb4fe7fc700 1 -- 172.21.15.55:0/344029575 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb4f0004300 con 0x7fb500107bf0 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb4fe7fc700 1 -- 172.21.15.55:0/344029575 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0011af0 con 0x7fb500107bf0 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb505f05700 1 -- 172.21.15.55:0/344029575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 msgr2=0x7fb500108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb505f05700 1 --2- 172.21.15.55:0/344029575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb500108010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00098a0 tx=0x7fb4f0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb505f05700 1 -- 172.21.15.55:0/344029575 shutdown_connections 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb505f05700 1 --2- 172.21.15.55:0/344029575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb500108010 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb505f05700 1 -- 172.21.15.55:0/344029575 >> 172.21.15.55:0/344029575 conn(0x7fb500103190 msgr2=0x7fb5001055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb505f05700 1 -- 172.21.15.55:0/344029575 shutdown_connections 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.152+0000 7fb505f05700 1 -- 172.21.15.55:0/344029575 wait complete. 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.153+0000 7fb505f05700 1 Processor -- start 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.153+0000 7fb505f05700 1 -- start start 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.153+0000 7fb505f05700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb50019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.952 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.153+0000 7fb505f05700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb50010c1a0 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb4ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb50019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb4ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb50019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60606/0 (socket says 172.21.15.55:60606) 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb4ff7fe700 1 -- 172.21.15.55:0/2280729530 learned_addr learned my addr 172.21.15.55:0/2280729530 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb4ff7fe700 1 -- 172.21.15.55:0/2280729530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50019d1e0 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb4ff7fe700 1 --2- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb50019cca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00117b0 tx=0x7fb4f0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb4fcff9700 1 -- 172.21.15.55:0/2280729530 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0003af0 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb4fcff9700 1 -- 172.21.15.55:0/2280729530 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb4f0020070 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50019d470 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb4fcff9700 1 -- 172.21.15.55:0/2280729530 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4f0019630 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.154+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5001a0160 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.155+0000 7fb4fcff9700 1 -- 172.21.15.55:0/2280729530 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb4f00198a0 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.155+0000 7fb4fcff9700 1 --2- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb4e8039600 0x7fb4e803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.155+0000 7fb4fcff9700 1 -- 172.21.15.55:0/2280729530 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4f004db80 con 0x7fb500107bf0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.155+0000 7fb4feffd700 1 --2- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb4e8039600 0x7fb4e803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:28.953 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.155+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb500064a80 con 0x7fb500107bf0 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.156+0000 7fb4feffd700 1 --2- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb4e8039600 0x7fb4e803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006fd0 tx=0x7fb4f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.161+0000 7fb4fcff9700 1 -- 172.21.15.55:0/2280729530 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb4f002bd00 con 0x7fb500107bf0 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.372+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb50002e700 con 0x7fb4e8039600 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.373+0000 7fb4fcff9700 1 -- 172.21.15.55:0/2280729530 <== mgr.14120 v2:172.21.15.55:6800/2562126894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb50002e700 con 0x7fb4e8039600 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.375+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb4e8039600 msgr2=0x7fb4e803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.375+0000 7fb505f05700 1 --2- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb4e8039600 0x7fb4e803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006fd0 tx=0x7fb4f4006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.375+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 msgr2=0x7fb50019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.375+0000 7fb505f05700 1 --2- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb50019cca0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb4f00117b0 tx=0x7fb4f0011b70 comp rx=0 tx=0).stop 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.375+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 shutdown_connections 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.375+0000 7fb505f05700 1 --2- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb4e8039600 0x7fb4e803bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.375+0000 7fb505f05700 1 --2- 172.21.15.55:0/2280729530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb500107bf0 0x7fb50019cca0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.375+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 >> 172.21.15.55:0/2280729530 conn(0x7fb500103190 msgr2=0x7fb500104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.376+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 shutdown_connections 2024-04-26T21:43:28.954 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:28.376+0000 7fb505f05700 1 -- 172.21.15.55:0/2280729530 wait complete. 2024-04-26T21:43:28.955 INFO:teuthology.orchestra.run.smithi055.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T21:43:28.955 INFO:teuthology.orchestra.run.smithi055.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T21:43:28.955 INFO:teuthology.orchestra.run.smithi055.stdout:Adding host smithi055... 2024-04-26T21:43:29.200 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:29 smithi055 ceph-mon[29221]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:30.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:30 smithi055 ceph-mon[29221]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi055", "addr": "172.21.15.55", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:31.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:31 smithi055 ceph-mon[29221]: Deploying cephadm binary to smithi055 2024-04-26T21:43:34.499 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout Added host 'smithi055' with addr '172.21.15.55' 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.422+0000 7fbe24655700 1 Processor -- start 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.422+0000 7fbe24655700 1 -- start start 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.422+0000 7fbe24655700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.422+0000 7fbe24655700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c109740 con 0x7fbe1c108d50 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.422+0000 7fbe223f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.423+0000 7fbe223f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60610/0 (socket says 172.21.15.55:60610) 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.423+0000 7fbe223f1700 1 -- 172.21.15.55:0/413880642 learned_addr learned my addr 172.21.15.55:0/413880642 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.423+0000 7fbe223f1700 1 -- 172.21.15.55:0/413880642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c109f50 con 0x7fbe1c108d50 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.423+0000 7fbe223f1700 1 --2- 172.21.15.55:0/413880642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c109170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009e80 tx=0x7fbe1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f53b46563ba1d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.423+0000 7fbe213ef700 1 -- 172.21.15.55:0/413880642 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe14017070 con 0x7fbe1c108d50 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.423+0000 7fbe213ef700 1 -- 172.21.15.55:0/413880642 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe1400b8c0 con 0x7fbe1c108d50 2024-04-26T21:43:34.500 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.423+0000 7fbe213ef700 1 -- 172.21.15.55:0/413880642 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe14004490 con 0x7fbe1c108d50 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.424+0000 7fbe24655700 1 -- 172.21.15.55:0/413880642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 msgr2=0x7fbe1c109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.424+0000 7fbe24655700 1 --2- 172.21.15.55:0/413880642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c109170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009e80 tx=0x7fbe1400b330 comp rx=0 tx=0).stop 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.424+0000 7fbe24655700 1 -- 172.21.15.55:0/413880642 shutdown_connections 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.424+0000 7fbe24655700 1 --2- 172.21.15.55:0/413880642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c109170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.424+0000 7fbe24655700 1 -- 172.21.15.55:0/413880642 >> 172.21.15.55:0/413880642 conn(0x7fbe1c1042f0 msgr2=0x7fbe1c106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.424+0000 7fbe24655700 1 -- 172.21.15.55:0/413880642 shutdown_connections 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.424+0000 7fbe24655700 1 -- 172.21.15.55:0/413880642 wait complete. 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe24655700 1 Processor -- start 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe24655700 1 -- start start 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe24655700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe24655700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c10d180 con 0x7fbe1c108d50 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe223f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe223f1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60614/0 (socket says 172.21.15.55:60614) 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe223f1700 1 -- 172.21.15.55:0/2299247592 learned_addr learned my addr 172.21.15.55:0/2299247592 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:34.501 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe223f1700 1 -- 172.21.15.55:0/2299247592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c19d0d0 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe223f1700 1 --2- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c19cb90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe14004a60 tx=0x7fbe1400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe137fe700 1 -- 172.21.15.55:0/2299247592 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe14020020 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.425+0000 7fbe137fe700 1 -- 172.21.15.55:0/2299247592 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe14003c60 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.426+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c19d360 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.426+0000 7fbe137fe700 1 -- 172.21.15.55:0/2299247592 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe14017040 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.426+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1a0050 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.427+0000 7fbe137fe700 1 -- 172.21.15.55:0/2299247592 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbe14023020 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.427+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1c10d300 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.427+0000 7fbe137fe700 1 --2- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fbe08039600 0x7fbe0803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.427+0000 7fbe137fe700 1 -- 172.21.15.55:0/2299247592 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbe1400f050 con 0x7fbe1c108d50 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.427+0000 7fbe21bf0700 1 --2- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fbe08039600 0x7fbe0803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.427+0000 7fbe21bf0700 1 --2- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fbe08039600 0x7fbe0803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006fd0 tx=0x7fbe0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:34.502 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.432+0000 7fbe137fe700 1 -- 172.21.15.55:0/2299247592 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbe1402a020 con 0x7fbe1c108d50 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:29.644+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi055", "addr": "172.21.15.55", "target": ["mon-mgr", ""]}) -- 0x7fbe1c105470 con 0x7fbe08039600 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.939+0000 7fbe137fe700 1 -- 172.21.15.55:0/2299247592 <== mgr.14120 v2:172.21.15.55:6800/2562126894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fbe1c105470 con 0x7fbe08039600 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fbe08039600 msgr2=0x7fbe0803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 --2- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fbe08039600 0x7fbe0803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006fd0 tx=0x7fbe0c006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 msgr2=0x7fbe1c19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 --2- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c19cb90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe14004a60 tx=0x7fbe1400bd50 comp rx=0 tx=0).stop 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 shutdown_connections 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 --2- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fbe08039600 0x7fbe0803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 --2- 172.21.15.55:0/2299247592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe1c108d50 0x7fbe1c19cb90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 >> 172.21.15.55:0/2299247592 conn(0x7fbe1c1042f0 msgr2=0x7fbe1c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 shutdown_connections 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:33.942+0000 7fbe24655700 1 -- 172.21.15.55:0/2299247592 wait complete. 2024-04-26T21:43:34.503 INFO:teuthology.orchestra.run.smithi055.stdout:Deploying unmanaged mon service... 2024-04-26T21:43:35.174 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:34 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:35.175 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:34 smithi055 ceph-mon[29221]: Added host smithi055 2024-04-26T21:43:35.175 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:34 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:35.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T21:43:35.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.941+0000 7f9ff6a34700 1 Processor -- start 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.941+0000 7f9ff6a34700 1 -- start start 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.941+0000 7f9ff6a34700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff0109570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.941+0000 7f9ff6a34700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0109b40 con 0x7f9ff0109150 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.941+0000 7f9feffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff0109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.941+0000 7f9feffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff0109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58990/0 (socket says 172.21.15.55:58990) 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.941+0000 7f9feffff700 1 -- 172.21.15.55:0/980732630 learned_addr learned my addr 172.21.15.55:0/980732630 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.941+0000 7f9feffff700 1 -- 172.21.15.55:0/980732630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff0109cc0 con 0x7f9ff0109150 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.942+0000 7f9feffff700 1 --2- 172.21.15.55:0/980732630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff0109570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00098a0 tx=0x7f9fe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d1d4fbb8a85428 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.942+0000 7f9feeffd700 1 -- 172.21.15.55:0/980732630 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe0004160 con 0x7f9ff0109150 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.942+0000 7f9feeffd700 1 -- 172.21.15.55:0/980732630 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9fe0004300 con 0x7f9ff0109150 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.942+0000 7f9feeffd700 1 -- 172.21.15.55:0/980732630 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe0011af0 con 0x7f9ff0109150 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.942+0000 7f9ff6a34700 1 -- 172.21.15.55:0/980732630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 msgr2=0x7f9ff0109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.942+0000 7f9ff6a34700 1 --2- 172.21.15.55:0/980732630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff0109570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00098a0 tx=0x7f9fe0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.943+0000 7f9ff6a34700 1 -- 172.21.15.55:0/980732630 shutdown_connections 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.943+0000 7f9ff6a34700 1 --2- 172.21.15.55:0/980732630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff0109570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.943+0000 7f9ff6a34700 1 -- 172.21.15.55:0/980732630 >> 172.21.15.55:0/980732630 conn(0x7f9ff0102940 msgr2=0x7f9ff0104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.943+0000 7f9ff6a34700 1 -- 172.21.15.55:0/980732630 shutdown_connections 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.943+0000 7f9ff6a34700 1 -- 172.21.15.55:0/980732630 wait complete. 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.943+0000 7f9ff6a34700 1 Processor -- start 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.943+0000 7f9ff6a34700 1 -- start start 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9ff6a34700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9ff6a34700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff010c0d0 con 0x7f9ff0109150 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9feffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9feffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59000/0 (socket says 172.21.15.55:59000) 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9feffff700 1 -- 172.21.15.55:0/2935871431 learned_addr learned my addr 172.21.15.55:0/2935871431 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9feffff700 1 -- 172.21.15.55:0/2935871431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff019d150 con 0x7f9ff0109150 2024-04-26T21:43:35.793 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9feffff700 1 --2- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff019cc10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0003f40 tx=0x7f9fe0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9fed7fa700 1 -- 172.21.15.55:0/2935871431 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe00038b0 con 0x7f9ff0109150 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9fed7fa700 1 -- 172.21.15.55:0/2935871431 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9fe0003a50 con 0x7f9ff0109150 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff019d3e0 con 0x7f9ff0109150 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9fed7fa700 1 -- 172.21.15.55:0/2935871431 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe001a5c0 con 0x7f9ff0109150 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.944+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff01a00d0 con 0x7f9ff0109150 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.945+0000 7f9fed7fa700 1 -- 172.21.15.55:0/2935871431 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9fe001a870 con 0x7f9ff0109150 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.945+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff0064a80 con 0x7f9ff0109150 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.946+0000 7f9fed7fa700 1 --2- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f9fd80395b0 0x7f9fd803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.946+0000 7f9fed7fa700 1 -- 172.21.15.55:0/2935871431 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9fe004dac0 con 0x7f9ff0109150 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.946+0000 7f9fef7fe700 1 --2- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f9fd80395b0 0x7f9fd803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.946+0000 7f9fef7fe700 1 --2- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f9fd80395b0 0x7f9fd803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006fd0 tx=0x7f9fe4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:35.794 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:34.951+0000 7f9fed7fa700 1 -- 172.21.15.55:0/2935871431 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9fe0019c70 con 0x7f9ff0109150 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.174+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9ff0103ad0 con 0x7f9fd80395b0 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.179+0000 7f9fed7fa700 1 -- 172.21.15.55:0/2935871431 <== mgr.14120 v2:172.21.15.55:6800/2562126894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9ff0103ad0 con 0x7f9fd80395b0 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f9fd80395b0 msgr2=0x7f9fd803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 --2- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f9fd80395b0 0x7f9fd803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4006fd0 tx=0x7f9fe4006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 msgr2=0x7f9ff019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 --2- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff019cc10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0003f40 tx=0x7f9fe0011890 comp rx=0 tx=0).stop 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 shutdown_connections 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 --2- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f9fd80395b0 0x7f9fd803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 --2- 172.21.15.55:0/2935871431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ff0109150 0x7f9ff019cc10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 >> 172.21.15.55:0/2935871431 conn(0x7f9ff0102940 msgr2=0x7f9ff01033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 shutdown_connections 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:35.181+0000 7f9ff6a34700 1 -- 172.21.15.55:0/2935871431 wait complete. 2024-04-26T21:43:35.795 INFO:teuthology.orchestra.run.smithi055.stdout:Deploying unmanaged mgr service... 2024-04-26T21:43:36.398 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:36 smithi055 ceph-mon[29221]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:36.398 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:36 smithi055 ceph-mon[29221]: Saving service mon spec with placement count:5 2024-04-26T21:43:36.398 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:36 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:37.661 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.252+0000 7f51c2654700 1 Processor -- start 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.252+0000 7f51c2654700 1 -- start start 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.252+0000 7f51c2654700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.252+0000 7f51c2654700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc07ce90 con 0x7f51bc1075e0 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.253+0000 7f51bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.253+0000 7f51bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59008/0 (socket says 172.21.15.55:59008) 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.253+0000 7f51bbfff700 1 -- 172.21.15.55:0/186409157 learned_addr learned my addr 172.21.15.55:0/186409157 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.253+0000 7f51bbfff700 1 -- 172.21.15.55:0/186409157 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc107f40 con 0x7f51bc1075e0 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.253+0000 7f51bbfff700 1 --2- 172.21.15.55:0/186409157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc107a00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0098a0 tx=0x7f51ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58042b9c9387119e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.253+0000 7f51baffd700 1 -- 172.21.15.55:0/186409157 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac004160 con 0x7f51bc1075e0 2024-04-26T21:43:37.662 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.253+0000 7f51baffd700 1 -- 172.21.15.55:0/186409157 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f51ac004300 con 0x7f51bc1075e0 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.253+0000 7f51baffd700 1 -- 172.21.15.55:0/186409157 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac011af0 con 0x7f51bc1075e0 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.254+0000 7f51c2654700 1 -- 172.21.15.55:0/186409157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 msgr2=0x7f51bc107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.254+0000 7f51c2654700 1 --2- 172.21.15.55:0/186409157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc107a00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0098a0 tx=0x7f51ac009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.254+0000 7f51c2654700 1 -- 172.21.15.55:0/186409157 shutdown_connections 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.254+0000 7f51c2654700 1 --2- 172.21.15.55:0/186409157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc107a00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.254+0000 7f51c2654700 1 -- 172.21.15.55:0/186409157 >> 172.21.15.55:0/186409157 conn(0x7f51bc103220 msgr2=0x7f51bc105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.254+0000 7f51c2654700 1 -- 172.21.15.55:0/186409157 shutdown_connections 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.254+0000 7f51c2654700 1 -- 172.21.15.55:0/186409157 wait complete. 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51c2654700 1 Processor -- start 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51c2654700 1 -- start start 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51c2654700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51c2654700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc108f00 con 0x7f51bc1075e0 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:37.663 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59022/0 (socket says 172.21.15.55:59022) 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51bbfff700 1 -- 172.21.15.55:0/2914270440 learned_addr learned my addr 172.21.15.55:0/2914270440 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51bbfff700 1 -- 172.21.15.55:0/2914270440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc19f2f0 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.255+0000 7f51bbfff700 1 --2- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc19edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f51ac003f40 tx=0x7f51ac011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.256+0000 7f51b97fa700 1 -- 172.21.15.55:0/2914270440 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac0038b0 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.256+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51bc19f580 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.256+0000 7f51b97fa700 1 -- 172.21.15.55:0/2914270440 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f51ac003a50 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.256+0000 7f51b97fa700 1 -- 172.21.15.55:0/2914270440 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51ac01a5c0 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.256+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51bc1a2230 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.257+0000 7f51b97fa700 1 -- 172.21.15.55:0/2914270440 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f51ac024430 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.257+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51bc064a80 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.257+0000 7f51b97fa700 1 --2- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f51a40395b0 0x7f51a403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.257+0000 7f51b97fa700 1 -- 172.21.15.55:0/2914270440 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f51ac04dab0 con 0x7f51bc1075e0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.257+0000 7f51bb7fe700 1 --2- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f51a40395b0 0x7f51a403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:37.664 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.257+0000 7f51bb7fe700 1 --2- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f51a40395b0 0x7f51a403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51b0006fd0 tx=0x7f51b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.262+0000 7f51b97fa700 1 -- 172.21.15.55:0/2914270440 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f51ac02d910 con 0x7f51bc1075e0 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.501+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f51bc02e840 con 0x7f51a40395b0 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.510+0000 7f51b97fa700 1 -- 172.21.15.55:0/2914270440 <== mgr.14120 v2:172.21.15.55:6800/2562126894 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f51bc02e840 con 0x7f51a40395b0 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f51a40395b0 msgr2=0x7f51a403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 --2- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f51a40395b0 0x7f51a403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51b0006fd0 tx=0x7f51b0006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 msgr2=0x7f51bc19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 --2- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc19edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f51ac003f40 tx=0x7f51ac011890 comp rx=0 tx=0).stop 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 shutdown_connections 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 --2- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f51a40395b0 0x7f51a403ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 --2- 172.21.15.55:0/2914270440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51bc1075e0 0x7f51bc19edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 >> 172.21.15.55:0/2914270440 conn(0x7f51bc103220 msgr2=0x7f51bc195a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 shutdown_connections 2024-04-26T21:43:37.665 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:36.513+0000 7f51c2654700 1 -- 172.21.15.55:0/2914270440 wait complete. 2024-04-26T21:43:37.783 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:37 smithi055 ceph-mon[29221]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:37.783 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:37 smithi055 ceph-mon[29221]: Saving service mgr spec with placement count:2 2024-04-26T21:43:37.783 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:37 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:39.378 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.428+0000 7ff6b4e7a700 1 Processor -- start 2024-04-26T21:43:39.378 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.428+0000 7ff6b4e7a700 1 -- start start 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.429+0000 7ff6b4e7a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b010bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.429+0000 7ff6b4e7a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b007cdf0 con 0x7ff6b0109850 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.429+0000 7ff6ae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b010bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.429+0000 7ff6ae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b010bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59026/0 (socket says 172.21.15.55:59026) 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.429+0000 7ff6ae59c700 1 -- 172.21.15.55:0/3568050651 learned_addr learned my addr 172.21.15.55:0/3568050651 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.429+0000 7ff6ae59c700 1 -- 172.21.15.55:0/3568050651 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b010c180 con 0x7ff6b0109850 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.429+0000 7ff6ae59c700 1 --2- 172.21.15.55:0/3568050651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b010bc40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff6980098a0 tx=0x7ff698009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4855c89d4e75a52d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.429+0000 7ff6ad59a700 1 -- 172.21.15.55:0/3568050651 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff698004160 con 0x7ff6b0109850 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6ad59a700 1 -- 172.21.15.55:0/3568050651 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff698004300 con 0x7ff6b0109850 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6ad59a700 1 -- 172.21.15.55:0/3568050651 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff698011a20 con 0x7ff6b0109850 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/3568050651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 msgr2=0x7ff6b010bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6b4e7a700 1 --2- 172.21.15.55:0/3568050651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b010bc40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff6980098a0 tx=0x7ff698009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/3568050651 shutdown_connections 2024-04-26T21:43:39.379 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6b4e7a700 1 --2- 172.21.15.55:0/3568050651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b010bc40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/3568050651 >> 172.21.15.55:0/3568050651 conn(0x7ff6b01031e0 msgr2=0x7ff6b0105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/3568050651 shutdown_connections 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.430+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/3568050651 wait complete. 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.431+0000 7ff6b4e7a700 1 Processor -- start 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.431+0000 7ff6b4e7a700 1 -- start start 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.431+0000 7ff6b4e7a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b01090d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.431+0000 7ff6b4e7a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b010d0c0 con 0x7ff6b0109850 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6ae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b01090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6ae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b01090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59038/0 (socket says 172.21.15.55:59038) 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6ae59c700 1 -- 172.21.15.55:0/527942564 learned_addr learned my addr 172.21.15.55:0/527942564 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6ae59c700 1 -- 172.21.15.55:0/527942564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b0109610 con 0x7ff6b0109850 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6ae59c700 1 --2- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b01090d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6980116e0 tx=0x7ff698011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6a77fe700 1 -- 172.21.15.55:0/527942564 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff698003a60 con 0x7ff6b0109850 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6a77fe700 1 -- 172.21.15.55:0/527942564 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff69801b8f0 con 0x7ff6b0109850 2024-04-26T21:43:39.380 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b0107800 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6a77fe700 1 -- 172.21.15.55:0/527942564 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff698019860 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.432+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b0107d40 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.433+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b0064a80 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.434+0000 7ff6a77fe700 1 -- 172.21.15.55:0/527942564 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff698019a00 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.434+0000 7ff6a77fe700 1 --2- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7ff69c0395b0 0x7ff69c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.434+0000 7ff6a77fe700 1 -- 172.21.15.55:0/527942564 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff69804dce0 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.434+0000 7ff6add9b700 1 --2- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7ff69c0395b0 0x7ff69c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.435+0000 7ff6add9b700 1 --2- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7ff69c0395b0 0x7ff69c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0006fd0 tx=0x7ff6a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.439+0000 7ff6a77fe700 1 -- 172.21.15.55:0/527942564 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff69801f030 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.646+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff6b0108460 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.648+0000 7ff6a77fe700 1 -- 172.21.15.55:0/527942564 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7ff698029260 con 0x7ff6b0109850 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7ff69c0395b0 msgr2=0x7ff69c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 --2- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7ff69c0395b0 0x7ff69c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0006fd0 tx=0x7ff6a0006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:39.381 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 msgr2=0x7ff6b01090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:39.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 --2- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b01090d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6980116e0 tx=0x7ff698011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:39.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 shutdown_connections 2024-04-26T21:43:39.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 --2- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7ff69c0395b0 0x7ff69c03ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:39.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 --2- 172.21.15.55:0/527942564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6b0109850 0x7ff6b01090d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:39.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 >> 172.21.15.55:0/527942564 conn(0x7ff6b01031e0 msgr2=0x7ff6b0103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:39.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 shutdown_connections 2024-04-26T21:43:39.382 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:38.651+0000 7ff6b4e7a700 1 -- 172.21.15.55:0/527942564 wait complete. 2024-04-26T21:43:39.586 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:39 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:39.586 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:39 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/527942564' entity='client.admin' 2024-04-26T21:43:39.587 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:39 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:41.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:41 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/207331744' entity='client.admin' 2024-04-26T21:43:41.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:41 smithi055 ceph-mon[29221]: from='mgr.14120 172.21.15.55:0/333439213' entity='mgr.x' 2024-04-26T21:43:41.220 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.800+0000 7fdf22b6d700 1 Processor -- start 2024-04-26T21:43:41.220 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.800+0000 7fdf22b6d700 1 -- start start 2024-04-26T21:43:41.220 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.800+0000 7fdf22b6d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c109110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.800+0000 7fdf22b6d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1096e0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.801+0000 7fdf20909700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.801+0000 7fdf20909700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c109110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59048/0 (socket says 172.21.15.55:59048) 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.801+0000 7fdf20909700 1 -- 172.21.15.55:0/2206376445 learned_addr learned my addr 172.21.15.55:0/2206376445 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.801+0000 7fdf20909700 1 -- 172.21.15.55:0/2206376445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c109ef0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.801+0000 7fdf20909700 1 --2- 172.21.15.55:0/2206376445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c109110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf100098a0 tx=0x7fdf10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d60a216f10d5ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.801+0000 7fdf1b7fe700 1 -- 172.21.15.55:0/2206376445 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf10004160 con 0x7fdf1c108cf0 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.801+0000 7fdf1b7fe700 1 -- 172.21.15.55:0/2206376445 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf10004300 con 0x7fdf1c108cf0 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.801+0000 7fdf1b7fe700 1 -- 172.21.15.55:0/2206376445 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf10011a20 con 0x7fdf1c108cf0 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 -- 172.21.15.55:0/2206376445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 msgr2=0x7fdf1c109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 --2- 172.21.15.55:0/2206376445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c109110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf100098a0 tx=0x7fdf10009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 -- 172.21.15.55:0/2206376445 shutdown_connections 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 --2- 172.21.15.55:0/2206376445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c109110 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.221 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 -- 172.21.15.55:0/2206376445 >> 172.21.15.55:0/2206376445 conn(0x7fdf1c1042b0 msgr2=0x7fdf1c1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 -- 172.21.15.55:0/2206376445 shutdown_connections 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 -- 172.21.15.55:0/2206376445 wait complete. 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 Processor -- start 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.802+0000 7fdf22b6d700 1 -- start start 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf22b6d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c07eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf22b6d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c10d2a0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf20909700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c07eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf20909700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c07eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59052/0 (socket says 172.21.15.55:59052) 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf20909700 1 -- 172.21.15.55:0/207331744 learned_addr learned my addr 172.21.15.55:0/207331744 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf20909700 1 -- 172.21.15.55:0/207331744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c07f0d0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf20909700 1 --2- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c07eb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf100116e0 tx=0x7fdf10011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf19ffb700 1 -- 172.21.15.55:0/207331744 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf100039e0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.222 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf19ffb700 1 -- 172.21.15.55:0/207331744 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1001b8f0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c07d2c0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.803+0000 7fdf19ffb700 1 -- 172.21.15.55:0/207331744 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf10023760 con 0x7fdf1c108cf0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.804+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c07d800 con 0x7fdf1c108cf0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.804+0000 7fdf19ffb700 1 -- 172.21.15.55:0/207331744 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdf100239d0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.805+0000 7fdf19ffb700 1 --2- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fdf0c039560 0x7fdf0c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.805+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c064a80 con 0x7fdf1c108cf0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.805+0000 7fdf19ffb700 1 -- 172.21.15.55:0/207331744 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf1004e120 con 0x7fdf1c108cf0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.805+0000 7fdf1bfff700 1 --2- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fdf0c039560 0x7fdf0c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.805+0000 7fdf1bfff700 1 --2- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fdf0c039560 0x7fdf0c03ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006fd0 tx=0x7fdf08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:39.810+0000 7fdf19ffb700 1 -- 172.21.15.55:0/207331744 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdf1001f030 con 0x7fdf1c108cf0 2024-04-26T21:43:41.223 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.034+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fdf1c07e110 con 0x7fdf1c108cf0 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.036+0000 7fdf19ffb700 1 -- 172.21.15.55:0/207331744 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fdf100105a0 con 0x7fdf1c108cf0 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fdf0c039560 msgr2=0x7fdf0c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 --2- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fdf0c039560 0x7fdf0c03ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006fd0 tx=0x7fdf08006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 msgr2=0x7fdf1c07eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 --2- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c07eb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf100116e0 tx=0x7fdf10011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 shutdown_connections 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 --2- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fdf0c039560 0x7fdf0c03ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 --2- 172.21.15.55:0/207331744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf1c108cf0 0x7fdf1c07eb90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 >> 172.21.15.55:0/207331744 conn(0x7fdf1c1042b0 msgr2=0x7fdf1c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 shutdown_connections 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:40.039+0000 7fdf22b6d700 1 -- 172.21.15.55:0/207331744 wait complete. 2024-04-26T21:43:41.224 INFO:teuthology.orchestra.run.smithi055.stdout:Enabling the dashboard module... 2024-04-26T21:43:42.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:42 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2707288572' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T21:43:42.187 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: ignoring --setuser ceph since I am not root 2024-04-26T21:43:42.187 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: ignoring --setgroup ceph since I am not root 2024-04-26T21:43:42.514 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:42.243+0000 7f816e97a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:43:42.514 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:42.434+0000 7f816e97a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:43:42.728 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.635+0000 7fb610d35700 1 Processor -- start 2024-04-26T21:43:42.728 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.635+0000 7fb610d35700 1 -- start start 2024-04-26T21:43:42.728 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.635+0000 7fb610d35700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c10bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.728 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.635+0000 7fb610d35700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c07ced0 con 0x7fb60c109760 2024-04-26T21:43:42.728 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.635+0000 7fb60a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c10bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.728 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.635+0000 7fb60a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c10bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53254/0 (socket says 172.21.15.55:53254) 2024-04-26T21:43:42.728 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.635+0000 7fb60a59c700 1 -- 172.21.15.55:0/1665971332 learned_addr learned my addr 172.21.15.55:0/1665971332 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:42.728 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.636+0000 7fb60a59c700 1 -- 172.21.15.55:0/1665971332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c10c090 con 0x7fb60c109760 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.636+0000 7fb60a59c700 1 --2- 172.21.15.55:0/1665971332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c10bb50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009e80 tx=0x7fb5fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795ab7287ff46fee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.636+0000 7fb60959a700 1 -- 172.21.15.55:0/1665971332 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5fc017070 con 0x7fb60c109760 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.636+0000 7fb60959a700 1 -- 172.21.15.55:0/1665971332 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5fc00b8c0 con 0x7fb60c109760 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.636+0000 7fb60959a700 1 -- 172.21.15.55:0/1665971332 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5fc004490 con 0x7fb60c109760 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.637+0000 7fb610d35700 1 -- 172.21.15.55:0/1665971332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 msgr2=0x7fb60c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.637+0000 7fb610d35700 1 --2- 172.21.15.55:0/1665971332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c10bb50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009e80 tx=0x7fb5fc00b330 comp rx=0 tx=0).stop 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.637+0000 7fb610d35700 1 -- 172.21.15.55:0/1665971332 shutdown_connections 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.637+0000 7fb610d35700 1 --2- 172.21.15.55:0/1665971332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c10bb50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.637+0000 7fb610d35700 1 -- 172.21.15.55:0/1665971332 >> 172.21.15.55:0/1665971332 conn(0x7fb60c103190 msgr2=0x7fb60c1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.637+0000 7fb610d35700 1 -- 172.21.15.55:0/1665971332 shutdown_connections 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.637+0000 7fb610d35700 1 -- 172.21.15.55:0/1665971332 wait complete. 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb610d35700 1 Processor -- start 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb610d35700 1 -- start start 2024-04-26T21:43:42.729 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb610d35700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb610d35700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c10d0a0 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb60a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb60a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53270/0 (socket says 172.21.15.55:53270) 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb60a59c700 1 -- 172.21.15.55:0/2707288572 learned_addr learned my addr 172.21.15.55:0/2707288572 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb60a59c700 1 -- 172.21.15.55:0/2707288572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c19d110 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.638+0000 7fb60a59c700 1 --2- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c19cbd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00b7b0 tx=0x7fb5fc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.639+0000 7fb6037fe700 1 -- 172.21.15.55:0/2707288572 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5fc020020 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.639+0000 7fb6037fe700 1 -- 172.21.15.55:0/2707288572 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5fc003c60 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.639+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c19d3a0 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.639+0000 7fb6037fe700 1 -- 172.21.15.55:0/2707288572 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5fc017040 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.639+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c1a0090 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.640+0000 7fb6037fe700 1 -- 172.21.15.55:0/2707288572 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb5fc023020 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.640+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60c064a80 con 0x7fb60c109760 2024-04-26T21:43:42.730 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.640+0000 7fb6037fe700 1 --2- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb5f8039600 0x7fb5f803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.640+0000 7fb6037fe700 1 -- 172.21.15.55:0/2707288572 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb5fc00f050 con 0x7fb60c109760 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.640+0000 7fb609d9b700 1 --2- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb5f8039600 0x7fb5f803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.641+0000 7fb609d9b700 1 --2- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb5f8039600 0x7fb5f803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4006fd0 tx=0x7fb5f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.645+0000 7fb6037fe700 1 -- 172.21.15.55:0/2707288572 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb5fc02a020 con 0x7fb60c109760 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:41.918+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fb60c1a0660 con 0x7fb60c109760 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.055+0000 7fb6037fe700 1 -- 172.21.15.55:0/2707288572 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fb5fc02b260 con 0x7fb60c109760 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.057+0000 7fb6037fe700 1 -- 172.21.15.55:0/2707288572 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fb5fc018bb0 con 0x7fb60c109760 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb5f8039600 msgr2=0x7fb5f803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 --2- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb5f8039600 0x7fb5f803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4006fd0 tx=0x7fb5f4006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 msgr2=0x7fb60c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 --2- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c19cbd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00b7b0 tx=0x7fb5fc00bd50 comp rx=0 tx=0).stop 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 shutdown_connections 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 --2- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7fb5f8039600 0x7fb5f803bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.731 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 --2- 172.21.15.55:0/2707288572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb60c109760 0x7fb60c19cbd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:42.732 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 >> 172.21.15.55:0/2707288572 conn(0x7fb60c103190 msgr2=0x7fb60c103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:42.732 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.058+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 shutdown_connections 2024-04-26T21:43:42.732 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:42.059+0000 7fb610d35700 1 -- 172.21.15.55:0/2707288572 wait complete. 2024-04-26T21:43:42.848 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:42.540+0000 7f816e97a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:43:43.139 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:43 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2707288572' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T21:43:43.139 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:43 smithi055 ceph-mon[29221]: mgrmap e9: x(active, since 20s) 2024-04-26T21:43:43.139 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:42.868+0000 7f816e97a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:43:43.139 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:42.957+0000 7f816e97a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.176+0000 7f01c53b7700 1 Processor -- start 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.177+0000 7f01c53b7700 1 -- start start 2024-04-26T21:43:44.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.177+0000 7f01c53b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c0107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.177+0000 7f01c53b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f01c007ce50 con 0x7f01c01075e0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.177+0000 7f01beffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c0107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.177+0000 7f01beffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c0107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53306/0 (socket says 172.21.15.55:53306) 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.177+0000 7f01beffd700 1 -- 172.21.15.55:0/1610506324 learned_addr learned my addr 172.21.15.55:0/1610506324 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.177+0000 7f01beffd700 1 -- 172.21.15.55:0/1610506324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c0107f40 con 0x7f01c01075e0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.178+0000 7f01beffd700 1 --2- 172.21.15.55:0/1610506324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c0107a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01b00098a0 tx=0x7f01b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d280d6bac73edd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.178+0000 7f01bdffb700 1 -- 172.21.15.55:0/1610506324 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b0004160 con 0x7f01c01075e0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.178+0000 7f01bdffb700 1 -- 172.21.15.55:0/1610506324 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b0004300 con 0x7f01c01075e0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.178+0000 7f01bdffb700 1 -- 172.21.15.55:0/1610506324 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b0011a20 con 0x7f01c01075e0 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.178+0000 7f01c53b7700 1 -- 172.21.15.55:0/1610506324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 msgr2=0x7f01c0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.178+0000 7f01c53b7700 1 --2- 172.21.15.55:0/1610506324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c0107a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f01b00098a0 tx=0x7f01b0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.179+0000 7f01c53b7700 1 -- 172.21.15.55:0/1610506324 shutdown_connections 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.179+0000 7f01c53b7700 1 --2- 172.21.15.55:0/1610506324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c0107a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.179+0000 7f01c53b7700 1 -- 172.21.15.55:0/1610506324 >> 172.21.15.55:0/1610506324 conn(0x7f01c01031e0 msgr2=0x7f01c0105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.179+0000 7f01c53b7700 1 -- 172.21.15.55:0/1610506324 shutdown_connections 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.179+0000 7f01c53b7700 1 -- 172.21.15.55:0/1610506324 wait complete. 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.179+0000 7f01c53b7700 1 Processor -- start 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.180+0000 7f01c53b7700 1 -- start start 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.180+0000 7f01c53b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.180+0000 7f01c53b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0108fb0 con 0x7f01c01075e0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.180+0000 7f01beffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.180+0000 7f01beffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53320/0 (socket says 172.21.15.55:53320) 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.180+0000 7f01beffd700 1 -- 172.21.15.55:0/188019178 learned_addr learned my addr 172.21.15.55:0/188019178 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.180+0000 7f01beffd700 1 -- 172.21.15.55:0/188019178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c019f2d0 con 0x7f01c01075e0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.180+0000 7f01beffd700 1 --2- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c019ed90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f01b00116e0 tx=0x7f01b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.181+0000 7f01a7fff700 1 -- 172.21.15.55:0/188019178 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b0003a60 con 0x7f01c01075e0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.181+0000 7f01a7fff700 1 -- 172.21.15.55:0/188019178 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b001b8f0 con 0x7f01c01075e0 2024-04-26T21:43:44.071 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.181+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c019f560 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.181+0000 7f01a7fff700 1 -- 172.21.15.55:0/188019178 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01b0019860 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.181+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c01a2210 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.182+0000 7f01a7fff700 1 -- 172.21.15.55:0/188019178 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f01b0019b10 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.182+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c0064a80 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.182+0000 7f01a7fff700 1 --2- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f01a8039610 0x7f01a803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.182+0000 7f01a7fff700 1 -- 172.21.15.55:0/188019178 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f01b004dd20 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.182+0000 7f01be7fc700 1 -- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f01a8039610 msgr2=0x7f01a803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/2562126894 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.182+0000 7f01be7fc700 1 --2- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f01a8039610 0x7f01a803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.187+0000 7f01a7fff700 1 -- 172.21.15.55:0/188019178 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f01b001f030 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.383+0000 7f01be7fc700 1 -- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f01a8039610 msgr2=0x7f01a803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/2562126894 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.383+0000 7f01be7fc700 1 --2- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f01a8039610 0x7f01a803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.497+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f01c00008d0 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.498+0000 7f01a7fff700 1 -- 172.21.15.55:0/188019178 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f01b0029260 con 0x7f01c01075e0 2024-04-26T21:43:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f01a8039610 msgr2=0x7f01a803bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 --2- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f01a8039610 0x7f01a803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 msgr2=0x7f01c019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 --2- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c019ed90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f01b00116e0 tx=0x7f01b0011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 shutdown_connections 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 --2- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f01a8039610 0x7f01a803bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 --2- 172.21.15.55:0/188019178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c01075e0 0x7f01c019ed90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 >> 172.21.15.55:0/188019178 conn(0x7f01c01031e0 msgr2=0x7f01c0195a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 shutdown_connections 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:43.500+0000 7f01c53b7700 1 -- 172.21.15.55:0/188019178 wait complete. 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for the mgr to restart... 2024-04-26T21:43:44.073 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mgr epoch 9... 2024-04-26T21:43:44.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:44 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/188019178' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T21:43:44.437 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:44 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:44.267+0000 7f816e97a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:43:44.823 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:44 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:44.447+0000 7f816e97a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:43:44.823 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:44 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:44.629+0000 7f816e97a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:43:45.186 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:44 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:44.931+0000 7f816e97a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:43:45.187 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:45 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:45.028+0000 7f816e97a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:43:45.686 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:45 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:45.240+0000 7f816e97a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:43:45.687 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:45 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:45.440+0000 7f816e97a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:43:46.436 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:46 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:46.069+0000 7f816e97a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:43:46.823 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:46 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:46.509+0000 7f816e97a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:43:46.823 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:46 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:46.611+0000 7f816e97a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:43:46.823 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:46 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:46.822+0000 7f816e97a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:43:47.626 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:47.367+0000 7f816e97a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:43:47.626 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:47.464+0000 7f816e97a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:43:47.936 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:47.720+0000 7f816e97a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:43:47.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:47.859+0000 7f816e97a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:43:48.392 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:47.959+0000 7f816e97a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:43:48.392 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:48.079+0000 7f816e97a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:43:48.392 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:48.392+0000 7f816e97a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:43:49.303 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:49.023+0000 7f816e97a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:43:49.303 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:43:49.249+0000 7f816e97a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:43:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: Active manager daemon x restarted 2024-04-26T21:43:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: Activating manager daemon x 2024-04-26T21:43:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T21:43:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: mgrmap e10: x(active, starting, since 0.00848296s) 2024-04-26T21:43:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:43:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T21:43:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T21:43:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T21:43:49.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T21:43:49.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: Manager daemon x is now available 2024-04-26T21:43:49.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:50.592 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T21:43:50.592 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T21:43:50.592 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:50 smithi055 ceph-mon[29221]: [26/Apr/2024:21:43:49] ENGINE Bus STARTING 2024-04-26T21:43:50.592 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:50 smithi055 ceph-mon[29221]: [26/Apr/2024:21:43:49] ENGINE Serving on https://172.21.15.55:7150 2024-04-26T21:43:50.592 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:50 smithi055 ceph-mon[29221]: [26/Apr/2024:21:43:49] ENGINE Serving on http://172.21.15.55:8765 2024-04-26T21:43:50.592 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:50 smithi055 ceph-mon[29221]: [26/Apr/2024:21:43:49] ENGINE Bus STARTED 2024-04-26T21:43:50.592 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:50 smithi055 ceph-mon[29221]: mgrmap e11: x(active, since 1.00646s) 2024-04-26T21:43:50.861 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-04-26T21:43:50.861 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T21:43:50.861 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T21:43:50.861 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-04-26T21:43:50.861 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.616+0000 7f48bb0b7700 1 Processor -- start 2024-04-26T21:43:50.861 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.616+0000 7f48bb0b7700 1 -- start start 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.616+0000 7f48bb0b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b41079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.616+0000 7f48bb0b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48b407ce70 con 0x7f48b41075b0 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.616+0000 7f48b8e53700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b41079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.616+0000 7f48b8e53700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b41079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53326/0 (socket says 172.21.15.55:53326) 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.616+0000 7f48b8e53700 1 -- 172.21.15.55:0/3708091588 learned_addr learned my addr 172.21.15.55:0/3708091588 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.617+0000 7f48b8e53700 1 -- 172.21.15.55:0/3708091588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b4107f10 con 0x7f48b41075b0 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.617+0000 7f48b8e53700 1 --2- 172.21.15.55:0/3708091588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b41079d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f48a80098a0 tx=0x7f48a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1112896f6beca0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.617+0000 7f48b37fe700 1 -- 172.21.15.55:0/3708091588 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a8004160 con 0x7f48b41075b0 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.617+0000 7f48b37fe700 1 -- 172.21.15.55:0/3708091588 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48a8004300 con 0x7f48b41075b0 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.617+0000 7f48b37fe700 1 -- 172.21.15.55:0/3708091588 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a8011a20 con 0x7f48b41075b0 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.618+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3708091588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 msgr2=0x7f48b41079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:50.862 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.618+0000 7f48bb0b7700 1 --2- 172.21.15.55:0/3708091588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b41079d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f48a80098a0 tx=0x7f48a8009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.618+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3708091588 shutdown_connections 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.618+0000 7f48bb0b7700 1 --2- 172.21.15.55:0/3708091588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b41079d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.618+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3708091588 >> 172.21.15.55:0/3708091588 conn(0x7f48b4103190 msgr2=0x7f48b41055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.618+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3708091588 shutdown_connections 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.618+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3708091588 wait complete. 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.619+0000 7f48bb0b7700 1 Processor -- start 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.619+0000 7f48bb0b7700 1 -- start start 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.619+0000 7f48bb0b7700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.619+0000 7f48bb0b7700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48b4108f80 con 0x7f48b41075b0 2024-04-26T21:43:50.863 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.619+0000 7f48b8e53700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.619+0000 7f48b8e53700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53338/0 (socket says 172.21.15.55:53338) 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.619+0000 7f48b8e53700 1 -- 172.21.15.55:0/3892564168 learned_addr learned my addr 172.21.15.55:0/3892564168 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.620+0000 7f48b8e53700 1 -- 172.21.15.55:0/3892564168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b419f300 con 0x7f48b41075b0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.620+0000 7f48b8e53700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b419edc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48a8004660 tx=0x7f48a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.620+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a8003aa0 con 0x7f48b41075b0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.620+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48a801b8f0 con 0x7f48b41075b0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.620+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a8023900 con 0x7f48b41075b0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.620+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b419f590 con 0x7f48b41075b0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.620+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b41a2280 con 0x7f48b41075b0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.621+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f48a8023b70 con 0x7f48b41075b0 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.621+0000 7f48b1ffb700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 0x7f48a403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.621+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 --> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f48a403c230 con 0x7f48a4039660 2024-04-26T21:43:50.864 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.621+0000 7f48b3fff700 1 -- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 msgr2=0x7f48a403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/2562126894 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.621+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f48a804dfa0 con 0x7f48b41075b0 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.621+0000 7f48b3fff700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 0x7f48a403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.822+0000 7f48b3fff700 1 -- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 msgr2=0x7f48a403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/2562126894 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:44.822+0000 7f48b3fff700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 0x7f48a403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:45.223+0000 7f48b3fff700 1 -- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 msgr2=0x7f48a403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/2562126894 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:45.223+0000 7f48b3fff700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 0x7f48a403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:46.024+0000 7f48b3fff700 1 -- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 msgr2=0x7f48a403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/2562126894 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:46.024+0000 7f48b3fff700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 0x7f48a403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:47.625+0000 7f48b3fff700 1 -- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 msgr2=0x7f48a403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.55:6800/2562126894 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:47.625+0000 7f48b3fff700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 0x7f48a403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:49.258+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f48a802c3f0 con 0x7f48b41075b0 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:49.258+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 msgr2=0x7f48a403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:43:50.865 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:49.258+0000 7f48b1ffb700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 0x7f48a403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.255+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f48a804ff80 con 0x7f48b41075b0 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.255+0000 7f48b1ffb700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48a403cff0 0x7f48a403f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.255+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f48a800af30 con 0x7f48a403cff0 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.256+0000 7f48b3fff700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48a403cff0 0x7f48a403f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.257+0000 7f48b3fff700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48a403cff0 0x7f48a403f3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f48a0003a10 tx=0x7f48a0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.257+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f48a800af30 con 0x7f48a403cff0 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f48b406a510 con 0x7f48a403cff0 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48b1ffb700 1 -- 172.21.15.55:0/3892564168 <== mgr.14152 v2:172.21.15.55:6800/3284422558 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f48b406a510 con 0x7f48a403cff0 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48a403cff0 msgr2=0x7f48a403f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48a403cff0 0x7f48a403f3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f48a0003a10 tx=0x7f48a0009270 comp rx=0 tx=0).stop 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 msgr2=0x7f48b419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b419edc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f48a8004660 tx=0x7f48a8011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:50.866 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 shutdown_connections 2024-04-26T21:43:50.867 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48a403cff0 0x7f48a403f3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:50.867 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48b41075b0 0x7f48b419edc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:50.867 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 --2- 172.21.15.55:0/3892564168 >> [v2:172.21.15.55:6800/2562126894,v1:172.21.15.55:6801/2562126894] conn(0x7f48a4039660 0x7f48a403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:50.867 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.266+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 >> 172.21.15.55:0/3892564168 conn(0x7f48b4103190 msgr2=0x7f48b4195a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:50.867 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.267+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 shutdown_connections 2024-04-26T21:43:50.867 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:50.267+0000 7f48bb0b7700 1 -- 172.21.15.55:0/3892564168 wait complete. 2024-04-26T21:43:50.867 INFO:teuthology.orchestra.run.smithi055.stdout:mgr epoch 9 is available 2024-04-26T21:43:50.867 INFO:teuthology.orchestra.run.smithi055.stdout:Generating a dashboard self-signed certificate... 2024-04-26T21:43:51.395 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:51 smithi055 ceph-mon[29221]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T21:43:51.395 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:51 smithi055 ceph-mon[29221]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T21:43:52.341 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T21:43:52.342 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.425+0000 7fa0c4f13700 1 Processor -- start 2024-04-26T21:43:52.342 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.425+0000 7fa0c4f13700 1 -- start start 2024-04-26T21:43:52.342 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.425+0000 7fa0c4f13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c010bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:52.342 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.425+0000 7fa0c4f13700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c007ce70 con 0x7fa0c0109810 2024-04-26T21:43:52.342 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.425+0000 7fa0be59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c010bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:52.342 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.425+0000 7fa0be59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c010bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52770/0 (socket says 172.21.15.55:52770) 2024-04-26T21:43:52.342 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.425+0000 7fa0be59c700 1 -- 172.21.15.55:0/1738140483 learned_addr learned my addr 172.21.15.55:0/1738140483 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:52.342 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.426+0000 7fa0be59c700 1 -- 172.21.15.55:0/1738140483 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c010c140 con 0x7fa0c0109810 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.426+0000 7fa0be59c700 1 --2- 172.21.15.55:0/1738140483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c010bc00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098a0 tx=0x7fa0b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbd096914798b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.426+0000 7fa0bd59a700 1 -- 172.21.15.55:0/1738140483 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0004160 con 0x7fa0c0109810 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.426+0000 7fa0bd59a700 1 -- 172.21.15.55:0/1738140483 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b0004300 con 0x7fa0c0109810 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.426+0000 7fa0bd59a700 1 -- 172.21.15.55:0/1738140483 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0011a20 con 0x7fa0c0109810 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.427+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1738140483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 msgr2=0x7fa0c010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.427+0000 7fa0c4f13700 1 --2- 172.21.15.55:0/1738140483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c010bc00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098a0 tx=0x7fa0b0009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.427+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1738140483 shutdown_connections 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.427+0000 7fa0c4f13700 1 --2- 172.21.15.55:0/1738140483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c010bc00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.427+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1738140483 >> 172.21.15.55:0/1738140483 conn(0x7fa0c0103220 msgr2=0x7fa0c0105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.427+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1738140483 shutdown_connections 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.427+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1738140483 wait complete. 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0c4f13700 1 Processor -- start 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0c4f13700 1 -- start start 2024-04-26T21:43:52.343 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0c4f13700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0c4f13700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010d0f0 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0be59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0be59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52780/0 (socket says 172.21.15.55:52780) 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0be59c700 1 -- 172.21.15.55:0/1931487731 learned_addr learned my addr 172.21.15.55:0/1931487731 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0be59c700 1 -- 172.21.15.55:0/1931487731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c019d080 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.428+0000 7fa0be59c700 1 --2- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c019cb40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0000c00 tx=0x7fa0b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.429+0000 7fa0b77fe700 1 -- 172.21.15.55:0/1931487731 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0003aa0 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.429+0000 7fa0b77fe700 1 -- 172.21.15.55:0/1931487731 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b001b8f0 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.429+0000 7fa0b77fe700 1 -- 172.21.15.55:0/1931487731 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b0023870 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.429+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c019d310 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.429+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c01a0000 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.430+0000 7fa0b77fe700 1 -- 172.21.15.55:0/1931487731 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa0b001b440 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.430+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c0064a80 con 0x7fa0c0109810 2024-04-26T21:43:52.344 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.430+0000 7fa0b77fe700 1 --2- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0ac039660 0x7fa0ac03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.430+0000 7fa0b77fe700 1 -- 172.21.15.55:0/1931487731 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa0b004dfa0 con 0x7fa0c0109810 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.430+0000 7fa0bdd9b700 1 --2- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0ac039660 0x7fa0ac03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.431+0000 7fa0bdd9b700 1 --2- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0ac039660 0x7fa0ac03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8006fd0 tx=0x7fa0a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.435+0000 7fa0b77fe700 1 -- 172.21.15.55:0/1931487731 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa0b001f030 con 0x7fa0c0109810 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.648+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fa0c0104330 con 0x7fa0ac039660 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.787+0000 7fa0b77fe700 1 -- 172.21.15.55:0/1931487731 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fa0c0104330 con 0x7fa0ac039660 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0ac039660 msgr2=0x7fa0ac03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 --2- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0ac039660 0x7fa0ac03bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8006fd0 tx=0x7fa0a8006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 msgr2=0x7fa0c019cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 --2- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c019cb40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0000c00 tx=0x7fa0b0011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 shutdown_connections 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 --2- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0ac039660 0x7fa0ac03bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 --2- 172.21.15.55:0/1931487731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0c0109810 0x7fa0c019cb40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:52.345 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 >> 172.21.15.55:0/1931487731 conn(0x7fa0c0103220 msgr2=0x7fa0c0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:52.346 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 shutdown_connections 2024-04-26T21:43:52.346 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:51.790+0000 7fa0c4f13700 1 -- 172.21.15.55:0/1931487731 wait complete. 2024-04-26T21:43:52.346 INFO:teuthology.orchestra.run.smithi055.stdout:Creating initial admin user... 2024-04-26T21:43:52.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:52 smithi055 ceph-mon[29221]: mgrmap e12: x(active, since 2s) 2024-04-26T21:43:52.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:52 smithi055 ceph-mon[29221]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:52.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:43:52.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:43:52.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:43:54.058 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:43:54.058 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:54 smithi055 ceph-mon[29221]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:43:54.058 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:43:54.066 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$mHemDYcXpemlCpnu3VxwEOeRB0uxFI7rz68iuuDIi3ly5gyz2N82W", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714167833, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T21:43:54.066 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.794+0000 7f7c65428700 1 Processor -- start 2024-04-26T21:43:54.066 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.794+0000 7f7c65428700 1 -- start start 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.794+0000 7f7c65428700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6010bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.794+0000 7f7c65428700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6007ced0 con 0x7f7c60109760 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.795+0000 7f7c5effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6010bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.795+0000 7f7c5effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6010bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52792/0 (socket says 172.21.15.55:52792) 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.795+0000 7f7c5effd700 1 -- 172.21.15.55:0/915889780 learned_addr learned my addr 172.21.15.55:0/915889780 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.795+0000 7f7c5effd700 1 -- 172.21.15.55:0/915889780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6010c090 con 0x7f7c60109760 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.795+0000 7f7c5effd700 1 --2- 172.21.15.55:0/915889780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6010bb50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c480098a0 tx=0x7f7c48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a54549683d0ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c5dffb700 1 -- 172.21.15.55:0/915889780 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48004160 con 0x7f7c60109760 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c5dffb700 1 -- 172.21.15.55:0/915889780 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c48004300 con 0x7f7c60109760 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c5dffb700 1 -- 172.21.15.55:0/915889780 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48011a20 con 0x7f7c60109760 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c65428700 1 -- 172.21.15.55:0/915889780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 msgr2=0x7f7c6010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c65428700 1 --2- 172.21.15.55:0/915889780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6010bb50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c480098a0 tx=0x7f7c48009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c65428700 1 -- 172.21.15.55:0/915889780 shutdown_connections 2024-04-26T21:43:54.067 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c65428700 1 --2- 172.21.15.55:0/915889780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6010bb50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c65428700 1 -- 172.21.15.55:0/915889780 >> 172.21.15.55:0/915889780 conn(0x7f7c60103190 msgr2=0x7f7c601055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c65428700 1 -- 172.21.15.55:0/915889780 shutdown_connections 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.796+0000 7f7c65428700 1 -- 172.21.15.55:0/915889780 wait complete. 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.797+0000 7f7c65428700 1 Processor -- start 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.797+0000 7f7c65428700 1 -- start start 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.797+0000 7f7c65428700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.797+0000 7f7c65428700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6010d0a0 con 0x7f7c60109760 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.797+0000 7f7c5effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c5effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52794/0 (socket says 172.21.15.55:52794) 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c5effd700 1 -- 172.21.15.55:0/1719763870 learned_addr learned my addr 172.21.15.55:0/1719763870 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c5effd700 1 -- 172.21.15.55:0/1719763870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6019d100 con 0x7f7c60109760 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c5effd700 1 --2- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6019cbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c480116e0 tx=0x7f7c48011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:54.068 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c57fff700 1 -- 172.21.15.55:0/1719763870 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c48003b20 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c57fff700 1 -- 172.21.15.55:0/1719763870 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4801b8f0 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6019d390 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c57fff700 1 -- 172.21.15.55:0/1719763870 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c480238c0 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.798+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c601a0080 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.799+0000 7f7c57fff700 1 -- 172.21.15.55:0/1719763870 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f7c48023b70 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.799+0000 7f7c57fff700 1 --2- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c4c0395b0 0x7f7c4c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.799+0000 7f7c57fff700 1 -- 172.21.15.55:0/1719763870 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7c4804e140 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.799+0000 7f7c5e7fc700 1 --2- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c4c0395b0 0x7f7c4c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.800+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c60064a80 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.800+0000 7f7c5e7fc700 1 --2- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c4c0395b0 0x7f7c4c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c50006fd0 tx=0x7f7c50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:52.805+0000 7f7c57fff700 1 -- 172.21.15.55:0/1719763870 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7c4801f030 con 0x7f7c60109760 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.055+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f7c601043b0 con 0x7f7c4c0395b0 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.303+0000 7f7c57fff700 1 -- 172.21.15.55:0/1719763870 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f7c601043b0 con 0x7f7c4c0395b0 2024-04-26T21:43:54.069 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c4c0395b0 msgr2=0x7f7c4c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 --2- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c4c0395b0 0x7f7c4c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7c50006fd0 tx=0x7f7c50006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 msgr2=0x7f7c6019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 --2- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6019cbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c480116e0 tx=0x7f7c48011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 shutdown_connections 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 --2- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c4c0395b0 0x7f7c4c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 --2- 172.21.15.55:0/1719763870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c60109760 0x7f7c6019cbc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 >> 172.21.15.55:0/1719763870 conn(0x7f7c60103190 msgr2=0x7f7c60103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 shutdown_connections 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:53.305+0000 7f7c65428700 1 -- 172.21.15.55:0/1719763870 wait complete. 2024-04-26T21:43:54.070 INFO:teuthology.orchestra.run.smithi055.stdout:Fetching dashboard port number... 2024-04-26T21:43:55.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1884892442' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.522+0000 7f987409c700 1 Processor -- start 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.523+0000 7f987409c700 1 -- start start 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.523+0000 7f987409c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c07d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.523+0000 7f987409c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f986c07d8d0 con 0x7f986c07eea0 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.523+0000 7f9871e38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c07d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.523+0000 7f9871e38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c07d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52810/0 (socket says 172.21.15.55:52810) 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.523+0000 7f9871e38700 1 -- 172.21.15.55:0/357040842 learned_addr learned my addr 172.21.15.55:0/357040842 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.523+0000 7f9871e38700 1 -- 172.21.15.55:0/357040842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c07da50 con 0x7f986c07eea0 2024-04-26T21:43:55.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.524+0000 7f9871e38700 1 --2- 172.21.15.55:0/357040842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c07d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f98680098a0 tx=0x7f9868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d525d0faf6e244ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.524+0000 7f9870e36700 1 -- 172.21.15.55:0/357040842 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9868004160 con 0x7f986c07eea0 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.524+0000 7f9870e36700 1 -- 172.21.15.55:0/357040842 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9868004300 con 0x7f986c07eea0 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.524+0000 7f9870e36700 1 -- 172.21.15.55:0/357040842 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9868011a20 con 0x7f986c07eea0 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 -- 172.21.15.55:0/357040842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 msgr2=0x7f986c07d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 --2- 172.21.15.55:0/357040842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c07d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f98680098a0 tx=0x7f9868009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 -- 172.21.15.55:0/357040842 shutdown_connections 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 --2- 172.21.15.55:0/357040842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c07d300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 -- 172.21.15.55:0/357040842 >> 172.21.15.55:0/357040842 conn(0x7f986c1042b0 msgr2=0x7f986c1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 -- 172.21.15.55:0/357040842 shutdown_connections 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 -- 172.21.15.55:0/357040842 wait complete. 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 Processor -- start 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.525+0000 7f987409c700 1 -- start start 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f987409c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.789 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f987409c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f986c10f310 con 0x7f986c07eea0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f9871e38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f9871e38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52822/0 (socket says 172.21.15.55:52822) 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f9871e38700 1 -- 172.21.15.55:0/1884892442 learned_addr learned my addr 172.21.15.55:0/1884892442 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f9871e38700 1 -- 172.21.15.55:0/1884892442 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c19f290 con 0x7f986c07eea0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f9871e38700 1 --2- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c19ed50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9868000c00 tx=0x7f9868011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f9862ffd700 1 -- 172.21.15.55:0/1884892442 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9868003aa0 con 0x7f986c07eea0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.526+0000 7f9862ffd700 1 -- 172.21.15.55:0/1884892442 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f986801b8f0 con 0x7f986c07eea0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.527+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986c19f520 con 0x7f986c07eea0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.527+0000 7f9862ffd700 1 -- 172.21.15.55:0/1884892442 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9868023870 con 0x7f986c07eea0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.527+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986c1a2210 con 0x7f986c07eea0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.527+0000 7f9862ffd700 1 -- 172.21.15.55:0/1884892442 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f9868023b70 con 0x7f986c07eea0 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.528+0000 7f9862ffd700 1 --2- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9858039600 0x7f985803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:55.790 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.528+0000 7f9862ffd700 1 -- 172.21.15.55:0/1884892442 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f986804ef10 con 0x7f986c07eea0 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.528+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986c064a80 con 0x7f986c07eea0 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.528+0000 7f9871637700 1 --2- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9858039600 0x7f985803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.528+0000 7f9871637700 1 --2- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9858039600 0x7f985803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f985c006fd0 tx=0x7f985c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.534+0000 7f9862ffd700 1 -- 172.21.15.55:0/1884892442 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f986801f030 con 0x7f986c07eea0 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.745+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f986c1a27e0 con 0x7f986c07eea0 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.746+0000 7f9862ffd700 1 -- 172.21.15.55:0/1884892442 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9868018e30 con 0x7f986c07eea0 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9858039600 msgr2=0x7f985803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 --2- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9858039600 0x7f985803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f985c006fd0 tx=0x7f985c006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 msgr2=0x7f986c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 --2- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c19ed50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9868000c00 tx=0x7f9868011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 shutdown_connections 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 --2- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9858039600 0x7f985803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.791 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 --2- 172.21.15.55:0/1884892442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f986c07eea0 0x7f986c19ed50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:55.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 >> 172.21.15.55:0/1884892442 conn(0x7f986c1042b0 msgr2=0x7f986c104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:55.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 shutdown_connections 2024-04-26T21:43:55.792 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:54.748+0000 7f987409c700 1 -- 172.21.15.55:0/1884892442 wait complete. 2024-04-26T21:43:55.797 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T21:43:55.797 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout disabled 2024-04-26T21:43:55.805 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T21:43:55.805 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-04-26T21:43:55.805 INFO:teuthology.orchestra.run.smithi055.stdout:firewalld.service is not enabled 2024-04-26T21:43:55.805 INFO:teuthology.orchestra.run.smithi055.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T21:43:55.807 INFO:teuthology.orchestra.run.smithi055.stdout:Ceph Dashboard is now available at: 2024-04-26T21:43:55.807 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:55.807 INFO:teuthology.orchestra.run.smithi055.stdout: URL: https://smithi055:8443/ 2024-04-26T21:43:55.807 INFO:teuthology.orchestra.run.smithi055.stdout: User: admin 2024-04-26T21:43:55.807 INFO:teuthology.orchestra.run.smithi055.stdout: Password: q47fsmqgjw 2024-04-26T21:43:55.807 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:55.808 INFO:teuthology.orchestra.run.smithi055.stdout:Saving cluster configuration to /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config directory 2024-04-26T21:43:55.808 INFO:teuthology.orchestra.run.smithi055.stdout:Enabling autotune for osd_memory_target 2024-04-26T21:43:57.264 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:57 smithi055 ceph-mon[29221]: mgrmap e13: x(active, since 6s) 2024-04-26T21:43:57.272 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.395+0000 7f5d888ae700 1 Processor -- start 2024-04-26T21:43:57.272 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.395+0000 7f5d888ae700 1 -- start start 2024-04-26T21:43:57.272 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.396+0000 7f5d888ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8407d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.272 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.396+0000 7f5d888ae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8407d910 con 0x7f5d8407eee0 2024-04-26T21:43:57.272 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.396+0000 7f5d8259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8407d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.272 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.396+0000 7f5d8259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8407d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52830/0 (socket says 172.21.15.55:52830) 2024-04-26T21:43:57.272 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.396+0000 7f5d8259c700 1 -- 172.21.15.55:0/3591596855 learned_addr learned my addr 172.21.15.55:0/3591596855 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:57.272 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.396+0000 7f5d8259c700 1 -- 172.21.15.55:0/3591596855 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8407da90 con 0x7f5d8407eee0 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.396+0000 7f5d8259c700 1 --2- 172.21.15.55:0/3591596855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8407d340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d740098a0 tx=0x7f5d74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33aa9dc73e447fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d8159a700 1 -- 172.21.15.55:0/3591596855 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d74004160 con 0x7f5d8407eee0 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d8159a700 1 -- 172.21.15.55:0/3591596855 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d74004300 con 0x7f5d8407eee0 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d8159a700 1 -- 172.21.15.55:0/3591596855 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d74011a20 con 0x7f5d8407eee0 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d888ae700 1 -- 172.21.15.55:0/3591596855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 msgr2=0x7f5d8407d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d888ae700 1 --2- 172.21.15.55:0/3591596855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8407d340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d740098a0 tx=0x7f5d74009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d888ae700 1 -- 172.21.15.55:0/3591596855 shutdown_connections 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d888ae700 1 --2- 172.21.15.55:0/3591596855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8407d340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d888ae700 1 -- 172.21.15.55:0/3591596855 >> 172.21.15.55:0/3591596855 conn(0x7f5d841042a0 msgr2=0x7f5d841066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d888ae700 1 -- 172.21.15.55:0/3591596855 shutdown_connections 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.397+0000 7f5d888ae700 1 -- 172.21.15.55:0/3591596855 wait complete. 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.398+0000 7f5d888ae700 1 Processor -- start 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.398+0000 7f5d888ae700 1 -- start start 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.398+0000 7f5d888ae700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.273 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.398+0000 7f5d888ae700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410f2c0 con 0x7f5d8407eee0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d8259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d8259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52840/0 (socket says 172.21.15.55:52840) 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d8259c700 1 -- 172.21.15.55:0/3709258055 learned_addr learned my addr 172.21.15.55:0/3709258055 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d8259c700 1 -- 172.21.15.55:0/3709258055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8419f2e0 con 0x7f5d8407eee0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d8259c700 1 --2- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8419eda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009520 tx=0x7f5d74011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d737fe700 1 -- 172.21.15.55:0/3709258055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d740039e0 con 0x7f5d8407eee0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d737fe700 1 -- 172.21.15.55:0/3709258055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d7401b8f0 con 0x7f5d8407eee0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8419f570 con 0x7f5d8407eee0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d737fe700 1 -- 172.21.15.55:0/3709258055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d74019770 con 0x7f5d8407eee0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.399+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a2220 con 0x7f5d8407eee0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.400+0000 7f5d737fe700 1 -- 172.21.15.55:0/3709258055 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5d74019a90 con 0x7f5d8407eee0 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.400+0000 7f5d737fe700 1 --2- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5d6c0396a0 0x7f5d6c03bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:57.274 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.400+0000 7f5d737fe700 1 -- 172.21.15.55:0/3709258055 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5d7404e1e0 con 0x7f5d8407eee0 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.400+0000 7f5d81d9b700 1 --2- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5d6c0396a0 0x7f5d6c03bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.400+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d84064a80 con 0x7f5d8407eee0 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.401+0000 7f5d81d9b700 1 --2- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5d6c0396a0 0x7f5d6c03bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006fd0 tx=0x7f5d78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.406+0000 7f5d737fe700 1 -- 172.21.15.55:0/3709258055 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5d7401f030 con 0x7f5d8407eee0 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.615+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f5d841a27d0 con 0x7f5d8407eee0 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.615+0000 7f5d737fe700 1 -- 172.21.15.55:0/3709258055 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f5d74019e20 con 0x7f5d8407eee0 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5d6c0396a0 msgr2=0x7f5d6c03bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 --2- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5d6c0396a0 0x7f5d6c03bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006fd0 tx=0x7f5d78006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 msgr2=0x7f5d8419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 --2- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8419eda0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009520 tx=0x7f5d74011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 shutdown_connections 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 --2- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5d6c0396a0 0x7f5d6c03bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 --2- 172.21.15.55:0/3709258055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5d8407eee0 0x7f5d8419eda0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 >> 172.21.15.55:0/3709258055 conn(0x7f5d841042a0 msgr2=0x7f5d84104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:57.275 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 shutdown_connections 2024-04-26T21:43:57.276 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:56.618+0000 7f5d888ae700 1 -- 172.21.15.55:0/3709258055 wait complete. 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.691+0000 7f9e3982e700 1 Processor -- start 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.691+0000 7f9e3982e700 1 -- start start 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.691+0000 7f9e3982e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.691+0000 7f9e3982e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9e341079a0 con 0x7f9e34106fb0 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.691+0000 7f9e32ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.691+0000 7f9e32ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52842/0 (socket says 172.21.15.55:52842) 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.691+0000 7f9e32ffd700 1 -- 172.21.15.55:0/365368275 learned_addr learned my addr 172.21.15.55:0/365368275 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.692+0000 7f9e32ffd700 1 -- 172.21.15.55:0/365368275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e34109f40 con 0x7f9e34106fb0 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.692+0000 7f9e32ffd700 1 --2- 172.21.15.55:0/365368275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341073d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e240098a0 tx=0x7f9e24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de97dad00cfb4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.692+0000 7f9e31ffb700 1 -- 172.21.15.55:0/365368275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e24004160 con 0x7f9e34106fb0 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.692+0000 7f9e31ffb700 1 -- 172.21.15.55:0/365368275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e24004300 con 0x7f9e34106fb0 2024-04-26T21:43:58.647 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.692+0000 7f9e31ffb700 1 -- 172.21.15.55:0/365368275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e24011a20 con 0x7f9e34106fb0 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.693+0000 7f9e3982e700 1 -- 172.21.15.55:0/365368275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 msgr2=0x7f9e341073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.693+0000 7f9e3982e700 1 --2- 172.21.15.55:0/365368275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341073d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e240098a0 tx=0x7f9e24009bb0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.693+0000 7f9e3982e700 1 -- 172.21.15.55:0/365368275 shutdown_connections 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.693+0000 7f9e3982e700 1 --2- 172.21.15.55:0/365368275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341073d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.693+0000 7f9e3982e700 1 -- 172.21.15.55:0/365368275 >> 172.21.15.55:0/365368275 conn(0x7f9e34102940 msgr2=0x7f9e34104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.693+0000 7f9e3982e700 1 -- 172.21.15.55:0/365368275 shutdown_connections 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.693+0000 7f9e3982e700 1 -- 172.21.15.55:0/365368275 wait complete. 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.693+0000 7f9e3982e700 1 Processor -- start 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e3982e700 1 -- start start 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e3982e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e3982e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3410d2f0 con 0x7f9e34106fb0 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e32ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e32ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341097c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52844/0 (socket says 172.21.15.55:52844) 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e32ffd700 1 -- 172.21.15.55:0/705124786 learned_addr learned my addr 172.21.15.55:0/705124786 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e32ffd700 1 -- 172.21.15.55:0/705124786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e34109d00 con 0x7f9e34106fb0 2024-04-26T21:43:58.648 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e32ffd700 1 --2- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341097c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e240116e0 tx=0x7f9e24011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e3882c700 1 -- 172.21.15.55:0/705124786 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e240039e0 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e3882c700 1 -- 172.21.15.55:0/705124786 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e2401b8f0 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e3882c700 1 -- 172.21.15.55:0/705124786 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e24023760 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.694+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e34107ef0 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.695+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e34108430 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.695+0000 7f9e3882c700 1 -- 172.21.15.55:0/705124786 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9e2401b440 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.695+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e34064a80 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.696+0000 7f9e3882c700 1 --2- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e1c039650 0x7f9e1c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.696+0000 7f9e3882c700 1 -- 172.21.15.55:0/705124786 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e2404e0e0 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.696+0000 7f9e327fc700 1 --2- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e1c039650 0x7f9e1c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.696+0000 7f9e327fc700 1 --2- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e1c039650 0x7f9e1c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e28006fd0 tx=0x7f9e28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:57.701+0000 7f9e3882c700 1 -- 172.21.15.55:0/705124786 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9e2401f030 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.023+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f9e341086b0 con 0x7f9e34106fb0 2024-04-26T21:43:58.649 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.025+0000 7f9e3882c700 1 -- 172.21.15.55:0/705124786 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f9e2402c3f0 con 0x7f9e34106fb0 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e1c039650 msgr2=0x7f9e1c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 --2- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e1c039650 0x7f9e1c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9e28006fd0 tx=0x7f9e28006e40 comp rx=0 tx=0).stop 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 msgr2=0x7f9e341097c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 --2- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341097c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e240116e0 tx=0x7f9e24011aa0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 shutdown_connections 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 --2- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e1c039650 0x7f9e1c03bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 --2- 172.21.15.55:0/705124786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e34106fb0 0x7f9e341097c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 >> 172.21.15.55:0/705124786 conn(0x7f9e34102940 msgr2=0x7f9e341032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 shutdown_connections 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr 2024-04-26T21:43:58.028+0000 7f9e3982e700 1 -- 172.21.15.55:0/705124786 wait complete. 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T21:43:58.650 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: ceph telemetry on 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout:For more information see: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:43:58.651 INFO:teuthology.orchestra.run.smithi055.stdout:Bootstrap complete. 2024-04-26T21:43:58.693 INFO:tasks.cephadm:Fetching config... 2024-04-26T21:43:58.693 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:43:58.693 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T21:43:58.711 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T21:43:58.711 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:43:58.711 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T21:43:58.767 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T21:43:58.768 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:43:58.768 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T21:43:58.832 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T21:43:58.832 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:43:58.833 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T21:43:58.888 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T21:43:58.888 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD0dx9JkRXk+yKQjZUFBN6ZdZHpT7lRv6tKyV53lc0IzWJIbkq04OASRwUFE3Uo6r4zkNdU0lugxSdjGFzDbqKQczIYs9g/zYMklFQVnjM+3MSNzrUf6hME/HCZciyG7rTgFVvxH9w7zqWY6wnsQkzJbGxhiNqGLlCXP7xKuPeVuSvqBHpDJP6H6uaJ1PjJWUuOaNDTyvlH6KpLyjtsb6kQdUAczawMZCNK0jwqA9oDWZfgGqcPZ743OqvYK4UN4PCSA4Vs4hQr8gl8p3uHv0YxCs7kkC+vMuwnzh9byNyU/pSA+nNVhmjY038WQv4RdjH1fkqoF6MzmFRl0GtPSYSVYIdF7fOU/cFoE6ZatpRux0M5o+Q435IyGiosizWT6zlsl9bvdb7UjNd7LmiZhqRhp4uyEs+7Vy+BgB3Yz+UXcxbBa6mdNg+7ZgkWZF14AQiP6YxMeyedxMk8uiOntJ4hHZtqJNYrBbQkPScaXHlxynFuIrq6gkEjoo35erHc6ac= ceph-c84f1140-0415-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:43:58.962 INFO:teuthology.orchestra.run.smithi055.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD0dx9JkRXk+yKQjZUFBN6ZdZHpT7lRv6tKyV53lc0IzWJIbkq04OASRwUFE3Uo6r4zkNdU0lugxSdjGFzDbqKQczIYs9g/zYMklFQVnjM+3MSNzrUf6hME/HCZciyG7rTgFVvxH9w7zqWY6wnsQkzJbGxhiNqGLlCXP7xKuPeVuSvqBHpDJP6H6uaJ1PjJWUuOaNDTyvlH6KpLyjtsb6kQdUAczawMZCNK0jwqA9oDWZfgGqcPZ743OqvYK4UN4PCSA4Vs4hQr8gl8p3uHv0YxCs7kkC+vMuwnzh9byNyU/pSA+nNVhmjY038WQv4RdjH1fkqoF6MzmFRl0GtPSYSVYIdF7fOU/cFoE6ZatpRux0M5o+Q435IyGiosizWT6zlsl9bvdb7UjNd7LmiZhqRhp4uyEs+7Vy+BgB3Yz+UXcxbBa6mdNg+7ZgkWZF14AQiP6YxMeyedxMk8uiOntJ4hHZtqJNYrBbQkPScaXHlxynFuIrq6gkEjoo35erHc6ac= ceph-c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:58.974 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD0dx9JkRXk+yKQjZUFBN6ZdZHpT7lRv6tKyV53lc0IzWJIbkq04OASRwUFE3Uo6r4zkNdU0lugxSdjGFzDbqKQczIYs9g/zYMklFQVnjM+3MSNzrUf6hME/HCZciyG7rTgFVvxH9w7zqWY6wnsQkzJbGxhiNqGLlCXP7xKuPeVuSvqBHpDJP6H6uaJ1PjJWUuOaNDTyvlH6KpLyjtsb6kQdUAczawMZCNK0jwqA9oDWZfgGqcPZ743OqvYK4UN4PCSA4Vs4hQr8gl8p3uHv0YxCs7kkC+vMuwnzh9byNyU/pSA+nNVhmjY038WQv4RdjH1fkqoF6MzmFRl0GtPSYSVYIdF7fOU/cFoE6ZatpRux0M5o+Q435IyGiosizWT6zlsl9bvdb7UjNd7LmiZhqRhp4uyEs+7Vy+BgB3Yz+UXcxbBa6mdNg+7ZgkWZF14AQiP6YxMeyedxMk8uiOntJ4hHZtqJNYrBbQkPScaXHlxynFuIrq6gkEjoo35erHc6ac= ceph-c84f1140-0415-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:43:59.016 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD0dx9JkRXk+yKQjZUFBN6ZdZHpT7lRv6tKyV53lc0IzWJIbkq04OASRwUFE3Uo6r4zkNdU0lugxSdjGFzDbqKQczIYs9g/zYMklFQVnjM+3MSNzrUf6hME/HCZciyG7rTgFVvxH9w7zqWY6wnsQkzJbGxhiNqGLlCXP7xKuPeVuSvqBHpDJP6H6uaJ1PjJWUuOaNDTyvlH6KpLyjtsb6kQdUAczawMZCNK0jwqA9oDWZfgGqcPZ743OqvYK4UN4PCSA4Vs4hQr8gl8p3uHv0YxCs7kkC+vMuwnzh9byNyU/pSA+nNVhmjY038WQv4RdjH1fkqoF6MzmFRl0GtPSYSVYIdF7fOU/cFoE6ZatpRux0M5o+Q435IyGiosizWT6zlsl9bvdb7UjNd7LmiZhqRhp4uyEs+7Vy+BgB3Yz+UXcxbBa6mdNg+7ZgkWZF14AQiP6YxMeyedxMk8uiOntJ4hHZtqJNYrBbQkPScaXHlxynFuIrq6gkEjoo35erHc6ac= ceph-c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:59.028 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD0dx9JkRXk+yKQjZUFBN6ZdZHpT7lRv6tKyV53lc0IzWJIbkq04OASRwUFE3Uo6r4zkNdU0lugxSdjGFzDbqKQczIYs9g/zYMklFQVnjM+3MSNzrUf6hME/HCZciyG7rTgFVvxH9w7zqWY6wnsQkzJbGxhiNqGLlCXP7xKuPeVuSvqBHpDJP6H6uaJ1PjJWUuOaNDTyvlH6KpLyjtsb6kQdUAczawMZCNK0jwqA9oDWZfgGqcPZ743OqvYK4UN4PCSA4Vs4hQr8gl8p3uHv0YxCs7kkC+vMuwnzh9byNyU/pSA+nNVhmjY038WQv4RdjH1fkqoF6MzmFRl0GtPSYSVYIdF7fOU/cFoE6ZatpRux0M5o+Q435IyGiosizWT6zlsl9bvdb7UjNd7LmiZhqRhp4uyEs+7Vy+BgB3Yz+UXcxbBa6mdNg+7ZgkWZF14AQiP6YxMeyedxMk8uiOntJ4hHZtqJNYrBbQkPScaXHlxynFuIrq6gkEjoo35erHc6ac= ceph-c84f1140-0415-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T21:43:59.067 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD0dx9JkRXk+yKQjZUFBN6ZdZHpT7lRv6tKyV53lc0IzWJIbkq04OASRwUFE3Uo6r4zkNdU0lugxSdjGFzDbqKQczIYs9g/zYMklFQVnjM+3MSNzrUf6hME/HCZciyG7rTgFVvxH9w7zqWY6wnsQkzJbGxhiNqGLlCXP7xKuPeVuSvqBHpDJP6H6uaJ1PjJWUuOaNDTyvlH6KpLyjtsb6kQdUAczawMZCNK0jwqA9oDWZfgGqcPZ743OqvYK4UN4PCSA4Vs4hQr8gl8p3uHv0YxCs7kkC+vMuwnzh9byNyU/pSA+nNVhmjY038WQv4RdjH1fkqoF6MzmFRl0GtPSYSVYIdF7fOU/cFoE6ZatpRux0M5o+Q435IyGiosizWT6zlsl9bvdb7UjNd7LmiZhqRhp4uyEs+7Vy+BgB3Yz+UXcxbBa6mdNg+7ZgkWZF14AQiP6YxMeyedxMk8uiOntJ4hHZtqJNYrBbQkPScaXHlxynFuIrq6gkEjoo35erHc6ac= ceph-c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:43:59.078 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T21:43:59.325 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:43:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:43:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/705124786' entity='client.admin' 2024-04-26T21:44:01.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.574+0000 7fe8b1253700 1 -- 172.21.15.55:0/2123559983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 msgr2=0x7fe8ac105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.574+0000 7fe8b1253700 1 --2- 172.21.15.55:0/2123559983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 0x7fe8ac105dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009910 tx=0x7fe89c009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:01.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.574+0000 7fe8b1253700 1 -- 172.21.15.55:0/2123559983 shutdown_connections 2024-04-26T21:44:01.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.574+0000 7fe8b1253700 1 --2- 172.21.15.55:0/2123559983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 0x7fe8ac105dc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.574+0000 7fe8b1253700 1 -- 172.21.15.55:0/2123559983 >> 172.21.15.55:0/2123559983 conn(0x7fe8ac0fd320 msgr2=0x7fe8ac0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:01.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.574+0000 7fe8b1253700 1 -- 172.21.15.55:0/2123559983 shutdown_connections 2024-04-26T21:44:01.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.574+0000 7fe8b1253700 1 -- 172.21.15.55:0/2123559983 wait complete. 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.575+0000 7fe8b1253700 1 Processor -- start 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.575+0000 7fe8b1253700 1 -- start start 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8b1253700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 0x7fe8ac198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8b1253700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac108dc0 con 0x7fe8ac1039d0 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 0x7fe8ac198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 0x7fe8ac198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47178/0 (socket says 172.21.15.55:47178) 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8aad9d700 1 -- 172.21.15.55:0/3958962853 learned_addr learned my addr 172.21.15.55:0/3958962853 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8aad9d700 1 -- 172.21.15.55:0/3958962853 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac1989d0 con 0x7fe8ac1039d0 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8aad9d700 1 --2- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 0x7fe8ac198490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005b40 tx=0x7fe89c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:01.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe893fff700 1 -- 172.21.15.55:0/3958962853 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe89c027070 con 0x7fe8ac1039d0 2024-04-26T21:44:01.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac198c60 con 0x7fe8ac1039d0 2024-04-26T21:44:01.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac19b900 con 0x7fe8ac1039d0 2024-04-26T21:44:01.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe893fff700 1 -- 172.21.15.55:0/3958962853 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe89c00be10 con 0x7fe8ac1039d0 2024-04-26T21:44:01.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.576+0000 7fe893fff700 1 -- 172.21.15.55:0/3958962853 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe89c022de0 con 0x7fe8ac1039d0 2024-04-26T21:44:01.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.577+0000 7fe893fff700 1 -- 172.21.15.55:0/3958962853 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe89c02a070 con 0x7fe8ac1039d0 2024-04-26T21:44:01.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.577+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ac109540 con 0x7fe8ac1039d0 2024-04-26T21:44:01.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.578+0000 7fe893fff700 1 --2- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe894039650 0x7fe89403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:01.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.578+0000 7fe893fff700 1 -- 172.21.15.55:0/3958962853 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe89c01e030 con 0x7fe8ac1039d0 2024-04-26T21:44:01.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.578+0000 7fe8aa59c700 1 --2- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe894039650 0x7fe89403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:01.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.578+0000 7fe8aa59c700 1 --2- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe894039650 0x7fe89403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0006fd0 tx=0x7fe8a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:01.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.587+0000 7fe893fff700 1 -- 172.21.15.55:0/3958962853 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe89c013070 con 0x7fe8ac1039d0 2024-04-26T21:44:01.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.793+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fe8ac02e5a0 con 0x7fe8ac1039d0 2024-04-26T21:44:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.796+0000 7fe893fff700 1 -- 172.21.15.55:0/3958962853 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fe89c00d340 con 0x7fe8ac1039d0 2024-04-26T21:44:01.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe894039650 msgr2=0x7fe89403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 --2- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe894039650 0x7fe89403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0006fd0 tx=0x7fe8a0006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:01.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 msgr2=0x7fe8ac198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:01.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 --2- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 0x7fe8ac198490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005b40 tx=0x7fe89c004f80 comp rx=0 tx=0).stop 2024-04-26T21:44:01.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 shutdown_connections 2024-04-26T21:44:01.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 --2- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe894039650 0x7fe89403bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 --2- 172.21.15.55:0/3958962853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ac1039d0 0x7fe8ac198490 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:01.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 >> 172.21.15.55:0/3958962853 conn(0x7fe8ac0fd320 msgr2=0x7fe8ac0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:01.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 shutdown_connections 2024-04-26T21:44:01.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:01.799+0000 7fe8b1253700 1 -- 172.21.15.55:0/3958962853 wait complete. 2024-04-26T21:44:01.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:01.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:01.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi055", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:01.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:01.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:01.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:02.418 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T21:44:02.421 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T21:44:02.667 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:03.123 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:02 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3958962853' entity='client.admin' 2024-04-26T21:44:03.123 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:03.123 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:04.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.202+0000 7f301029f700 1 -- 172.21.15.55:0/3920670948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 msgr2=0x7f3008103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.202+0000 7f301029f700 1 --2- 172.21.15.55:0/3920670948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 0x7f3008103290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3004009910 tx=0x7f3004009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.202+0000 7f301029f700 1 -- 172.21.15.55:0/3920670948 shutdown_connections 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.202+0000 7f301029f700 1 --2- 172.21.15.55:0/3920670948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 0x7f3008103290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.202+0000 7f301029f700 1 -- 172.21.15.55:0/3920670948 >> 172.21.15.55:0/3920670948 conn(0x7f30080fe420 msgr2=0x7f30081008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.202+0000 7f301029f700 1 -- 172.21.15.55:0/3920670948 shutdown_connections 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.202+0000 7f301029f700 1 -- 172.21.15.55:0/3920670948 wait complete. 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.203+0000 7f301029f700 1 Processor -- start 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.203+0000 7f301029f700 1 -- start start 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.203+0000 7f301029f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 0x7f30081984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.203+0000 7f301029f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3008108d70 con 0x7f3008102e70 2024-04-26T21:44:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.203+0000 7f300e03b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 0x7f30081984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.203+0000 7f300e03b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 0x7f30081984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47204/0 (socket says 172.21.15.55:47204) 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.203+0000 7f300e03b700 1 -- 172.21.15.55:0/1325921691 learned_addr learned my addr 172.21.15.55:0/1325921691 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.203+0000 7f300e03b700 1 -- 172.21.15.55:0/1325921691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3008198a10 con 0x7f3008102e70 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.204+0000 7f300e03b700 1 --2- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 0x7f30081984d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3004005800 tx=0x7f3004009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.204+0000 7f2fff7fe700 1 -- 172.21.15.55:0/1325921691 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3004027070 con 0x7f3008102e70 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.204+0000 7f2fff7fe700 1 -- 172.21.15.55:0/1325921691 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300400be10 con 0x7f3008102e70 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.204+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3008198ca0 con 0x7f3008102e70 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.204+0000 7f2fff7fe700 1 -- 172.21.15.55:0/1325921691 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f300400f460 con 0x7f3008102e70 2024-04-26T21:44:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.204+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300819b990 con 0x7f3008102e70 2024-04-26T21:44:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.205+0000 7f2fff7fe700 1 -- 172.21.15.55:0/1325921691 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f300402a070 con 0x7f3008102e70 2024-04-26T21:44:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.205+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30081094f0 con 0x7f3008102e70 2024-04-26T21:44:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.205+0000 7f2fff7fe700 1 --2- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ff40396b0 0x7f2ff403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.205+0000 7f2fff7fe700 1 -- 172.21.15.55:0/1325921691 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f300401e030 con 0x7f3008102e70 2024-04-26T21:44:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.205+0000 7f300d83a700 1 --2- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ff40396b0 0x7f2ff403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.205+0000 7f300d83a700 1 --2- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ff40396b0 0x7f2ff403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006fd0 tx=0x7f2ff8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:04.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.210+0000 7f2fff7fe700 1 -- 172.21.15.55:0/1325921691 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3004013070 con 0x7f3008102e70 2024-04-26T21:44:04.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.421+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f300802e5a0 con 0x7f2ff40396b0 2024-04-26T21:44:04.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.425+0000 7f2fff7fe700 1 -- 172.21.15.55:0/1325921691 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f300802e5a0 con 0x7f2ff40396b0 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ff40396b0 msgr2=0x7f2ff403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 --2- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ff40396b0 0x7f2ff403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006fd0 tx=0x7f2ff8006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 msgr2=0x7f30081984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 --2- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 0x7f30081984d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3004005800 tx=0x7f3004009f60 comp rx=0 tx=0).stop 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 shutdown_connections 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 --2- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ff40396b0 0x7f2ff403bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 --2- 172.21.15.55:0/1325921691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3008102e70 0x7f30081984d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 >> 172.21.15.55:0/1325921691 conn(0x7f30080fe420 msgr2=0x7f30080fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 shutdown_connections 2024-04-26T21:44:04.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:04.427+0000 7f301029f700 1 -- 172.21.15.55:0/1325921691 wait complete. 2024-04-26T21:44:05.109 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi064 2024-04-26T21:44:05.109 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:44:05.109 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:44:05.128 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:44:05.128 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:05.186 INFO:tasks.cephadm:Adding host smithi064 to orchestrator... 2024-04-26T21:44:05.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch host add smithi064 2024-04-26T21:44:05.439 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: Updating smithi055:/etc/ceph/ceph.conf 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: Updating smithi055:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: Updating smithi055:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: Updating smithi055:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.841+0000 7f9f50da0700 1 -- 172.21.15.55:0/1688608389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 msgr2=0x7f9f4c109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.841+0000 7f9f50da0700 1 --2- 172.21.15.55:0/1688608389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 0x7f9f4c109100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009910 tx=0x7f9f3c009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.841+0000 7f9f50da0700 1 -- 172.21.15.55:0/1688608389 shutdown_connections 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.841+0000 7f9f50da0700 1 --2- 172.21.15.55:0/1688608389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 0x7f9f4c109100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.841+0000 7f9f50da0700 1 -- 172.21.15.55:0/1688608389 >> 172.21.15.55:0/1688608389 conn(0x7f9f4c0776d0 msgr2=0x7f9f4c077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.841+0000 7f9f50da0700 1 -- 172.21.15.55:0/1688608389 shutdown_connections 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.841+0000 7f9f50da0700 1 -- 172.21.15.55:0/1688608389 wait complete. 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.842+0000 7f9f50da0700 1 Processor -- start 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.842+0000 7f9f50da0700 1 -- start start 2024-04-26T21:44:06.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.842+0000 7f9f50da0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 0x7f9f4c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.842+0000 7f9f50da0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c10d150 con 0x7f9f4c108d20 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.842+0000 7f9f4b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 0x7f9f4c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.842+0000 7f9f4b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 0x7f9f4c198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47230/0 (socket says 172.21.15.55:47230) 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.842+0000 7f9f4b7fe700 1 -- 172.21.15.55:0/543542053 learned_addr learned my addr 172.21.15.55:0/543542053 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.843+0000 7f9f4b7fe700 1 -- 172.21.15.55:0/543542053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c198bd0 con 0x7f9f4c108d20 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.843+0000 7f9f4b7fe700 1 --2- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 0x7f9f4c198690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c006010 tx=0x7f9f3c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.843+0000 7f9f48ff9700 1 -- 172.21.15.55:0/543542053 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f3c026070 con 0x7f9f4c108d20 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.843+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c198ec0 con 0x7f9f4c108d20 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.843+0000 7f9f48ff9700 1 -- 172.21.15.55:0/543542053 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f3c00bba0 con 0x7f9f4c108d20 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.843+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c19cf60 con 0x7f9f4c108d20 2024-04-26T21:44:06.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.843+0000 7f9f48ff9700 1 -- 172.21.15.55:0/543542053 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f3c0038b0 con 0x7f9f4c108d20 2024-04-26T21:44:06.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.844+0000 7f9f48ff9700 1 -- 172.21.15.55:0/543542053 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9f3c029070 con 0x7f9f4c108d20 2024-04-26T21:44:06.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.844+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4c1043d0 con 0x7f9f4c108d20 2024-04-26T21:44:06.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.844+0000 7f9f48ff9700 1 --2- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f34039650 0x7f9f3403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:06.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.844+0000 7f9f48ff9700 1 -- 172.21.15.55:0/543542053 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9f3c01e070 con 0x7f9f4c108d20 2024-04-26T21:44:06.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.844+0000 7f9f4affd700 1 --2- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f34039650 0x7f9f3403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:06.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.845+0000 7f9f4affd700 1 --2- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f34039650 0x7f9f3403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f40006fd0 tx=0x7f9f40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:06.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:06.849+0000 7f9f48ff9700 1 -- 172.21.15.55:0/543542053 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9f3c013070 con 0x7f9f4c108d20 2024-04-26T21:44:07.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:07.061+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}) -- 0x7f9f4c1999f0 con 0x7f9f34039650 2024-04-26T21:44:07.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:07 smithi055 ceph-mon[29221]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:08.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:08 smithi055 ceph-mon[29221]: Deploying cephadm binary to smithi064 2024-04-26T21:44:10.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:10 smithi055 ceph-mon[29221]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:11.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.410+0000 7f9f48ff9700 1 -- 172.21.15.55:0/543542053 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9f4c1999f0 con 0x7f9f34039650 2024-04-26T21:44:11.410 INFO:teuthology.orchestra.run.smithi055.stdout:Added host 'smithi064' with addr '172.21.15.64' 2024-04-26T21:44:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f34039650 msgr2=0x7f9f3403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 --2- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f34039650 0x7f9f3403bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f40006fd0 tx=0x7f9f40006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 msgr2=0x7f9f4c198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 --2- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 0x7f9f4c198690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c006010 tx=0x7f9f3c005780 comp rx=0 tx=0).stop 2024-04-26T21:44:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 shutdown_connections 2024-04-26T21:44:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 --2- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f34039650 0x7f9f3403bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 --2- 172.21.15.55:0/543542053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f4c108d20 0x7f9f4c198690 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:11.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 >> 172.21.15.55:0/543542053 conn(0x7f9f4c0776d0 msgr2=0x7f9f4c0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:11.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 shutdown_connections 2024-04-26T21:44:11.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:11.412+0000 7f9f50da0700 1 -- 172.21.15.55:0/543542053 wait complete. 2024-04-26T21:44:11.918 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:44:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:12.615 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:12 smithi055 ceph-mon[29221]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:12.615 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:12 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:12.615 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:12 smithi055 ceph-mon[29221]: Added host smithi064 2024-04-26T21:44:12.615 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:12 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:12.615 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:12 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:13.639 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:13.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.645+0000 7fae1a503700 1 -- 172.21.15.55:0/2869493266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 msgr2=0x7fae14107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.645+0000 7fae1a503700 1 --2- 172.21.15.55:0/2869493266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 0x7fae14107950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae0400aa20 tx=0x7fae0400ad30 comp rx=0 tx=0).stop 2024-04-26T21:44:13.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.645+0000 7fae1a503700 1 -- 172.21.15.55:0/2869493266 shutdown_connections 2024-04-26T21:44:13.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.645+0000 7fae1a503700 1 --2- 172.21.15.55:0/2869493266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 0x7fae14107950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.645+0000 7fae1a503700 1 -- 172.21.15.55:0/2869493266 >> 172.21.15.55:0/2869493266 conn(0x7fae140784a0 msgr2=0x7fae140788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:13.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.645+0000 7fae1a503700 1 -- 172.21.15.55:0/2869493266 shutdown_connections 2024-04-26T21:44:13.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.645+0000 7fae1a503700 1 -- 172.21.15.55:0/2869493266 wait complete. 2024-04-26T21:44:13.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.646+0000 7fae1a503700 1 Processor -- start 2024-04-26T21:44:13.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.646+0000 7fae1a503700 1 -- start start 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.646+0000 7fae1a503700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 0x7fae14196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.646+0000 7fae1a503700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fae1410b170 con 0x7fae14107570 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.646+0000 7fae13fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 0x7fae14196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.646+0000 7fae13fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 0x7fae14196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40720/0 (socket says 172.21.15.55:40720) 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.646+0000 7fae13fff700 1 -- 172.21.15.55:0/2765719498 learned_addr learned my addr 172.21.15.55:0/2765719498 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.646+0000 7fae13fff700 1 -- 172.21.15.55:0/2765719498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae14196b50 con 0x7fae14107570 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.647+0000 7fae13fff700 1 --2- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 0x7fae14196610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae0400c6d0 tx=0x7fae04005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.647+0000 7fae117fa700 1 -- 172.21.15.55:0/2765719498 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae04026070 con 0x7fae14107570 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.647+0000 7fae117fa700 1 -- 172.21.15.55:0/2765719498 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae040055b0 con 0x7fae14107570 2024-04-26T21:44:13.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.647+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae14196de0 con 0x7fae14107570 2024-04-26T21:44:13.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.647+0000 7fae117fa700 1 -- 172.21.15.55:0/2765719498 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae0400e5e0 con 0x7fae14107570 2024-04-26T21:44:13.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.647+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae14197240 con 0x7fae14107570 2024-04-26T21:44:13.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.648+0000 7fae117fa700 1 -- 172.21.15.55:0/2765719498 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fae04029070 con 0x7fae14107570 2024-04-26T21:44:13.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.648+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae14102be0 con 0x7fae14107570 2024-04-26T21:44:13.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.648+0000 7fae117fa700 1 --2- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fadfc0396b0 0x7fadfc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:13.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.648+0000 7fae117fa700 1 -- 172.21.15.55:0/2765719498 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fae0404e9f0 con 0x7fae14107570 2024-04-26T21:44:13.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.648+0000 7fae137fe700 1 --2- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fadfc0396b0 0x7fadfc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:13.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.649+0000 7fae137fe700 1 --2- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fadfc0396b0 0x7fadfc03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fae08006fd0 tx=0x7fae08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:13.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.653+0000 7fae117fa700 1 -- 172.21.15.55:0/2765719498 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fae0400e7b0 con 0x7fae14107570 2024-04-26T21:44:13.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.871+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fae14197990 con 0x7fadfc0396b0 2024-04-26T21:44:13.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.872+0000 7fae117fa700 1 -- 172.21.15.55:0/2765719498 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fae14197990 con 0x7fadfc0396b0 2024-04-26T21:44:13.873 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:44:13.873 INFO:teuthology.orchestra.run.smithi055.stdout:[{"addr": "172.21.15.55", "hostname": "smithi055", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}] 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fadfc0396b0 msgr2=0x7fadfc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 --2- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fadfc0396b0 0x7fadfc03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fae08006fd0 tx=0x7fae08006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 msgr2=0x7fae14196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 --2- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 0x7fae14196610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae0400c6d0 tx=0x7fae04005050 comp rx=0 tx=0).stop 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 shutdown_connections 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 --2- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fadfc0396b0 0x7fadfc03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 --2- 172.21.15.55:0/2765719498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae14107570 0x7fae14196610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 >> 172.21.15.55:0/2765719498 conn(0x7fae140784a0 msgr2=0x7fae140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 shutdown_connections 2024-04-26T21:44:13.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:13.874+0000 7fae1a503700 1 -- 172.21.15.55:0/2765719498 wait complete. 2024-04-26T21:44:14.507 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-04-26T21:44:14.507 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:44:14.507 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-04-26T21:44:14.525 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:44:14.525 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:14.583 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-04-26T21:44:14.583 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch host add smithi165 2024-04-26T21:44:14.832 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:14.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:14 smithi055 ceph-mon[29221]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:14.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:14 smithi055 ceph-mon[29221]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:14.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.327+0000 7fdffd351700 1 -- 172.21.15.55:0/684647315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 msgr2=0x7fdff81023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.327+0000 7fdffd351700 1 --2- 172.21.15.55:0/684647315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 0x7fdff81023b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009910 tx=0x7fdfe8009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.327+0000 7fdffd351700 1 -- 172.21.15.55:0/684647315 shutdown_connections 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.327+0000 7fdffd351700 1 --2- 172.21.15.55:0/684647315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 0x7fdff81023b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.327+0000 7fdffd351700 1 -- 172.21.15.55:0/684647315 >> 172.21.15.55:0/684647315 conn(0x7fdff80fd880 msgr2=0x7fdff80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.328+0000 7fdffd351700 1 -- 172.21.15.55:0/684647315 shutdown_connections 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.328+0000 7fdffd351700 1 -- 172.21.15.55:0/684647315 wait complete. 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.328+0000 7fdffd351700 1 Processor -- start 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.328+0000 7fdffd351700 1 -- start start 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdffd351700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 0x7fdff819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdffd351700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdff810c250 con 0x7fdff8101fd0 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdff6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 0x7fdff819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdff6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 0x7fdff819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40738/0 (socket says 172.21.15.55:40738) 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdff6ffd700 1 -- 172.21.15.55:0/2759374934 learned_addr learned my addr 172.21.15.55:0/2759374934 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:44:16.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdff6ffd700 1 -- 172.21.15.55:0/2759374934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff819eb60 con 0x7fdff8101fd0 2024-04-26T21:44:16.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdff6ffd700 1 --2- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 0x7fdff819e620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80056d0 tx=0x7fdfe80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdfdffff700 1 -- 172.21.15.55:0/2759374934 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfe8026070 con 0x7fdff8101fd0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdfdffff700 1 -- 172.21.15.55:0/2759374934 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe800bba0 con 0x7fdff8101fd0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff819ee50 con 0x7fdff8101fd0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdfdffff700 1 -- 172.21.15.55:0/2759374934 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfe8003920 con 0x7fdff8101fd0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.329+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff81988c0 con 0x7fdff8101fd0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.330+0000 7fdfdffff700 1 -- 172.21.15.55:0/2759374934 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdfe8029070 con 0x7fdff8101fd0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.330+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff81034b0 con 0x7fdff8101fd0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.330+0000 7fdfdffff700 1 --2- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfe0039650 0x7fdfe003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.330+0000 7fdfdffff700 1 -- 172.21.15.55:0/2759374934 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdfe801e070 con 0x7fdff8101fd0 2024-04-26T21:44:16.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.331+0000 7fdff67fc700 1 --2- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfe0039650 0x7fdfe003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:16.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.331+0000 7fdff67fc700 1 --2- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfe0039650 0x7fdfe003bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdfec006fd0 tx=0x7fdfec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:16.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.335+0000 7fdfdffff700 1 -- 172.21.15.55:0/2759374934 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdfe8013070 con 0x7fdff8101fd0 2024-04-26T21:44:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:16.550+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}) -- 0x7fdff8199c20 con 0x7fdfe0039650 2024-04-26T21:44:16.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:16 smithi055 ceph-mon[29221]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:17.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:17 smithi055 ceph-mon[29221]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:17.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:17.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:17.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:17.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:17.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:17.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:17.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:18.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:18 smithi055 ceph-mon[29221]: Deploying cephadm binary to smithi165 2024-04-26T21:44:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:18 smithi055 ceph-mon[29221]: Updating smithi064:/etc/ceph/ceph.conf 2024-04-26T21:44:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:18 smithi055 ceph-mon[29221]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:18 smithi055 ceph-mon[29221]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:18 smithi055 ceph-mon[29221]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:20.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.866+0000 7fdfdffff700 1 -- 172.21.15.55:0/2759374934 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fdff8199c20 con 0x7fdfe0039650 2024-04-26T21:44:20.867 INFO:teuthology.orchestra.run.smithi055.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-04-26T21:44:20.867 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:20 smithi055 ceph-mon[29221]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:20.867 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:20.867 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:20.867 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:20.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfe0039650 msgr2=0x7fdfe003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 --2- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfe0039650 0x7fdfe003bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdfec006fd0 tx=0x7fdfec006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 msgr2=0x7fdff819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 --2- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 0x7fdff819e620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80056d0 tx=0x7fdfe80057b0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 shutdown_connections 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 --2- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfe0039650 0x7fdfe003bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 --2- 172.21.15.55:0/2759374934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdff8101fd0 0x7fdff819e620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 >> 172.21.15.55:0/2759374934 conn(0x7fdff80fd880 msgr2=0x7fdff806b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 shutdown_connections 2024-04-26T21:44:20.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:20.868+0000 7fdffd351700 1 -- 172.21.15.55:0/2759374934 wait complete. 2024-04-26T21:44:21.482 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T21:44:21.728 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:22.135 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:21 smithi055 ceph-mon[29221]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:22.136 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:22.136 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:21 smithi055 ceph-mon[29221]: Added host smithi165 2024-04-26T21:44:22.136 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:22.136 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:23.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.148+0000 7f8b24f35700 1 -- 172.21.15.55:0/3370176189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 msgr2=0x7f8b20103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.148+0000 7f8b24f35700 1 --2- 172.21.15.55:0/3370176189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 0x7f8b20103470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009910 tx=0x7f8b10009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:23.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.148+0000 7f8b24f35700 1 -- 172.21.15.55:0/3370176189 shutdown_connections 2024-04-26T21:44:23.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.148+0000 7f8b24f35700 1 --2- 172.21.15.55:0/3370176189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 0x7f8b20103470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.148+0000 7f8b24f35700 1 -- 172.21.15.55:0/3370176189 >> 172.21.15.55:0/3370176189 conn(0x7f8b200fe960 msgr2=0x7f8b20100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:23.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.148+0000 7f8b24f35700 1 -- 172.21.15.55:0/3370176189 shutdown_connections 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.148+0000 7f8b24f35700 1 -- 172.21.15.55:0/3370176189 wait complete. 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.149+0000 7f8b24f35700 1 Processor -- start 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.149+0000 7f8b24f35700 1 -- start start 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.149+0000 7f8b24f35700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 0x7f8b2019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.149+0000 7f8b24f35700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2010d310 con 0x7f8b20103090 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b1e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 0x7f8b2019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b1e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 0x7f8b2019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54968/0 (socket says 172.21.15.55:54968) 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b1e59c700 1 -- 172.21.15.55:0/3933579421 learned_addr learned my addr 172.21.15.55:0/3933579421 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b1e59c700 1 -- 172.21.15.55:0/3933579421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2019eaf0 con 0x7f8b20103090 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b1e59c700 1 --2- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 0x7f8b2019e5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b100056d0 tx=0x7f8b100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b0f7fe700 1 -- 172.21.15.55:0/3933579421 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b10026070 con 0x7f8b20103090 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b0f7fe700 1 -- 172.21.15.55:0/3933579421 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b1000bba0 con 0x7f8b20103090 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2019ede0 con 0x7f8b20103090 2024-04-26T21:44:23.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b0f7fe700 1 -- 172.21.15.55:0/3933579421 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b10003920 con 0x7f8b20103090 2024-04-26T21:44:23.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.150+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b20198850 con 0x7f8b20103090 2024-04-26T21:44:23.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.151+0000 7f8b0f7fe700 1 -- 172.21.15.55:0/3933579421 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8b10029070 con 0x7f8b20103090 2024-04-26T21:44:23.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.151+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b20104570 con 0x7f8b20103090 2024-04-26T21:44:23.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.151+0000 7f8b0f7fe700 1 --2- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b08039650 0x7f8b0803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:23.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.151+0000 7f8b0f7fe700 1 -- 172.21.15.55:0/3933579421 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8b1001e070 con 0x7f8b20103090 2024-04-26T21:44:23.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.151+0000 7f8b1dd9b700 1 --2- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b08039650 0x7f8b0803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:23.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.152+0000 7f8b1dd9b700 1 --2- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b08039650 0x7f8b0803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b14006fd0 tx=0x7f8b14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:23.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.157+0000 7f8b0f7fe700 1 -- 172.21.15.55:0/3933579421 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8b10013070 con 0x7f8b20103090 2024-04-26T21:44:23.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:22 smithi055 ceph-mon[29221]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:23.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:23.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.371+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b20199be0 con 0x7f8b08039650 2024-04-26T21:44:23.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.372+0000 7f8b0f7fe700 1 -- 172.21.15.55:0/3933579421 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f8b20199be0 con 0x7f8b08039650 2024-04-26T21:44:23.373 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:44:23.373 INFO:teuthology.orchestra.run.smithi055.stdout:[{"addr": "172.21.15.55", "hostname": "smithi055", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-04-26T21:44:23.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.374+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b08039650 msgr2=0x7f8b0803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.374+0000 7f8b24f35700 1 --2- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b08039650 0x7f8b0803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b14006fd0 tx=0x7f8b14006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:23.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.374+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 msgr2=0x7f8b2019e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:23.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.374+0000 7f8b24f35700 1 --2- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 0x7f8b2019e5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b100056d0 tx=0x7f8b100057b0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.375+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 shutdown_connections 2024-04-26T21:44:23.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.375+0000 7f8b24f35700 1 --2- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b08039650 0x7f8b0803bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.375+0000 7f8b24f35700 1 --2- 172.21.15.55:0/3933579421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b20103090 0x7f8b2019e5b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:23.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.375+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 >> 172.21.15.55:0/3933579421 conn(0x7f8b200fe960 msgr2=0x7f8b20104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:23.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.375+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 shutdown_connections 2024-04-26T21:44:23.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:23.375+0000 7f8b24f35700 1 -- 172.21.15.55:0/3933579421 wait complete. 2024-04-26T21:44:24.000 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T21:44:24.000 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T21:44:24.241 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:24.650 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:24 smithi055 ceph-mon[29221]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:24.650 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:24 smithi055 ceph-mon[29221]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:24.650 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:24 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:25.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.748+0000 7f0fa97e9700 1 -- 172.21.15.55:0/1199374058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 msgr2=0x7f0fa4103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:25.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.748+0000 7f0fa97e9700 1 --2- 172.21.15.55:0/1199374058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 0x7f0fa4103470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009910 tx=0x7f0f94009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:25.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.748+0000 7f0fa97e9700 1 -- 172.21.15.55:0/1199374058 shutdown_connections 2024-04-26T21:44:25.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.748+0000 7f0fa97e9700 1 --2- 172.21.15.55:0/1199374058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 0x7f0fa4103470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:25.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.748+0000 7f0fa97e9700 1 -- 172.21.15.55:0/1199374058 >> 172.21.15.55:0/1199374058 conn(0x7f0fa40fe960 msgr2=0x7f0fa4100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:25.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.748+0000 7f0fa97e9700 1 -- 172.21.15.55:0/1199374058 shutdown_connections 2024-04-26T21:44:25.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.748+0000 7f0fa97e9700 1 -- 172.21.15.55:0/1199374058 wait complete. 2024-04-26T21:44:25.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.749+0000 7f0fa97e9700 1 Processor -- start 2024-04-26T21:44:25.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.749+0000 7f0fa97e9700 1 -- start start 2024-04-26T21:44:25.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.749+0000 7f0fa97e9700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 0x7f0fa419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.749+0000 7f0fa97e9700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa410d310 con 0x7f0fa4103090 2024-04-26T21:44:25.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0fa2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 0x7f0fa419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0fa2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 0x7f0fa419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54978/0 (socket says 172.21.15.55:54978) 2024-04-26T21:44:25.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0fa2ffd700 1 -- 172.21.15.55:0/3816709804 learned_addr learned my addr 172.21.15.55:0/3816709804 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:44:25.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0fa2ffd700 1 -- 172.21.15.55:0/3816709804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa419eaf0 con 0x7f0fa4103090 2024-04-26T21:44:25.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0fa2ffd700 1 --2- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 0x7f0fa419e5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f940056d0 tx=0x7f0f940057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0f8bfff700 1 -- 172.21.15.55:0/3816709804 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f94026070 con 0x7f0fa4103090 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0f8bfff700 1 -- 172.21.15.55:0/3816709804 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9400bba0 con 0x7f0fa4103090 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa419ede0 con 0x7f0fa4103090 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0f8bfff700 1 -- 172.21.15.55:0/3816709804 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f94003920 con 0x7f0fa4103090 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.750+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa4198850 con 0x7f0fa4103090 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.751+0000 7f0f8bfff700 1 -- 172.21.15.55:0/3816709804 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0f94029070 con 0x7f0fa4103090 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.751+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa4104570 con 0x7f0fa4103090 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.751+0000 7f0f8bfff700 1 --2- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f8c0396b0 0x7f0f8c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.751+0000 7f0f8bfff700 1 -- 172.21.15.55:0/3816709804 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0f9401e070 con 0x7f0fa4103090 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.751+0000 7f0fa27fc700 1 --2- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f8c0396b0 0x7f0f8c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:25.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.752+0000 7f0fa27fc700 1 --2- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f8c0396b0 0x7f0f8c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f98006fd0 tx=0x7f0f98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:25.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.756+0000 7f0f8bfff700 1 -- 172.21.15.55:0/3816709804 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0f94013070 con 0x7f0fa4103090 2024-04-26T21:44:25.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:25.970+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0fa40008d0 con 0x7f0fa4103090 2024-04-26T21:44:26.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.505+0000 7f0f8bfff700 1 -- 172.21.15.55:0/3816709804 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f0f94030090 con 0x7f0fa4103090 2024-04-26T21:44:26.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.508+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f8c0396b0 msgr2=0x7f0f8c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.508+0000 7f0fa97e9700 1 --2- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f8c0396b0 0x7f0f8c03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f98006fd0 tx=0x7f0f98006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:26.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.508+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 msgr2=0x7f0fa419e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:26.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.508+0000 7f0fa97e9700 1 --2- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 0x7f0fa419e5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0f940056d0 tx=0x7f0f940057b0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.508+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 shutdown_connections 2024-04-26T21:44:26.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.508+0000 7f0fa97e9700 1 --2- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f8c0396b0 0x7f0f8c03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.508+0000 7f0fa97e9700 1 --2- 172.21.15.55:0/3816709804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0fa4103090 0x7f0fa419e5b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:26.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.508+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 >> 172.21.15.55:0/3816709804 conn(0x7f0fa40fe960 msgr2=0x7f0fa4104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:26.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.509+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 shutdown_connections 2024-04-26T21:44:26.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:26.509+0000 7f0fa97e9700 1 -- 172.21.15.55:0/3816709804 wait complete. 2024-04-26T21:44:26.510 INFO:teuthology.orchestra.run.smithi055.stderr:adjusted tunables profile to default 2024-04-26T21:44:26.802 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:26 smithi055 ceph-mon[29221]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:26.802 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:26 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3816709804' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:44:27.062 INFO:tasks.cephadm:Adding mon.a on smithi055 2024-04-26T21:44:27.062 INFO:tasks.cephadm:Adding mon.b on smithi064 2024-04-26T21:44:27.062 INFO:tasks.cephadm:Adding mon.c on smithi165 2024-04-26T21:44:27.063 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c' 2024-04-26T21:44:27.329 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:44:27.377 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3816709804' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:27.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:27 smithi055 ceph-mon[29221]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T21:44:28.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.976+0000 7f48e8aed700 1 -- 172.21.15.165:0/1134082475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 msgr2=0x7f48e41023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:28.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.976+0000 7f48e8aed700 1 --2- 172.21.15.165:0/1134082475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 0x7f48e41023b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009910 tx=0x7f48d4009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:28.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.976+0000 7f48e8aed700 1 -- 172.21.15.165:0/1134082475 shutdown_connections 2024-04-26T21:44:28.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.976+0000 7f48e8aed700 1 --2- 172.21.15.165:0/1134082475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 0x7f48e41023b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:28.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.976+0000 7f48e8aed700 1 -- 172.21.15.165:0/1134082475 >> 172.21.15.165:0/1134082475 conn(0x7f48e40fd880 msgr2=0x7f48e40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:28.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.976+0000 7f48e8aed700 1 -- 172.21.15.165:0/1134082475 shutdown_connections 2024-04-26T21:44:28.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.976+0000 7f48e8aed700 1 -- 172.21.15.165:0/1134082475 wait complete. 2024-04-26T21:44:28.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.977+0000 7f48e8aed700 1 Processor -- start 2024-04-26T21:44:28.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.977+0000 7f48e8aed700 1 -- start start 2024-04-26T21:44:28.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.977+0000 7f48e8aed700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 0x7f48e419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:28.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.977+0000 7f48e8aed700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48e410c250 con 0x7f48e4101fd0 2024-04-26T21:44:28.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.977+0000 7f48e259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 0x7f48e419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:28.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.977+0000 7f48e259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 0x7f48e419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.165:34944/0 (socket says 172.21.15.165:34944) 2024-04-26T21:44:28.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.977+0000 7f48e259c700 1 -- 172.21.15.165:0/1302356871 learned_addr learned my addr 172.21.15.165:0/1302356871 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:44:28.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.978+0000 7f48e259c700 1 -- 172.21.15.165:0/1302356871 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e419d970 con 0x7f48e4101fd0 2024-04-26T21:44:28.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.978+0000 7f48e259c700 1 --2- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 0x7f48e419d430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f48d40056d0 tx=0x7f48d40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:28.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.978+0000 7f48d37fe700 1 -- 172.21.15.165:0/1302356871 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d4026070 con 0x7f48e4101fd0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.978+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e419dc60 con 0x7f48e4101fd0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.978+0000 7f48d37fe700 1 -- 172.21.15.165:0/1302356871 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d400bba0 con 0x7f48e4101fd0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.978+0000 7f48d37fe700 1 -- 172.21.15.165:0/1302356871 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d4003920 con 0x7f48e4101fd0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.978+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e41975f0 con 0x7f48e4101fd0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.979+0000 7f48d37fe700 1 -- 172.21.15.165:0/1302356871 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f48d4029070 con 0x7f48e4101fd0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.979+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e41034b0 con 0x7f48e4101fd0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.979+0000 7f48d37fe700 1 --2- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48cc039650 0x7f48cc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.979+0000 7f48d37fe700 1 -- 172.21.15.165:0/1302356871 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f48d401e070 con 0x7f48e4101fd0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.980+0000 7f48e1d9b700 1 --2- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48cc039650 0x7f48cc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:28.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.980+0000 7f48e1d9b700 1 --2- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48cc039650 0x7f48cc03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006fd0 tx=0x7f48d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:28.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:28.985+0000 7f48d37fe700 1 -- 172.21.15.165:0/1302356871 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f48d4013070 con 0x7f48e4101fd0 2024-04-26T21:44:29.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.197+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}) -- 0x7f48e4198b20 con 0x7f48cc039650 2024-04-26T21:44:29.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.207+0000 7f48d37fe700 1 -- 172.21.15.165:0/1302356871 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f48e4198b20 con 0x7f48cc039650 2024-04-26T21:44:29.207 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mon update... 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48cc039650 msgr2=0x7f48cc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 --2- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48cc039650 0x7f48cc03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f48d8006fd0 tx=0x7f48d8006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 msgr2=0x7f48e419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 --2- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 0x7f48e419d430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f48d40056d0 tx=0x7f48d40057b0 comp rx=0 tx=0).stop 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 shutdown_connections 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 --2- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48cc039650 0x7f48cc03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 --2- 172.21.15.165:0/1302356871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48e4101fd0 0x7f48e419d430 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 >> 172.21.15.165:0/1302356871 conn(0x7f48e40fd880 msgr2=0x7f48e41039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 shutdown_connections 2024-04-26T21:44:29.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:29.208+0000 7f48e8aed700 1 -- 172.21.15.165:0/1302356871 wait complete. 2024-04-26T21:44:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:28 smithi055 ceph-mon[29221]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:28 smithi055 ceph-mon[29221]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:28 smithi055 ceph-mon[29221]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:28 smithi055 ceph-mon[29221]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:29.811 DEBUG:teuthology.orchestra.run.smithi064:mon.b> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T21:44:29.813 DEBUG:teuthology.orchestra.run.smithi165:mon.c> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T21:44:29.816 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:44:29.816 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:44:30.541 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: Saving service mon spec with placement smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c;count:3 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: Deploying daemon mon.c on smithi165 2024-04-26T21:44:30.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:30 smithi055 ceph-mon[29221]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:32.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:32 smithi055 ceph-mon[29221]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.535+0000 7f1d6adc4700 1 -- 172.21.15.165:0/1350186728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 msgr2=0x7f1d641023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.535+0000 7f1d6adc4700 1 --2- 172.21.15.165:0/1350186728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 0x7f1d641023b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009910 tx=0x7f1d58009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.535+0000 7f1d6adc4700 1 -- 172.21.15.165:0/1350186728 shutdown_connections 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.535+0000 7f1d6adc4700 1 --2- 172.21.15.165:0/1350186728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 0x7f1d641023b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.535+0000 7f1d6adc4700 1 -- 172.21.15.165:0/1350186728 >> 172.21.15.165:0/1350186728 conn(0x7f1d64077880 msgr2=0x7f1d64077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.535+0000 7f1d6adc4700 1 -- 172.21.15.165:0/1350186728 shutdown_connections 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.535+0000 7f1d6adc4700 1 -- 172.21.15.165:0/1350186728 wait complete. 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.536+0000 7f1d6adc4700 1 Processor -- start 2024-04-26T21:44:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.536+0000 7f1d6adc4700 1 -- start start 2024-04-26T21:44:33.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.536+0000 7f1d6adc4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 0x7f1d6419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.536+0000 7f1d6adc4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6410c200 con 0x7f1d64101fd0 2024-04-26T21:44:33.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.536+0000 7f1d68b60700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 0x7f1d6419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.536+0000 7f1d68b60700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 0x7f1d6419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.165:46766/0 (socket says 172.21.15.165:46766) 2024-04-26T21:44:33.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d68b60700 1 -- 172.21.15.165:0/167169726 learned_addr learned my addr 172.21.15.165:0/167169726 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:44:33.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d68b60700 1 -- 172.21.15.165:0/167169726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6419eaf0 con 0x7f1d64101fd0 2024-04-26T21:44:33.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d68b60700 1 --2- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 0x7f1d6419e5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d58006010 tx=0x7f1d58005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d61ffb700 1 -- 172.21.15.165:0/167169726 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d58026070 con 0x7f1d64101fd0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6419ede0 con 0x7f1d64101fd0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d61ffb700 1 -- 172.21.15.165:0/167169726 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5800bba0 con 0x7f1d64101fd0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d61ffb700 1 -- 172.21.15.165:0/167169726 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d58003910 con 0x7f1d64101fd0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d64198850 con 0x7f1d64101fd0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d61ffb700 1 -- 172.21.15.165:0/167169726 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1d58029070 con 0x7f1d64101fd0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.538+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d64103460 con 0x7f1d64101fd0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.539+0000 7f1d61ffb700 1 --2- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1d54039650 0x7f1d5403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.539+0000 7f1d61ffb700 1 -- 172.21.15.165:0/167169726 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1d5801e070 con 0x7f1d64101fd0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.539+0000 7f1d63fff700 1 --2- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1d54039650 0x7f1d5403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:33.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.540+0000 7f1d63fff700 1 --2- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1d54039650 0x7f1d5403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d50006fd0 tx=0x7f1d50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:33.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.544+0000 7f1d61ffb700 1 -- 172.21.15.165:0/167169726 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1d58013070 con 0x7f1d64101fd0 2024-04-26T21:44:33.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.836+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1d640008d0 con 0x7f1d64101fd0 2024-04-26T21:44:33.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.837+0000 7f1d61ffb700 1 -- 172.21.15.165:0/167169726 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f1d58030020 con 0x7f1d64101fd0 2024-04-26T21:44:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:44:33.837 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":1,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","modified":"2024-04-26T21:42:49.108371Z","created":"2024-04-26T21:42:49.108371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T21:44:33.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1d54039650 msgr2=0x7f1d5403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 --2- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1d54039650 0x7f1d5403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d50006fd0 tx=0x7f1d50006e40 comp rx=0 tx=0).stop 2024-04-26T21:44:33.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 msgr2=0x7f1d6419e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:33.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 --2- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 0x7f1d6419e5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d58006010 tx=0x7f1d58005780 comp rx=0 tx=0).stop 2024-04-26T21:44:33.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 shutdown_connections 2024-04-26T21:44:33.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 --2- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1d54039650 0x7f1d5403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 --2- 172.21.15.165:0/167169726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d64101fd0 0x7f1d6419e5b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:33.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 >> 172.21.15.165:0/167169726 conn(0x7f1d64077880 msgr2=0x7f1d6406a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:33.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 shutdown_connections 2024-04-26T21:44:33.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:33.839+0000 7f1d6adc4700 1 -- 172.21.15.165:0/167169726 wait complete. 2024-04-26T21:44:33.840 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 1 2024-04-26T21:44:33.956 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:44:33.956 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:44:34.301 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:44:34.301 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:44:34.301 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:44:34.301 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:44:34.301 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Deploying cephadm binary to smithi165 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Updating smithi064:/etc/ceph/ceph.conf 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:34.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Added host smithi165 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:34.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3816709804' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3816709804' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:34.305 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Saving service mon spec with placement smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c;count:3 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:34.306 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: Deploying daemon mon.c on smithi165 2024-04-26T21:44:34.307 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.307 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:34.307 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 ceph-mon[33296]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T21:44:34.307 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 bash[33264]: 2db75f7cb6093c3d3c696b34530d53df5f45535a01349296dd2c80649ddb9c8c 2024-04-26T21:44:34.307 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:33 smithi165 systemd[1]: Started Ceph mon.c for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:44:35.546 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:44:35.546 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:44:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:44:39.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:39.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:39.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: mon.a calling monitor election 2024-04-26T21:44:39.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: mon.c calling monitor election 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: monmap epoch 2 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: last_changed 2024-04-26T21:44:33.961880+0000 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: created 2024-04-26T21:42:49.108371+0000 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: min_mon_release 19 (squid) 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: election_strategy: 1 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-26T21:44:39.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: fsmap 2024-04-26T21:44:39.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:39.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: mgrmap e13: x(active, since 49s) 2024-04-26T21:44:39.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: overall HEALTH_OK 2024-04-26T21:44:39.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:39.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:39.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:39.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:39.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: mon.a calling monitor election 2024-04-26T21:44:39.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: mon.c calling monitor election 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: monmap epoch 2 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: last_changed 2024-04-26T21:44:33.961880+0000 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: created 2024-04-26T21:42:49.108371+0000 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: min_mon_release 19 (squid) 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: election_strategy: 1 2024-04-26T21:44:39.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-04-26T21:44:39.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-26T21:44:39.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: fsmap 2024-04-26T21:44:39.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:39.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: mgrmap e13: x(active, since 49s) 2024-04-26T21:44:39.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: overall HEALTH_OK 2024-04-26T21:44:39.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:39.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.206+0000 7f8aeefd5700 1 -- 172.21.15.165:0/343415825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 msgr2=0x7f8ad8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.206+0000 7f8aeefd5700 1 --2- 172.21.15.165:0/343415825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 0x7f8ad8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8101d50 tx=0x7f8adc00feb0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.206+0000 7f8aeefd5700 1 -- 172.21.15.165:0/343415825 shutdown_connections 2024-04-26T21:44:40.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.206+0000 7f8aeefd5700 1 --2- 172.21.15.165:0/343415825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 0x7f8ad8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.206+0000 7f8aeefd5700 1 -- 172.21.15.165:0/343415825 >> 172.21.15.165:0/343415825 conn(0x7f8ae80fd950 msgr2=0x7f8ae80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:40.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.207+0000 7f8aeefd5700 1 -- 172.21.15.165:0/343415825 shutdown_connections 2024-04-26T21:44:40.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.207+0000 7f8aeefd5700 1 -- 172.21.15.165:0/343415825 wait complete. 2024-04-26T21:44:40.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8aeefd5700 1 Processor -- start 2024-04-26T21:44:40.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8aeefd5700 1 -- start start 2024-04-26T21:44:40.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8aeefd5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 0x7f8ae819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8aeefd5700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 0x7f8ae81976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8aeefd5700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8109d70 con 0x7f8ad80035c0 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8aeefd5700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8109bf0 con 0x7f8ae819db30 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8ae7fff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 0x7f8ae81976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8ae7fff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 0x7f8ae81976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48844/0 (socket says 172.21.15.165:48844) 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8ae7fff700 1 -- 172.21.15.165:0/1111373552 learned_addr learned my addr 172.21.15.165:0/1111373552 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8ae7fff700 1 -- 172.21.15.165:0/1111373552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 msgr2=0x7f8ae81976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8ae7fff700 1 -- 172.21.15.165:0/1111373552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 msgr2=0x7f8ae81976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8ae7fff700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 0x7f8ae81976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8ae7fff700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 0x7f8ae81976c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.208+0000 7f8aecd71700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 0x7f8ae819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.209+0000 7f8aecd71700 1 -- 172.21.15.165:0/1111373552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 msgr2=0x7f8ae81976c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.209+0000 7f8aecd71700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 0x7f8ae81976c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.209+0000 7f8aecd71700 1 -- 172.21.15.165:0/1111373552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae819e0b0 con 0x7f8ad80035c0 2024-04-26T21:44:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.209+0000 7f8aecd71700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 0x7f8ae819d5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8adc010010 tx=0x7f8adc0116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.209+0000 7f8ae5ffb700 1 -- 172.21.15.165:0/1111373552 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8adc013960 con 0x7f8ad80035c0 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.209+0000 7f8ae5ffb700 1 -- 172.21.15.165:0/1111373552 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8adc0044e0 con 0x7f8ad80035c0 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.209+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae8197c60 con 0x7f8ad80035c0 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.209+0000 7f8ae5ffb700 1 -- 172.21.15.165:0/1111373552 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8adc011b80 con 0x7f8ad80035c0 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.210+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae81981f0 con 0x7f8ad80035c0 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.211+0000 7f8ae5ffb700 1 -- 172.21.15.165:0/1111373552 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8adc023070 con 0x7f8ad80035c0 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.211+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae8103560 con 0x7f8ad80035c0 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.211+0000 7f8ae5ffb700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8acc0393f0 0x7f8acc03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.211+0000 7f8ae5ffb700 1 -- 172.21.15.165:0/1111373552 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8adc020050 con 0x7f8ad80035c0 2024-04-26T21:44:40.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.211+0000 7f8ae7fff700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8acc0393f0 0x7f8acc03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:40.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.212+0000 7f8ae7fff700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8acc0393f0 0x7f8acc03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009180 tx=0x7f8ad4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:40.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.217+0000 7f8ae5ffb700 1 -- 172.21.15.165:0/1111373552 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8adc028020 con 0x7f8ad80035c0 2024-04-26T21:44:40.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:40.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:40.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:40.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:40 smithi165 ceph-mon[33296]: Deploying daemon mon.b on smithi064 2024-04-26T21:44:40.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:40 smithi055 ceph-mon[29221]: Deploying daemon mon.b on smithi064 2024-04-26T21:44:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:40.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.500+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8ae8198500 con 0x7f8ad80035c0 2024-04-26T21:44:40.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.500+0000 7f8ae5ffb700 1 -- 172.21.15.165:0/1111373552 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f8adc01a390 con 0x7f8ad80035c0 2024-04-26T21:44:40.500 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:44:40.501 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","modified":"2024-04-26T21:44:33.961880Z","created":"2024-04-26T21:42:49.108371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T21:44:40.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8acc0393f0 msgr2=0x7f8acc03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8acc0393f0 0x7f8acc03b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009180 tx=0x7f8ad4008040 comp rx=0 tx=0).stop 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 msgr2=0x7f8ae819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 0x7f8ae819d5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8adc010010 tx=0x7f8adc0116a0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 shutdown_connections 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8acc0393f0 0x7f8acc03b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ad80035c0 0x7f8ae819d5f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 --2- 172.21.15.165:0/1111373552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ae819db30 0x7f8ae81976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 >> 172.21.15.165:0/1111373552 conn(0x7f8ae80fd950 msgr2=0x7f8ae80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 shutdown_connections 2024-04-26T21:44:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:40.503+0000 7f8aeefd5700 1 -- 172.21.15.165:0/1111373552 wait complete. 2024-04-26T21:44:40.504 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-04-26T21:44:41.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:41 smithi165 ceph-mon[33296]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:41.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:41 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/1111373552' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:41.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:41 smithi055 ceph-mon[29221]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:41.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:41 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/1111373552' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:42.112 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T21:44:42.113 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T21:44:42.357 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:44:42.435 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 systemd[1]: Starting Ceph mon.b for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:44:42.725 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 podman[32747]: 2024-04-26 21:44:42.621139397 +0000 UTC m=+0.106442227 container create b8ed5b06c3bf864fa36b6f2c21201de1f1ee6165148bac8d2b921099559fc414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:44:42.725 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 podman[32747]: 2024-04-26 21:44:42.53355794 +0000 UTC m=+0.018860784 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:44:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 podman[32747]: 2024-04-26 21:44:42.784012333 +0000 UTC m=+0.269315164 container init b8ed5b06c3bf864fa36b6f2c21201de1f1ee6165148bac8d2b921099559fc414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T21:44:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 podman[32747]: 2024-04-26 21:44:42.787624736 +0000 UTC m=+0.272927575 container start b8ed5b06c3bf864fa36b6f2c21201de1f1ee6165148bac8d2b921099559fc414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T21:44:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T21:44:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-26T21:44:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pidfile_write: ignore empty --pid-file 2024-04-26T21:44:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: load: jerasure load: lrc 2024-04-26T21:44:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: RocksDB version: 7.9.2 2024-04-26T21:44:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Git sha 0 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: DB SUMMARY 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: DB Session ID: EF62L05BILCHGT3RKCKZ 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: CURRENT file: CURRENT 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.error_if_exists: 0 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.create_if_missing: 0 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.paranoid_checks: 1 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.env: 0x55b87b351cc0 2024-04-26T21:44:43.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T21:44:43.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.info_log: 0x55b87d8e9d80 2024-04-26T21:44:43.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T21:44:43.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.statistics: (nil) 2024-04-26T21:44:43.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.use_fsync: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_log_file_size: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.allow_fallocate: 1 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.use_direct_reads: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.db_log_dir: 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.wal_dir: 2024-04-26T21:44:43.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.write_buffer_manager: 0x55b87d8bb0e0 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T21:44:43.011 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.unordered_write: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.row_cache: None 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.wal_filter: None 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.two_write_queues: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.wal_compression: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.atomic_flush: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T21:44:43.012 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.log_readahead_size: 0 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_background_jobs: 2 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_background_compactions: -1 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_subcompactions: 1 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T21:44:43.013 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_open_files: -1 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_background_flushes: -1 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Compression algorithms supported: 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T21:44:43.014 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: kZSTD supported: 0 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: kXpressCompression supported: 0 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: kZlibCompression supported: 1 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: kSnappyCompression supported: 1 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: kLZ4Compression supported: 1 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: kBZip2Compression supported: 0 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.merge_operator: 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_filter: None 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T21:44:43.015 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b87d8e9f40) 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_type: 0 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: data_block_index_type: 0 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_shortening: 1 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: checksum: 4 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: no_block_cache: 0 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache: 0x55b87d8ed1f0 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-04-26T21:44:43.016 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_options: 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: capacity : 536870912 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: num_shard_bits : 4 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: strict_capacity_limit : 0 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_cache_compressed: (nil) 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: persistent_cache: (nil) 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_size: 4096 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_size_deviation: 10 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_restart_interval: 16 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: index_block_restart_interval: 1 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: metadata_block_size: 4096 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: partition_filters: 0 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: use_delta_encoding: 1 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: filter_policy: bloomfilter 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: whole_key_filtering: 1 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: verify_compression: 0 2024-04-26T21:44:43.017 INFO:journalctl@ceph.mon.b.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout: format_version: 5 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout: enable_index_compression: 1 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout: block_align: 0 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout: max_auto_readahead_size: 262144 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout: prepopulate_block_cache: 0 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout: initial_auto_readahead_size: 8192 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression: NoCompression 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.num_levels: 7 2024-04-26T21:44:43.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T21:44:43.019 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T21:44:43.020 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T21:44:43.021 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T21:44:43.022 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.inplace_update_support: 0 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T21:44:43.023 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.bloom_locality: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.max_successive_merges: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.ttl: 2592000 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.enable_blob_files: false 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.min_blob_size: 0 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T21:44:43.024 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fd42dd7c-53c4-48df-ad5f-f0397c470d1b 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167882818750, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167882819187, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714167882, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fd42dd7c-53c4-48df-ad5f-f0397c470d1b", "db_session_id": "EF62L05BILCHGT3RKCKZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T21:44:43.025 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714167882819277, "job": 1, "event": "recovery_finished"} 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b87da0a000 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: DB pointer 0x55b87d9f6000 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** DB Stats ** 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:44:43.026 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** Compaction Stats [default] ** 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** Compaction Stats [default] ** 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T21:44:43.027 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: Block cache BinnedLRUCache@0x55b87d8ed1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-04-26T21:44:43.028 INFO:journalctl@ceph.mon.b.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: using public_addr v2:172.21.15.64:0/0 -> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: starting mon.b rank -1 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(???) e0 preinit fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).mds e1 new map 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout: e1 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout: legacy client fscid: -1 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout: 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout: No filesystems configured 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T21:44:43.029 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.030 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Deploying cephadm binary to smithi165 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Updating smithi064:/etc/ceph/ceph.conf 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:43.031 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Added host smithi165 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3816709804' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3816709804' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.032 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Saving service mon spec with placement smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c;count:3 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:43.033 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Deploying daemon mon.c on smithi165 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.a calling monitor election 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.034 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.c calling monitor election 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: monmap epoch 2 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: last_changed 2024-04-26T21:44:33.961880+0000 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: created 2024-04-26T21:42:49.108371+0000 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: min_mon_release 19 (squid) 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: election_strategy: 1 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-26T21:44:43.035 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: fsmap 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mgrmap e13: x(active, since 49s) 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: overall HEALTH_OK 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: Deploying daemon mon.b on smithi064 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/1111373552' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 ceph-mon[32765]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 bash[32747]: b8ed5b06c3bf864fa36b6f2c21201de1f1ee6165148bac8d2b921099559fc414 2024-04-26T21:44:43.036 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:42 smithi064 systemd[1]: Started Ceph mon.b for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: mon.a calling monitor election 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: mon.c calling monitor election 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: mon.b calling monitor election 2024-04-26T21:44:48.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: monmap epoch 3 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: last_changed 2024-04-26T21:44:42.841901+0000 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: created 2024-04-26T21:42:49.108371+0000 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: min_mon_release 19 (squid) 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: election_strategy: 1 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: fsmap 2024-04-26T21:44:48.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:48.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: mgrmap e13: x(active, since 58s) 2024-04-26T21:44:48.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: overall HEALTH_OK 2024-04-26T21:44:48.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.191+0000 7f3f3601d700 1 -- 172.21.15.165:0/582298961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 msgr2=0x7f3f18005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.191+0000 7f3f3601d700 1 --2- 172.21.15.165:0/582298961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 0x7f3f18005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3f3006b710 tx=0x7f3f2400fe60 comp rx=0 tx=0).stop 2024-04-26T21:44:48.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.191+0000 7f3f3601d700 1 -- 172.21.15.165:0/582298961 shutdown_connections 2024-04-26T21:44:48.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.191+0000 7f3f3601d700 1 --2- 172.21.15.165:0/582298961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 0x7f3f18005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.191+0000 7f3f3601d700 1 -- 172.21.15.165:0/582298961 >> 172.21.15.165:0/582298961 conn(0x7f3f300fd960 msgr2=0x7f3f300ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:48.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.191+0000 7f3f3601d700 1 -- 172.21.15.165:0/582298961 shutdown_connections 2024-04-26T21:44:48.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.191+0000 7f3f3601d700 1 -- 172.21.15.165:0/582298961 wait complete. 2024-04-26T21:44:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.192+0000 7f3f3601d700 1 Processor -- start 2024-04-26T21:44:48.193 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:48.193 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: mon.a calling monitor election 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: mon.c calling monitor election 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: mon.b calling monitor election 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:44:48.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: monmap epoch 3 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: last_changed 2024-04-26T21:44:42.841901+0000 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: created 2024-04-26T21:42:49.108371+0000 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: min_mon_release 19 (squid) 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: election_strategy: 1 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: fsmap 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: mgrmap e13: x(active, since 58s) 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: overall HEALTH_OK 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.196 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.196 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.192+0000 7f3f3601d700 1 -- start start 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.192+0000 7f3f3601d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 0x7f3f3019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.192+0000 7f3f3601d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 0x7f3f301989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.192+0000 7f3f3601d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 0x7f3f30199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.192+0000 7f3f3601d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3010ae30 con 0x7f3f180035c0 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.192+0000 7f3f3601d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3010acb0 con 0x7f3f3019ed20 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.192+0000 7f3f3601d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3010afb0 con 0x7f3f30198ee0 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 0x7f3f3019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2ffff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 0x7f3f30199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 0x7f3f3019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.165:39624/0 (socket says 172.21.15.165:39624) 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 -- 172.21.15.165:0/2758727863 learned_addr learned my addr 172.21.15.165:0/2758727863 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2effd700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 0x7f3f301989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2ffff700 1 -- 172.21.15.165:0/2758727863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 msgr2=0x7f3f30199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2ffff700 1 -- 172.21.15.165:0/2758727863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 msgr2=0x7f3f30199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2ffff700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 0x7f3f30199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2effd700 1 -- 172.21.15.165:0/2758727863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 msgr2=0x7f3f301989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2ffff700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 0x7f3f30199390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2effd700 1 -- 172.21.15.165:0/2758727863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 msgr2=0x7f3f301989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2effd700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 0x7f3f301989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2effd700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 0x7f3f301989a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T21:44:48.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 -- 172.21.15.165:0/2758727863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 msgr2=0x7f3f30199390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 0x7f3f30199390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 -- 172.21.15.165:0/2758727863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 msgr2=0x7f3f301989a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 0x7f3f301989a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 -- 172.21.15.165:0/2758727863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f301a56b0 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2f7fe700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 0x7f3f3019e7e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f24001eb0 tx=0x7f3f24003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.193+0000 7f3f2cff9700 1 -- 172.21.15.165:0/2758727863 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24022040 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.194+0000 7f3f2cff9700 1 -- 172.21.15.165:0/2758727863 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2402c070 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.194+0000 7f3f2cff9700 1 -- 172.21.15.165:0/2758727863 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2401de60 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.194+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f30101c80 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.194+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301020b0 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.195+0000 7f3f2cff9700 1 -- 172.21.15.165:0/2758727863 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3f240314a0 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.195+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f30064a80 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.195+0000 7f3f2cff9700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f1c039ae0 0x7f3f1c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.195+0000 7f3f2cff9700 1 -- 172.21.15.165:0/2758727863 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3f2404f540 con 0x7f3f180035c0 2024-04-26T21:44:48.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.195+0000 7f3f2effd700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f1c039ae0 0x7f3f1c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:48.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.196+0000 7f3f2effd700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f1c039ae0 0x7f3f1c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009850 tx=0x7f3f20008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:48.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.200+0000 7f3f2cff9700 1 -- 172.21.15.165:0/2758727863 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3f24020120 con 0x7f3f180035c0 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: mon.a calling monitor election 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: mon.c calling monitor election 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: mon.b calling monitor election 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:48.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: monmap epoch 3 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: last_changed 2024-04-26T21:44:42.841901+0000 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: created 2024-04-26T21:42:49.108371+0000 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: min_mon_release 19 (squid) 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: election_strategy: 1 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: 0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: 2: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.b 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: fsmap 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T21:44:48.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: mgrmap e13: x(active, since 58s) 2024-04-26T21:44:48.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: overall HEALTH_OK 2024-04-26T21:44:48.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:48.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.491+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3f301024f0 con 0x7f3f180035c0 2024-04-26T21:44:48.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.492+0000 7f3f2cff9700 1 -- 172.21.15.165:0/2758727863 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f3f2402fbd0 con 0x7f3f180035c0 2024-04-26T21:44:48.493 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-26T21:44:48.493 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":3,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","modified":"2024-04-26T21:44:42.841901Z","created":"2024-04-26T21:42:49.108371Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f1c039ae0 msgr2=0x7f3f1c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f1c039ae0 0x7f3f1c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009850 tx=0x7f3f20008040 comp rx=0 tx=0).stop 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 msgr2=0x7f3f3019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 0x7f3f3019e7e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f24001eb0 tx=0x7f3f24003d50 comp rx=0 tx=0).stop 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 shutdown_connections 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f30198ee0 0x7f3f30199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f180035c0 0x7f3f3019e7e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f1c039ae0 0x7f3f1c03bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 --2- 172.21.15.165:0/2758727863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3019ed20 0x7f3f301989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:48.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 >> 172.21.15.165:0/2758727863 conn(0x7f3f300fd960 msgr2=0x7f3f30104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:48.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 shutdown_connections 2024-04-26T21:44:48.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:48.494+0000 7f3f3601d700 1 -- 172.21.15.165:0/2758727863 wait complete. 2024-04-26T21:44:48.496 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 3 2024-04-26T21:44:49.073 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:48 smithi165 ceph-mon[33296]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:49.073 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:49.073 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:48 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/2758727863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:49.073 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:49.099 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T21:44:49.099 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T21:44:49.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:48 smithi055 ceph-mon[29221]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:48 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/2758727863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:48 smithi064 ceph-mon[32765]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:48 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/2758727863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T21:44:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:44:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:50.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: Updating smithi055:/etc/ceph/ceph.conf 2024-04-26T21:44:50.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: Updating smithi064:/etc/ceph/ceph.conf 2024-04-26T21:44:50.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:50 smithi165 ceph-mon[33296]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: Updating smithi055:/etc/ceph/ceph.conf 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: Updating smithi064:/etc/ceph/ceph.conf 2024-04-26T21:44:50.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:50 smithi055 ceph-mon[29221]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T21:44:50.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:50.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:50.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:50.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:50.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: Updating smithi055:/etc/ceph/ceph.conf 2024-04-26T21:44:50.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: Updating smithi064:/etc/ceph/ceph.conf 2024-04-26T21:44:50.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:50 smithi064 ceph-mon[32765]: Updating smithi165:/etc/ceph/ceph.conf 2024-04-26T21:44:50.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.954+0000 7f91b5462700 1 -- 172.21.15.55:0/3717050056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0103ff0 msgr2=0x7f91b01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.954+0000 7f91b5462700 1 --2- 172.21.15.55:0/3717050056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0103ff0 0x7f91b01043d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009910 tx=0x7f91a4009c20 comp rx=0 tx=0).stop 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.954+0000 7f91b5462700 1 -- 172.21.15.55:0/3717050056 shutdown_connections 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.954+0000 7f91b5462700 1 --2- 172.21.15.55:0/3717050056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0103ff0 0x7f91b01043d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.954+0000 7f91b5462700 1 -- 172.21.15.55:0/3717050056 >> 172.21.15.55:0/3717050056 conn(0x7f91b00fd820 msgr2=0x7f91b00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.954+0000 7f91b5462700 1 -- 172.21.15.55:0/3717050056 shutdown_connections 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.954+0000 7f91b5462700 1 -- 172.21.15.55:0/3717050056 wait complete. 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.954+0000 7f91b5462700 1 Processor -- start 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91b5462700 1 -- start start 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91b5462700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b0103ff0 0x7f91b0101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91b5462700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0102190 0x7f91b0102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91b5462700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91b01a4d90 0x7f91b01a7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91b5462700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91b010d370 con 0x7f91b0102190 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91b5462700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91b010d1f0 con 0x7f91b01a4d90 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91b5462700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f91b010d4f0 con 0x7f91b0103ff0 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91ae7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0102190 0x7f91b0102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91ae7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0102190 0x7f91b0102610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43534/0 (socket says 172.21.15.55:43534) 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91ae7fc700 1 -- 172.21.15.55:0/1910635635 learned_addr learned my addr 172.21.15.55:0/1910635635 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91af7fe700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91b01a4d90 0x7f91b01a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91aeffd700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b0103ff0 0x7f91b0101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91ae7fc700 1 -- 172.21.15.55:0/1910635635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b0103ff0 msgr2=0x7f91b0101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.955+0000 7f91ae7fc700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b0103ff0 0x7f91b0101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f91ae7fc700 1 -- 172.21.15.55:0/1910635635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91b01a4d90 msgr2=0x7f91b01a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f91ae7fc700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91b01a4d90 0x7f91b01a7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f91ae7fc700 1 -- 172.21.15.55:0/1910635635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b01a78b0 con 0x7f91b0102190 2024-04-26T21:44:50.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f91ae7fc700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0102190 0x7f91b0102610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91a000b780 tx=0x7f91a000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f9197fff700 1 -- 172.21.15.55:0/1910635635 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0004230 con 0x7f91b0102190 2024-04-26T21:44:50.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b01a7ba0 con 0x7f91b0102190 2024-04-26T21:44:50.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f9197fff700 1 -- 172.21.15.55:0/1910635635 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a00043d0 con 0x7f91b0102190 2024-04-26T21:44:50.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f9197fff700 1 -- 172.21.15.55:0/1910635635 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0011ad0 con 0x7f91b0102190 2024-04-26T21:44:50.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.956+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b01a80e0 con 0x7f91b0102190 2024-04-26T21:44:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.957+0000 7f9197fff700 1 -- 172.21.15.55:0/1910635635 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f91a0011c70 con 0x7f91b0102190 2024-04-26T21:44:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.957+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b0064a80 con 0x7f91b0102190 2024-04-26T21:44:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.957+0000 7f9197fff700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9198039ae0 0x7f919803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.957+0000 7f9197fff700 1 -- 172.21.15.55:0/1910635635 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f91a004c3b0 con 0x7f91b0102190 2024-04-26T21:44:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.957+0000 7f91aeffd700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9198039ae0 0x7f919803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.958+0000 7f91aeffd700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9198039ae0 0x7f919803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009750 tx=0x7f91a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:50.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:50.963+0000 7f9197fff700 1 -- 172.21.15.55:0/1910635635 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f91a0018510 con 0x7f91b0102190 2024-04-26T21:44:51.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.185+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f91b01a8310 con 0x7f91b0102190 2024-04-26T21:44:51.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.185+0000 7f9197fff700 1 -- 172.21.15.55:0/1910635635 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f91a001fad0 con 0x7f91b0102190 2024-04-26T21:44:51.186 INFO:teuthology.orchestra.run.smithi055.stdout:# minimal ceph.conf for c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:51.186 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-04-26T21:44:51.186 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:44:51.186 INFO:teuthology.orchestra.run.smithi055.stdout: mon_host = [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-04-26T21:44:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.187+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9198039ae0 msgr2=0x7f919803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.187+0000 7f91b5462700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9198039ae0 0x7f919803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009750 tx=0x7f91a401a040 comp rx=0 tx=0).stop 2024-04-26T21:44:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0102190 msgr2=0x7f91b0102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:51.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0102190 0x7f91b0102610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f91a000b780 tx=0x7f91a000ba90 comp rx=0 tx=0).stop 2024-04-26T21:44:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 shutdown_connections 2024-04-26T21:44:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9198039ae0 0x7f919803bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91b01a4d90 0x7f91b01a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b0103ff0 0x7f91b0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 --2- 172.21.15.55:0/1910635635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91b0102190 0x7f91b0102610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 >> 172.21.15.55:0/1910635635 conn(0x7f91b00fd820 msgr2=0x7f91b0106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 shutdown_connections 2024-04-26T21:44:51.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:44:51.188+0000 7f91b5462700 1 -- 172.21.15.55:0/1910635635 wait complete. 2024-04-26T21:44:51.753 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: Updating smithi055:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.753 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.754 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.755 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.755 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:44:51.755 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:51.755 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:51.755 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:51.755 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: Reconfiguring daemon mon.a on smithi055 2024-04-26T21:44:51.755 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:51 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1910635635' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: Updating smithi055:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: Reconfiguring daemon mon.a on smithi055 2024-04-26T21:44:51.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:51 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1910635635' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:51.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: Updating smithi055:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: Updating smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: Updating smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T21:44:51.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.829 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:51.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T21:44:51.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:51.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:51.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:51.830 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: Reconfiguring daemon mon.a on smithi055 2024-04-26T21:44:51.831 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:51 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1910635635' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:52.451 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T21:44:52.451 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:44:52.451 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:44:52.480 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:44:52.480 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:52.543 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:44:52.543 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:44:52.575 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:44:52.575 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:52.644 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:44:52.644 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T21:44:52.676 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:44:52.676 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T21:44:52.743 INFO:tasks.cephadm:Adding mgr.x on smithi055 2024-04-26T21:44:52.743 INFO:tasks.cephadm:Adding mgr.y on smithi064 2024-04-26T21:44:52.743 INFO:tasks.cephadm:Adding mgr.z on smithi165 2024-04-26T21:44:52.743 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi055=x;smithi064=y;smithi165=z' 2024-04-26T21:44:52.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:52 smithi064 ceph-mon[32765]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:52.901 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:52 smithi055 ceph-mon[29221]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:53.018 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:44:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:52 smithi165 ceph-mon[33296]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:54.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:54.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:54.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:53 smithi064 ceph-mon[32765]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:44:54.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:54.218 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:54.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:54.219 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:53 smithi064 ceph-mon[32765]: Reconfiguring daemon mon.b on smithi064 2024-04-26T21:44:54.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:54.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:54.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:53 smithi165 ceph-mon[33296]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:44:54.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:54.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:54.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:54.275 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:53 smithi165 ceph-mon[33296]: Reconfiguring daemon mon.b on smithi064 2024-04-26T21:44:54.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:54.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:54.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:53 smithi055 ceph-mon[29221]: Reconfiguring mon.b (monmap changed)... 2024-04-26T21:44:54.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:54.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:54.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:54.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:53 smithi055 ceph-mon[29221]: Reconfiguring daemon mon.b on smithi064 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.472+0000 7f591f62d700 1 -- 172.21.15.165:0/373447257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918102fb0 msgr2=0x7f59181033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.472+0000 7f591f62d700 1 --2- 172.21.15.165:0/373447257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918102fb0 0x7f59181033d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f591000b0a0 tx=0x7f591000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.472+0000 7f591f62d700 1 -- 172.21.15.165:0/373447257 shutdown_connections 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.472+0000 7f591f62d700 1 --2- 172.21.15.165:0/373447257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918102fb0 0x7f59181033d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.472+0000 7f591f62d700 1 -- 172.21.15.165:0/373447257 >> 172.21.15.165:0/373447257 conn(0x7f59180fe550 msgr2=0x7f5918100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.472+0000 7f591f62d700 1 -- 172.21.15.165:0/373447257 shutdown_connections 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.472+0000 7f591f62d700 1 -- 172.21.15.165:0/373447257 wait complete. 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591f62d700 1 Processor -- start 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591f62d700 1 -- start start 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591f62d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5918102fb0 0x7f59181986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591f62d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918198c20 0x7f591819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591f62d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59181990a0 0x7f59181a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591f62d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5918108ea0 con 0x7f5918198c20 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591f62d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5918108d20 con 0x7f5918102fb0 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591f62d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5918109020 con 0x7f59181990a0 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59181990a0 0x7f59181a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59181990a0 0x7f59181a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57428/0 (socket says 172.21.15.165:57428) 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 -- 172.21.15.165:0/2941755961 learned_addr learned my addr 172.21.15.165:0/2941755961 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591d3c9700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5918102fb0 0x7f59181986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591cbc8700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918198c20 0x7f591819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 -- 172.21.15.165:0/2941755961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5918102fb0 msgr2=0x7f59181986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5918102fb0 0x7f59181986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 -- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918198c20 msgr2=0x7f591819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918198c20 0x7f591819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 -- 172.21.15.165:0/2941755961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59181a0af0 con 0x7f59181990a0 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591d3c9700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5918102fb0 0x7f59181986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591cbc8700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918198c20 0x7f591819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:44:54.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.473+0000 7f591dbca700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59181990a0 0x7f59181a0220 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f591400b550 tx=0x7f591400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.474+0000 7f590e7fc700 1 -- 172.21.15.165:0/2941755961 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914015020 con 0x7f59181990a0 2024-04-26T21:44:54.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.474+0000 7f590e7fc700 1 -- 172.21.15.165:0/2941755961 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5914011ce0 con 0x7f59181990a0 2024-04-26T21:44:54.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.474+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59181a0de0 con 0x7f59181990a0 2024-04-26T21:44:54.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.474+0000 7f590e7fc700 1 -- 172.21.15.165:0/2941755961 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914010da0 con 0x7f59181990a0 2024-04-26T21:44:54.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.474+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181a12c0 con 0x7f59181990a0 2024-04-26T21:44:54.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.475+0000 7f590e7fc700 1 -- 172.21.15.165:0/2941755961 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5914018020 con 0x7f59181990a0 2024-04-26T21:44:54.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.475+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5918064a80 con 0x7f59181990a0 2024-04-26T21:44:54.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.475+0000 7f590e7fc700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5904039ae0 0x7f590403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:44:54.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.475+0000 7f590e7fc700 1 -- 172.21.15.165:0/2941755961 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f591404c230 con 0x7f59181990a0 2024-04-26T21:44:54.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.475+0000 7f591d3c9700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5904039ae0 0x7f590403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:44:54.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.476+0000 7f591d3c9700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5904039ae0 0x7f590403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5910003f40 tx=0x7f5910006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:44:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.480+0000 7f590e7fc700 1 -- 172.21.15.165:0/2941755961 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f591401f020 con 0x7f59181990a0 2024-04-26T21:44:54.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.700+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi055=x;smithi064=y;smithi165=z", "target": ["mon-mgr", ""]}) -- 0x7f5918063860 con 0x7f5904039ae0 2024-04-26T21:44:54.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.711+0000 7f590e7fc700 1 -- 172.21.15.165:0/2941755961 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5918063860 con 0x7f5904039ae0 2024-04-26T21:44:54.711 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mgr update... 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5904039ae0 msgr2=0x7f590403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5904039ae0 0x7f590403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5910003f40 tx=0x7f5910006210 comp rx=0 tx=0).stop 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59181990a0 msgr2=0x7f59181a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59181990a0 0x7f59181a0220 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f591400b550 tx=0x7f591400b910 comp rx=0 tx=0).stop 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 shutdown_connections 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59181990a0 0x7f59181a0220 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5904039ae0 0x7f590403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5918102fb0 0x7f59181986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 --2- 172.21.15.165:0/2941755961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5918198c20 0x7f591819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:44:54.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 >> 172.21.15.165:0/2941755961 conn(0x7f59180fe550 msgr2=0x7f59180ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:44:54.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 shutdown_connections 2024-04-26T21:44:54.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:44:54.713+0000 7f591f62d700 1 -- 172.21.15.165:0/2941755961 wait complete. 2024-04-26T21:44:55.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:54 smithi064 ceph-mon[32765]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:55.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:55.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:55.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:55.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.299 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:54 smithi165 ceph-mon[33296]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:55.299 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.299 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.299 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:55.299 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:55.299 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:55.299 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.325 DEBUG:teuthology.orchestra.run.smithi064:mgr.y> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T21:44:55.327 DEBUG:teuthology.orchestra.run.smithi165:mgr.z> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T21:44:55.330 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T21:44:55.330 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:44:55.330 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:44:55.349 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:44:55.350 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 2024-04-26T21:44:55.408 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:44:55.408 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:55.408 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T21:44:55.408 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:55.409 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:55.409 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-04-26 21:43:59.690295835 +0000 2024-04-26T21:44:55.409 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-04-26 21:39:21.253270278 +0000 2024-04-26T21:44:55.409 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-04-26 21:39:21.253270278 +0000 2024-04-26T21:44:55.409 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-04-26 21:39:21.253270278 +0000 2024-04-26T21:44:55.409 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:44:55.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:54 smithi055 ceph-mon[29221]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:55.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T21:44:55.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T21:44:55.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:55.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:55.464 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-04-26T21:44:55.464 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-04-26T21:44:55.464 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000324687 s, 1.6 MB/s 2024-04-26T21:44:55.465 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:44:55.525 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 2024-04-26T21:44:55.583 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:44:55.584 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:55.584 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T21:44:55.584 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:55.584 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:55.584 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-04-26 21:43:59.722295224 +0000 2024-04-26T21:44:55.584 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-04-26 21:39:21.604263081 +0000 2024-04-26T21:44:55.584 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-04-26 21:39:21.604263081 +0000 2024-04-26T21:44:55.584 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-04-26 21:39:21.604263081 +0000 2024-04-26T21:44:55.584 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:44:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-04-26T21:44:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-04-26T21:44:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000282135 s, 1.8 MB/s 2024-04-26T21:44:55.652 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:44:55.710 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-04-26 21:43:59.751294670 +0000 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-04-26 21:39:21.956255863 +0000 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-04-26 21:39:21.956255863 +0000 2024-04-26T21:44:55.768 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-04-26 21:39:21.956255863 +0000 2024-04-26T21:44:55.769 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:44:55.834 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-04-26T21:44:55.834 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-04-26T21:44:55.834 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000291236 s, 1.8 MB/s 2024-04-26T21:44:55.836 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:44:55.895 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-04-26 21:43:59.782294078 +0000 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-04-26 21:39:22.314248523 +0000 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-04-26 21:39:22.314248523 +0000 2024-04-26T21:44:55.954 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-04-26 21:39:22.314248523 +0000 2024-04-26T21:44:55.955 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-04-26T21:44:56.021 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000300903 s, 1.7 MB/s 2024-04-26T21:44:56.022 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:44:56.080 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-04-26T21:44:56.080 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:44:56.099 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:44:56.100 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-04-26T21:44:56.158 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:44:56.158 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:56.158 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T21:44:56.158 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:56.158 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:56.159 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-04-26 21:44:42.313580284 +0000 2024-04-26T21:44:56.159 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-04-26 21:39:41.707440155 +0000 2024-04-26T21:44:56.159 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-04-26 21:39:41.707440155 +0000 2024-04-26T21:44:56.159 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-04-26 21:39:41.707440155 +0000 2024-04-26T21:44:56.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:44:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:55 smithi055 ceph-mon[29221]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:44:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:55 smithi055 ceph-mon[29221]: Reconfiguring daemon mon.c on smithi165 2024-04-26T21:44:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:55 smithi055 ceph-mon[29221]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi055=x;smithi064=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:55 smithi055 ceph-mon[29221]: Saving service mgr spec with placement smithi055=x;smithi064=y;smithi165=z;count:3 2024-04-26T21:44:56.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:55 smithi064 ceph-mon[32765]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:44:56.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:55 smithi064 ceph-mon[32765]: Reconfiguring daemon mon.c on smithi165 2024-04-26T21:44:56.223 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:55 smithi064 ceph-mon[32765]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi055=x;smithi064=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:56.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:55 smithi064 ceph-mon[32765]: Saving service mgr spec with placement smithi055=x;smithi064=y;smithi165=z;count:3 2024-04-26T21:44:56.226 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-04-26T21:44:56.226 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-04-26T21:44:56.226 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000291969 s, 1.8 MB/s 2024-04-26T21:44:56.227 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:44:56.287 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-04-26T21:44:56.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:55 smithi165 ceph-mon[33296]: Reconfiguring mon.c (monmap changed)... 2024-04-26T21:44:56.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:55 smithi165 ceph-mon[33296]: Reconfiguring daemon mon.c on smithi165 2024-04-26T21:44:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:55 smithi165 ceph-mon[33296]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi055=x;smithi064=y;smithi165=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:44:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:55 smithi165 ceph-mon[33296]: Saving service mgr spec with placement smithi055=x;smithi064=y;smithi165=z;count:3 2024-04-26T21:44:56.345 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:44:56.346 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T21:44:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-04-26 21:44:42.313580284 +0000 2024-04-26T21:44:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-04-26 21:39:42.051432848 +0000 2024-04-26T21:44:56.346 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-04-26 21:39:42.051432848 +0000 2024-04-26T21:44:56.346 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-04-26 21:39:42.051432848 +0000 2024-04-26T21:44:56.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:44:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-04-26T21:44:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-04-26T21:44:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000206822 s, 2.5 MB/s 2024-04-26T21:44:56.413 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:44:56.473 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-04-26T21:44:56.531 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:44:56.531 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:56.531 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T21:44:56.531 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:56.531 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-04-26 21:44:42.314580264 +0000 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-04-26 21:39:42.396425519 +0000 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-04-26 21:39:42.396425519 +0000 2024-04-26T21:44:56.532 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-04-26 21:39:42.396425519 +0000 2024-04-26T21:44:56.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:44:56.597 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-04-26T21:44:56.597 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-04-26T21:44:56.597 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000236096 s, 2.2 MB/s 2024-04-26T21:44:56.599 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:44:56.657 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-04-26T21:44:56.714 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:44:56.714 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:56.714 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T21:44:56.714 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:56.714 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:56.714 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-04-26 21:44:42.314580264 +0000 2024-04-26T21:44:56.714 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-04-26 21:39:42.745418106 +0000 2024-04-26T21:44:56.714 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-04-26 21:39:42.745418106 +0000 2024-04-26T21:44:56.715 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-04-26 21:39:42.745418106 +0000 2024-04-26T21:44:56.715 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:44:56.779 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-04-26T21:44:56.779 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-04-26T21:44:56.779 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000194073 s, 2.6 MB/s 2024-04-26T21:44:56.781 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:44:56.838 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-26T21:44:56.838 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T21:44:56.856 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T21:44:56.856 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-04-26T21:44:56.912 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T21:44:56.912 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T21:44:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-26 21:44:33.257159400 +0000 2024-04-26T21:44:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-26 21:39:29.350166990 +0000 2024-04-26T21:44:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-26 21:39:29.350166990 +0000 2024-04-26T21:44:56.913 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-26 21:39:29.350166990 +0000 2024-04-26T21:44:56.913 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T21:44:56.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-26T21:44:56.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-26T21:44:56.978 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000253176 s, 2.0 MB/s 2024-04-26T21:44:56.979 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T21:44:57.036 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-04-26T21:44:57.092 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T21:44:57.092 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:57.092 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-26T21:44:57.092 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:57.092 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:57.093 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-26 21:44:33.258159385 +0000 2024-04-26T21:44:57.093 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-26 21:39:29.689161694 +0000 2024-04-26T21:44:57.093 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-26 21:39:29.689161694 +0000 2024-04-26T21:44:57.093 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-26 21:39:29.689161694 +0000 2024-04-26T21:44:57.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T21:44:57.155 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-26T21:44:57.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-26T21:44:57.156 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000197134 s, 2.6 MB/s 2024-04-26T21:44:57.157 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T21:44:57.213 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-04-26T21:44:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:56 smithi064 ceph-mon[32765]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:57.267 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T21:44:57.267 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:57.267 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T21:44:57.267 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:57.268 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:57.268 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-26 21:44:33.258159385 +0000 2024-04-26T21:44:57.268 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-26 21:39:30.039156226 +0000 2024-04-26T21:44:57.268 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-26 21:39:30.039156226 +0000 2024-04-26T21:44:57.268 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-26 21:39:30.039156226 +0000 2024-04-26T21:44:57.268 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T21:44:57.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:56 smithi165 ceph-mon[33296]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:57.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:57.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-26T21:44:57.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-26T21:44:57.330 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000197628 s, 2.6 MB/s 2024-04-26T21:44:57.331 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T21:44:57.387 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-04-26T21:44:57.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:56 smithi055 ceph-mon[29221]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:57.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:57.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:57.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:57.441 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T21:44:57.441 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T21:44:57.441 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T21:44:57.441 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:44:57.441 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T21:44:57.441 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-26 21:44:33.258159385 +0000 2024-04-26T21:44:57.442 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-26 21:39:30.367151102 +0000 2024-04-26T21:44:57.442 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-26 21:39:30.367151102 +0000 2024-04-26T21:44:57.442 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-26 21:39:30.367151102 +0000 2024-04-26T21:44:57.442 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T21:44:57.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-26T21:44:57.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-26T21:44:57.504 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000220148 s, 2.3 MB/s 2024-04-26T21:44:57.505 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T21:44:57.561 INFO:tasks.cephadm:Deploying osd.0 on smithi055 with /dev/vg_nvme/lv_4... 2024-04-26T21:44:57.561 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:44:57.805 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:44:58.042 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:58.042 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:58.042 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:58.042 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:58.042 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:58.042 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:58.042 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:58.042 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:57 smithi165 ceph-mon[33296]: Deploying daemon mgr.z on smithi165 2024-04-26T21:44:58.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:58.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:58.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:58.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:58.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:58.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:58.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:58.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:57 smithi055 ceph-mon[29221]: Deploying daemon mgr.z on smithi165 2024-04-26T21:44:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:44:58.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:44:58.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:44:58.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:44:58.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:58.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:57 smithi064 ceph-mon[32765]: Deploying daemon mgr.z on smithi165 2024-04-26T21:44:59.302 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:44:59 smithi165 ceph-mon[33296]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:44:59 smithi055 ceph-mon[29221]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:44:59 smithi064 ceph-mon[32765]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:44:59.577 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:44:59 smithi165 ceph-mgr[35147]: -- 172.21.15.165:0/515907022 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557239a2a5a0 con 0x5572399fe800 2024-04-26T21:44:59.606 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:44:59.606 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:44:59.606 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-04-26T21:44:59.607 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-04-26T21:44:59.607 INFO:teuthology.orchestra.run.smithi055.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012715 s, 825 MB/s 2024-04-26T21:44:59.607 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-04-26T21:44:59.870 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:44:59 smithi165 bash[35127]: fa46b4a04d462701b54d8d6d84c3636cd1ab874ae58c31364b83558d16709be1 2024-04-26T21:44:59.870 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:44:59 smithi165 systemd[1]: Started Ceph mgr.z for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:44:59.870 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:44:59 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:44:59.681+0000 7f2319070200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:44:59.870 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:44:59 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:44:59.869+0000 7f2319070200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:45:00.245 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi055:vg_nvme/lv_4 2024-04-26T21:45:00.281 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:44:59 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:44:59.971+0000 7f2319070200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:45:00.281 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:00.280+0000 7f2319070200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:45:00.487 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:45:00.576 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:00.365+0000 7f2319070200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:00.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:00 smithi055 ceph-mon[29221]: Deploying daemon mgr.y on smithi064 2024-04-26T21:45:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:45:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:45:01.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:45:01.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:01.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:00 smithi064 ceph-mon[32765]: Deploying daemon mgr.y on smithi064 2024-04-26T21:45:01.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:45:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T21:45:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:45:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:00 smithi165 ceph-mon[33296]: Deploying daemon mgr.y on smithi064 2024-04-26T21:45:01.827 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:01 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:01.568+0000 7f2319070200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:45:01.827 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:01 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:01.741+0000 7f2319070200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:45:01.995 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:01 smithi064 systemd[1]: Starting Ceph mgr.y for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:45:02.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 -- 172.21.15.55:0/4081412126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0103580 msgr2=0x7f99b01039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 --2- 172.21.15.55:0/4081412126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0103580 0x7f99b01039e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f99a40099c0 tx=0x7f99a4009cd0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 -- 172.21.15.55:0/4081412126 shutdown_connections 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 --2- 172.21.15.55:0/4081412126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0103f20 0x7f99b010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 --2- 172.21.15.55:0/4081412126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99b0102380 0x7f99b01027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 --2- 172.21.15.55:0/4081412126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0103580 0x7f99b01039e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 -- 172.21.15.55:0/4081412126 >> 172.21.15.55:0/4081412126 conn(0x7f99b00fd940 msgr2=0x7f99b00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 -- 172.21.15.55:0/4081412126 shutdown_connections 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.022+0000 7f99b69a3700 1 -- 172.21.15.55:0/4081412126 wait complete. 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99b69a3700 1 Processor -- start 2024-04-26T21:45:02.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99b69a3700 1 -- start start 2024-04-26T21:45:02.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99b69a3700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0102380 0x7f99b006c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99b69a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0103580 0x7f99b019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99b69a3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99b0103f20 0x7f99b01a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99b69a3700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010de70 con 0x7f99b0102380 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99b69a3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010dcf0 con 0x7f99b0103580 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99b69a3700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010dff0 con 0x7f99b0103f20 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99affff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0102380 0x7f99b006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99affff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0102380 0x7f99b006c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51024/0 (socket says 172.21.15.55:51024) 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.023+0000 7f99affff700 1 -- 172.21.15.55:0/403100507 learned_addr learned my addr 172.21.15.55:0/403100507 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99af7fe700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0103580 0x7f99b019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99b4f40700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99b0103f20 0x7f99b01a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99affff700 1 -- 172.21.15.55:0/403100507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99b0103f20 msgr2=0x7f99b01a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99affff700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99b0103f20 0x7f99b01a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99affff700 1 -- 172.21.15.55:0/403100507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0103580 msgr2=0x7f99b019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99affff700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0103580 0x7f99b019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99affff700 1 -- 172.21.15.55:0/403100507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b01a1240 con 0x7f99b0102380 2024-04-26T21:45:02.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99affff700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0102380 0x7f99b006c670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e950 tx=0x7f99a000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0004230 con 0x7f99b0102380 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b01a1530 con 0x7f99b0102380 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a00043d0 con 0x7f99b0102380 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0003d60 con 0x7f99b0102380 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.024+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b01a19f0 con 0x7f99b0102380 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.025+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f99a0021440 con 0x7f99b0102380 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.025+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b006a110 con 0x7f99b0102380 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.025+0000 7f99ad7fa700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9998039890 0x7f999803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.025+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f99a004cd60 con 0x7f99b0102380 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.026+0000 7f99af7fe700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9998039890 0x7f999803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:02.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.026+0000 7f99af7fe700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9998039890 0x7f999803bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:02.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.031+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f99a000fa10 con 0x7f99b0102380 2024-04-26T21:45:02.220 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:01 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:01.916+0000 7f2319070200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:45:02.220 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:02.219+0000 7f2319070200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:45:02.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:02.252+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f99b0063860 con 0x7f9998039890 2024-04-26T21:45:02.257 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:01 smithi064 podman[34145]: 2024-04-26 21:45:01.897845205 +0000 UTC m=+0.018526516 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:45:02.257 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 podman[34145]: 2024-04-26 21:45:02.001277672 +0000 UTC m=+0.121959019 container create 1e53c41971930f7f67a3383fe4e07b0b7d3aabeb1ea86a9dc70b17b307a7f419 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:45:02.257 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 podman[34145]: 2024-04-26 21:45:02.238963238 +0000 UTC m=+0.359644547 container init 1e53c41971930f7f67a3383fe4e07b0b7d3aabeb1ea86a9dc70b17b307a7f419 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:45:02.257 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 podman[34145]: 2024-04-26 21:45:02.241994712 +0000 UTC m=+0.362676021 container start 1e53c41971930f7f67a3383fe4e07b0b7d3aabeb1ea86a9dc70b17b307a7f419 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:45:02.531 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:02.315+0000 7f2319070200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:45:02.532 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:02.531+0000 7f2319070200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:45:02.594 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mgr[34164]: -- 172.21.15.64:0/2480846765 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56099f01c5a0 con 0x56099eff0c00 2024-04-26T21:45:02.594 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 bash[34145]: 1e53c41971930f7f67a3383fe4e07b0b7d3aabeb1ea86a9dc70b17b307a7f419 2024-04-26T21:45:02.594 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 systemd[1]: Started Ceph mgr.y for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:45:02.594 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:02.401+0000 7f149aef1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T21:45:02.595 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:02.593+0000 7f149aef1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:02.748+0000 7f2319070200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:02.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:03.007 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:02.704+0000 7f149aef1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T21:45:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:03.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:03.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:03.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:03.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:03.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:03.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:03.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:03.014 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:03.014 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:03.014 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:03.015 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:03.015 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:03.015 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:03.015 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:03.015 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:03.015 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:03.410 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:03.034+0000 7f149aef1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T21:45:03.410 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:03.130+0000 7f149aef1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T21:45:03.696 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:03.387+0000 7f2319070200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:45:03.745 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:03 smithi055 ceph-mon[29221]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:04.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:03 smithi064 ceph-mon[32765]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:04.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:03 smithi165 ceph-mon[33296]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:04.077 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:03.830+0000 7f2319070200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:45:04.077 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:03.937+0000 7f2319070200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:45:04.577 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:04.144+0000 7f2319070200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:45:04.757 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:04.431+0000 7f149aef1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T21:45:04.757 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:04.605+0000 7f149aef1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T21:45:05.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:04.679+0000 7f2319070200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:45:05.021 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:04.773+0000 7f2319070200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:45:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:05.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:05.022 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.082 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:04.781+0000 7f149aef1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T21:45:05.082 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:05.082+0000 7f149aef1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T21:45:05.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:05.082 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:05.083 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:05.327 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:05.019+0000 7f2319070200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:45:05.327 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:05.152+0000 7f2319070200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:45:05.327 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:05.247+0000 7f2319070200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:45:05.386 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:05.177+0000 7f149aef1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T21:45:05.387 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:05.387+0000 7f149aef1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T21:45:05.669 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:05.363+0000 7f2319070200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:45:05.669 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:05.668+0000 7f2319070200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:45:05.757 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:05.584+0000 7f149aef1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T21:45:06.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-mon[33296]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:45:06.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-mon[33296]: Reconfiguring daemon mgr.x on smithi055 2024-04-26T21:45:06.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2355945894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e004f5b1-6940-4e27-9d6a-7b4c81d08f27"}]: dispatch 2024-04-26T21:45:06.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2355945894' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e004f5b1-6940-4e27-9d6a-7b4c81d08f27"}]': finished 2024-04-26T21:45:06.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-mon[33296]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:45:06.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:06.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:05 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2509300965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:06.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:05 smithi055 ceph-mon[29221]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:45:06.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:05 smithi055 ceph-mon[29221]: Reconfiguring daemon mgr.x on smithi055 2024-04-26T21:45:06.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:05 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2355945894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e004f5b1-6940-4e27-9d6a-7b4c81d08f27"}]: dispatch 2024-04-26T21:45:06.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:05 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2355945894' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e004f5b1-6940-4e27-9d6a-7b4c81d08f27"}]': finished 2024-04-26T21:45:06.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:05 smithi055 ceph-mon[29221]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:45:06.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:06.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:05 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2509300965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:06.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-mon[32765]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T21:45:06.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-mon[32765]: Reconfiguring daemon mgr.x on smithi055 2024-04-26T21:45:06.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2355945894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e004f5b1-6940-4e27-9d6a-7b4c81d08f27"}]: dispatch 2024-04-26T21:45:06.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2355945894' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e004f5b1-6940-4e27-9d6a-7b4c81d08f27"}]': finished 2024-04-26T21:45:06.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-mon[32765]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T21:45:06.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:06.212 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:05 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2509300965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:06.507 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:06.211+0000 7f149aef1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T21:45:06.577 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:06 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:06.270+0000 7f2319070200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:45:06.577 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 21:45:06 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z[35143]: 2024-04-26T21:45:06.485+0000 7f2319070200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:45:06.932 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:06.646+0000 7f149aef1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T21:45:06.932 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:06.746+0000 7f149aef1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T21:45:06.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:06.939+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f99a0052020 con 0x7f99b0102380 2024-04-26T21:45:07.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:06 smithi055 ceph-mon[29221]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:06 smithi055 ceph-mon[29221]: Standby manager daemon z started 2024-04-26T21:45:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:06 smithi055 ceph-mon[29221]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:45:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:06 smithi055 ceph-mon[29221]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:06 smithi055 ceph-mon[29221]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:45:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:06 smithi055 ceph-mon[29221]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:07.257 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:06.957+0000 7f149aef1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T21:45:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-mon[32765]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-mon[32765]: Standby manager daemon z started 2024-04-26T21:45:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-mon[32765]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:45:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-mon[32765]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:07.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-mon[32765]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:45:07.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:06 smithi064 ceph-mon[32765]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:07.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:06 smithi165 ceph-mon[33296]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:07.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:06 smithi165 ceph-mon[33296]: Standby manager daemon z started 2024-04-26T21:45:07.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:06 smithi165 ceph-mon[33296]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T21:45:07.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:06 smithi165 ceph-mon[33296]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:07.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:06 smithi165 ceph-mon[33296]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T21:45:07.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:06 smithi165 ceph-mon[33296]: from='mgr.? 172.21.15.165:0/3823685256' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:07.757 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:07.499+0000 7f149aef1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T21:45:07.757 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:07.594+0000 7f149aef1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T21:45:08.200 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:07 smithi064 ceph-mon[32765]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-26T21:45:08.200 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:45:08.201 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:07.844+0000 7f149aef1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T21:45:08.201 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:07.984+0000 7f149aef1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T21:45:08.201 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:08.082+0000 7f149aef1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T21:45:08.201 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:08.200+0000 7f149aef1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T21:45:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:07 smithi165 ceph-mon[33296]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-26T21:45:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:45:08.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:07 smithi055 ceph-mon[29221]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-26T21:45:08.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T21:45:08.949 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:08.511+0000 7f149aef1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T21:45:09.177 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:08 smithi055 ceph-mon[29221]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:08 smithi064 ceph-mon[32765]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:09.257 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:09.137+0000 7f149aef1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T21:45:09.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:08 smithi165 ceph-mon[33296]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:09.757 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 21:45:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y[34160]: 2024-04-26T21:45:09.357+0000 7f149aef1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T21:45:09.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:09.961+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f99a0016370 con 0x7f99b0102380 2024-04-26T21:45:10.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:09 smithi064 ceph-mon[32765]: Standby manager daemon y started 2024-04-26T21:45:10.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:09 smithi064 ceph-mon[32765]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:45:10.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:09 smithi064 ceph-mon[32765]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:10.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:09 smithi064 ceph-mon[32765]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:45:10.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:09 smithi064 ceph-mon[32765]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:10.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:09 smithi165 ceph-mon[33296]: Standby manager daemon y started 2024-04-26T21:45:10.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:09 smithi165 ceph-mon[33296]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:45:10.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:09 smithi165 ceph-mon[33296]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:10.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:09 smithi165 ceph-mon[33296]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:45:10.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:09 smithi165 ceph-mon[33296]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:10.360 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:09 smithi055 ceph-mon[29221]: Standby manager daemon y started 2024-04-26T21:45:10.360 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:09 smithi055 ceph-mon[29221]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T21:45:10.360 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:09 smithi055 ceph-mon[29221]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T21:45:10.360 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:09 smithi055 ceph-mon[29221]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T21:45:10.360 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:09 smithi055 ceph-mon[29221]: from='mgr.? 172.21.15.64:0/1874049583' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T21:45:11.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:11.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-26T21:45:11.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:45:11.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:11.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:11.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:11.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:11.252 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:11.253 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-26T21:45:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:45:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:11.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:11.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:11.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:11.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-26T21:45:11.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T21:45:11.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:11.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:11.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:11.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:11.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:11.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:13.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:12 smithi055 ceph-mon[29221]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:13.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:12 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:45:13.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:12 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:12 smithi064 ceph-mon[32765]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:12 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:45:13.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:12 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:13.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:12 smithi165 ceph-mon[33296]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:13.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:12 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T21:45:13.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:12 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:14.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:13 smithi055 ceph-mon[29221]: Deploying daemon osd.0 on smithi055 2024-04-26T21:45:14.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:13 smithi064 ceph-mon[32765]: Deploying daemon osd.0 on smithi055 2024-04-26T21:45:14.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:13 smithi165 ceph-mon[33296]: Deploying daemon osd.0 on smithi055 2024-04-26T21:45:15.126 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:14 smithi055 ceph-mon[29221]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:15.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:14 smithi064 ceph-mon[32765]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:15.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:14 smithi165 ceph-mon[33296]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:17.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:16 smithi064 ceph-mon[32765]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:17.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:16 smithi165 ceph-mon[33296]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:17.418 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:16 smithi055 ceph-mon[29221]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:19.241 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:18 smithi055 ceph-mon[29221]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:19.241 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:19.241 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:19.241 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:18 smithi064 ceph-mon[32765]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:19.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:18 smithi165 ceph-mon[33296]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:19.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:19.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:19.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:21.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:20 smithi064 ceph-mon[32765]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:21.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:20 smithi165 ceph-mon[33296]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:21.345 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:20 smithi055 ceph-mon[29221]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:21.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.411+0000 7f99ad7fa700 1 -- 172.21.15.55:0/403100507 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f99b0063860 con 0x7f9998039890 2024-04-26T21:45:21.413 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 0 on host 'smithi055' 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9998039890 msgr2=0x7f999803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9998039890 0x7f999803bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000c00 tx=0x7f99a4005fb0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0102380 msgr2=0x7f99b006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0102380 0x7f99b006c670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e950 tx=0x7f99a000ec60 comp rx=0 tx=0).stop 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 shutdown_connections 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99b0103f20 0x7f99b01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99b0102380 0x7f99b006c670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9998039890 0x7f999803bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 --2- 172.21.15.55:0/403100507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b0103580 0x7f99b019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 >> 172.21.15.55:0/403100507 conn(0x7f99b00fd940 msgr2=0x7f99b0106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 shutdown_connections 2024-04-26T21:45:21.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:45:21.414+0000 7f99b69a3700 1 -- 172.21.15.55:0/403100507 wait complete. 2024-04-26T21:45:22.390 DEBUG:teuthology.orchestra.run.smithi055:osd.0> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T21:45:22.393 INFO:tasks.cephadm:Deploying osd.1 on smithi064 with /dev/vg_nvme/lv_4... 2024-04-26T21:45:22.393 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:45:22.646 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:45:22.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:22 smithi055 ceph-mon[29221]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:22.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:22 smithi055 ceph-mon[29221]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:45:22.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:22 smithi064 ceph-mon[32765]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:22.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:22 smithi064 ceph-mon[32765]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:45:22.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:22 smithi165 ceph-mon[33296]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:22.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:22.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:22 smithi165 ceph-mon[33296]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T21:45:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:24 smithi055 ceph-mon[29221]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:45:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:24 smithi055 ceph-mon[29221]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:45:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:24 smithi055 ceph-mon[29221]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:45:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:24 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:24.187 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:45:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0[41522]: 2024-04-26T21:45:24.066+0000 7f5b1cbd1700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T21:45:24.188 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:45:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0[41522]: 2024-04-26T21:45:24.070+0000 7f5b189c4700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:24.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:24 smithi064 ceph-mon[32765]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:45:24.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:24 smithi064 ceph-mon[32765]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:45:24.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:24 smithi064 ceph-mon[32765]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:45:24.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:24 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:24.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:24 smithi165 ceph-mon[33296]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T21:45:24.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:24 smithi165 ceph-mon[33296]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T21:45:24.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:24 smithi165 ceph-mon[33296]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:45:24.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:24 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:45:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:45:24.466 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-04-26T21:45:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-04-26T21:45:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121218 s, 865 MB/s 2024-04-26T21:45:24.466 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-04-26T21:45:25.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-04-26T21:45:25.320 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:45:25.424 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:25 smithi055 ceph-mon[29221]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:25.424 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:25 smithi055 ceph-mon[29221]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:45:25.424 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:25 smithi055 ceph-mon[29221]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:45:25.424 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:25.424 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:25.424 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:25.443 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:25 smithi064 ceph-mon[32765]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:25.443 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:25 smithi064 ceph-mon[32765]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:45:25.443 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:25 smithi064 ceph-mon[32765]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:45:25.443 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:25.443 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:25.443 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:25.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:25 smithi165 ceph-mon[33296]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:25.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:25 smithi165 ceph-mon[33296]: from='osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:45:25.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:25 smithi165 ceph-mon[33296]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T21:45:25.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:25.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:25.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:26.414 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:45:26.414 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:45:26.414 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] boot 2024-04-26T21:45:26.414 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:45:26.414 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:26.415 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.415 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.415 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:26.415 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.415 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:26.415 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:26.415 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.415 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] boot 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:26.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:26.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: osd.0 [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] boot 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:26.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:26.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:26.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:26.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/4054576954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb01046a0 msgr2=0x7f1eb0104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:26.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/4054576954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb01046a0 0x7f1eb0104b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0099c0 tx=0x7f1e9c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/4054576954 shutdown_connections 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/4054576954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb0105040 0x7f1eb010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/4054576954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eb01034a0 0x7f1eb01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/4054576954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb01046a0 0x7f1eb0104b00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/4054576954 >> 172.21.15.64:0/4054576954 conn(0x7f1eb00fea40 msgr2=0x7f1eb0100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/4054576954 shutdown_connections 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/4054576954 wait complete. 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 Processor -- start 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- start start 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eb01034a0 0x7f1eb0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb01046a0 0x7f1eb0199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb0105040 0x7f1eb01a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb010efc0 con 0x7f1eb01034a0 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb010ee40 con 0x7f1eb0105040 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.732+0000 7f1eb6b0c700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb010f140 con 0x7f1eb01046a0 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb0105040 0x7f1eb01a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb0105040 0x7f1eb01a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37148/0 (socket says 172.21.15.64:37148) 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 -- 172.21.15.64:0/2915523839 learned_addr learned my addr 172.21.15.64:0/2915523839 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb48a8700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eb01034a0 0x7f1eb0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eaffff700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb01046a0 0x7f1eb0199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 -- 172.21.15.64:0/2915523839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb01046a0 msgr2=0x7f1eb0199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb01046a0 0x7f1eb0199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 -- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eb01034a0 msgr2=0x7f1eb0199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eb01034a0 0x7f1eb0199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 -- 172.21.15.64:0/2915523839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb01a12c0 con 0x7f1eb0105040 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb50a9700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb0105040 0x7f1eb01a0c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8007ee0 tx=0x7f1ea800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eadffb700 1 -- 172.21.15.64:0/2915523839 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800dbd0 con 0x7f1eb0105040 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eadffb700 1 -- 172.21.15.64:0/2915523839 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea800dd70 con 0x7f1eb0105040 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb01a15b0 con 0x7f1eb0105040 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eadffb700 1 -- 172.21.15.64:0/2915523839 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8005990 con 0x7f1eb0105040 2024-04-26T21:45:26.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.733+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01a1af0 con 0x7f1eb0105040 2024-04-26T21:45:26.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.734+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb006a110 con 0x7f1eb0105040 2024-04-26T21:45:26.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.735+0000 7f1eadffb700 1 -- 172.21.15.64:0/2915523839 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f1ea801a020 con 0x7f1eb0105040 2024-04-26T21:45:26.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.736+0000 7f1eadffb700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1ea00a3c30 0x7f1ea00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:26.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.736+0000 7f1eadffb700 1 -- 172.21.15.64:0/2915523839 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f1ea80ce0b0 con 0x7f1eb0105040 2024-04-26T21:45:26.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.736+0000 7f1eb48a8700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1ea00a3c30 0x7f1ea00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:26.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.737+0000 7f1eb48a8700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1ea00a3c30 0x7f1ea00a60f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0104500 tx=0x7f1ea4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:26.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.741+0000 7f1eadffb700 1 -- 172.21.15.64:0/2915523839 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1ea8010070 con 0x7f1eb0105040 2024-04-26T21:45:26.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:26.958+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1eb0063860 con 0x7f1ea00a3c30 2024-04-26T21:45:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi055 2024-04-26T21:45:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi055 to 17093M 2024-04-26T21:45:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:45:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:27.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:27.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:27.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:27.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:27.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi055 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi055 to 17093M 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi055 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi055 to 17093M 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:27.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:28.224 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:28 smithi064 ceph-mon[32765]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:28.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:28 smithi055 ceph-mon[29221]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:28.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:28 smithi165 ceph-mon[33296]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:29.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:29 smithi055 ceph-mon[29221]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:29.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:29 smithi165 ceph-mon[33296]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:29.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:29 smithi064 ceph-mon[32765]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:30.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:30 smithi064 ceph-mon[32765]: from='client.? 172.21.15.64:0/3326072884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]: dispatch 2024-04-26T21:45:30.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:30 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]: dispatch 2024-04-26T21:45:30.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:30 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]': finished 2024-04-26T21:45:30.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:30 smithi064 ceph-mon[32765]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:45:30.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:30.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:30 smithi055 ceph-mon[29221]: from='client.? 172.21.15.64:0/3326072884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]: dispatch 2024-04-26T21:45:30.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:30 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]: dispatch 2024-04-26T21:45:30.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:30 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]': finished 2024-04-26T21:45:30.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:30 smithi055 ceph-mon[29221]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:45:30.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:30.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:30 smithi165 ceph-mon[33296]: from='client.? 172.21.15.64:0/3326072884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]: dispatch 2024-04-26T21:45:30.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:30 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]: dispatch 2024-04-26T21:45:30.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:30 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6715741-0a37-48df-8dd4-d07f539706f6"}]': finished 2024-04-26T21:45:30.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:30 smithi165 ceph-mon[33296]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T21:45:30.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:31.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:31 smithi055 ceph-mon[29221]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:31.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:31 smithi055 ceph-mon[29221]: from='client.? 172.21.15.64:0/1445105014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:31.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:31 smithi064 ceph-mon[32765]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:31.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:31 smithi064 ceph-mon[32765]: from='client.? 172.21.15.64:0/1445105014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:31.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:31 smithi165 ceph-mon[33296]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:31.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:31 smithi165 ceph-mon[33296]: from='client.? 172.21.15.64:0/1445105014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:33 smithi055 ceph-mon[29221]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:33 smithi064 ceph-mon[32765]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:33.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:33 smithi165 ceph-mon[33296]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:35.380 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:35 smithi064 ceph-mon[32765]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:35.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:35 smithi055 ceph-mon[29221]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:35 smithi165 ceph-mon[33296]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:37.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:37 smithi064 ceph-mon[32765]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:37.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:45:37.431 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:37.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:37 smithi055 ceph-mon[29221]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:37.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:45:37.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:37 smithi165 ceph-mon[33296]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T21:45:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:38.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:38 smithi064 ceph-mon[32765]: Deploying daemon osd.1 on smithi064 2024-04-26T21:45:38.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:38 smithi055 ceph-mon[29221]: Deploying daemon osd.1 on smithi064 2024-04-26T21:45:38.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:38 smithi165 ceph-mon[33296]: Deploying daemon osd.1 on smithi064 2024-04-26T21:45:39.422 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:39 smithi064 ceph-mon[32765]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:39.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:39 smithi055 ceph-mon[29221]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:39.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:39 smithi165 ceph-mon[33296]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:40.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:40 smithi055 ceph-mon[29221]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:40 smithi064 ceph-mon[32765]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:40.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:40 smithi165 ceph-mon[33296]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:43.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:42 smithi165 ceph-mon[33296]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:43.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:42 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:43.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:42 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:43.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:42 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:43.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:42 smithi055 ceph-mon[29221]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:43.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:42 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:43.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:42 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:43.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:42 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:42 smithi064 ceph-mon[32765]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:45.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:44 smithi064 ceph-mon[32765]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:45.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.072+0000 7f1eadffb700 1 -- 172.21.15.64:0/2915523839 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1eb0063860 con 0x7f1ea00a3c30 2024-04-26T21:45:45.074 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 1 on host 'smithi064' 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1ea00a3c30 msgr2=0x7f1ea00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1ea00a3c30 0x7f1ea00a60f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0104500 tx=0x7f1ea4009190 comp rx=0 tx=0).stop 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb0105040 msgr2=0x7f1eb01a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb0105040 0x7f1eb01a0c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8007ee0 tx=0x7f1ea800d3b0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 shutdown_connections 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1ea00a3c30 0x7f1ea00a60f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb0105040 0x7f1eb01a0c20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1eb01034a0 0x7f1eb0199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 --2- 172.21.15.64:0/2915523839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1eb01046a0 0x7f1eb0199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 >> 172.21.15.64:0/2915523839 conn(0x7f1eb00fea40 msgr2=0x7f1eb0107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 shutdown_connections 2024-04-26T21:45:45.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:45:45.075+0000 7f1eb6b0c700 1 -- 172.21.15.64:0/2915523839 wait complete. 2024-04-26T21:45:45.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:44 smithi055 ceph-mon[29221]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:45.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:44 smithi165 ceph-mon[33296]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:45.573 DEBUG:teuthology.orchestra.run.smithi064:osd.1> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T21:45:45.575 INFO:tasks.cephadm:Deploying osd.2 on smithi165 with /dev/vg_nvme/lv_4... 2024-04-26T21:45:45.576 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T21:45:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:45:46.007 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:45:45 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1[37384]: 2024-04-26T21:45:45.719+0000 7faf226417c0 -1 osd.1 0 log_to_monitors true 2024-04-26T21:45:46.277 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.277 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.277 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.277 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.277 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:46 smithi165 ceph-mon[33296]: from='osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:46.277 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:46 smithi165 ceph-mon[33296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:46.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:46 smithi064 ceph-mon[32765]: from='osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:46.313 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:46 smithi064 ceph-mon[32765]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:46.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:46.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:46 smithi055 ceph-mon[29221]: from='osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:46.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:46 smithi055 ceph-mon[29221]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T21:45:47.291 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:47 smithi165 ceph-mon[33296]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:47.291 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:47 smithi165 ceph-mon[33296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:45:47.291 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:47 smithi165 ceph-mon[33296]: from='osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:45:47.291 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:47 smithi165 ceph-mon[33296]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:45:47.291 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:47 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:47.291 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:47 smithi165 ceph-mon[33296]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:45:47.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:47 smithi064 ceph-mon[32765]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:47.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:47 smithi064 ceph-mon[32765]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:45:47.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:47 smithi064 ceph-mon[32765]: from='osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:45:47.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:47 smithi064 ceph-mon[32765]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:45:47.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:47 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:47.376 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:47 smithi064 ceph-mon[32765]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:45:47.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:47 smithi055 ceph-mon[29221]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:47.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:47 smithi055 ceph-mon[29221]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T21:45:47.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:47 smithi055 ceph-mon[29221]: from='osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:45:47.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:47 smithi055 ceph-mon[29221]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T21:45:47.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:47 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:47.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:47 smithi055 ceph-mon[29221]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:45:47.836 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T21:45:47.836 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T21:45:47.836 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-26T21:45:47.836 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-26T21:45:47.836 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114163 s, 918 MB/s 2024-04-26T21:45:47.836 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-26T21:45:48.007 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:45:47 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1[37384]: 2024-04-26T21:45:47.679+0000 7faf17ca6700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T21:45:48.007 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:45:47 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1[37384]: 2024-04-26T21:45:47.682+0000 7faf13a99700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:45:48.516 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-04-26T21:45:48.759 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:45:48.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:48 smithi165 ceph-mon[33296]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:48.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:48 smithi165 ceph-mon[33296]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:45:48.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:48 smithi165 ceph-mon[33296]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:45:48.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:48.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:48.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:48 smithi064 ceph-mon[32765]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:48.850 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:48 smithi064 ceph-mon[32765]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:45:48.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:48 smithi064 ceph-mon[32765]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:45:48.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:48.851 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:48.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:48 smithi055 ceph-mon[29221]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T21:45:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:48 smithi055 ceph-mon[29221]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:45:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:48 smithi055 ceph-mon[29221]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T21:45:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:45:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:45:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] boot 2024-04-26T21:45:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:45:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi064 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi064 to 17088M 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:49.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.003 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] boot 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi064 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi064 to 17088M 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:50.004 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:50.005 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.005 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:50.005 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:50.005 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:50.005 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:45:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:45:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: osd.1 [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] boot 2024-04-26T21:45:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T21:45:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T21:45:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi064 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi064 to 17088M 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 -- 172.21.15.165:0/3777605194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b8103fa0 msgr2=0x7fc7b8106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 --2- 172.21.15.165:0/3777605194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b8103fa0 0x7fc7b8106390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009990 tx=0x7fc7ac009ca0 comp rx=0 tx=0).stop 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 -- 172.21.15.165:0/3777605194 shutdown_connections 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 --2- 172.21.15.165:0/3777605194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7b8109200 0x7fc7b810b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 --2- 172.21.15.165:0/3777605194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b8103fa0 0x7fc7b8106390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 --2- 172.21.15.165:0/3777605194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b81068d0 0x7fc7b8108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 -- 172.21.15.165:0/3777605194 >> 172.21.15.165:0/3777605194 conn(0x7fc7b80fd950 msgr2=0x7fc7b80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 -- 172.21.15.165:0/3777605194 shutdown_connections 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 -- 172.21.15.165:0/3777605194 wait complete. 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 Processor -- start 2024-04-26T21:45:50.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 -- start start 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8103fa0 0x7fc7b8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b81068d0 0x7fc7b8199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.318+0000 7fc7bee82700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7b8109200 0x7fc7b81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7bee82700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810ef50 con 0x7fc7b8109200 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7bee82700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810edd0 con 0x7fc7b8103fa0 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7bee82700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810f0d0 con 0x7fc7b81068d0 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b81068d0 0x7fc7b8199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b81068d0 0x7fc7b8199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42712/0 (socket says 172.21.15.165:42712) 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 -- 172.21.15.165:0/94679624 learned_addr learned my addr 172.21.15.165:0/94679624 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7bcc1e700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8103fa0 0x7fc7b8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7bd41f700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7b8109200 0x7fc7b81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 -- 172.21.15.165:0/94679624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8103fa0 msgr2=0x7fc7b8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8103fa0 0x7fc7b8199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:50.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 -- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7b8109200 msgr2=0x7fc7b81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7b8109200 0x7fc7b81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 -- 172.21.15.165:0/94679624 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b81a1270 con 0x7fc7b81068d0 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7bd41f700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7b8109200 0x7fc7b81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b7fff700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b81068d0 0x7fc7b8199b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400d9b0 tx=0x7fc7a400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b5ffb700 1 -- 172.21.15.165:0/94679624 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4017070 con 0x7fc7b81068d0 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b5ffb700 1 -- 172.21.15.165:0/94679624 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4005020 con 0x7fc7b81068d0 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b81a1560 con 0x7fc7b81068d0 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.319+0000 7fc7b5ffb700 1 -- 172.21.15.165:0/94679624 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4009b50 con 0x7fc7b81068d0 2024-04-26T21:45:50.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.320+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a1a50 con 0x7fc7b81068d0 2024-04-26T21:45:50.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.320+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b806a110 con 0x7fc7b81068d0 2024-04-26T21:45:50.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.321+0000 7fc7b5ffb700 1 -- 172.21.15.165:0/94679624 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fc7a40051d0 con 0x7fc7b81068d0 2024-04-26T21:45:50.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.322+0000 7fc7b5ffb700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7a80a3bb0 0x7fc7a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:45:50.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.322+0000 7fc7b5ffb700 1 -- 172.21.15.165:0/94679624 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fc7a40cea50 con 0x7fc7b81068d0 2024-04-26T21:45:50.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.322+0000 7fc7bcc1e700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7a80a3bb0 0x7fc7a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:45:50.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.323+0000 7fc7bcc1e700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7a80a3bb0 0x7fc7a80a6070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00b5c0 tx=0x7fc7ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:45:50.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.327+0000 7fc7b5ffb700 1 -- 172.21.15.165:0/94679624 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc7a4020020 con 0x7fc7b81068d0 2024-04-26T21:45:50.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:45:50.530+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc7b8063860 con 0x7fc7a80a3bb0 2024-04-26T21:45:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:50 smithi064 ceph-mon[32765]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:50 smithi064 ceph-mon[32765]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:45:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:51.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:50 smithi165 ceph-mon[33296]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:51.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:50 smithi165 ceph-mon[33296]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:45:51.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:51.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:51.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:51.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:50 smithi055 ceph-mon[29221]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:51.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:50 smithi055 ceph-mon[29221]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T21:45:51.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:45:51.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:45:51.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:51 smithi064 ceph-mon[32765]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:52.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:51 smithi165 ceph-mon[33296]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:52.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:51 smithi055 ceph-mon[29221]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:45:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:52 smithi064 ceph-mon[32765]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:52 smithi165 ceph-mon[33296]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:53.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:52 smithi055 ceph-mon[29221]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:53 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/4271671070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]: dispatch 2024-04-26T21:45:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:53 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]: dispatch 2024-04-26T21:45:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:53 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]': finished 2024-04-26T21:45:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:53 smithi064 ceph-mon[32765]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:45:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:45:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:53 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/4271671070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]: dispatch 2024-04-26T21:45:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:53 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]: dispatch 2024-04-26T21:45:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:53 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]': finished 2024-04-26T21:45:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:53 smithi165 ceph-mon[33296]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:45:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:45:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:53 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/4271671070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]: dispatch 2024-04-26T21:45:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:53 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]: dispatch 2024-04-26T21:45:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:53 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15bb5be1-55c9-4a43-8253-fcefc1d5041f"}]': finished 2024-04-26T21:45:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:53 smithi055 ceph-mon[29221]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T21:45:54.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:45:55.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:54 smithi064 ceph-mon[32765]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:55.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:54 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/1570017812' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:55.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:54 smithi165 ceph-mon[33296]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:55.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:54 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/1570017812' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:55.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:54 smithi055 ceph-mon[29221]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:55.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:54 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/1570017812' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:45:57.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:56 smithi064 ceph-mon[32765]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:57.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:56 smithi165 ceph-mon[33296]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:56 smithi055 ceph-mon[29221]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:59.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:45:58 smithi064 ceph-mon[32765]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:59.043 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:45:58 smithi165 ceph-mon[33296]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:45:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:45:58 smithi055 ceph-mon[29221]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:00 smithi064 ceph-mon[32765]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:46:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:01.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:00 smithi165 ceph-mon[33296]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:01.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:46:01.054 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:00 smithi055 ceph-mon[29221]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T21:46:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:02.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:01 smithi064 ceph-mon[32765]: Deploying daemon osd.2 on smithi165 2024-04-26T21:46:02.057 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:01 smithi165 ceph-mon[33296]: Deploying daemon osd.2 on smithi165 2024-04-26T21:46:02.087 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:01 smithi055 ceph-mon[29221]: Deploying daemon osd.2 on smithi165 2024-04-26T21:46:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:02 smithi064 ceph-mon[32765]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:03.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:02 smithi165 ceph-mon[33296]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:03.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:02 smithi055 ceph-mon[29221]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:04 smithi064 ceph-mon[32765]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:05.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:04 smithi165 ceph-mon[33296]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:05.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:04 smithi055 ceph-mon[29221]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:06.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:06 smithi165 ceph-mon[33296]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:06.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:06 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:06.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:06 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:06.922 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:06 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:06 smithi064 ceph-mon[32765]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:06 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:06 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:06 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:07.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:06 smithi055 ceph-mon[29221]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:06 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:06 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:06 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:08 smithi064 ceph-mon[32765]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:08 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:08 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:08 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:09.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:08 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:09.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:08 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:08 smithi165 ceph-mon[33296]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:09.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:08 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:08 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:08 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:09.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:08 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:09.023 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:08 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:08 smithi055 ceph-mon[29221]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:09.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:08 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:08 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:08 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:09.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:08 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:09.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:08 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:09.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.697+0000 7fc7b5ffb700 1 -- 172.21.15.165:0/94679624 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc7b8063860 con 0x7fc7a80a3bb0 2024-04-26T21:46:09.699 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 2 on host 'smithi165' 2024-04-26T21:46:09.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7a80a3bb0 msgr2=0x7fc7a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7a80a3bb0 0x7fc7a80a6070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00b5c0 tx=0x7fc7ac005fb0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b81068d0 msgr2=0x7fc7b8199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:09.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b81068d0 0x7fc7b8199b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400d9b0 tx=0x7fc7a400dcc0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 shutdown_connections 2024-04-26T21:46:09.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7b8109200 0x7fc7b81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7a80a3bb0 0x7fc7a80a6070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8103fa0 0x7fc7b8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 --2- 172.21.15.165:0/94679624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7b81068d0 0x7fc7b8199b50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:09.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.699+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 >> 172.21.15.165:0/94679624 conn(0x7fc7b80fd950 msgr2=0x7fc7b8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:09.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.700+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 shutdown_connections 2024-04-26T21:46:09.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:46:09.700+0000 7fc7bee82700 1 -- 172.21.15.165:0/94679624 wait complete. 2024-04-26T21:46:10.514 DEBUG:teuthology.orchestra.run.smithi165:osd.2> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T21:46:10.516 INFO:tasks.cephadm:Deploying osd.3 on smithi055 with /dev/vg_nvme/lv_3... 2024-04-26T21:46:10.517 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:46:10.739 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:10 smithi165 ceph-mon[33296]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:10.739 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:10.739 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:10.739 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:10.739 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:10 smithi165 ceph-mon[33296]: from='osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:10.739 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:10 smithi165 ceph-mon[33296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:10.774 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:46:10.887 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:10 smithi055 ceph-mon[29221]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:10.887 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:10.887 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:10.887 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:10.887 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:10 smithi055 ceph-mon[29221]: from='osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:10.887 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:10 smithi055 ceph-mon[29221]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:10 smithi064 ceph-mon[32765]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:11.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:11.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:10 smithi064 ceph-mon[32765]: from='osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:11.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:10 smithi064 ceph-mon[32765]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T21:46:12.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:11 smithi064 ceph-mon[32765]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:46:12.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:11 smithi064 ceph-mon[32765]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:46:12.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:11 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:12.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:11 smithi064 ceph-mon[32765]: from='osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:46:12.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:11 smithi064 ceph-mon[32765]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:46:12.036 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:11 smithi055 ceph-mon[29221]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:46:12.036 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:11 smithi055 ceph-mon[29221]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:46:12.036 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:11 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:12.036 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:11 smithi055 ceph-mon[29221]: from='osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:46:12.036 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:11 smithi055 ceph-mon[29221]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:46:12.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:11 smithi165 ceph-mon[33296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T21:46:12.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:11 smithi165 ceph-mon[33296]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T21:46:12.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:11 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:12.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:11 smithi165 ceph-mon[33296]: from='osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:46:12.039 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:11 smithi165 ceph-mon[33296]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:46:12.039 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2[38444]: 2024-04-26T21:46:11.782+0000 7f864d482700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T21:46:12.040 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2[38444]: 2024-04-26T21:46:11.786+0000 7f8649275700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:46:12.787 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:46:12.787 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:46:12.787 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-04-26T21:46:12.787 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-04-26T21:46:12.787 INFO:teuthology.orchestra.run.smithi055.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0153563 s, 683 MB/s 2024-04-26T21:46:12.788 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-04-26T21:46:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:12 smithi165 ceph-mon[33296]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:12 smithi165 ceph-mon[33296]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:46:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:12 smithi165 ceph-mon[33296]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:46:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:12 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:13.117 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:12 smithi055 ceph-mon[29221]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:13.117 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:12 smithi055 ceph-mon[29221]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:46:13.117 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:12 smithi055 ceph-mon[29221]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:46:13.117 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:12 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:12 smithi064 ceph-mon[32765]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T21:46:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:12 smithi064 ceph-mon[32765]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:46:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:12 smithi064 ceph-mon[32765]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T21:46:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:12 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:13.424 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi055:vg_nvme/lv_3 2024-04-26T21:46:13.691 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:46:13.974 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:46:13.974 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:46:13.974 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] boot 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:13.975 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] boot 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:14.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:14.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:14.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: osd.2 [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] boot 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:14.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:15.029 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:14 smithi055 ceph-mon[29221]: pgmap v80: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:15.029 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:14 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi165 2024-04-26T21:46:15.030 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:14 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-26T21:46:15.030 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:15.030 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:14 smithi055 ceph-mon[29221]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:46:15.030 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:15.030 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:46:14 smithi055 sudo[45763]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:15.030 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:46:14 smithi055 sudo[45763]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.030 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:46:14 smithi055 sudo[45763]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.030 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:46:14 smithi055 sudo[45763]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.030 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:46:14 smithi055 sudo[45766]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:15.030 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:46:14 smithi055 sudo[45766]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.030 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:46:14 smithi055 sudo[45766]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.030 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 21:46:14 smithi055 sudo[45766]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.056 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42178]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:15.056 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42178]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.056 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42178]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.056 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42178]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.056 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:14 smithi165 ceph-mon[33296]: pgmap v80: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:15.056 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:14 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi165 2024-04-26T21:46:15.056 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:14 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-26T21:46:15.056 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:15.056 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:14 smithi165 ceph-mon[33296]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:46:15.057 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:15.184 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:14 smithi064 ceph-mon[32765]: pgmap v80: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:15.184 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:14 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi165 2024-04-26T21:46:15.184 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:14 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi165 to 17088M 2024-04-26T21:46:15.184 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:15.184 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:14 smithi064 ceph-mon[32765]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T21:46:15.184 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T21:46:15.184 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:46:14 smithi064 sudo[41060]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:15.184 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:46:14 smithi064 sudo[41060]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.184 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:46:14 smithi064 sudo[41060]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.184 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:46:14 smithi064 sudo[41060]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.185 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41063]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:15.185 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41063]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.185 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41063]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.185 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41063]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.281+0000 7f899858c700 1 -- 172.21.15.55:0/4252854458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 msgr2=0x7f8990106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:15.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.281+0000 7f899858c700 1 --2- 172.21.15.55:0/4252854458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 0x7f8990106300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f898c009990 tx=0x7f898c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:46:15.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.282+0000 7f899858c700 1 -- 172.21.15.55:0/4252854458 shutdown_connections 2024-04-26T21:46:15.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.282+0000 7f899858c700 1 --2- 172.21.15.55:0/4252854458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990109170 0x7f899010b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:15.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.282+0000 7f899858c700 1 --2- 172.21.15.55:0/4252854458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 0x7f8990106300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:15.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.282+0000 7f899858c700 1 --2- 172.21.15.55:0/4252854458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8990106840 0x7f8990108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:15.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.282+0000 7f899858c700 1 -- 172.21.15.55:0/4252854458 >> 172.21.15.55:0/4252854458 conn(0x7f89900fd940 msgr2=0x7f89900ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:15.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.282+0000 7f899858c700 1 -- 172.21.15.55:0/4252854458 shutdown_connections 2024-04-26T21:46:15.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 sudo[45773]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:15.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 sudo[45773]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.284 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 sudo[45773]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.284 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 sudo[45773]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.284 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 sudo[45776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:15.284 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 sudo[45776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.284 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 sudo[45776]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.284 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 sudo[45776]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.282+0000 7f899858c700 1 -- 172.21.15.55:0/4252854458 wait complete. 2024-04-26T21:46:15.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f899858c700 1 Processor -- start 2024-04-26T21:46:15.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f899858c700 1 -- start start 2024-04-26T21:46:15.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f899858c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 0x7f89901995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:15.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f899858c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8990106840 0x7f8990199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:15.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f899858c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990109170 0x7f89901a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:15.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f899858c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f899010ef00 con 0x7f8990103f10 2024-04-26T21:46:15.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f899858c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f899010ed80 con 0x7f8990106840 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f899858c700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f899010f080 con 0x7f8990109170 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f8996328700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 0x7f89901995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f8996328700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 0x7f89901995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36398/0 (socket says 172.21.15.55:36398) 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f8996328700 1 -- 172.21.15.55:0/3097853309 learned_addr learned my addr 172.21.15.55:0/3097853309 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.283+0000 7f8996b29700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990109170 0x7f89901a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f8995b27700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8990106840 0x7f8990199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f8996328700 1 -- 172.21.15.55:0/3097853309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990109170 msgr2=0x7f89901a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f8996328700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990109170 0x7f89901a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f8996328700 1 -- 172.21.15.55:0/3097853309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8990106840 msgr2=0x7f8990199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f8996328700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8990106840 0x7f8990199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f8996328700 1 -- 172.21.15.55:0/3097853309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89901a11a0 con 0x7f8990103f10 2024-04-26T21:46:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f8996b29700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990109170 0x7f89901a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:15.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f8996328700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 0x7f89901995e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f898c0038a0 tx=0x7f898c0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:15.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f89877fe700 1 -- 172.21.15.55:0/3097853309 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c027070 con 0x7f8990103f10 2024-04-26T21:46:15.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f89877fe700 1 -- 172.21.15.55:0/3097853309 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898c022470 con 0x7f8990103f10 2024-04-26T21:46:15.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89901a1430 con 0x7f8990103f10 2024-04-26T21:46:15.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f89877fe700 1 -- 172.21.15.55:0/3097853309 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00f6c0 con 0x7f8990103f10 2024-04-26T21:46:15.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.284+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901a1880 con 0x7f8990103f10 2024-04-26T21:46:15.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.285+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899006a110 con 0x7f8990103f10 2024-04-26T21:46:15.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.287+0000 7f89877fe700 1 -- 172.21.15.55:0/3097853309 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f898c02a070 con 0x7f8990103f10 2024-04-26T21:46:15.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.288+0000 7f89877fe700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f897c0a3bb0 0x7f897c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:15.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.288+0000 7f8995b27700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f897c0a3bb0 0x7f897c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:15.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.288+0000 7f89877fe700 1 -- 172.21.15.55:0/3097853309 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f898c0d0340 con 0x7f8990103f10 2024-04-26T21:46:15.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.288+0000 7f8995b27700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f897c0a3bb0 0x7f897c0a6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8980005fd0 tx=0x7f898000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:15.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.292+0000 7f89877fe700 1 -- 172.21.15.55:0/3097853309 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f898c013070 con 0x7f8990103f10 2024-04-26T21:46:15.327 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42181]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:15.327 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42181]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.327 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42181]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.327 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42181]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42184]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42184]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42184]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42184]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 sudo[42187]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41066]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T21:46:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41066]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41066]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41066]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41069]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T21:46:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41069]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T21:46:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41069]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T21:46:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 sudo[41069]: pam_unix(sudo:session): session closed for user root 2024-04-26T21:46:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:15.510+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8990063860 con 0x7f897c0a3bb0 2024-04-26T21:46:16.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:16.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:46:16.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.079 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:16.079 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:16.079 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:16.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:16.108+0000 7f89877fe700 1 -- 172.21.15.55:0/3097853309 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f898c022620 con 0x7f8990103f10 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.110 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:16.111 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:16.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T21:46:16.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T21:46:16.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T21:46:16.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T21:46:16.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T21:46:16.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T21:46:16.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:16.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:16.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:17.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:16 smithi165 ceph-mon[33296]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:17.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:16 smithi165 ceph-mon[33296]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:17.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:16 smithi165 ceph-mon[33296]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:46:17.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:16 smithi165 ceph-mon[33296]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:46:17.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:16 smithi055 ceph-mon[29221]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:17.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:16 smithi055 ceph-mon[29221]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:17.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:16 smithi055 ceph-mon[29221]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:46:17.167 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:16 smithi055 ceph-mon[29221]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:46:17.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:16 smithi064 ceph-mon[32765]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:17.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:16 smithi064 ceph-mon[32765]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:17.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:16 smithi064 ceph-mon[32765]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T21:46:17.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:16 smithi064 ceph-mon[32765]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T21:46:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:18 smithi165 ceph-mon[33296]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:18 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1508315289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcc948f-ba34-488a-be7b-ec4579b3a4c6"}]: dispatch 2024-04-26T21:46:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:18 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1508315289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fcc948f-ba34-488a-be7b-ec4579b3a4c6"}]': finished 2024-04-26T21:46:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:18 smithi165 ceph-mon[33296]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:46:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:19.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:18 smithi055 ceph-mon[29221]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:18 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1508315289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcc948f-ba34-488a-be7b-ec4579b3a4c6"}]: dispatch 2024-04-26T21:46:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:18 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1508315289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fcc948f-ba34-488a-be7b-ec4579b3a4c6"}]': finished 2024-04-26T21:46:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:18 smithi055 ceph-mon[29221]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:46:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:18 smithi064 ceph-mon[32765]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:18 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1508315289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcc948f-ba34-488a-be7b-ec4579b3a4c6"}]: dispatch 2024-04-26T21:46:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:18 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1508315289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fcc948f-ba34-488a-be7b-ec4579b3a4c6"}]': finished 2024-04-26T21:46:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:18 smithi064 ceph-mon[32765]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T21:46:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:20.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:19 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3679245370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:20.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:19 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3679245370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:20.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:19 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3679245370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:21.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:20 smithi165 ceph-mon[33296]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:21.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:20 smithi055 ceph-mon[29221]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:21.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:20 smithi064 ceph-mon[32765]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:22.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:22 smithi055 ceph-mon[29221]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:23.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:22 smithi064 ceph-mon[32765]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:23.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:22 smithi165 ceph-mon[33296]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:25.075 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:24 smithi055 ceph-mon[29221]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:25.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:24 smithi064 ceph-mon[32765]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:25.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:24 smithi165 ceph-mon[33296]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:26.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:46:26.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:26.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:46:26.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:26.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T21:46:26.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:26.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:26 smithi055 ceph-mon[29221]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:26.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:26 smithi055 ceph-mon[29221]: Deploying daemon osd.3 on smithi055 2024-04-26T21:46:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:26 smithi064 ceph-mon[32765]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:26 smithi064 ceph-mon[32765]: Deploying daemon osd.3 on smithi055 2024-04-26T21:46:27.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:26 smithi165 ceph-mon[33296]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:26 smithi165 ceph-mon[33296]: Deploying daemon osd.3 on smithi055 2024-04-26T21:46:29.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:28 smithi165 ceph-mon[33296]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:29.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:28 smithi055 ceph-mon[29221]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:29.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:28 smithi064 ceph-mon[32765]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:31.112 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:30 smithi055 ceph-mon[29221]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:31.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:30 smithi064 ceph-mon[32765]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:31.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:30 smithi165 ceph-mon[33296]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:32.374 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:32.374 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:32.374 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:32.374 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:32 smithi055 ceph-mon[29221]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:32.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:32.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:32.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:32.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:32 smithi064 ceph-mon[32765]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:32.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:32.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:32.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:32.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:32 smithi165 ceph-mon[33296]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:34.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:34 smithi055 ceph-mon[29221]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:34.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:34.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:34.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:34.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:34.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:34 smithi064 ceph-mon[32765]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:34.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:34 smithi165 ceph-mon[33296]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:34.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:34.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:34.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:34.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:34.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:35.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.085+0000 7f89877fe700 1 -- 172.21.15.55:0/3097853309 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8990063860 con 0x7f897c0a3bb0 2024-04-26T21:46:35.087 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 3 on host 'smithi055' 2024-04-26T21:46:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f897c0a3bb0 msgr2=0x7f897c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f897c0a3bb0 0x7f897c0a6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8980005fd0 tx=0x7f898000a560 comp rx=0 tx=0).stop 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 msgr2=0x7f89901995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 0x7f89901995e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f898c0038a0 tx=0x7f898c0039f0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 shutdown_connections 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8990109170 0x7f89901a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8990103f10 0x7f89901995e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f897c0a3bb0 0x7f897c0a6070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 --2- 172.21.15.55:0/3097853309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8990106840 0x7f8990199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 >> 172.21.15.55:0/3097853309 conn(0x7f89900fd940 msgr2=0x7f89901047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 shutdown_connections 2024-04-26T21:46:35.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:46:35.088+0000 7f899858c700 1 -- 172.21.15.55:0/3097853309 wait complete. 2024-04-26T21:46:35.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:35 smithi055 ceph-mon[29221]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:46:35.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:35.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:35.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:35.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:35 smithi064 ceph-mon[32765]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:46:35.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:35.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:35.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:35.793 DEBUG:teuthology.orchestra.run.smithi055:osd.3> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T21:46:35.796 INFO:tasks.cephadm:Deploying osd.4 on smithi064 with /dev/vg_nvme/lv_3... 2024-04-26T21:46:35.796 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:46:35.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:35 smithi165 ceph-mon[33296]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T21:46:35.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:35.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:35.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:36.051 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:46:36.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:36 smithi064 ceph-mon[32765]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:36.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:36 smithi064 ceph-mon[32765]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:46:36.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:36 smithi064 ceph-mon[32765]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:46:36.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:36 smithi064 ceph-mon[32765]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:46:36.494 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:36.651 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:36 smithi055 ceph-mon[29221]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:36.651 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:36 smithi055 ceph-mon[29221]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:46:36.651 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:36 smithi055 ceph-mon[29221]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:46:36.651 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:36 smithi055 ceph-mon[29221]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:46:36.651 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:36.651 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 21:46:36 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3[48042]: 2024-04-26T21:46:36.378+0000 7fe7e742b700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T21:46:36.652 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 21:46:36 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3[48042]: 2024-04-26T21:46:36.383+0000 7fe7e321e700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:46:36.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:36 smithi165 ceph-mon[33296]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:36.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:36 smithi165 ceph-mon[33296]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T21:46:36.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:36 smithi165 ceph-mon[33296]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T21:46:36.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:36 smithi165 ceph-mon[33296]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:46:36.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:37.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:37 smithi055 ceph-mon[29221]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:46:37.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:37 smithi055 ceph-mon[29221]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:46:37.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:37.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:37 smithi064 ceph-mon[32765]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:46:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:37 smithi064 ceph-mon[32765]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:46:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:37.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:37 smithi165 ceph-mon[33296]: from='osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:46:37.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:37 smithi165 ceph-mon[33296]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T21:46:37.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:37.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:46:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:46:37.968 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-04-26T21:46:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-04-26T21:46:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120864 s, 868 MB/s 2024-04-26T21:46:37.968 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-04-26T21:46:38.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:38 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:46:38.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:38 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:46:38.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:38 smithi064 ceph-mon[32765]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:38.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:38 smithi064 ceph-mon[32765]: osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] boot 2024-04-26T21:46:38.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:38 smithi064 ceph-mon[32765]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:46:38.632 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.659 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-04-26T21:46:38.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:38 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:46:38.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:38 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:46:38.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:38 smithi055 ceph-mon[29221]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:38.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:38 smithi055 ceph-mon[29221]: osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] boot 2024-04-26T21:46:38.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:38 smithi055 ceph-mon[29221]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:46:38.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:38 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:46:38.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:38 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:46:38.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:38 smithi165 ceph-mon[33296]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T21:46:38.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:38 smithi165 ceph-mon[33296]: osd.3 [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] boot 2024-04-26T21:46:38.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:38 smithi165 ceph-mon[33296]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T21:46:38.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T21:46:38.909 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:46:39.652 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:46:39.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi055 2024-04-26T21:46:39.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:39.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:39.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi055 to 8546M 2024-04-26T21:46:39.653 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.654 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:39.654 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:39.654 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi055 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi055 to 8546M 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi055 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi055 to 8546M 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:39.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 -- 172.21.15.64:0/1093397040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6901046a0 msgr2=0x7fa690104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 --2- 172.21.15.64:0/1093397040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6901046a0 0x7fa690104b00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6840099c0 tx=0x7fa684009cd0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 -- 172.21.15.64:0/1093397040 shutdown_connections 2024-04-26T21:46:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 --2- 172.21.15.64:0/1093397040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa690105040 0x7fa69010b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 --2- 172.21.15.64:0/1093397040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6901034a0 0x7fa6901038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 --2- 172.21.15.64:0/1093397040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6901046a0 0x7fa690104b00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 -- 172.21.15.64:0/1093397040 >> 172.21.15.64:0/1093397040 conn(0x7fa6900fea40 msgr2=0x7fa690100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:46:40.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 -- 172.21.15.64:0/1093397040 shutdown_connections 2024-04-26T21:46:40.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 -- 172.21.15.64:0/1093397040 wait complete. 2024-04-26T21:46:40.566 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:40 smithi064 ceph-mon[32765]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:40.566 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:40 smithi064 ceph-mon[32765]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 Processor -- start 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.564+0000 7fa696369700 1 -- start start 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa696369700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6901034a0 0x7fa69019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa696369700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6901046a0 0x7fa6901a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa696369700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa690105040 0x7fa6901a7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa696369700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa69010ef40 con 0x7fa6901034a0 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa696369700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa69010edc0 con 0x7fa690105040 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa696369700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa69010f0c0 con 0x7fa6901046a0 2024-04-26T21:46:40.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa690105040 0x7fa6901a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa690105040 0x7fa6901a7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55544/0 (socket says 172.21.15.64:55544) 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 -- 172.21.15.64:0/874151212 learned_addr learned my addr 172.21.15.64:0/874151212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa68ffff700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6901034a0 0x7fa69019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa68f7fe700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6901046a0 0x7fa6901a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 -- 172.21.15.64:0/874151212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6901046a0 msgr2=0x7fa6901a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6901046a0 0x7fa6901a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 -- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6901034a0 msgr2=0x7fa69019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6901034a0 0x7fa69019fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 -- 172.21.15.64:0/874151212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6901a7820 con 0x7fa690105040 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa68f7fe700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6901046a0 0x7fa6901a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.565+0000 7fa694906700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa690105040 0x7fa6901a7180 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa688007ee0 tx=0x7fa68800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.566+0000 7fa68d7fa700 1 -- 172.21.15.64:0/874151212 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68800dbd0 con 0x7fa690105040 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.566+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6901a7b10 con 0x7fa690105040 2024-04-26T21:46:40.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.566+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6901a80a0 con 0x7fa690105040 2024-04-26T21:46:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.566+0000 7fa68d7fa700 1 -- 172.21.15.64:0/874151212 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa68800dd70 con 0x7fa690105040 2024-04-26T21:46:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.566+0000 7fa68d7fa700 1 -- 172.21.15.64:0/874151212 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688005990 con 0x7fa690105040 2024-04-26T21:46:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.567+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa69006a110 con 0x7fa690105040 2024-04-26T21:46:40.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.568+0000 7fa68d7fa700 1 -- 172.21.15.64:0/874151212 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa68801a020 con 0x7fa690105040 2024-04-26T21:46:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.569+0000 7fa68d7fa700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6780a3cc0 0x7fa6780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:46:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.569+0000 7fa68d7fa700 1 -- 172.21.15.64:0/874151212 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fa6880cef60 con 0x7fa690105040 2024-04-26T21:46:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.569+0000 7fa68ffff700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6780a3cc0 0x7fa6780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:46:40.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.570+0000 7fa68ffff700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6780a3cc0 0x7fa6780a6180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa690104500 tx=0x7fa68000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:46:40.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.573+0000 7fa68d7fa700 1 -- 172.21.15.64:0/874151212 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa688010070 con 0x7fa690105040 2024-04-26T21:46:40.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:40 smithi055 ceph-mon[29221]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:40.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:40 smithi055 ceph-mon[29221]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:46:40.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:40 smithi165 ceph-mon[33296]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:40.748 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:40 smithi165 ceph-mon[33296]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T21:46:40.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:46:40.789+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa690063920 con 0x7fa6780a3cc0 2024-04-26T21:46:41.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:41 smithi055 ceph-mon[29221]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:46:41.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:41 smithi055 ceph-mon[29221]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:41.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:41 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:41.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:41 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:41.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:41 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:41.734 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:41 smithi064 ceph-mon[32765]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:46:41.734 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:41 smithi064 ceph-mon[32765]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:41.734 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:41 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:41.734 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:41 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:41.734 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:41 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:41.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:41 smithi165 ceph-mon[33296]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T21:46:41.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:41 smithi165 ceph-mon[33296]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:46:41.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:41 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:46:41.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:41 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:46:41.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:41 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:42.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:42 smithi055 ceph-mon[29221]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:42.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:42 smithi055 ceph-mon[29221]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:46:42.688 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:42 smithi064 ceph-mon[32765]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:42.688 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:42 smithi064 ceph-mon[32765]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:46:42.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:42 smithi165 ceph-mon[33296]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:42 smithi165 ceph-mon[33296]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T21:46:44.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:44 smithi064 ceph-mon[32765]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:44.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:44 smithi064 ceph-mon[32765]: from='client.? 172.21.15.64:0/1629767293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]: dispatch 2024-04-26T21:46:44.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:44 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]: dispatch 2024-04-26T21:46:44.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:44 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]': finished 2024-04-26T21:46:44.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:44 smithi064 ceph-mon[32765]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:46:44.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:44 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:46:44.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:44 smithi165 ceph-mon[33296]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:44.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:44 smithi165 ceph-mon[33296]: from='client.? 172.21.15.64:0/1629767293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]: dispatch 2024-04-26T21:46:44.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:44 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]: dispatch 2024-04-26T21:46:44.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:44 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]': finished 2024-04-26T21:46:44.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:44 smithi165 ceph-mon[33296]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:46:44.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:44 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:46:44.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:44 smithi055 ceph-mon[29221]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:44.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:44 smithi055 ceph-mon[29221]: from='client.? 172.21.15.64:0/1629767293' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]: dispatch 2024-04-26T21:46:44.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:44 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]: dispatch 2024-04-26T21:46:44.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:44 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "daf304fa-67ee-45a4-a3ca-8140292c27ef"}]': finished 2024-04-26T21:46:44.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:44 smithi055 ceph-mon[29221]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T21:46:44.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:44 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:46:45.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:45 smithi064 ceph-mon[32765]: from='client.? 172.21.15.64:0/1858383411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:45.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:45 smithi165 ceph-mon[33296]: from='client.? 172.21.15.64:0/1858383411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:45.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:45 smithi055 ceph-mon[29221]: from='client.? 172.21.15.64:0/1858383411' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:46:46.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:46 smithi064 ceph-mon[32765]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-26T21:46:46.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:46 smithi165 ceph-mon[33296]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-26T21:46:46.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:46 smithi055 ceph-mon[29221]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-26T21:46:48.737 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:48 smithi064 ceph-mon[32765]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:46:48.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:48 smithi165 ceph-mon[33296]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:46:48.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:48 smithi055 ceph-mon[29221]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-26T21:46:50.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:50 smithi064 ceph-mon[32765]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:46:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:50 smithi165 ceph-mon[33296]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:46:50.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:50 smithi055 ceph-mon[29221]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:46:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:46:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:51.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:46:51.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:51.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T21:46:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:52.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:52 smithi064 ceph-mon[32765]: Deploying daemon osd.4 on smithi064 2024-04-26T21:46:52.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:52 smithi064 ceph-mon[32765]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T21:46:52.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:52 smithi165 ceph-mon[33296]: Deploying daemon osd.4 on smithi064 2024-04-26T21:46:52.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:52 smithi165 ceph-mon[33296]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T21:46:52.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:52 smithi055 ceph-mon[29221]: Deploying daemon osd.4 on smithi064 2024-04-26T21:46:52.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:52 smithi055 ceph-mon[29221]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T21:46:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:54 smithi064 ceph-mon[32765]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:46:54.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:54 smithi165 ceph-mon[33296]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:46:54.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:54 smithi055 ceph-mon[29221]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:46:56.621 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:56 smithi064 ceph-mon[32765]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-26T21:46:56.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:56 smithi165 ceph-mon[33296]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-26T21:46:56.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:56 smithi055 ceph-mon[29221]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-26T21:46:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:57.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:57.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:57.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:57.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:57.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:57.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:46:59.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:46:58 smithi064 ceph-mon[32765]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:46:58 smithi055 ceph-mon[29221]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:46:59.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:46:58 smithi165 ceph-mon[33296]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:00.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.460+0000 7fa68d7fa700 1 -- 172.21.15.64:0/874151212 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa690063920 con 0x7fa6780a3cc0 2024-04-26T21:47:00.460 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 4 on host 'smithi064' 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6780a3cc0 msgr2=0x7fa6780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6780a3cc0 0x7fa6780a6180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa690104500 tx=0x7fa68000a380 comp rx=0 tx=0).stop 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa690105040 msgr2=0x7fa6901a7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa690105040 0x7fa6901a7180 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa688007ee0 tx=0x7fa68800d3b0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 shutdown_connections 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6780a3cc0 0x7fa6780a6180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa690105040 0x7fa6901a7180 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6901034a0 0x7fa69019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 --2- 172.21.15.64:0/874151212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6901046a0 0x7fa6901a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 >> 172.21.15.64:0/874151212 conn(0x7fa6900fea40 msgr2=0x7fa690105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 shutdown_connections 2024-04-26T21:47:00.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:00.462+0000 7fa696369700 1 -- 172.21.15.64:0/874151212 wait complete. 2024-04-26T21:47:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:00.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:00.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:00.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:00.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:00.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:00.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:01.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:01.092 DEBUG:teuthology.orchestra.run.smithi064:osd.4> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T21:47:01.094 INFO:tasks.cephadm:Deploying osd.5 on smithi165 with /dev/vg_nvme/lv_3... 2024-04-26T21:47:01.094 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T21:47:01.338 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:47:01.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:01 smithi064 ceph-mon[32765]: from='osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:01.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:01 smithi064 ceph-mon[32765]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:01.785 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:01 smithi165 ceph-mon[33296]: from='osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:01.786 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:01 smithi165 ceph-mon[33296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:01.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:01 smithi055 ceph-mon[29221]: from='osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:01.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:01 smithi055 ceph-mon[29221]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T21:47:02.716 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:02 smithi165 ceph-mon[33296]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:02.716 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:02 smithi165 ceph-mon[33296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:47:02.716 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:02 smithi165 ceph-mon[33296]: from='osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:47:02.716 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:02 smithi165 ceph-mon[33296]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:47:02.716 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:02 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:02.716 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:02 smithi165 ceph-mon[33296]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:47:02.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:02 smithi055 ceph-mon[29221]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:02.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:02 smithi055 ceph-mon[29221]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:47:02.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:02 smithi055 ceph-mon[29221]: from='osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:47:02.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:02 smithi055 ceph-mon[29221]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:47:02.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:02 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:02.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:02 smithi055 ceph-mon[29221]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:47:02.959 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:02 smithi064 ceph-mon[32765]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:02.959 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:02 smithi064 ceph-mon[32765]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T21:47:02.959 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:02 smithi064 ceph-mon[32765]: from='osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:47:02.959 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:02 smithi064 ceph-mon[32765]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T21:47:02.959 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:02 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:02.959 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:02 smithi064 ceph-mon[32765]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:47:02.960 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 21:47:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4[43877]: 2024-04-26T21:47:02.644+0000 7ff0060ba700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T21:47:02.960 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 21:47:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4[43877]: 2024-04-26T21:47:02.652+0000 7ff001ead700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:47:03.323 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T21:47:03.323 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T21:47:03.323 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-26T21:47:03.323 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-26T21:47:03.324 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114218 s, 918 MB/s 2024-04-26T21:47:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-26T21:47:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:03 smithi055 ceph-mon[29221]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:47:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:03 smithi055 ceph-mon[29221]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:47:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:04.004 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:03 smithi064 ceph-mon[32765]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:47:04.004 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:03 smithi064 ceph-mon[32765]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:47:04.004 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:04.004 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:04.020 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:03 smithi165 ceph-mon[33296]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:47:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:03 smithi165 ceph-mon[33296]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T21:47:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:04.021 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:04.048 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-04-26T21:47:04.312 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] boot 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi064 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:04.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:04.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:04.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:04.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi064 to 8544M 2024-04-26T21:47:04.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:04.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:04.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:04.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:04.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:47:04.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:47:04.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:04.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] boot 2024-04-26T21:47:04.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:47:04.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:04.966 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi064 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi064 to 8544M 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:04.967 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: osd.4 [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] boot 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi064 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:05.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:05.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi064 to 8544M 2024-04-26T21:47:05.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:05.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:05.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:05.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:05.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:05 smithi165 ceph-mon[33296]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:47:05.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:05 smithi055 ceph-mon[29221]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:47:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:05 smithi064 ceph-mon[32765]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/3956706124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 msgr2=0x7f8b48104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/3956706124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 0x7f8b48104b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009990 tx=0x7f8b38009ca0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/3956706124 shutdown_connections 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/3956706124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b48105040 0x7f8b4810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/3956706124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b481034a0 0x7f8b481038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/3956706124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 0x7f8b48104b00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/3956706124 >> 172.21.15.165:0/3956706124 conn(0x7f8b480fea40 msgr2=0x7f8b48100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/3956706124 shutdown_connections 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.025+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/3956706124 wait complete. 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4f5ef700 1 Processor -- start 2024-04-26T21:47:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4f5ef700 1 -- start start 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4f5ef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b481034a0 0x7f8b480771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4f5ef700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 0x7f8b48075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4f5ef700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b48105040 0x7f8b48075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4f5ef700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4810ef90 con 0x7f8b48105040 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4f5ef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4810ee10 con 0x7f8b481034a0 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4f5ef700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4810f110 con 0x7f8b481046a0 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 0x7f8b48075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 0x7f8b48075810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41238/0 (socket says 172.21.15.165:41238) 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 -- 172.21.15.165:0/2887125443 learned_addr learned my addr 172.21.15.165:0/2887125443 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4d38b700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b481034a0 0x7f8b480771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4db8c700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b48105040 0x7f8b48075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 -- 172.21.15.165:0/2887125443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b481034a0 msgr2=0x7f8b480771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b481034a0 0x7f8b480771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 -- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b48105040 msgr2=0x7f8b48075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b48105040 0x7f8b48075d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 -- 172.21.15.165:0/2887125443 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b48076600 con 0x7f8b481046a0 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4db8c700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b48105040 0x7f8b48075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.026+0000 7f8b4cb8a700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 0x7f8b48075810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b380098e0 tx=0x7f8b380047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.027+0000 7f8b3e7fc700 1 -- 172.21.15.165:0/2887125443 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38027070 con 0x7f8b481046a0 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.027+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b481a3750 con 0x7f8b481046a0 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.027+0000 7f8b3e7fc700 1 -- 172.21.15.165:0/2887125443 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b380040b0 con 0x7f8b481046a0 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.027+0000 7f8b3e7fc700 1 -- 172.21.15.165:0/2887125443 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38022e60 con 0x7f8b481046a0 2024-04-26T21:47:06.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.027+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b481a3ba0 con 0x7f8b481046a0 2024-04-26T21:47:06.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.028+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4806a110 con 0x7f8b481046a0 2024-04-26T21:47:06.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.029+0000 7f8b3e7fc700 1 -- 172.21.15.165:0/2887125443 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8b3802a070 con 0x7f8b481046a0 2024-04-26T21:47:06.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.029+0000 7f8b3e7fc700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b340a3c40 0x7f8b340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:06.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.029+0000 7f8b3e7fc700 1 -- 172.21.15.165:0/2887125443 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f8b380d0100 con 0x7f8b481046a0 2024-04-26T21:47:06.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.029+0000 7f8b4d38b700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b340a3c40 0x7f8b340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:06.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.030+0000 7f8b4d38b700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b340a3c40 0x7f8b340a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b48104500 tx=0x7f8b4400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:06.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.034+0000 7f8b3e7fc700 1 -- 172.21.15.165:0/2887125443 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8b38013070 con 0x7f8b481046a0 2024-04-26T21:47:06.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:06.237+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8b48063860 con 0x7f8b340a3c40 2024-04-26T21:47:06.983 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:06 smithi165 ceph-mon[33296]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:06.983 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:06 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:06.983 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:06 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:06.983 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:06 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:06 smithi064 ceph-mon[32765]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:06 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:06 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:06 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:06 smithi055 ceph-mon[29221]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:06 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:06 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:06 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:07 smithi064 ceph-mon[32765]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:08.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:07 smithi165 ceph-mon[33296]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:08.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:07 smithi055 ceph-mon[29221]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:08 smithi064 ceph-mon[32765]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:09.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:08 smithi165 ceph-mon[33296]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:09.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:08 smithi055 ceph-mon[29221]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:09.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:09 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/4009316684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]: dispatch 2024-04-26T21:47:09.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:09 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]: dispatch 2024-04-26T21:47:09.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:09 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]': finished 2024-04-26T21:47:09.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:09 smithi165 ceph-mon[33296]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:47:09.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:09 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:10.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:09 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/4009316684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]: dispatch 2024-04-26T21:47:10.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:09 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]: dispatch 2024-04-26T21:47:10.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:09 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]': finished 2024-04-26T21:47:10.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:09 smithi064 ceph-mon[32765]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:47:10.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:09 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:10.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:09 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/4009316684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]: dispatch 2024-04-26T21:47:10.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:09 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]: dispatch 2024-04-26T21:47:10.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:09 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f42d9d6-e34c-4085-809b-fe73d7da4281"}]': finished 2024-04-26T21:47:10.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:09 smithi055 ceph-mon[29221]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T21:47:10.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:09 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:10 smithi064 ceph-mon[32765]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:10 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/1274140650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:11.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:10 smithi165 ceph-mon[33296]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:10 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/1274140650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:11.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:10 smithi055 ceph-mon[29221]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:11.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:10 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/1274140650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:13.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:12 smithi064 ceph-mon[32765]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:13.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:12 smithi165 ceph-mon[33296]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:13.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:12 smithi055 ceph-mon[29221]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:14.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:14 smithi165 ceph-mon[33296]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:14.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:14 smithi055 ceph-mon[29221]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:15.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:14 smithi064 ceph-mon[32765]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:17.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:16 smithi064 ceph-mon[32765]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:17.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:16 smithi165 ceph-mon[33296]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:17.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:16 smithi055 ceph-mon[29221]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:47:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:17 smithi064 ceph-mon[32765]: Deploying daemon osd.5 on smithi165 2024-04-26T21:47:18.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:47:18.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:18.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:17 smithi165 ceph-mon[33296]: Deploying daemon osd.5 on smithi165 2024-04-26T21:47:18.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T21:47:18.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:18.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:17 smithi055 ceph-mon[29221]: Deploying daemon osd.5 on smithi165 2024-04-26T21:47:18.907 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:18 smithi165 ceph-mon[33296]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:18 smithi064 ceph-mon[32765]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:19.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:18 smithi055 ceph-mon[29221]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:21.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:20 smithi064 ceph-mon[32765]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:21.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:20 smithi165 ceph-mon[33296]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:21.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:20 smithi055 ceph-mon[29221]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:23.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:22 smithi064 ceph-mon[32765]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:23.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:23.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:23.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:23.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:22 smithi165 ceph-mon[33296]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:23.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:23.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:23.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:23.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:22 smithi055 ceph-mon[29221]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:23.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:23.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:23.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:25.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:24 smithi165 ceph-mon[33296]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:25.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:24 smithi055 ceph-mon[29221]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:25.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:24 smithi064 ceph-mon[32765]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:26.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.607+0000 7f8b3e7fc700 1 -- 172.21.15.165:0/2887125443 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8b48063860 con 0x7f8b340a3c40 2024-04-26T21:47:26.608 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 5 on host 'smithi165' 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b340a3c40 msgr2=0x7f8b340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b340a3c40 0x7f8b340a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b48104500 tx=0x7f8b4400b410 comp rx=0 tx=0).stop 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 msgr2=0x7f8b48075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 0x7f8b48075810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b380098e0 tx=0x7f8b380047b0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 shutdown_connections 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b48105040 0x7f8b48075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b340a3c40 0x7f8b340a6100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b481034a0 0x7f8b480771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 --2- 172.21.15.165:0/2887125443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481046a0 0x7f8b48075810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 >> 172.21.15.165:0/2887125443 conn(0x7f8b480fea40 msgr2=0x7f8b48107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:26.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 shutdown_connections 2024-04-26T21:47:26.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:47:26.609+0000 7f8b4f5ef700 1 -- 172.21.15.165:0/2887125443 wait complete. 2024-04-26T21:47:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:47:26.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:47:26.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:26.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:26.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:47:27.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:47:27.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:27.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:27.310 DEBUG:teuthology.orchestra.run.smithi165:osd.5> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T21:47:27.312 INFO:tasks.cephadm:Deploying osd.6 on smithi055 with /dev/vg_nvme/lv_2... 2024-04-26T21:47:27.313 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:47:27.573 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:47:27.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:27 smithi165 ceph-mon[33296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:47:27.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:27 smithi165 ceph-mon[33296]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:47:27.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:27.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:27 smithi165 ceph-mon[33296]: from='osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:47:27.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:27 smithi165 ceph-mon[33296]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:47:27.828 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 21:47:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T21:47:27.775+0000 7f503700d700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T21:47:27.828 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 21:47:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T21:47:27.781+0000 7f5032e00700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:47:27.954 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:27 smithi055 ceph-mon[29221]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:47:27.954 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:27 smithi055 ceph-mon[29221]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:47:27.954 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:27.954 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:27 smithi055 ceph-mon[29221]: from='osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:47:27.954 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:27 smithi055 ceph-mon[29221]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:47:28.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:27 smithi064 ceph-mon[32765]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T21:47:28.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:27 smithi064 ceph-mon[32765]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T21:47:28.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:28.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:27 smithi064 ceph-mon[32765]: from='osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:47:28.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:27 smithi064 ceph-mon[32765]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:47:28.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:28 smithi165 ceph-mon[33296]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:28 smithi165 ceph-mon[33296]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:47:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:28 smithi165 ceph-mon[33296]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:47:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:28 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:28 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:28.989 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:28 smithi055 ceph-mon[29221]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:28.989 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:28 smithi055 ceph-mon[29221]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:47:28.989 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:28 smithi055 ceph-mon[29221]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:47:28.989 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:28.990 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:29.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:28 smithi064 ceph-mon[32765]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-04-26T21:47:29.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:28 smithi064 ceph-mon[32765]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:47:29.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:28 smithi064 ceph-mon[32765]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T21:47:29.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:28 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:29.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:28 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:29.733 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:47:29.734 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:47:29.734 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-04-26T21:47:29.734 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-04-26T21:47:29.734 INFO:teuthology.orchestra.run.smithi055.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00944898 s, 1.1 GB/s 2024-04-26T21:47:29.734 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-04-26T21:47:30.057 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:29 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:47:30.057 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:29 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:47:30.057 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:29 smithi165 ceph-mon[33296]: osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] boot 2024-04-26T21:47:30.057 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:29 smithi165 ceph-mon[33296]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T21:47:30.057 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:29 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:30.057 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:29 smithi165 ceph-mon[33296]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:47:30.091 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:29 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:47:30.091 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:29 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:47:30.091 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:29 smithi055 ceph-mon[29221]: osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] boot 2024-04-26T21:47:30.091 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:29 smithi055 ceph-mon[29221]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T21:47:30.091 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:29 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:30.091 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:29 smithi055 ceph-mon[29221]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:47:30.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:29 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:47:30.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:29 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:47:30.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:29 smithi064 ceph-mon[32765]: osd.5 [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] boot 2024-04-26T21:47:30.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:29 smithi064 ceph-mon[32765]: osdmap e38: 6 total, 6 up, 6 in 2024-04-26T21:47:30.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:29 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T21:47:30.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:29 smithi064 ceph-mon[32765]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T21:47:30.369 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi055:vg_nvme/lv_2 2024-04-26T21:47:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:47:31.036 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:31.037 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:30 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:31.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:47:31.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:31.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:31.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:31.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:31.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:30 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.158 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:31.158 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T21:47:31.158 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:31.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:31.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:31.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:31.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:30 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:31.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:31 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi165 2024-04-26T21:47:31.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:31 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-26T21:47:31.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:31 smithi055 ceph-mon[29221]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:47:32.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:31 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi165 2024-04-26T21:47:32.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:31 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-26T21:47:32.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:31 smithi165 ceph-mon[33296]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.170+0000 7fdfee0f6700 1 -- 172.21.15.55:0/4235676283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 msgr2=0x7fdfe81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.170+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/4235676283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 0x7fdfe81027a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80099c0 tx=0x7fdfd8009cd0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 -- 172.21.15.55:0/4235676283 shutdown_connections 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/4235676283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe8103f20 0x7fdfe810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/4235676283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 0x7fdfe81027a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/4235676283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfe8103580 0x7fdfe81039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 -- 172.21.15.55:0/4235676283 >> 172.21.15.55:0/4235676283 conn(0x7fdfe80fd920 msgr2=0x7fdfe80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 -- 172.21.15.55:0/4235676283 shutdown_connections 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 -- 172.21.15.55:0/4235676283 wait complete. 2024-04-26T21:47:32.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 Processor -- start 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.171+0000 7fdfee0f6700 1 -- start start 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfee0f6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 0x7fdfe81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfee0f6700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfe8103580 0x7fdfe8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfee0f6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe8103f20 0x7fdfe819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfee0f6700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810de60 con 0x7fdfe8102380 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfee0f6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810dce0 con 0x7fdfe8103f20 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfee0f6700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe810dfe0 con 0x7fdfe8103580 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 0x7fdfe81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 0x7fdfe81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45410/0 (socket says 172.21.15.55:45410) 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 -- 172.21.15.55:0/1042030293 learned_addr learned my addr 172.21.15.55:0/1042030293 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe7fff700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe8103f20 0x7fdfe819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe6ffd700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfe8103580 0x7fdfe8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 -- 172.21.15.55:0/1042030293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfe8103580 msgr2=0x7fdfe8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfe8103580 0x7fdfe8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 -- 172.21.15.55:0/1042030293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe8103f20 msgr2=0x7fdfe819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe8103f20 0x7fdfe819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 -- 172.21.15.55:0/1042030293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81a0210 con 0x7fdfe8102380 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe7fff700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe8103f20 0x7fdfe819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe77fe700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 0x7fdfe81985b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdfd801abe0 tx=0x7fdfd801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe4ff9700 1 -- 172.21.15.55:0/1042030293 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd802a070 con 0x7fdfe8102380 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe806b100 con 0x7fdfe8102380 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe4ff9700 1 -- 172.21.15.55:0/1042030293 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd800f910 con 0x7fdfe8102380 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.172+0000 7fdfe4ff9700 1 -- 172.21.15.55:0/1042030293 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800bea0 con 0x7fdfe8102380 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.173+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe806b630 con 0x7fdfe8102380 2024-04-26T21:47:32.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.173+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe806a110 con 0x7fdfe8102380 2024-04-26T21:47:32.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.174+0000 7fdfe4ff9700 1 -- 172.21.15.55:0/1042030293 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfd802d070 con 0x7fdfe8102380 2024-04-26T21:47:32.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.175+0000 7fdfe4ff9700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfd00a3c40 0x7fdfd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:32.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.175+0000 7fdfe6ffd700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfd00a3c40 0x7fdfd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:32.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.175+0000 7fdfe4ff9700 1 -- 172.21.15.55:0/1042030293 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7fdfd80d24b0 con 0x7fdfe8102380 2024-04-26T21:47:32.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.176+0000 7fdfe6ffd700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfd00a3c40 0x7fdfd00a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc005950 tx=0x7fdfdc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:32.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.180+0000 7fdfe4ff9700 1 -- 172.21.15.55:0/1042030293 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdfd8062060 con 0x7fdfe8102380 2024-04-26T21:47:32.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:31 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi165 2024-04-26T21:47:32.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:31 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi165 to 8544M 2024-04-26T21:47:32.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:31 smithi064 ceph-mon[32765]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T21:47:32.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:32.398+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdfe806b890 con 0x7fdfd00a3c40 2024-04-26T21:47:33.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:32 smithi165 ceph-mon[33296]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:33.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:33.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:33.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:33.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:32 smithi165 ceph-mon[33296]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:47:33.167 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:32 smithi055 ceph-mon[29221]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:33.168 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:33.168 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:33.168 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:33.168 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:32 smithi055 ceph-mon[29221]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:47:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:32 smithi064 ceph-mon[32765]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:33.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:32 smithi064 ceph-mon[32765]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T21:47:34.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:33 smithi165 ceph-mon[33296]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:34.135 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:33 smithi055 ceph-mon[29221]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:34.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:33 smithi064 ceph-mon[32765]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:47:35.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:34 smithi165 ceph-mon[33296]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:35.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:34 smithi055 ceph-mon[29221]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:35.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:34 smithi064 ceph-mon[32765]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:36.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:35 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3125293451' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c68fd703-709c-4adf-8437-6cdafc737f7d"}]: dispatch 2024-04-26T21:47:36.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:35 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3125293451' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c68fd703-709c-4adf-8437-6cdafc737f7d"}]': finished 2024-04-26T21:47:36.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:35 smithi165 ceph-mon[33296]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T21:47:36.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:36.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:35 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3125293451' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c68fd703-709c-4adf-8437-6cdafc737f7d"}]: dispatch 2024-04-26T21:47:36.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:35 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3125293451' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c68fd703-709c-4adf-8437-6cdafc737f7d"}]': finished 2024-04-26T21:47:36.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:35 smithi055 ceph-mon[29221]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T21:47:36.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:36.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:35 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3125293451' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c68fd703-709c-4adf-8437-6cdafc737f7d"}]: dispatch 2024-04-26T21:47:36.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:35 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3125293451' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c68fd703-709c-4adf-8437-6cdafc737f7d"}]': finished 2024-04-26T21:47:36.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:35 smithi064 ceph-mon[32765]: osdmap e43: 7 total, 6 up, 7 in 2024-04-26T21:47:36.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:36 smithi165 ceph-mon[33296]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:36 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3743042295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:37.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:36 smithi055 ceph-mon[29221]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:37.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:36 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3743042295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:37.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:36 smithi064 ceph-mon[32765]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:37.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:36 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3743042295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:47:39.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:38 smithi165 ceph-mon[33296]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:39.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:38 smithi055 ceph-mon[29221]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:38 smithi064 ceph-mon[32765]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:47:41.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:40 smithi165 ceph-mon[33296]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-26T21:47:41.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:40 smithi055 ceph-mon[29221]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-26T21:47:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:40 smithi064 ceph-mon[32765]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-26T21:47:43.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:42 smithi165 ceph-mon[33296]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T21:47:43.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:42 smithi055 ceph-mon[29221]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T21:47:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:42 smithi064 ceph-mon[32765]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T21:47:43.903 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:47:43.903 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:43.903 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:43 smithi055 ceph-mon[29221]: Deploying daemon osd.6 on smithi055 2024-04-26T21:47:44.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:47:44.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:44.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:43 smithi064 ceph-mon[32765]: Deploying daemon osd.6 on smithi055 2024-04-26T21:47:44.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T21:47:44.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:44.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:43 smithi165 ceph-mon[33296]: Deploying daemon osd.6 on smithi055 2024-04-26T21:47:44.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:44 smithi055 ceph-mon[29221]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:47:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:44 smithi064 ceph-mon[32765]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:47:45.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:44 smithi165 ceph-mon[33296]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T21:47:47.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:46 smithi055 ceph-mon[29221]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:46 smithi064 ceph-mon[32765]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:47.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:46 smithi165 ceph-mon[33296]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:49.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:48 smithi055 ceph-mon[29221]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:48 smithi064 ceph-mon[32765]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:49.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:48 smithi165 ceph-mon[33296]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:49.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:50.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:50 smithi055 ceph-mon[29221]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:50 smithi064 ceph-mon[32765]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:51.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:50 smithi165 ceph-mon[33296]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:52.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.635+0000 7fdfe4ff9700 1 -- 172.21.15.55:0/1042030293 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdfe806b890 con 0x7fdfd00a3c40 2024-04-26T21:47:52.636 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 6 on host 'smithi055' 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfd00a3c40 msgr2=0x7fdfd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfd00a3c40 0x7fdfd00a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc005950 tx=0x7fdfdc0058e0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 msgr2=0x7fdfe81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 0x7fdfe81985b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdfd801abe0 tx=0x7fdfd801ac10 comp rx=0 tx=0).stop 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 shutdown_connections 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdfd00a3c40 0x7fdfd00a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe8103f20 0x7fdfe819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe8102380 0x7fdfe81985b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 --2- 172.21.15.55:0/1042030293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfe8103580 0x7fdfe8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 >> 172.21.15.55:0/1042030293 conn(0x7fdfe80fd920 msgr2=0x7fdfe81069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 shutdown_connections 2024-04-26T21:47:52.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:47:52.637+0000 7fdfee0f6700 1 -- 172.21.15.55:0/1042030293 wait complete. 2024-04-26T21:47:52.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:52 smithi055 ceph-mon[29221]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:52.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:52.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:52.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:52.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:52.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:52.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:52 smithi055 ceph-mon[29221]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:47:52.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:52 smithi064 ceph-mon[32765]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:52 smithi064 ceph-mon[32765]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:47:53.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:52 smithi165 ceph-mon[33296]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:52 smithi165 ceph-mon[33296]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T21:47:53.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.452 DEBUG:teuthology.orchestra.run.smithi055:osd.6> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T21:47:53.454 INFO:tasks.cephadm:Deploying osd.7 on smithi064 with /dev/vg_nvme/lv_2... 2024-04-26T21:47:53.455 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:47:53.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:53.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:53 smithi055 ceph-mon[29221]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:47:53.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:53 smithi055 ceph-mon[29221]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:47:53.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:53 smithi055 ceph-mon[29221]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:47:53.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:53.712 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:47:53.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:53.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:53.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:53 smithi064 ceph-mon[32765]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:47:53.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:53 smithi064 ceph-mon[32765]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:47:53.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:53 smithi064 ceph-mon[32765]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:47:53.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:54.012 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 21:47:53 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T21:47:53.860+0000 7fd67e552700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T21:47:54.012 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 21:47:53 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T21:47:53.871+0000 7fd67a345700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:47:54.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:53 smithi165 ceph-mon[33296]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T21:47:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:53 smithi165 ceph-mon[33296]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T21:47:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:53 smithi165 ceph-mon[33296]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:47:54.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:54.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:54 smithi055 ceph-mon[29221]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:54.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:54 smithi055 ceph-mon[29221]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:47:54.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:54 smithi055 ceph-mon[29221]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:47:54.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:54.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:55.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:54 smithi064 ceph-mon[32765]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:55.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:54 smithi064 ceph-mon[32765]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:47:55.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:54 smithi064 ceph-mon[32765]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:47:55.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:55.211 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:55.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:54 smithi165 ceph-mon[33296]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:55.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:54 smithi165 ceph-mon[33296]: from='osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:47:55.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:54 smithi165 ceph-mon[33296]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T21:47:55.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:55.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:47:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:47:55.801 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-04-26T21:47:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-04-26T21:47:55.801 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140228 s, 748 MB/s 2024-04-26T21:47:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-04-26T21:47:56.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:55 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:47:56.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:55 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:47:56.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:55 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:56.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:55 smithi064 ceph-mon[32765]: osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] boot 2024-04-26T21:47:56.108 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:55 smithi064 ceph-mon[32765]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T21:47:56.109 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:55 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:56.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:55 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:47:56.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:55 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:47:56.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:55 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:56.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:55 smithi055 ceph-mon[29221]: osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] boot 2024-04-26T21:47:56.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:55 smithi055 ceph-mon[29221]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T21:47:56.166 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:55 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:56.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:55 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:47:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:55 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:47:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:55 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:55 smithi165 ceph-mon[33296]: osd.6 [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] boot 2024-04-26T21:47:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:55 smithi165 ceph-mon[33296]: osdmap e46: 7 total, 7 up, 7 in 2024-04-26T21:47:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:55 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T21:47:56.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-04-26T21:47:56.680 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:47:57.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:57.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:57.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:57.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:47:58.121 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:57 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi055 2024-04-26T21:47:58.122 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:57 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi055 to 5697M 2024-04-26T21:47:58.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:57 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi055 2024-04-26T21:47:58.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:57 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi055 to 5697M 2024-04-26T21:47:58.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:57 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi055 2024-04-26T21:47:58.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:57 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi055 to 5697M 2024-04-26T21:47:58.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 -- 172.21.15.64:0/1989959890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 msgr2=0x7f93741027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 --2- 172.21.15.64:0/1989959890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 0x7f93741027a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f935c0099c0 tx=0x7f935c009cd0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 -- 172.21.15.64:0/1989959890 shutdown_connections 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 --2- 172.21.15.64:0/1989959890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9374103f20 0x7f937410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 --2- 172.21.15.64:0/1989959890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 0x7f93741027a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 --2- 172.21.15.64:0/1989959890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9374103580 0x7f93741039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 -- 172.21.15.64:0/1989959890 >> 172.21.15.64:0/1989959890 conn(0x7f93740fd920 msgr2=0x7f93740ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 -- 172.21.15.64:0/1989959890 shutdown_connections 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.337+0000 7f937a9c1700 1 -- 172.21.15.64:0/1989959890 wait complete. 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f937a9c1700 1 Processor -- start 2024-04-26T21:47:58.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f937a9c1700 1 -- start start 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f937a9c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 0x7f93741062c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f937a9c1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9374103580 0x7f9374104960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f937a9c1700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9374103f20 0x7f9374104ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f937a9c1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f937410de20 con 0x7f9374103580 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f937a9c1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f937410dca0 con 0x7f9374102380 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f937a9c1700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f937410dfa0 con 0x7f9374103f20 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f9373fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 0x7f93741062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.338+0000 7f9373fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 0x7f93741062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48708/0 (socket says 172.21.15.64:48708) 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f9373fff700 1 -- 172.21.15.64:0/2342452997 learned_addr learned my addr 172.21.15.64:0/2342452997 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-26T21:47:58.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f93737fe700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9374103580 0x7f9374104960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f9378f5e700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9374103f20 0x7f9374104ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f9373fff700 1 -- 172.21.15.64:0/2342452997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9374103f20 msgr2=0x7f9374104ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f9373fff700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9374103f20 0x7f9374104ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f9373fff700 1 -- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9374103580 msgr2=0x7f9374104960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f9373fff700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9374103580 0x7f9374104960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f9373fff700 1 -- 172.21.15.64:0/2342452997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9374105770 con 0x7f9374102380 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f9373fff700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 0x7f93741062c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f935c0098e0 tx=0x7f935c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f93717fa700 1 -- 172.21.15.64:0/2342452997 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c026070 con 0x7f9374102380 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93741a3760 con 0x7f9374102380 2024-04-26T21:47:58.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f93717fa700 1 -- 172.21.15.64:0/2342452997 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c004120 con 0x7f9374102380 2024-04-26T21:47:58.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f93717fa700 1 -- 172.21.15.64:0/2342452997 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c0048e0 con 0x7f9374102380 2024-04-26T21:47:58.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.339+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93741a3c60 con 0x7f9374102380 2024-04-26T21:47:58.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.340+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937406a110 con 0x7f9374102380 2024-04-26T21:47:58.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.341+0000 7f93717fa700 1 -- 172.21.15.64:0/2342452997 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f935c029070 con 0x7f9374102380 2024-04-26T21:47:58.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.342+0000 7f93717fa700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93600a3c40 0x7f93600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:47:58.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.342+0000 7f93717fa700 1 -- 172.21.15.64:0/2342452997 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f935c0d00a0 con 0x7f9374102380 2024-04-26T21:47:58.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.342+0000 7f93737fe700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93600a3c40 0x7f93600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:47:58.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.343+0000 7f93737fe700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93600a3c40 0x7f93600a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9364005950 tx=0x7f936400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:47:58.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.347+0000 7f93717fa700 1 -- 172.21.15.64:0/2342452997 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f935c05e060 con 0x7f9374102380 2024-04-26T21:47:58.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:47:58.565+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f937406b810 con 0x7f93600a3c40 2024-04-26T21:47:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:58 smithi055 ceph-mon[29221]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:47:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:58 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:58 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:58 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:59.189 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:58 smithi064 ceph-mon[32765]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:47:59.189 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:58 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:59.189 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:58 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:59.189 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:58 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:59.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:58 smithi165 ceph-mon[33296]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:47:59.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:58 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:47:59.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:58 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:47:59.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:58 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:00.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:47:59 smithi055 ceph-mon[29221]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:00.192 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:47:59 smithi064 ceph-mon[32765]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:00.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:47:59 smithi165 ceph-mon[33296]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:00 smithi055 ceph-mon[29221]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:00 smithi064 ceph-mon[32765]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:01.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:00 smithi165 ceph-mon[33296]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:02.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:01 smithi055 ceph-mon[29221]: from='client.? 172.21.15.64:0/4148198852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]: dispatch 2024-04-26T21:48:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:01 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]: dispatch 2024-04-26T21:48:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:01 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]': finished 2024-04-26T21:48:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:01 smithi055 ceph-mon[29221]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T21:48:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:01 smithi064 ceph-mon[32765]: from='client.? 172.21.15.64:0/4148198852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]: dispatch 2024-04-26T21:48:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:01 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]: dispatch 2024-04-26T21:48:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:01 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]': finished 2024-04-26T21:48:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:01 smithi064 ceph-mon[32765]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T21:48:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:01 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:02.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:01 smithi165 ceph-mon[33296]: from='client.? 172.21.15.64:0/4148198852' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]: dispatch 2024-04-26T21:48:02.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:01 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]: dispatch 2024-04-26T21:48:02.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:01 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d96f9d6b-186d-48ba-9ade-8ce41775d9d3"}]': finished 2024-04-26T21:48:02.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:01 smithi165 ceph-mon[33296]: osdmap e48: 8 total, 7 up, 8 in 2024-04-26T21:48:02.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:01 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:03.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:02 smithi055 ceph-mon[29221]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:03.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:02 smithi055 ceph-mon[29221]: from='client.? 172.21.15.64:0/2446292812' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:03.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:02 smithi064 ceph-mon[32765]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:03.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:02 smithi064 ceph-mon[32765]: from='client.? 172.21.15.64:0/2446292812' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:03.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:02 smithi165 ceph-mon[33296]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:03.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:02 smithi165 ceph-mon[33296]: from='client.? 172.21.15.64:0/2446292812' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:05.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:04 smithi055 ceph-mon[29221]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:05.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:04 smithi064 ceph-mon[32765]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:05.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:04 smithi165 ceph-mon[33296]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:07.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:06 smithi055 ceph-mon[29221]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:07.195 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:06 smithi064 ceph-mon[32765]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:07.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:06 smithi165 ceph-mon[33296]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:09.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:08 smithi064 ceph-mon[32765]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:09.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:08 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:48:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:08 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:09.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:08 smithi165 ceph-mon[33296]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:09.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:08 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:48:09.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:08 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:09.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:08 smithi055 ceph-mon[29221]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:09.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:08 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T21:48:09.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:08 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:10.209 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:09 smithi064 ceph-mon[32765]: Deploying daemon osd.7 on smithi064 2024-04-26T21:48:10.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:09 smithi165 ceph-mon[33296]: Deploying daemon osd.7 on smithi064 2024-04-26T21:48:10.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:09 smithi055 ceph-mon[29221]: Deploying daemon osd.7 on smithi064 2024-04-26T21:48:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:10 smithi064 ceph-mon[32765]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:11.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:10 smithi165 ceph-mon[33296]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:11.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:10 smithi055 ceph-mon[29221]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:13.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:12 smithi064 ceph-mon[32765]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:12 smithi165 ceph-mon[33296]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:13.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:12 smithi055 ceph-mon[29221]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:15.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:14 smithi064 ceph-mon[32765]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:15.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:15.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:15.182 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:14 smithi165 ceph-mon[33296]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:15.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:15.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:14 smithi055 ceph-mon[29221]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:17.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:16 smithi064 ceph-mon[32765]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:17.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:16 smithi165 ceph-mon[33296]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:17.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:16 smithi055 ceph-mon[29221]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:18.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.031+0000 7f93717fa700 1 -- 172.21.15.64:0/2342452997 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f937406b810 con 0x7f93600a3c40 2024-04-26T21:48:18.033 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 7 on host 'smithi064' 2024-04-26T21:48:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93600a3c40 msgr2=0x7f93600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93600a3c40 0x7f93600a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9364005950 tx=0x7f936400b410 comp rx=0 tx=0).stop 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 msgr2=0x7f93741062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 0x7f93741062c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f935c0098e0 tx=0x7f935c005e80 comp rx=0 tx=0).stop 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 shutdown_connections 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9374103f20 0x7f9374104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93600a3c40 0x7f93600a6100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9374102380 0x7f93741062c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 --2- 172.21.15.64:0/2342452997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9374103580 0x7f9374104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 >> 172.21.15.64:0/2342452997 conn(0x7f93740fd920 msgr2=0x7f937406b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 shutdown_connections 2024-04-26T21:48:18.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:48:18.034+0000 7f937a9c1700 1 -- 172.21.15.64:0/2342452997 wait complete. 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:48:18.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:18 smithi165 ceph-mon[33296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:48:18.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:48:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:18 smithi055 ceph-mon[29221]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:48:18.718 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:18 smithi064 ceph-mon[32765]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T21:48:18.745 DEBUG:teuthology.orchestra.run.smithi064:osd.7> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T21:48:18.747 INFO:tasks.cephadm:Deploying osd.8 on smithi165 with /dev/vg_nvme/lv_2... 2024-04-26T21:48:18.747 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T21:48:18.998 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:48:20.187 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:19 smithi064 ceph-mon[32765]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:48:20.187 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:19 smithi064 ceph-mon[32765]: from='osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:48:20.187 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:19 smithi064 ceph-mon[32765]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:48:20.187 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:19 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:20.187 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:19 smithi064 ceph-mon[32765]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:48:20.188 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 21:48:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T21:48:20.000+0000 7fb867e6a700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T21:48:20.188 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 21:48:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T21:48:20.009+0000 7fb863c5d700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:48:20.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:19 smithi165 ceph-mon[33296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:48:20.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:19 smithi165 ceph-mon[33296]: from='osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:48:20.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:19 smithi165 ceph-mon[33296]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:48:20.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:19 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:20.276 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:19 smithi165 ceph-mon[33296]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:48:20.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:19 smithi055 ceph-mon[29221]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T21:48:20.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:19 smithi055 ceph-mon[29221]: from='osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:48:20.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:19 smithi055 ceph-mon[29221]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T21:48:20.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:19 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:20.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:19 smithi055 ceph-mon[29221]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi165.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012692 s, 826 MB/s 2024-04-26T21:48:21.025 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-26T21:48:21.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:21 smithi064 ceph-mon[32765]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:21.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:21 smithi064 ceph-mon[32765]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:48:21.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:21 smithi064 ceph-mon[32765]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:48:21.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:21 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:21.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:21 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:21.322 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:21 smithi165 ceph-mon[33296]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:21.322 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:21 smithi165 ceph-mon[33296]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:48:21.322 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:21 smithi165 ceph-mon[33296]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:48:21.322 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:21 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:21.322 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:21 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:21.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:21 smithi055 ceph-mon[29221]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-04-26T21:48:21.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:21 smithi055 ceph-mon[29221]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:48:21.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:21 smithi055 ceph-mon[29221]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T21:48:21.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:21.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:21.761 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-04-26T21:48:22.002 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:48:22.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:48:22.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] boot 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:22.026 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:22 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] boot 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:22.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:22.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:22 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:48:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:48:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: osd.7 [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] boot 2024-04-26T21:48:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: osdmap e51: 8 total, 8 up, 8 in 2024-04-26T21:48:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T21:48:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:22.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:22.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:22.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:22.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:22 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:23.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:23 smithi165 ceph-mon[33296]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:23.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:23 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi064 2024-04-26T21:48:23.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:23 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi064 to 5696M 2024-04-26T21:48:23.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:23 smithi165 ceph-mon[33296]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:48:23.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:23 smithi165 ceph-mon[33296]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:48:23.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:23 smithi055 ceph-mon[29221]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:23.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:23 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi064 2024-04-26T21:48:23.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:23 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi064 to 5696M 2024-04-26T21:48:23.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:23 smithi055 ceph-mon[29221]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:48:23.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:23 smithi055 ceph-mon[29221]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:48:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:23 smithi064 ceph-mon[32765]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:23 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi064 2024-04-26T21:48:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:23 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi064 to 5696M 2024-04-26T21:48:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:23 smithi064 ceph-mon[32765]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T21:48:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:23 smithi064 ceph-mon[32765]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 -- 172.21.15.165:0/1196302404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2441035a0 msgr2=0x7fd244103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 --2- 172.21.15.165:0/1196302404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2441035a0 0x7fd244103a00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd234009990 tx=0x7fd234009ca0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 -- 172.21.15.165:0/1196302404 shutdown_connections 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 --2- 172.21.15.165:0/1196302404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd244103f40 0x7fd24410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 --2- 172.21.15.165:0/1196302404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2441023a0 0x7fd2441027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 --2- 172.21.15.165:0/1196302404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2441035a0 0x7fd244103a00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 -- 172.21.15.165:0/1196302404 >> 172.21.15.165:0/1196302404 conn(0x7fd2440fd980 msgr2=0x7fd2440ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 -- 172.21.15.165:0/1196302404 shutdown_connections 2024-04-26T21:48:23.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.627+0000 7fd24b826700 1 -- 172.21.15.165:0/1196302404 wait complete. 2024-04-26T21:48:23.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd24b826700 1 Processor -- start 2024-04-26T21:48:23.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd24b826700 1 -- start start 2024-04-26T21:48:23.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd24b826700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2441023a0 0x7fd244199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd24b826700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2441035a0 0x7fd244199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd24b826700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd244103f40 0x7fd2441a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd24b826700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410de40 con 0x7fd2441023a0 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd24b826700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410dcc0 con 0x7fd2441035a0 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd24b826700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410dfc0 con 0x7fd244103f40 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd249dc3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd244103f40 0x7fd2441a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd249dc3700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd244103f40 0x7fd2441a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36674/0 (socket says 172.21.15.165:36674) 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd249dc3700 1 -- 172.21.15.165:0/2325795610 learned_addr learned my addr 172.21.15.165:0/2325795610 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd248dc1700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2441035a0 0x7fd244199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd2495c2700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2441023a0 0x7fd244199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd249dc3700 1 -- 172.21.15.165:0/2325795610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2441035a0 msgr2=0x7fd244199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd249dc3700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2441035a0 0x7fd244199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd249dc3700 1 -- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2441023a0 msgr2=0x7fd244199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd249dc3700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2441023a0 0x7fd244199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd249dc3700 1 -- 172.21.15.165:0/2325795610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2441a12b0 con 0x7fd244103f40 2024-04-26T21:48:23.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.628+0000 7fd2495c2700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2441023a0 0x7fd244199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:48:23.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.629+0000 7fd249dc3700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd244103f40 0x7fd2441a0c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00ef60 tx=0x7fd23c00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:23.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.629+0000 7fd23a7fc700 1 -- 172.21.15.165:0/2325795610 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0092e0 con 0x7fd244103f40 2024-04-26T21:48:23.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.629+0000 7fd23a7fc700 1 -- 172.21.15.165:0/2325795610 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd23c004bb0 con 0x7fd244103f40 2024-04-26T21:48:23.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.629+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2441a1540 con 0x7fd244103f40 2024-04-26T21:48:23.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.629+0000 7fd23a7fc700 1 -- 172.21.15.165:0/2325795610 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c004280 con 0x7fd244103f40 2024-04-26T21:48:23.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.629+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2441a1a30 con 0x7fd244103f40 2024-04-26T21:48:23.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.630+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24406a110 con 0x7fd244103f40 2024-04-26T21:48:23.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.631+0000 7fd23a7fc700 1 -- 172.21.15.165:0/2325795610 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd23c01b030 con 0x7fd244103f40 2024-04-26T21:48:23.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.632+0000 7fd23a7fc700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2300a3c40 0x7fd2300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:23.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.632+0000 7fd23a7fc700 1 -- 172.21.15.165:0/2325795610 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7fd23c0cfdc0 con 0x7fd244103f40 2024-04-26T21:48:23.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.632+0000 7fd2495c2700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2300a3c40 0x7fd2300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:23.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.632+0000 7fd2495c2700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2300a3c40 0x7fd2300a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd240005fd0 tx=0x7fd240005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:23.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.636+0000 7fd23a7fc700 1 -- 172.21.15.165:0/2325795610 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd23c022020 con 0x7fd244103f40 2024-04-26T21:48:23.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:23.843+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd2441a1d50 con 0x7fd2300a3c40 2024-04-26T21:48:24.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:24 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:24.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:24 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:24.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:24 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:24.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:24 smithi165 ceph-mon[33296]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:48:24.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:24 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:24.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:24 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:24.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:24 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:24.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:24 smithi055 ceph-mon[29221]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:48:24.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:24 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:24.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:24 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:24.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:24 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:24.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:24 smithi064 ceph-mon[32765]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T21:48:25.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:25 smithi165 ceph-mon[33296]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:25.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:25 smithi165 ceph-mon[33296]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:25.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:25 smithi165 ceph-mon[33296]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:48:25.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:25 smithi055 ceph-mon[29221]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:25.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:25 smithi055 ceph-mon[29221]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:25.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:25 smithi055 ceph-mon[29221]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:48:25.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:25 smithi064 ceph-mon[32765]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:25.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:25 smithi064 ceph-mon[32765]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:25.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:25 smithi064 ceph-mon[32765]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T21:48:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:27 smithi165 ceph-mon[33296]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:27 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/3679519865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]: dispatch 2024-04-26T21:48:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:27 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]: dispatch 2024-04-26T21:48:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:27 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]': finished 2024-04-26T21:48:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:27 smithi165 ceph-mon[33296]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T21:48:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:27.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:27 smithi055 ceph-mon[29221]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:27 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/3679519865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]: dispatch 2024-04-26T21:48:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:27 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]: dispatch 2024-04-26T21:48:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:27 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]': finished 2024-04-26T21:48:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:27 smithi055 ceph-mon[29221]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T21:48:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:27 smithi064 ceph-mon[32765]: pgmap v183: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:27 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/3679519865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]: dispatch 2024-04-26T21:48:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:27 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]: dispatch 2024-04-26T21:48:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:27 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f954a5a8-dd31-40f3-88eb-34444e538d2a"}]': finished 2024-04-26T21:48:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:27 smithi064 ceph-mon[32765]: osdmap e56: 9 total, 8 up, 9 in 2024-04-26T21:48:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:28.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:28 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/2962651997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:28.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:28 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/2962651997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:28.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:28 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/2962651997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:29.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:29 smithi165 ceph-mon[33296]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:29.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:29 smithi055 ceph-mon[29221]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:29.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:29 smithi064 ceph-mon[32765]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:31.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:31 smithi165 ceph-mon[33296]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:31.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:31 smithi055 ceph-mon[29221]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:31.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:31 smithi064 ceph-mon[32765]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:33.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:33 smithi165 ceph-mon[33296]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:33.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:33 smithi055 ceph-mon[29221]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:33 smithi064 ceph-mon[32765]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:35 smithi165 ceph-mon[33296]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:48:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:35 smithi165 ceph-mon[33296]: Deploying daemon osd.8 on smithi165 2024-04-26T21:48:35.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:35 smithi055 ceph-mon[29221]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:48:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:35 smithi055 ceph-mon[29221]: Deploying daemon osd.8 on smithi165 2024-04-26T21:48:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:35 smithi064 ceph-mon[32765]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T21:48:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:35 smithi064 ceph-mon[32765]: Deploying daemon osd.8 on smithi165 2024-04-26T21:48:37.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:37 smithi165 ceph-mon[33296]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:37.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:37 smithi055 ceph-mon[29221]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:37 smithi064 ceph-mon[32765]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:39.272 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:39 smithi165 ceph-mon[33296]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:39.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:39 smithi055 ceph-mon[29221]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:39.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:39 smithi064 ceph-mon[32765]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:41.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:40 smithi165 ceph-mon[33296]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:41.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:41.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:41.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:40 smithi064 ceph-mon[32765]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:41.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:40 smithi055 ceph-mon[29221]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:41.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:41.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:41.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:43.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:42 smithi165 ceph-mon[33296]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:42 smithi064 ceph-mon[32765]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:43.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:42 smithi055 ceph-mon[29221]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:43.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.965+0000 7fd23a7fc700 1 -- 172.21.15.165:0/2325795610 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd2441a1d50 con 0x7fd2300a3c40 2024-04-26T21:48:43.967 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 8 on host 'smithi165' 2024-04-26T21:48:43.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.967+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2300a3c40 msgr2=0x7fd2300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:43.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.967+0000 7fd24b826700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2300a3c40 0x7fd2300a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd240005fd0 tx=0x7fd240005f00 comp rx=0 tx=0).stop 2024-04-26T21:48:43.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.967+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd244103f40 msgr2=0x7fd2441a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:43.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.967+0000 7fd24b826700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd244103f40 0x7fd2441a0c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00ef60 tx=0x7fd23c00ef90 comp rx=0 tx=0).stop 2024-04-26T21:48:43.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.968+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 shutdown_connections 2024-04-26T21:48:43.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.968+0000 7fd24b826700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd244103f40 0x7fd2441a0c10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:43.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.968+0000 7fd24b826700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2441023a0 0x7fd244199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:43.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.968+0000 7fd24b826700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2300a3c40 0x7fd2300a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:43.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.968+0000 7fd24b826700 1 --2- 172.21.15.165:0/2325795610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2441035a0 0x7fd244199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:43.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.968+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 >> 172.21.15.165:0/2325795610 conn(0x7fd2440fd980 msgr2=0x7fd2440ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:43.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.968+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 shutdown_connections 2024-04-26T21:48:43.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:48:43.968+0000 7fd24b826700 1 -- 172.21.15.165:0/2325795610 wait complete. 2024-04-26T21:48:44.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:44.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:44.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:48:44.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:48:44.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.290 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:44.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:44.667 DEBUG:teuthology.orchestra.run.smithi165:osd.8> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T21:48:44.670 INFO:tasks.cephadm:Deploying osd.9 on smithi055 with /dev/vg_nvme/lv_1... 2024-04-26T21:48:44.670 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:48:44.929 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:48:45.152 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:44 smithi165 ceph-mon[33296]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:45.152 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:44 smithi165 ceph-mon[33296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:48:45.153 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:44 smithi165 ceph-mon[33296]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:48:45.153 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:44 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:45.153 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:44 smithi165 ceph-mon[33296]: from='osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:48:45.153 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:44 smithi165 ceph-mon[33296]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:48:45.238 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:44 smithi055 ceph-mon[29221]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:45.238 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:44 smithi055 ceph-mon[29221]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:48:45.238 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:44 smithi055 ceph-mon[29221]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:48:45.238 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:44 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:45.238 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:44 smithi055 ceph-mon[29221]: from='osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:48:45.238 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:44 smithi055 ceph-mon[29221]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:48:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:44 smithi064 ceph-mon[32765]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:44 smithi064 ceph-mon[32765]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T21:48:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:44 smithi064 ceph-mon[32765]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T21:48:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:44 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:44 smithi064 ceph-mon[32765]: from='osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:48:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:44 smithi064 ceph-mon[32765]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:48:45.577 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 21:48:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T21:48:45.385+0000 7f15d12df700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T21:48:45.577 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 21:48:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T21:48:45.397+0000 7f15c90ca700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:48:46.240 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:45 smithi165 ceph-mon[33296]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:48:46.240 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:45 smithi165 ceph-mon[33296]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:48:46.240 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:45 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:46.240 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:45 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:46.240 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:45 smithi165 ceph-mon[33296]: from='osd.8 ' entity='osd.8' 2024-04-26T21:48:46.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:45 smithi064 ceph-mon[32765]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:48:46.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:45 smithi064 ceph-mon[32765]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:48:46.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:45 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:46.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:45 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:46.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:45 smithi064 ceph-mon[32765]: from='osd.8 ' entity='osd.8' 2024-04-26T21:48:46.350 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:45 smithi055 ceph-mon[29221]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:48:46.350 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:45 smithi055 ceph-mon[29221]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T21:48:46.350 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:45 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:46.350 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:45 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:46.350 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:45 smithi055 ceph-mon[29221]: from='osd.8 ' entity='osd.8' 2024-04-26T21:48:47.069 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:48:47.069 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:48:47.069 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-04-26T21:48:47.069 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-04-26T21:48:47.069 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00719384 s, 1.5 GB/s 2024-04-26T21:48:47.069 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-04-26T21:48:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:46 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:48:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:46 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:48:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:46 smithi064 ceph-mon[32765]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:46 smithi064 ceph-mon[32765]: osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] boot 2024-04-26T21:48:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:46 smithi064 ceph-mon[32765]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T21:48:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:47.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:46 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:48:47.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:46 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:48:47.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:46 smithi165 ceph-mon[33296]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:47.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:47.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:46 smithi165 ceph-mon[33296]: osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] boot 2024-04-26T21:48:47.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:46 smithi165 ceph-mon[33296]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T21:48:47.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:47.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:46 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:48:47.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:46 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:48:47.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:46 smithi055 ceph-mon[29221]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-04-26T21:48:47.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:47.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:46 smithi055 ceph-mon[29221]: osd.8 [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] boot 2024-04-26T21:48:47.378 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:46 smithi055 ceph-mon[29221]: osdmap e59: 9 total, 9 up, 9 in 2024-04-26T21:48:47.378 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T21:48:47.750 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi055:vg_nvme/lv_1 2024-04-26T21:48:48.007 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:48:48.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:48 smithi064 ceph-mon[32765]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:48:48.272 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:48 smithi055 ceph-mon[29221]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:48:48.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:48 smithi165 ceph-mon[33296]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi165 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:49.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi165 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:49.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:48:49.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi165 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi165 to 5696M 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:49.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:48:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 -- 172.21.15.55:0/1093138410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3790105040 msgr2=0x7f379010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 --2- 172.21.15.55:0/1093138410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3790105040 0x7f379010b8f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3788009fc0 tx=0x7f378800b330 comp rx=0 tx=0).stop 2024-04-26T21:48:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 -- 172.21.15.55:0/1093138410 shutdown_connections 2024-04-26T21:48:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 --2- 172.21.15.55:0/1093138410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3790105040 0x7f379010b8f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 --2- 172.21.15.55:0/1093138410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37901034a0 0x7f37901038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 --2- 172.21.15.55:0/1093138410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f37901046a0 0x7f3790104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 -- 172.21.15.55:0/1093138410 >> 172.21.15.55:0/1093138410 conn(0x7f37900fea40 msgr2=0x7f3790100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:48:49.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 -- 172.21.15.55:0/1093138410 shutdown_connections 2024-04-26T21:48:49.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.659+0000 7f3794f6a700 1 -- 172.21.15.55:0/1093138410 wait complete. 2024-04-26T21:48:49.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f3794f6a700 1 Processor -- start 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f3794f6a700 1 -- start start 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f3794f6a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37901034a0 0x7f3790199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f3794f6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37901046a0 0x7f37901999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f3794f6a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3790105040 0x7f37901a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f3794f6a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f379010ef20 con 0x7f37901034a0 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f3794f6a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f379010eda0 con 0x7f37901046a0 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f3794f6a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f379010f0a0 con 0x7f3790105040 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f378e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37901034a0 0x7f3790199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f378e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37901034a0 0x7f3790199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46408/0 (socket says 172.21.15.55:46408) 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f378e59c700 1 -- 172.21.15.55:0/2105834693 learned_addr learned my addr 172.21.15.55:0/2105834693 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.660+0000 7f378dd9b700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37901046a0 0x7f37901999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f378ed9d700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3790105040 0x7f37901a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:49.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f378e59c700 1 -- 172.21.15.55:0/2105834693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3790105040 msgr2=0x7f37901a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f378e59c700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3790105040 0x7f37901a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f378e59c700 1 -- 172.21.15.55:0/2105834693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37901046a0 msgr2=0x7f37901999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f378e59c700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37901046a0 0x7f37901999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f378e59c700 1 -- 172.21.15.55:0/2105834693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37901a10e0 con 0x7f37901034a0 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f378ed9d700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3790105040 0x7f37901a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f378e59c700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37901034a0 0x7f3790199480 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f378000ea20 tx=0x7f378000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f377f7fe700 1 -- 172.21.15.55:0/2105834693 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378000cbd0 con 0x7f37901034a0 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f377f7fe700 1 -- 172.21.15.55:0/2105834693 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f378000ee50 con 0x7f37901034a0 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37901a13d0 con 0x7f37901034a0 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f377f7fe700 1 -- 172.21.15.55:0/2105834693 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780003ce0 con 0x7f37901034a0 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.661+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37901a1910 con 0x7f37901034a0 2024-04-26T21:48:49.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.662+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379006a110 con 0x7f37901034a0 2024-04-26T21:48:49.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.663+0000 7f377f7fe700 1 -- 172.21.15.55:0/2105834693 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3780003e80 con 0x7f37901034a0 2024-04-26T21:48:49.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.664+0000 7f377f7fe700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f37780a3c40 0x7f37780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:48:49.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.664+0000 7f378dd9b700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f37780a3c40 0x7f37780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:48:49.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.664+0000 7f377f7fe700 1 -- 172.21.15.55:0/2105834693 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7f37800cfd30 con 0x7f37901034a0 2024-04-26T21:48:49.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.665+0000 7f378dd9b700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f37780a3c40 0x7f37780a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f378400ac50 tx=0x7f378400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:48:49.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.668+0000 7f377f7fe700 1 -- 172.21.15.55:0/2105834693 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f378009d0b0 con 0x7f37901034a0 2024-04-26T21:48:49.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:48:49.886+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3790063860 con 0x7f37780a3c40 2024-04-26T21:48:50.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:50 smithi055 ceph-mon[29221]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:48:50.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:50.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:50.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:50.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:50 smithi064 ceph-mon[32765]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:48:50.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:50.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:50.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:50.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:50 smithi165 ceph-mon[33296]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T21:48:50.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:48:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:48:50.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:51.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:51 smithi055 ceph-mon[29221]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:48:51.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:51 smithi055 ceph-mon[29221]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:51.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:51 smithi055 ceph-mon[29221]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:48:51.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:51 smithi064 ceph-mon[32765]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:48:51.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:51 smithi064 ceph-mon[32765]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:51.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:51 smithi064 ceph-mon[32765]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:48:51.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:51 smithi165 ceph-mon[33296]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:48:51.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:51 smithi165 ceph-mon[33296]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:48:51.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:51 smithi165 ceph-mon[33296]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T21:48:53.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:53 smithi055 ceph-mon[29221]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-26T21:48:53.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:53 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4253397799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9769f00-30fe-4ecd-9e58-b81cc6e94d63"}]: dispatch 2024-04-26T21:48:53.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:53 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4253397799' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9769f00-30fe-4ecd-9e58-b81cc6e94d63"}]': finished 2024-04-26T21:48:53.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:53 smithi055 ceph-mon[29221]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T21:48:53.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:48:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:53 smithi064 ceph-mon[32765]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-26T21:48:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:53 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4253397799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9769f00-30fe-4ecd-9e58-b81cc6e94d63"}]: dispatch 2024-04-26T21:48:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:53 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4253397799' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9769f00-30fe-4ecd-9e58-b81cc6e94d63"}]': finished 2024-04-26T21:48:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:53 smithi064 ceph-mon[32765]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T21:48:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:48:53.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:53 smithi165 ceph-mon[33296]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-04-26T21:48:53.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:53 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4253397799' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9769f00-30fe-4ecd-9e58-b81cc6e94d63"}]: dispatch 2024-04-26T21:48:53.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:53 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4253397799' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9769f00-30fe-4ecd-9e58-b81cc6e94d63"}]': finished 2024-04-26T21:48:53.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:53 smithi165 ceph-mon[33296]: osdmap e64: 10 total, 9 up, 10 in 2024-04-26T21:48:53.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:48:54.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:54 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/472706186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:54.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:54 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/472706186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:54.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:54 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/472706186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:48:55.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:55 smithi055 ceph-mon[29221]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:48:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:55 smithi064 ceph-mon[32765]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:48:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:55 smithi165 ceph-mon[33296]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-26T21:48:57.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:57 smithi055 ceph-mon[29221]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-26T21:48:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:57 smithi064 ceph-mon[32765]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-26T21:48:57.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:57 smithi165 ceph-mon[33296]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-26T21:48:59.173 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:48:59 smithi055 ceph-mon[29221]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:48:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:48:59 smithi064 ceph-mon[32765]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:48:59.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:48:59 smithi165 ceph-mon[33296]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T21:49:01.414 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:01 smithi055 ceph-mon[29221]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T21:49:01.414 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:49:01.414 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:01 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:01 smithi064 ceph-mon[32765]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T21:49:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:01 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:49:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:01 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:01.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:01 smithi165 ceph-mon[33296]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-26T21:49:01.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:01 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T21:49:01.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:01 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:02.398 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:02 smithi055 ceph-mon[29221]: Deploying daemon osd.9 on smithi055 2024-04-26T21:49:02.399 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:02 smithi055 ceph-mon[29221]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:02.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:02 smithi064 ceph-mon[32765]: Deploying daemon osd.9 on smithi055 2024-04-26T21:49:02.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:02 smithi064 ceph-mon[32765]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:02.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:02 smithi165 ceph-mon[33296]: Deploying daemon osd.9 on smithi055 2024-04-26T21:49:02.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:02 smithi165 ceph-mon[33296]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:04.625 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:04 smithi055 ceph-mon[29221]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:04.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:04 smithi064 ceph-mon[32765]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:04.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:04 smithi165 ceph-mon[33296]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:06.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:06 smithi055 ceph-mon[29221]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:06 smithi064 ceph-mon[32765]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:06.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:06 smithi165 ceph-mon[33296]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:07.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:07.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:07.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:07.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:07.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:07.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:08.553 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:08 smithi055 ceph-mon[29221]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:08.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:08 smithi165 ceph-mon[33296]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:08 smithi064 ceph-mon[32765]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.563+0000 7f377f7fe700 1 -- 172.21.15.55:0/2105834693 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3790063860 con 0x7f37780a3c40 2024-04-26T21:49:10.564 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 9 on host 'smithi055' 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f37780a3c40 msgr2=0x7f37780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f37780a3c40 0x7f37780a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f378400ac50 tx=0x7f378400a380 comp rx=0 tx=0).stop 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37901034a0 msgr2=0x7f3790199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37901034a0 0x7f3790199480 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f378000ea20 tx=0x7f378000ed30 comp rx=0 tx=0).stop 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 shutdown_connections 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3790105040 0x7f37901a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37901034a0 0x7f3790199480 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f37780a3c40 0x7f37780a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 --2- 172.21.15.55:0/2105834693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37901046a0 0x7f37901999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 >> 172.21.15.55:0/2105834693 conn(0x7f37900fea40 msgr2=0x7f3790107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 shutdown_connections 2024-04-26T21:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:49:10.566+0000 7f3794f6a700 1 -- 172.21.15.55:0/2105834693 wait complete. 2024-04-26T21:49:10.876 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:10.876 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:10.877 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:10.877 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:10.877 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:10.877 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:10.877 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:49:10.877 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:10.877 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:10.877 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:10 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:49:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:10 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:11.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:11.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:10 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:11.224 DEBUG:teuthology.orchestra.run.smithi055:osd.9> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T21:49:11.226 INFO:tasks.cephadm:Deploying osd.10 on smithi064 with /dev/vg_nvme/lv_1... 2024-04-26T21:49:11.226 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:49:11.483 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:49:11.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:11 smithi064 ceph-mon[32765]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:49:11.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:11 smithi064 ceph-mon[32765]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:49:11.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:11 smithi064 ceph-mon[32765]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:49:11.858 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:11 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:11.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:11 smithi055 ceph-mon[29221]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:49:11.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:11 smithi055 ceph-mon[29221]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:49:11.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:11 smithi055 ceph-mon[29221]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:49:11.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:11 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:12.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:11 smithi165 ceph-mon[33296]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T21:49:12.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:11 smithi165 ceph-mon[33296]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T21:49:12.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:11 smithi165 ceph-mon[33296]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-04-26T21:49:12.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:11 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:12.437 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 21:49:12 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T21:49:12.142+0000 7f35f9d10700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T21:49:12.437 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 21:49:12 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T21:49:12.158+0000 7f35f1afb700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:49:12.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:12 smithi055 ceph-mon[29221]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:12.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:12 smithi055 ceph-mon[29221]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:49:12.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:12 smithi055 ceph-mon[29221]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:49:12.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:12 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:12.927 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:12 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:12.928 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:12 smithi055 ceph-mon[29221]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' 2024-04-26T21:49:12.982 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:12 smithi064 ceph-mon[32765]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:12.982 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:12 smithi064 ceph-mon[32765]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:49:12.982 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:12 smithi064 ceph-mon[32765]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:49:12.982 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:12 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:12.982 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:12 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:12.982 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:12 smithi064 ceph-mon[32765]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' 2024-04-26T21:49:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:12 smithi165 ceph-mon[33296]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:12 smithi165 ceph-mon[33296]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-04-26T21:49:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:12 smithi165 ceph-mon[33296]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T21:49:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:12 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:12 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:13.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:12 smithi165 ceph-mon[33296]: from='osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676]' entity='osd.9' 2024-04-26T21:49:13.699 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:49:13.699 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:49:13.699 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-04-26T21:49:13.699 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-04-26T21:49:13.699 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115829 s, 905 MB/s 2024-04-26T21:49:13.700 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-04-26T21:49:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:13 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:49:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:13 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:49:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:13 smithi064 ceph-mon[32765]: osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] boot 2024-04-26T21:49:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:13 smithi064 ceph-mon[32765]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T21:49:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:13 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:13 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:49:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:13 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:49:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:13 smithi165 ceph-mon[33296]: osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] boot 2024-04-26T21:49:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:13 smithi165 ceph-mon[33296]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T21:49:14.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:13 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:14.104 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:13 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:49:14.104 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:13 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:49:14.104 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:14.104 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:13 smithi055 ceph-mon[29221]: osd.9 [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] boot 2024-04-26T21:49:14.104 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:13 smithi055 ceph-mon[29221]: osdmap e67: 10 total, 10 up, 10 in 2024-04-26T21:49:14.104 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:13 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T21:49:14.434 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-04-26T21:49:14.692 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T21:49:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:49:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:15.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:15.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:15 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:15.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:15.509 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:15 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:15.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:15.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:15.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:15 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:16.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.278+0000 7f46c1396700 1 -- 172.21.15.64:0/2512009545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc1050b0 msgr2=0x7f46bc10b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.278+0000 7f46c1396700 1 --2- 172.21.15.64:0/2512009545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc1050b0 0x7f46bc10b960 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f46b400b0a0 tx=0x7f46b400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.278+0000 7f46c1396700 1 -- 172.21.15.64:0/2512009545 shutdown_connections 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.278+0000 7f46c1396700 1 --2- 172.21.15.64:0/2512009545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc1050b0 0x7f46bc10b960 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.278+0000 7f46c1396700 1 --2- 172.21.15.64:0/2512009545 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46bc101730 0x7f46bc101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.278+0000 7f46c1396700 1 --2- 172.21.15.64:0/2512009545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc104710 0x7f46bc104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.279+0000 7f46c1396700 1 -- 172.21.15.64:0/2512009545 >> 172.21.15.64:0/2512009545 conn(0x7f46bc0fd0b0 msgr2=0x7f46bc0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.279+0000 7f46c1396700 1 -- 172.21.15.64:0/2512009545 shutdown_connections 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.279+0000 7f46c1396700 1 -- 172.21.15.64:0/2512009545 wait complete. 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.279+0000 7f46c1396700 1 Processor -- start 2024-04-26T21:49:16.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.279+0000 7f46c1396700 1 -- start start 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.279+0000 7f46c1396700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc101730 0x7f46bc104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.279+0000 7f46c1396700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc104710 0x7f46bc1026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46c1396700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46bc1050b0 0x7f46bc102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46c1396700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc10f040 con 0x7f46bc104710 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46c1396700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc10eec0 con 0x7f46bc101730 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46c1396700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc10f1c0 con 0x7f46bc1050b0 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46bb7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46bc1050b0 0x7f46bc102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46baffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc101730 0x7f46bc104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46ba7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc104710 0x7f46bc1026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:16.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46bb7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46bc1050b0 0x7f46bc102c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.64:33194/0 (socket says 172.21.15.64:33194) 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46bb7fe700 1 -- 172.21.15.64:0/729946554 learned_addr learned my addr 172.21.15.64:0/729946554 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46baffd700 1 -- 172.21.15.64:0/729946554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46bc1050b0 msgr2=0x7f46bc102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46baffd700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46bc1050b0 0x7f46bc102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46baffd700 1 -- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc104710 msgr2=0x7f46bc1026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46baffd700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc104710 0x7f46bc1026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46baffd700 1 -- 172.21.15.64:0/729946554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc1034b0 con 0x7f46bc101730 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46baffd700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc101730 0x7f46bc104070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f46bc06af00 tx=0x7f46ac00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.280+0000 7f46a3fff700 1 -- 172.21.15.64:0/729946554 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac015020 con 0x7f46bc101730 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.281+0000 7f46a3fff700 1 -- 172.21.15.64:0/729946554 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46ac00be30 con 0x7f46bc101730 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.281+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc1a38a0 con 0x7f46bc101730 2024-04-26T21:49:16.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.281+0000 7f46a3fff700 1 -- 172.21.15.64:0/729946554 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac010bb0 con 0x7f46bc101730 2024-04-26T21:49:16.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.281+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc1a3de0 con 0x7f46bc101730 2024-04-26T21:49:16.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.282+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46bc06a110 con 0x7f46bc101730 2024-04-26T21:49:16.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.283+0000 7f46a3fff700 1 -- 172.21.15.64:0/729946554 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f46ac018020 con 0x7f46bc101730 2024-04-26T21:49:16.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.283+0000 7f46a3fff700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46a40a3c40 0x7f46a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:16.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.283+0000 7f46a3fff700 1 -- 172.21.15.64:0/729946554 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f46ac0cf7d0 con 0x7f46bc101730 2024-04-26T21:49:16.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.284+0000 7f46ba7fc700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46a40a3c40 0x7f46a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:16.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.284+0000 7f46ba7fc700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46a40a3c40 0x7f46a40a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009a10 tx=0x7f46b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:49:16.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.287+0000 7f46a3fff700 1 -- 172.21.15.64:0/729946554 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f46ac01f020 con 0x7f46bc101730 2024-04-26T21:49:16.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:16 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi055 2024-04-26T21:49:16.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:16 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi055 to 4273M 2024-04-26T21:49:16.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:16.504+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f46bc063860 con 0x7f46a40a3c40 2024-04-26T21:49:16.504 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:16 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi055 2024-04-26T21:49:16.504 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:16 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi055 to 4273M 2024-04-26T21:49:16.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:16 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi055 2024-04-26T21:49:16.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:16 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi055 to 4273M 2024-04-26T21:49:17.404 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:17 smithi064 ceph-mon[32765]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:17.404 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:17.404 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:17.404 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:17.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:17 smithi055 ceph-mon[29221]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:17.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:17.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:17 smithi165 ceph-mon[33296]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:17.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:17.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:17.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:18.474 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:18 smithi064 ceph-mon[32765]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:18.474 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:18 smithi064 ceph-mon[32765]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:18.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:18 smithi165 ceph-mon[33296]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:18.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:18 smithi165 ceph-mon[33296]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:18.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:18 smithi055 ceph-mon[29221]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:18.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:18 smithi055 ceph-mon[29221]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:19.672 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:19 smithi064 ceph-mon[32765]: from='client.? 172.21.15.64:0/3972919806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]: dispatch 2024-04-26T21:49:19.672 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:19 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]: dispatch 2024-04-26T21:49:19.672 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:19 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]': finished 2024-04-26T21:49:19.672 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:19 smithi064 ceph-mon[32765]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T21:49:19.672 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:19 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:19.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:19 smithi055 ceph-mon[29221]: from='client.? 172.21.15.64:0/3972919806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]: dispatch 2024-04-26T21:49:19.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:19 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]: dispatch 2024-04-26T21:49:19.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:19 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]': finished 2024-04-26T21:49:19.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:19 smithi055 ceph-mon[29221]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T21:49:19.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:19 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:19.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:19 smithi165 ceph-mon[33296]: from='client.? 172.21.15.64:0/3972919806' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]: dispatch 2024-04-26T21:49:19.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:19 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]: dispatch 2024-04-26T21:49:19.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:19 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5"}]': finished 2024-04-26T21:49:19.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:19 smithi165 ceph-mon[33296]: osdmap e69: 11 total, 10 up, 11 in 2024-04-26T21:49:19.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:19 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:20 smithi055 ceph-mon[29221]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:20 smithi055 ceph-mon[29221]: from='client.? 172.21.15.64:0/936222309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:20.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:20 smithi064 ceph-mon[32765]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:20.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:20 smithi064 ceph-mon[32765]: from='client.? 172.21.15.64:0/936222309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:20.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:20 smithi165 ceph-mon[33296]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:20.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:20 smithi165 ceph-mon[33296]: from='client.? 172.21.15.64:0/936222309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:22.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:22 smithi055 ceph-mon[29221]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:22.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:22 smithi064 ceph-mon[32765]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:22.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:22 smithi165 ceph-mon[33296]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:24.585 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:24 smithi064 ceph-mon[32765]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:24.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:24 smithi055 ceph-mon[29221]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:24.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:24 smithi165 ceph-mon[33296]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:26.609 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:26 smithi064 ceph-mon[32765]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:26.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:26 smithi055 ceph-mon[29221]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:26.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:26 smithi165 ceph-mon[33296]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:27.553 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:49:27.553 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:27 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:27.553 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:27 smithi064 ceph-mon[32765]: Deploying daemon osd.10 on smithi064 2024-04-26T21:49:27.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:49:27.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:27 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:27.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:27 smithi055 ceph-mon[29221]: Deploying daemon osd.10 on smithi064 2024-04-26T21:49:27.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T21:49:27.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:27 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:27.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:27 smithi165 ceph-mon[33296]: Deploying daemon osd.10 on smithi064 2024-04-26T21:49:28.627 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:28 smithi064 ceph-mon[32765]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:28.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:28 smithi055 ceph-mon[29221]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:28.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:28 smithi165 ceph-mon[33296]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:30 smithi064 ceph-mon[32765]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:30.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:30 smithi165 ceph-mon[33296]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:30.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:30 smithi055 ceph-mon[29221]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:32.486 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:32 smithi064 ceph-mon[32765]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:32.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:32 smithi165 ceph-mon[33296]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:32.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:32 smithi055 ceph-mon[29221]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:33.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:33 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:33.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:33 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:33.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:33 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:33.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:33.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:33.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:33.838 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:33 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:33.838 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:33 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:33.838 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:33 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:34 smithi064 ceph-mon[32765]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:34.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:34 smithi165 ceph-mon[33296]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:34.849 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:34 smithi055 ceph-mon[29221]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:36.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.424+0000 7f46a3fff700 1 -- 172.21.15.64:0/729946554 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f46bc063860 con 0x7f46a40a3c40 2024-04-26T21:49:36.425 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 10 on host 'smithi064' 2024-04-26T21:49:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46a40a3c40 msgr2=0x7f46a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46a40a3c40 0x7f46a40a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009a10 tx=0x7f46b0009190 comp rx=0 tx=0).stop 2024-04-26T21:49:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc101730 msgr2=0x7f46bc104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc101730 0x7f46bc104070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f46bc06af00 tx=0x7f46ac00b730 comp rx=0 tx=0).stop 2024-04-26T21:49:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 shutdown_connections 2024-04-26T21:49:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46bc1050b0 0x7f46bc102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:36.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46a40a3c40 0x7f46a40a6100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:36.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46bc101730 0x7f46bc104070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:36.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 --2- 172.21.15.64:0/729946554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46bc104710 0x7f46bc1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:36.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.426+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 >> 172.21.15.64:0/729946554 conn(0x7f46bc0fd0b0 msgr2=0x7f46bc107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:49:36.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.427+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 shutdown_connections 2024-04-26T21:49:36.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-04-26T21:49:36.427+0000 7f46c1396700 1 -- 172.21.15.64:0/729946554 wait complete. 2024-04-26T21:49:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:49:36.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:49:36.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:36 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:36.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:36.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:36.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:36.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:49:36.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:49:36.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:36.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:36 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:37.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:37.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:36 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:37.103 DEBUG:teuthology.orchestra.run.smithi064:osd.10> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T21:49:37.105 INFO:tasks.cephadm:Deploying osd.11 on smithi165 with /dev/vg_nvme/lv_1... 2024-04-26T21:49:37.105 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T21:49:37.351 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:49:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:37 smithi064 ceph-mon[32765]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:49:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:37 smithi064 ceph-mon[32765]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:49:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:37 smithi064 ceph-mon[32765]: from='osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:49:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:37 smithi064 ceph-mon[32765]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:49:37.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:37 smithi165 ceph-mon[33296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:49:37.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:37 smithi165 ceph-mon[33296]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:49:37.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:37.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:37 smithi165 ceph-mon[33296]: from='osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:49:37.821 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:37 smithi165 ceph-mon[33296]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:49:37.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:37 smithi055 ceph-mon[29221]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T21:49:37.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:37 smithi055 ceph-mon[29221]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T21:49:37.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:37.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:37 smithi055 ceph-mon[29221]: from='osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:49:37.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:37 smithi055 ceph-mon[29221]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-04-26T21:49:38.246 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 21:49:38 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T21:49:38.095+0000 7fa4760bc700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T21:49:38.246 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 21:49:38 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T21:49:38.108+0000 7fa46d6a6700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:49:38.750 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:38 smithi165 ceph-mon[33296]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:38.751 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:38 smithi165 ceph-mon[33296]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:49:38.751 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:38 smithi165 ceph-mon[33296]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:49:38.751 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:38.751 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:38.751 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:38 smithi165 ceph-mon[33296]: from='osd.10 ' entity='osd.10' 2024-04-26T21:49:38.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:38 smithi055 ceph-mon[29221]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:38.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:38 smithi055 ceph-mon[29221]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:49:38.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:38 smithi055 ceph-mon[29221]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:49:38.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:38.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:38.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:38 smithi055 ceph-mon[29221]: from='osd.10 ' entity='osd.10' 2024-04-26T21:49:39.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:38 smithi064 ceph-mon[32765]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 894 GiB / 894 GiB avail 2024-04-26T21:49:39.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:38 smithi064 ceph-mon[32765]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-04-26T21:49:39.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:38 smithi064 ceph-mon[32765]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T21:49:39.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:39.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:39.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:38 smithi064 ceph-mon[32765]: from='osd.10 ' entity='osd.10' 2024-04-26T21:49:39.458 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T21:49:39.458 INFO:teuthology.orchestra.run.smithi165.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T21:49:39.458 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10+0 records in 2024-04-26T21:49:39.458 INFO:teuthology.orchestra.run.smithi165.stderr:10+0 records out 2024-04-26T21:49:39.458 INFO:teuthology.orchestra.run.smithi165.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128642 s, 815 MB/s 2024-04-26T21:49:39.458 INFO:teuthology.orchestra.run.smithi165.stderr:--> Zapping successful for: 2024-04-26T21:49:39.767 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:39 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:49:39.767 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:39 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:49:39.767 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:39 smithi165 ceph-mon[33296]: osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] boot 2024-04-26T21:49:39.767 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:39 smithi165 ceph-mon[33296]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T21:49:39.767 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:39 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:39.767 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:39 smithi165 ceph-mon[33296]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:49:39.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:39 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:49:39.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:39 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:49:39.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:39 smithi055 ceph-mon[29221]: osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] boot 2024-04-26T21:49:39.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:39 smithi055 ceph-mon[29221]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T21:49:39.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:39 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:39.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:39 smithi055 ceph-mon[29221]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:49:40.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:39 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:49:40.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:39 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:49:40.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:39 smithi064 ceph-mon[32765]: osd.10 [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] boot 2024-04-26T21:49:40.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:39 smithi064 ceph-mon[32765]: osdmap e72: 11 total, 11 up, 11 in 2024-04-26T21:49:40.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:39 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T21:49:40.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:39 smithi064 ceph-mon[32765]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T21:49:40.113 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-04-26T21:49:40.362 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:49:40.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:40.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:40.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:40.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:40.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:40.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:40.814 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:40.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:40.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:40.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:40.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:40.815 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:40 smithi165 ceph-mon[33296]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:49:40.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:40.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:40.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:40 smithi055 ceph-mon[29221]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:49:41.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:41.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:49:41.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:40 smithi064 ceph-mon[32765]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T21:49:41.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:41 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi064 2024-04-26T21:49:41.934 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:41 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi064 to 4272M 2024-04-26T21:49:41.934 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:41 smithi165 ceph-mon[33296]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:49:41.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:41 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi064 2024-04-26T21:49:41.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:41 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi064 to 4272M 2024-04-26T21:49:41.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:41 smithi055 ceph-mon[29221]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:49:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:41 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi064 2024-04-26T21:49:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:41 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi064 to 4272M 2024-04-26T21:49:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:41 smithi064 ceph-mon[32765]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T21:49:42.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 -- 172.21.15.165:0/899711762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c102380 msgr2=0x7f767c1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:42.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 --2- 172.21.15.165:0/899711762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c102380 0x7f767c1027a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f766c009990 tx=0x7f766c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:49:42.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 -- 172.21.15.165:0/899711762 shutdown_connections 2024-04-26T21:49:42.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 --2- 172.21.15.165:0/899711762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f767c103f20 0x7f767c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:42.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 --2- 172.21.15.165:0/899711762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c102380 0x7f767c1027a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:42.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 --2- 172.21.15.165:0/899711762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767c103580 0x7f767c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 -- 172.21.15.165:0/899711762 >> 172.21.15.165:0/899711762 conn(0x7f767c0fd920 msgr2=0x7f767c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 -- 172.21.15.165:0/899711762 shutdown_connections 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 -- 172.21.15.165:0/899711762 wait complete. 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 Processor -- start 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.167+0000 7f768256b700 1 -- start start 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f768256b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f767c102380 0x7f767c06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f768256b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c103580 0x7f767c19e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f768256b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767c103f20 0x7f767c1a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f768256b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f767c10ddf0 con 0x7f767c102380 2024-04-26T21:49:42.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f768256b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f767c10dc70 con 0x7f767c103f20 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f768256b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f767c10df70 con 0x7f767c103580 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c103580 0x7f767c19e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c103580 0x7f767c19e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44662/0 (socket says 172.21.15.165:44662) 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 -- 172.21.15.165:0/2261441441 learned_addr learned my addr 172.21.15.165:0/2261441441 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767bfff700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f767c102380 0x7f767c06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f7680b08700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767c103f20 0x7f767c1a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 -- 172.21.15.165:0/2261441441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767c103f20 msgr2=0x7f767c1a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767c103f20 0x7f767c1a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 -- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f767c102380 msgr2=0x7f767c06c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f767c102380 0x7f767c06c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:49:42.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 -- 172.21.15.165:0/2261441441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c1a1190 con 0x7f767c103580 2024-04-26T21:49:42.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.168+0000 7f767b7fe700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c103580 0x7f767c19e2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f767000c9b0 tx=0x7f767000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:49:42.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.169+0000 7f76797fa700 1 -- 172.21.15.165:0/2261441441 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670007b00 con 0x7f767c103580 2024-04-26T21:49:42.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.169+0000 7f76797fa700 1 -- 172.21.15.165:0/2261441441 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7670007ca0 con 0x7f767c103580 2024-04-26T21:49:42.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.169+0000 7f76797fa700 1 -- 172.21.15.165:0/2261441441 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670018660 con 0x7f767c103580 2024-04-26T21:49:42.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.169+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c1a1480 con 0x7f767c103580 2024-04-26T21:49:42.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.169+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c1a19c0 con 0x7f767c103580 2024-04-26T21:49:42.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.170+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767c06a110 con 0x7f767c103580 2024-04-26T21:49:42.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.171+0000 7f76797fa700 1 -- 172.21.15.165:0/2261441441 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f767000f450 con 0x7f767c103580 2024-04-26T21:49:42.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.171+0000 7f76797fa700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76640a3c40 0x7f76640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:49:42.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.172+0000 7f76797fa700 1 -- 172.21.15.165:0/2261441441 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f76700cfd70 con 0x7f767c103580 2024-04-26T21:49:42.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.172+0000 7f767bfff700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76640a3c40 0x7f76640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:49:42.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.172+0000 7f767bfff700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76640a3c40 0x7f76640a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f766c0038a0 tx=0x7f766c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:49:42.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.176+0000 7f76797fa700 1 -- 172.21.15.165:0/2261441441 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7670014030 con 0x7f767c103580 2024-04-26T21:49:42.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:49:42.382+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f767c063860 con 0x7f76640a3c40 2024-04-26T21:49:42.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:42 smithi055 ceph-mon[29221]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-04-26T21:49:42.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:42 smithi055 ceph-mon[29221]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:49:42.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:42 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:42.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:42 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:42.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:42 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:42.958 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:42 smithi064 ceph-mon[32765]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-04-26T21:49:42.958 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:42 smithi064 ceph-mon[32765]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:49:42.958 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:42.959 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:42.959 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:42 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:42.994 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:42 smithi165 ceph-mon[33296]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-04-26T21:49:42.994 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:42 smithi165 ceph-mon[33296]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T21:49:42.994 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:42 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T21:49:42.994 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:42 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T21:49:42.995 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:42 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:43.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:43 smithi055 ceph-mon[29221]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:44.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:43 smithi064 ceph-mon[32765]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:44.032 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:43 smithi165 ceph-mon[33296]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:49:44.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:44 smithi055 ceph-mon[29221]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-26T21:49:45.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:44 smithi064 ceph-mon[32765]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-26T21:49:45.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:44 smithi165 ceph-mon[33296]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-26T21:49:45.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:45 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/3317039314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]: dispatch 2024-04-26T21:49:45.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:45 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]: dispatch 2024-04-26T21:49:45.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:45 smithi055 ceph-mon[29221]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]': finished 2024-04-26T21:49:45.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:45 smithi055 ceph-mon[29221]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T21:49:45.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:45 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:49:46.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:45 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/3317039314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]: dispatch 2024-04-26T21:49:46.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:45 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]: dispatch 2024-04-26T21:49:46.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:45 smithi064 ceph-mon[32765]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]': finished 2024-04-26T21:49:46.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:45 smithi064 ceph-mon[32765]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T21:49:46.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:45 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:49:46.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:45 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/3317039314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]: dispatch 2024-04-26T21:49:46.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:45 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]: dispatch 2024-04-26T21:49:46.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:45 smithi165 ceph-mon[33296]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09405356-ca7d-48a4-8cc2-caf08446e063"}]': finished 2024-04-26T21:49:46.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:45 smithi165 ceph-mon[33296]: osdmap e77: 12 total, 11 up, 12 in 2024-04-26T21:49:46.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:45 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:49:46.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:46 smithi055 ceph-mon[29221]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:49:46.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:46 smithi055 ceph-mon[29221]: from='client.? 172.21.15.165:0/3328699250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:47.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:46 smithi064 ceph-mon[32765]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:49:47.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:46 smithi064 ceph-mon[32765]: from='client.? 172.21.15.165:0/3328699250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:47.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:46 smithi165 ceph-mon[33296]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T21:49:47.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:46 smithi165 ceph-mon[33296]: from='client.? 172.21.15.165:0/3328699250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T21:49:48.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:48 smithi055 ceph-mon[29221]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:48 smithi064 ceph-mon[32765]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:49.043 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:48 smithi165 ceph-mon[33296]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:50.795 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:50 smithi165 ceph-mon[33296]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:50.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:50 smithi055 ceph-mon[29221]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:50 smithi064 ceph-mon[32765]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:52.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:52 smithi055 ceph-mon[29221]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:52 smithi064 ceph-mon[32765]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:53.007 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:52 smithi165 ceph-mon[33296]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:53.917 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:49:53.918 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:53 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:53.918 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:53 smithi165 ceph-mon[33296]: Deploying daemon osd.11 on smithi165 2024-04-26T21:49:53.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:49:53.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:53 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:53.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:53 smithi055 ceph-mon[29221]: Deploying daemon osd.11 on smithi165 2024-04-26T21:49:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T21:49:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:53 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:53 smithi064 ceph-mon[32765]: Deploying daemon osd.11 on smithi165 2024-04-26T21:49:54.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:54 smithi165 ceph-mon[33296]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:55.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:54 smithi064 ceph-mon[32765]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:55.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:54 smithi055 ceph-mon[29221]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:57.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:56 smithi064 ceph-mon[32765]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:57.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:56 smithi165 ceph-mon[33296]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:57.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:56 smithi055 ceph-mon[29221]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:58.898 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:49:58 smithi165 ceph-mon[33296]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:59.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:49:58 smithi064 ceph-mon[32765]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:49:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:49:58 smithi055 ceph-mon[29221]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:00.515 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:00.515 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:00.515 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:00.515 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:00 smithi165 ceph-mon[33296]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:00.515 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:00 smithi165 ceph-mon[33296]: overall HEALTH_OK 2024-04-26T21:50:00.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:00.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:00.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:00.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:00 smithi055 ceph-mon[29221]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:00.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:00 smithi055 ceph-mon[29221]: overall HEALTH_OK 2024-04-26T21:50:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:00 smithi064 ceph-mon[32765]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:00 smithi064 ceph-mon[32765]: overall HEALTH_OK 2024-04-26T21:50:02.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:02 smithi055 ceph-mon[29221]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:02.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:02 smithi064 ceph-mon[32765]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:02.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:02 smithi165 ceph-mon[33296]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:03.308 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 11 on host 'smithi165' 2024-04-26T21:50:03.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.307+0000 7f76797fa700 1 -- 172.21.15.165:0/2261441441 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f767c063860 con 0x7f76640a3c40 2024-04-26T21:50:03.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.309+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76640a3c40 msgr2=0x7f76640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:03.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.309+0000 7f768256b700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76640a3c40 0x7f76640a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f766c0038a0 tx=0x7f766c005fb0 comp rx=0 tx=0).stop 2024-04-26T21:50:03.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.309+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c103580 msgr2=0x7f767c19e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.309+0000 7f768256b700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c103580 0x7f767c19e2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f767000c9b0 tx=0x7f767000ccc0 comp rx=0 tx=0).stop 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.310+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 shutdown_connections 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.310+0000 7f768256b700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76640a3c40 0x7f76640a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.310+0000 7f768256b700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767c103f20 0x7f767c1a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.310+0000 7f768256b700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f767c102380 0x7f767c06c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.310+0000 7f768256b700 1 --2- 172.21.15.165:0/2261441441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f767c103580 0x7f767c19e2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.310+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 >> 172.21.15.165:0/2261441441 conn(0x7f767c0fd920 msgr2=0x7f767c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.310+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 shutdown_connections 2024-04-26T21:50:03.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:03.310+0000 7f768256b700 1 -- 172.21.15.165:0/2261441441 wait complete. 2024-04-26T21:50:03.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:03.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:03.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:50:03.826 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:03 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:50:03.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:50:03.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:03 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:03.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:03.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:03 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:03.987 DEBUG:teuthology.orchestra.run.smithi165:osd.11> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T21:50:03.990 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T21:50:03.990 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T21:50:04.250 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:50:04.634 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:04 smithi055 ceph-mon[29221]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:04.635 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:04 smithi055 ceph-mon[29221]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:50:04.635 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:04 smithi055 ceph-mon[29221]: from='osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:50:04.635 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:04 smithi055 ceph-mon[29221]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:50:04.635 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:04 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:04.635 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:04 smithi055 ceph-mon[29221]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:50:04.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:04 smithi165 ceph-mon[33296]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:04.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:04 smithi165 ceph-mon[33296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:50:04.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:04 smithi165 ceph-mon[33296]: from='osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:50:04.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:04 smithi165 ceph-mon[33296]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:50:04.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:04 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:04.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:04 smithi165 ceph-mon[33296]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:50:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:04 smithi064 ceph-mon[32765]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:04 smithi064 ceph-mon[32765]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T21:50:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:04 smithi064 ceph-mon[32765]: from='osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:50:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:04 smithi064 ceph-mon[32765]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T21:50:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:04 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:04 smithi064 ceph-mon[32765]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]: dispatch 2024-04-26T21:50:05.327 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 21:50:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T21:50:04.951+0000 7f8d85f04700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T21:50:05.327 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 21:50:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T21:50:04.967+0000 7f8d7d4ee700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T21:50:05.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:05 smithi055 ceph-mon[29221]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:50:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:05 smithi055 ceph-mon[29221]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:50:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:05 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:05 smithi055 ceph-mon[29221]: from='osd.11 ' entity='osd.11' 2024-04-26T21:50:05.757 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:05 smithi165 ceph-mon[33296]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:50:05.757 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:05 smithi165 ceph-mon[33296]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:50:05.757 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:05 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:05.757 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:05 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:05.757 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:05 smithi165 ceph-mon[33296]: from='osd.11 ' entity='osd.11' 2024-04-26T21:50:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:05 smithi064 ceph-mon[32765]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-04-26T21:50:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:05 smithi064 ceph-mon[32765]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T21:50:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:05 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:05 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:05 smithi064 ceph-mon[32765]: from='osd.11 ' entity='osd.11' 2024-04-26T21:50:06.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 -- 172.21.15.55:0/2193556999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 msgr2=0x7f11a810b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:06.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 --2- 172.21.15.55:0/2193556999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 0x7f11a810b5e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f11a000b0a0 tx=0x7f11a000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 -- 172.21.15.55:0/2193556999 shutdown_connections 2024-04-26T21:50:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 --2- 172.21.15.55:0/2193556999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 0x7f11a810b5e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 --2- 172.21.15.55:0/2193556999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a8103f10 0x7f11a8106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 --2- 172.21.15.55:0/2193556999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8106840 0x7f11a8108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 -- 172.21.15.55:0/2193556999 >> 172.21.15.55:0/2193556999 conn(0x7f11a80fd920 msgr2=0x7f11a80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 -- 172.21.15.55:0/2193556999 shutdown_connections 2024-04-26T21:50:06.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 -- 172.21.15.55:0/2193556999 wait complete. 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.107+0000 7f11acb31700 1 Processor -- start 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11acb31700 1 -- start start 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11acb31700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8103f10 0x7f11a81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11acb31700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a8106840 0x7f11a8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11acb31700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 0x7f11a81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11acb31700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810ef40 con 0x7f11a8109170 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11acb31700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810edc0 con 0x7f11a8106840 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11acb31700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810f0c0 con 0x7f11a8103f10 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 0x7f11a81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:06.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 0x7f11a81a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40068/0 (socket says 172.21.15.55:40068) 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 -- 172.21.15.55:0/3063411105 learned_addr learned my addr 172.21.15.55:0/3063411105 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a659c700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8103f10 0x7f11a81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a5d9b700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a8106840 0x7f11a8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 -- 172.21.15.55:0/3063411105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8103f10 msgr2=0x7f11a81995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8103f10 0x7f11a81995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 -- 172.21.15.55:0/3063411105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a8106840 msgr2=0x7f11a8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a8106840 0x7f11a8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 -- 172.21.15.55:0/3063411105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a81a1210 con 0x7f11a8109170 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11a6d9d700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 0x7f11a81a0b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11a0006130 tx=0x7f11a0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11977fe700 1 -- 172.21.15.55:0/3063411105 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0021070 con 0x7f11a8109170 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.108+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a81a14a0 con 0x7f11a8109170 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.109+0000 7f11977fe700 1 -- 172.21.15.55:0/3063411105 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11a0007e30 con 0x7f11a8109170 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.109+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a19a0 con 0x7f11a8109170 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.109+0000 7f11977fe700 1 -- 172.21.15.55:0/3063411105 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0004790 con 0x7f11a8109170 2024-04-26T21:50:06.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.109+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1188005360 con 0x7f11a8109170 2024-04-26T21:50:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.111+0000 7f11977fe700 1 -- 172.21.15.55:0/3063411105 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f11a0007980 con 0x7f11a8109170 2024-04-26T21:50:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.111+0000 7f11977fe700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11900a3c40 0x7f11900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.111+0000 7f11a659c700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11900a3c40 0x7f11900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.111+0000 7f11977fe700 1 -- 172.21.15.55:0/3063411105 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f11a000f020 con 0x7f11a8109170 2024-04-26T21:50:06.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.112+0000 7f11a659c700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11900a3c40 0x7f11900a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1198009c60 tx=0x7f1198009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:06.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.115+0000 7f11977fe700 1 -- 172.21.15.55:0/3063411105 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f11a002a020 con 0x7f11a8109170 2024-04-26T21:50:06.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.325+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1188005920 con 0x7f11a8109170 2024-04-26T21:50:06.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.326+0000 7f11977fe700 1 -- 172.21.15.55:0/3063411105 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f11a009e610 con 0x7f11a8109170 2024-04-26T21:50:06.326 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:06.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11900a3c40 msgr2=0x7f11900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11900a3c40 0x7f11900a6100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1198009c60 tx=0x7f1198009260 comp rx=0 tx=0).stop 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 msgr2=0x7f11a81a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 0x7f11a81a0b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11a0006130 tx=0x7f11a0004db0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 shutdown_connections 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a8109170 0x7f11a81a0b70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8103f10 0x7f11a81995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11900a3c40 0x7f11900a6100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 --2- 172.21.15.55:0/3063411105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11a8106840 0x7f11a8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 >> 172.21.15.55:0/3063411105 conn(0x7f11a80fd920 msgr2=0x7f11a8104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 shutdown_connections 2024-04-26T21:50:06.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:06.328+0000 7f11acb31700 1 -- 172.21.15.55:0/3063411105 wait complete. 2024-04-26T21:50:06.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:06 smithi055 ceph-mon[29221]: purged_snaps scrub starts 2024-04-26T21:50:06.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:06 smithi055 ceph-mon[29221]: purged_snaps scrub ok 2024-04-26T21:50:06.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:06 smithi055 ceph-mon[29221]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:06.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:06 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:06.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:06 smithi055 ceph-mon[29221]: osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] boot 2024-04-26T21:50:06.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:06 smithi055 ceph-mon[29221]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T21:50:06.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:06 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:06.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:06 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3063411105' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:50:06.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:06 smithi165 ceph-mon[33296]: purged_snaps scrub starts 2024-04-26T21:50:06.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:06 smithi165 ceph-mon[33296]: purged_snaps scrub ok 2024-04-26T21:50:06.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:06 smithi165 ceph-mon[33296]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:06.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:06 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:06.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:06 smithi165 ceph-mon[33296]: osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] boot 2024-04-26T21:50:06.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:06 smithi165 ceph-mon[33296]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T21:50:06.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:06 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:06.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:06 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3063411105' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:50:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:06 smithi064 ceph-mon[32765]: purged_snaps scrub starts 2024-04-26T21:50:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:06 smithi064 ceph-mon[32765]: purged_snaps scrub ok 2024-04-26T21:50:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:06 smithi064 ceph-mon[32765]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-04-26T21:50:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:06 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:06 smithi064 ceph-mon[32765]: osd.11 [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] boot 2024-04-26T21:50:07.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:06 smithi064 ceph-mon[32765]: osdmap e80: 12 total, 12 up, 12 in 2024-04-26T21:50:07.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:06 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T21:50:07.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:06 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3063411105' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T21:50:07.266 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1714168205,"num_in_osds":12,"osd_in_since":1714168185,"num_remapped_pgs":0} 2024-04-26T21:50:07.266 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:50:07.514 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:50:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:50:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:08.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:08.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:07 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.299 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:50:08.299 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.299 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.299 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.300 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.300 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.300 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.300 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.300 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:08.300 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:08.300 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:07 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:07 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:08 smithi064 ceph-mon[32765]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:08 smithi064 ceph-mon[32765]: Detected new or changed devices on smithi165 2024-04-26T21:50:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:08 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-26T21:50:09.272 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:08 smithi055 ceph-mon[29221]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:09.272 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:08 smithi055 ceph-mon[29221]: Detected new or changed devices on smithi165 2024-04-26T21:50:09.272 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:08 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-26T21:50:09.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.278+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/2746208575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f681034a0 msgr2=0x7f1f68103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:09.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.278+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/2746208575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f681034a0 0x7f1f68103920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009990 tx=0x7f1f5c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.278+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/2746208575 shutdown_connections 2024-04-26T21:50:09.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.279+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/2746208575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f68103e60 0x7f1f681102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.279+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/2746208575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f681094e0 0x7f1f681098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.279+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/2746208575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f681034a0 0x7f1f68103920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.279+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/2746208575 >> 172.21.15.55:0/2746208575 conn(0x7f1f6807a590 msgr2=0x7f1f6807a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:09.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.279+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/2746208575 shutdown_connections 2024-04-26T21:50:09.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.279+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/2746208575 wait complete. 2024-04-26T21:50:09.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.279+0000 7f1f6e1c2700 1 Processor -- start 2024-04-26T21:50:09.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.279+0000 7f1f6e1c2700 1 -- start start 2024-04-26T21:50:09.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f6e1c2700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f681034a0 0x7f1f6819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:09.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f6e1c2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f68103e60 0x7f1f6819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:09.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f6e1c2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f681094e0 0x7f1f68199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:09.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f6e1c2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68113610 con 0x7f1f68103e60 2024-04-26T21:50:09.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f6e1c2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68113490 con 0x7f1f681094e0 2024-04-26T21:50:09.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f6e1c2700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68113790 con 0x7f1f681034a0 2024-04-26T21:50:09.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f68103e60 0x7f1f6819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f68103e60 0x7f1f6819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40086/0 (socket says 172.21.15.55:40086) 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 -- 172.21.15.55:0/1865844093 learned_addr learned my addr 172.21.15.55:0/1865844093 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f67fff700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f681094e0 0x7f1f68199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f677fe700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f681034a0 0x7f1f6819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 -- 172.21.15.55:0/1865844093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f681034a0 msgr2=0x7f1f6819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f681034a0 0x7f1f6819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 -- 172.21.15.55:0/1865844093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f681094e0 msgr2=0x7f1f68199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f681094e0 0x7f1f68199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 -- 172.21.15.55:0/1865844093 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6819a0f0 con 0x7f1f68103e60 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.280+0000 7f1f66ffd700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f68103e60 0x7f1f6819fd10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000c00 tx=0x7f1f5c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.281+0000 7f1f64ff9700 1 -- 172.21.15.55:0/1865844093 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c027070 con 0x7f1f68103e60 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.281+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6819a380 con 0x7f1f68103e60 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.281+0000 7f1f64ff9700 1 -- 172.21.15.55:0/1865844093 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f5c003f20 con 0x7f1f68103e60 2024-04-26T21:50:09.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.281+0000 7f1f64ff9700 1 -- 172.21.15.55:0/1865844093 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c022ca0 con 0x7f1f68103e60 2024-04-26T21:50:09.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.281+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f681a66a0 con 0x7f1f68103e60 2024-04-26T21:50:09.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.282+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6810b950 con 0x7f1f68103e60 2024-04-26T21:50:09.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.283+0000 7f1f64ff9700 1 -- 172.21.15.55:0/1865844093 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f5c02a070 con 0x7f1f68103e60 2024-04-26T21:50:09.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.283+0000 7f1f64ff9700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1f500a3c40 0x7f1f500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:09.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.284+0000 7f1f677fe700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1f500a3c40 0x7f1f500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:09.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.284+0000 7f1f64ff9700 1 -- 172.21.15.55:0/1865844093 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1f5c0d0e60 con 0x7f1f68103e60 2024-04-26T21:50:09.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.284+0000 7f1f677fe700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1f500a3c40 0x7f1f500a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f6819aad0 tx=0x7f1f580073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:09.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.288+0000 7f1f64ff9700 1 -- 172.21.15.55:0/1865844093 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1f5c05f060 con 0x7f1f68103e60 2024-04-26T21:50:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:08 smithi165 ceph-mon[33296]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:08 smithi165 ceph-mon[33296]: Detected new or changed devices on smithi165 2024-04-26T21:50:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:08 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi165 to 4272M 2024-04-26T21:50:09.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.497+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f6819aec0 con 0x7f1f68103e60 2024-04-26T21:50:09.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.498+0000 7f1f64ff9700 1 -- 172.21.15.55:0/1865844093 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18422 (secure 0 0 0) 0x7f1f5c013070 con 0x7f1f68103e60 2024-04-26T21:50:09.499 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:09.499 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":81,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:50:06.966268+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1f500a3c40 msgr2=0x7f1f500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1f500a3c40 0x7f1f500a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f6819aad0 tx=0x7f1f580073d0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f68103e60 msgr2=0x7f1f6819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f68103e60 0x7f1f6819fd10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000c00 tx=0x7f1f5c0047b0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 shutdown_connections 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1f500a3c40 0x7f1f500a6100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f681094e0 0x7f1f68199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f681034a0 0x7f1f6819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 --2- 172.21.15.55:0/1865844093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f68103e60 0x7f1f6819fd10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 >> 172.21.15.55:0/1865844093 conn(0x7f1f6807a590 msgr2=0x7f1f681022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 shutdown_connections 2024-04-26T21:50:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:09.500+0000 7f1f6e1c2700 1 -- 172.21.15.55:0/1865844093 wait complete. 2024-04-26T21:50:10.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:09 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1865844093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:10.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:09 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1865844093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:10.370 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:09 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1865844093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:10.404 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:50:10.404 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T21:50:10.655 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:50:11.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:10 smithi055 ceph-mon[29221]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:10 smithi064 ceph-mon[32765]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:11.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:10 smithi165 ceph-mon[33296]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:12.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 -- 172.21.15.55:0/1518188429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd0103e60 msgr2=0x7febd01102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:12.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 --2- 172.21.15.55:0/1518188429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd0103e60 0x7febd01102b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7febc800b0a0 tx=0x7febc800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 -- 172.21.15.55:0/1518188429 shutdown_connections 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 --2- 172.21.15.55:0/1518188429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd0103e60 0x7febd01102b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 --2- 172.21.15.55:0/1518188429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febd01094e0 0x7febd01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 --2- 172.21.15.55:0/1518188429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01034a0 0x7febd0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 -- 172.21.15.55:0/1518188429 >> 172.21.15.55:0/1518188429 conn(0x7febd007a590 msgr2=0x7febd007a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 -- 172.21.15.55:0/1518188429 shutdown_connections 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.489+0000 7febd6f9d700 1 -- 172.21.15.55:0/1518188429 wait complete. 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.490+0000 7febd6f9d700 1 Processor -- start 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.490+0000 7febd6f9d700 1 -- start start 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd6f9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01034a0 0x7febd019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd6f9d700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febd0103e60 0x7febd019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd6f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd01094e0 0x7febd01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd6f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7febd01136b0 con 0x7febd01094e0 2024-04-26T21:50:12.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd6f9d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febd0113530 con 0x7febd01034a0 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd6f9d700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7febd0113830 con 0x7febd0103e60 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd01094e0 0x7febd01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd01094e0 0x7febd01a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58284/0 (socket says 172.21.15.55:58284) 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 -- 172.21.15.55:0/2572562269 learned_addr learned my addr 172.21.15.55:0/2572562269 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febcffff700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febd0103e60 0x7febd019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd4d39700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01034a0 0x7febd019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 -- 172.21.15.55:0/2572562269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febd0103e60 msgr2=0x7febd019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febd0103e60 0x7febd019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 -- 172.21.15.55:0/2572562269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01034a0 msgr2=0x7febd019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01034a0 0x7febd019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 -- 172.21.15.55:0/2572562269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd01a0d20 con 0x7febd01094e0 2024-04-26T21:50:12.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd553a700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd01094e0 0x7febd01a05e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7febc8009fd0 tx=0x7febc8004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:12.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febcdffb700 1 -- 172.21.15.55:0/2572562269 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8021070 con 0x7febd01094e0 2024-04-26T21:50:12.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd01a0fb0 con 0x7febd01094e0 2024-04-26T21:50:12.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.491+0000 7febcdffb700 1 -- 172.21.15.55:0/2572562269 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc8007b80 con 0x7febd01094e0 2024-04-26T21:50:12.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.492+0000 7febcdffb700 1 -- 172.21.15.55:0/2572562269 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80052a0 con 0x7febd01094e0 2024-04-26T21:50:12.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.492+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd01a8890 con 0x7febd01094e0 2024-04-26T21:50:12.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.493+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd006a110 con 0x7febd01094e0 2024-04-26T21:50:12.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.494+0000 7febcdffb700 1 -- 172.21.15.55:0/2572562269 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7febc8024070 con 0x7febd01094e0 2024-04-26T21:50:12.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.495+0000 7febcdffb700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7febb80a3c40 0x7febb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:12.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.495+0000 7febcdffb700 1 -- 172.21.15.55:0/2572562269 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7febc800f020 con 0x7febd01094e0 2024-04-26T21:50:12.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.495+0000 7febd4d39700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7febb80a3c40 0x7febb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:12.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.495+0000 7febd4d39700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7febb80a3c40 0x7febb80a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7febc0009a10 tx=0x7febc0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:12.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.498+0000 7febcdffb700 1 -- 172.21.15.55:0/2572562269 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7febc802b020 con 0x7febd01094e0 2024-04-26T21:50:12.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.704+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7febd01a8a60 con 0x7febd01094e0 2024-04-26T21:50:12.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.705+0000 7febcdffb700 1 -- 172.21.15.55:0/2572562269 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7febc80074e0 con 0x7febd01094e0 2024-04-26T21:50:12.705 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7febb80a3c40 msgr2=0x7febb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7febb80a3c40 0x7febb80a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7febc0009a10 tx=0x7febc0009190 comp rx=0 tx=0).stop 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd01094e0 msgr2=0x7febd01a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd01094e0 0x7febd01a05e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7febc8009fd0 tx=0x7febc8004dc0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 shutdown_connections 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febd01094e0 0x7febd01a05e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7febb80a3c40 0x7febb80a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febd01034a0 0x7febd019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 --2- 172.21.15.55:0/2572562269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febd0103e60 0x7febd019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 >> 172.21.15.55:0/2572562269 conn(0x7febd007a590 msgr2=0x7febd010e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 shutdown_connections 2024-04-26T21:50:12.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:12.707+0000 7febd6f9d700 1 -- 172.21.15.55:0/2572562269 wait complete. 2024-04-26T21:50:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:12 smithi064 ceph-mon[32765]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:13.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:12 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2572562269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:12 smithi165 ceph-mon[33296]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:12 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2572562269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:13.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:12 smithi055 ceph-mon[29221]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:13.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:12 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2572562269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.a on smithi055 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.d on smithi055 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.g on smithi055 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.j on smithi055 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.b on smithi064 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.e on smithi064 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.h on smithi064 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.k on smithi064 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.c on smithi165 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.f on smithi165 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.i on smithi165 2024-04-26T21:50:13.519 INFO:tasks.cephadm:Adding mds.l on smithi165 2024-04-26T21:50:13.520 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l' 2024-04-26T21:50:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T21:50:15.245 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:14 smithi165 ceph-mon[33296]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:15.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:14 smithi064 ceph-mon[32765]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:15.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:14 smithi055 ceph-mon[29221]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/645847162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01035a0 msgr2=0x7fb9a0103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/645847162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01035a0 0x7fb9a0103a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009990 tx=0x7fb99c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/645847162 shutdown_connections 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/645847162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a0103f40 0x7fb9a010a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/645847162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a01023a0 0x7fb9a01027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/645847162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01035a0 0x7fb9a0103a00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/645847162 >> 172.21.15.165:0/645847162 conn(0x7fb9a00fd980 msgr2=0x7fb9a00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/645847162 shutdown_connections 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.484+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/645847162 wait complete. 2024-04-26T21:50:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a7dbd700 1 Processor -- start 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a7dbd700 1 -- start start 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a7dbd700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01023a0 0x7fb9a01995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a7dbd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a01035a0 0x7fb9a0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a7dbd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a0103f40 0x7fb9a01a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a7dbd700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a010de30 con 0x7fb9a0103f40 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a7dbd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a010dcb0 con 0x7fb9a01035a0 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a7dbd700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a010dfb0 con 0x7fb9a01023a0 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01023a0 0x7fb9a01995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01023a0 0x7fb9a01995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48192/0 (socket says 172.21.15.165:48192) 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 -- 172.21.15.165:0/2381320214 learned_addr learned my addr 172.21.15.165:0/2381320214 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-26T21:50:15.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5358700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a01035a0 0x7fb9a0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a635a700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a0103f40 0x7fb9a01a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 -- 172.21.15.165:0/2381320214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a01035a0 msgr2=0x7fb9a0199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a01035a0 0x7fb9a0199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 -- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a0103f40 msgr2=0x7fb9a01a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a0103f40 0x7fb9a01a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 -- 172.21.15.165:0/2381320214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a01a1240 con 0x7fb9a01023a0 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.485+0000 7fb9a5b59700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01023a0 0x7fb9a01995e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb99000d9b0 tx=0x7fb99000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.486+0000 7fb996ffd700 1 -- 172.21.15.165:0/2381320214 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990017070 con 0x7fb9a01023a0 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.486+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a01a1530 con 0x7fb9a01023a0 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.486+0000 7fb996ffd700 1 -- 172.21.15.165:0/2381320214 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb990009de0 con 0x7fb9a01023a0 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.486+0000 7fb996ffd700 1 -- 172.21.15.165:0/2381320214 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99000f3c0 con 0x7fb9a01023a0 2024-04-26T21:50:15.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.486+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a1a20 con 0x7fb9a01023a0 2024-04-26T21:50:15.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.487+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a006a110 con 0x7fb9a01023a0 2024-04-26T21:50:15.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.488+0000 7fb996ffd700 1 -- 172.21.15.165:0/2381320214 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb990009930 con 0x7fb9a01023a0 2024-04-26T21:50:15.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.488+0000 7fb996ffd700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb98c0a3c40 0x7fb98c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:15.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.488+0000 7fb996ffd700 1 -- 172.21.15.165:0/2381320214 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb9900cfac0 con 0x7fb9a01023a0 2024-04-26T21:50:15.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.489+0000 7fb9a5358700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb98c0a3c40 0x7fb98c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:15.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.489+0000 7fb9a5358700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb98c0a3c40 0x7fb98c0a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00b5c0 tx=0x7fb99c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:15.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.493+0000 7fb996ffd700 1 -- 172.21.15.165:0/2381320214 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb99009c940 con 0x7fb9a01023a0 2024-04-26T21:50:15.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.703+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}) -- 0x7fb9a006c2a0 con 0x7fb98c0a3c40 2024-04-26T21:50:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.711+0000 7fb996ffd700 1 -- 172.21.15.165:0/2381320214 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb9a006c2a0 con 0x7fb98c0a3c40 2024-04-26T21:50:15.713 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mds.all update... 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb98c0a3c40 msgr2=0x7fb98c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb98c0a3c40 0x7fb98c0a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00b5c0 tx=0x7fb99c005fb0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01023a0 msgr2=0x7fb9a01995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01023a0 0x7fb9a01995e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb99000d9b0 tx=0x7fb99000dcc0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 shutdown_connections 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9a0103f40 0x7fb9a01a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9a01023a0 0x7fb9a01995e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb98c0a3c40 0x7fb98c0a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 --2- 172.21.15.165:0/2381320214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb9a01035a0 0x7fb9a0199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 >> 172.21.15.165:0/2381320214 conn(0x7fb9a00fd980 msgr2=0x7fb9a006bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 shutdown_connections 2024-04-26T21:50:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-26T21:50:15.714+0000 7fb9a7dbd700 1 -- 172.21.15.165:0/2381320214 wait complete. 2024-04-26T21:50:16.439 DEBUG:teuthology.orchestra.run.smithi055:mds.a> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T21:50:16.442 DEBUG:teuthology.orchestra.run.smithi055:mds.d> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T21:50:16.486 DEBUG:teuthology.orchestra.run.smithi055:mds.g> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T21:50:16.531 DEBUG:teuthology.orchestra.run.smithi055:mds.j> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T21:50:16.575 DEBUG:teuthology.orchestra.run.smithi064:mds.b> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T21:50:16.578 DEBUG:teuthology.orchestra.run.smithi064:mds.e> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T21:50:16.620 DEBUG:teuthology.orchestra.run.smithi064:mds.h> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T21:50:16.663 DEBUG:teuthology.orchestra.run.smithi064:mds.k> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T21:50:16.707 DEBUG:teuthology.orchestra.run.smithi165:mds.c> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T21:50:16.710 DEBUG:teuthology.orchestra.run.smithi165:mds.f> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T21:50:16.711 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: Saving service mds.all spec with placement smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:16.712 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:16 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:16.713 DEBUG:teuthology.orchestra.run.smithi165:mds.i> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T21:50:16.756 DEBUG:teuthology.orchestra.run.smithi165:mds.l> sudo journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T21:50:16.799 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T21:50:16.800 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:50:16.824 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:16.824 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:16.824 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: Saving service mds.all spec with placement smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-04-26T21:50:16.824 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:16.825 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:16.825 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:16.825 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:16.825 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:16.825 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:16.825 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:16.825 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:16 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1328434409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 msgr2=0x7f35f8101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1328434409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 0x7f35f8101cb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009900 tx=0x7f35e802a410 comp rx=0 tx=0).stop 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1328434409 shutdown_connections 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1328434409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f8103440 0x7f35f8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1328434409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35f8102a80 0x7f35f8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1328434409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 0x7f35f8101cb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1328434409 >> 172.21.15.55:0/1328434409 conn(0x7f35f80fce20 msgr2=0x7f35f80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1328434409 shutdown_connections 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1328434409 wait complete. 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.917+0000 7f35fe5bf640 1 Processor -- start 2024-04-26T21:50:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35fe5bf640 1 -- start start 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35fe5bf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 0x7f35f8105820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35fe5bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f8102a80 0x7f35f8103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35fe5bf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35f8103440 0x7f35f81043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35fe5bf640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f35f810d3e0 con 0x7f35f8101890 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35fe5bf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35f810d260 con 0x7f35f8102a80 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35fe5bf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f35f810d560 con 0x7f35f8103440 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 0x7f35f8105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f8102a80 0x7f35f8103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 0x7f35f8105820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58298/0 (socket says 172.21.15.55:58298) 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 -- 172.21.15.55:0/1775895220 learned_addr learned my addr 172.21.15.55:0/1775895220 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35fcb5c640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35f8103440 0x7f35f81043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 -- 172.21.15.55:0/1775895220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35f8103440 msgr2=0x7f35f81043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35f8103440 0x7f35f81043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 -- 172.21.15.55:0/1775895220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f8102a80 msgr2=0x7f35f8103e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f8102a80 0x7f35f8103e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 -- 172.21.15.55:0/1775895220 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f8104c30 con 0x7f35f8101890 2024-04-26T21:50:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.918+0000 7f35f7fff640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 0x7f35f8105820 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35e8031500 tx=0x7f35e8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.919+0000 7f35f57fa640 1 -- 172.21.15.55:0/1775895220 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e803f070 con 0x7f35f8101890 2024-04-26T21:50:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.919+0000 7f35f57fa640 1 -- 172.21.15.55:0/1775895220 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e8003e70 con 0x7f35f8101890 2024-04-26T21:50:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.919+0000 7f35f57fa640 1 -- 172.21.15.55:0/1775895220 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8035030 con 0x7f35f8101890 2024-04-26T21:50:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.919+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f81a2cf0 con 0x7f35f8101890 2024-04-26T21:50:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.919+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f81a3180 con 0x7f35f8101890 2024-04-26T21:50:16.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.920+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f81a33b0 con 0x7f35f8101890 2024-04-26T21:50:16.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.921+0000 7f35f57fa640 1 -- 172.21.15.55:0/1775895220 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f35e8042070 con 0x7f35f8101890 2024-04-26T21:50:16.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.921+0000 7f35f57fa640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f35e00a3c30 0x7f35e00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:16.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.921+0000 7f35f77fe640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f35e00a3c30 0x7f35e00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:16.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.921+0000 7f35f57fa640 1 -- 172.21.15.55:0/1775895220 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f35e80f0a60 con 0x7f35f8101890 2024-04-26T21:50:16.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.922+0000 7f35f77fe640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f35e00a3c30 0x7f35e00a60f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f35f8105590 tx=0x7f35ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:16.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:16.926+0000 7f35f57fa640 1 -- 172.21.15.55:0/1775895220 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f35e80bd290 con 0x7f35f8101890 2024-04-26T21:50:17.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:17.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:50:17.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: Saving service mds.all spec with placement smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-04-26T21:50:17.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:17.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:17.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:17.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:17.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:17.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:17.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:17.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:16 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:17.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.112+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35f81a3740 con 0x7f35f8101890 2024-04-26T21:50:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.113+0000 7f35f57fa640 1 -- 172.21.15.55:0/1775895220 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18422 (secure 0 0 0) 0x7f35e80eee20 con 0x7f35f8101890 2024-04-26T21:50:17.114 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:17.114 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":81,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:50:06.966268+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f35e00a3c30 msgr2=0x7f35e00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f35e00a3c30 0x7f35e00a60f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f35f8105590 tx=0x7f35ec00b410 comp rx=0 tx=0).stop 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 msgr2=0x7f35f8105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 0x7f35f8105820 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35e8031500 tx=0x7f35e8003c80 comp rx=0 tx=0).stop 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 shutdown_connections 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f35e00a3c30 0x7f35e00a60f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35f8103440 0x7f35f81043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f8102a80 0x7f35f8103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 --2- 172.21.15.55:0/1775895220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35f8101890 0x7f35f8105820 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 >> 172.21.15.55:0/1775895220 conn(0x7f35f80fce20 msgr2=0x7f35f80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 shutdown_connections 2024-04-26T21:50:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.115+0000 7f35fe5bf640 1 -- 172.21.15.55:0/1775895220 wait complete. 2024-04-26T21:50:17.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:50:17.128 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:50:17.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 -- 172.21.15.55:0/3966987340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c101260 msgr2=0x7ff60c101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 --2- 172.21.15.55:0/3966987340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c101260 0x7ff60c101680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009ca0 tx=0x7ff5fc02a950 comp rx=0 tx=0).stop 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 -- 172.21.15.55:0/3966987340 shutdown_connections 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 --2- 172.21.15.55:0/3966987340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c102560 0x7ff60c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 --2- 172.21.15.55:0/3966987340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff60c101bc0 0x7ff60c102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 --2- 172.21.15.55:0/3966987340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c101260 0x7ff60c101680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 -- 172.21.15.55:0/3966987340 >> 172.21.15.55:0/3966987340 conn(0x7ff60c0fcde0 msgr2=0x7ff60c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 -- 172.21.15.55:0/3966987340 shutdown_connections 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.271+0000 7ff61172b640 1 -- 172.21.15.55:0/3966987340 wait complete. 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff61172b640 1 Processor -- start 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff61172b640 1 -- start start 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff61172b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c101260 0x7ff60c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff61172b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff60c101bc0 0x7ff60c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff61172b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c102560 0x7ff60c1a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:17.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff61172b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c1106c0 con 0x7ff60c102560 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff61172b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c110540 con 0x7ff60c101bc0 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff61172b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c110840 con 0x7ff60c101260 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff60b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c102560 0x7ff60c1a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff60b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c102560 0x7ff60c1a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58318/0 (socket says 172.21.15.55:58318) 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff60b7fe640 1 -- 172.21.15.55:0/2398855119 learned_addr learned my addr 172.21.15.55:0/2398855119 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.272+0000 7ff60affd640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c101260 0x7ff60c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff60a7fc640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff60c101bc0 0x7ff60c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff60b7fe640 1 -- 172.21.15.55:0/2398855119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c101260 msgr2=0x7ff60c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff60b7fe640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c101260 0x7ff60c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff60b7fe640 1 -- 172.21.15.55:0/2398855119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff60c101bc0 msgr2=0x7ff60c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff60b7fe640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff60c101bc0 0x7ff60c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff60b7fe640 1 -- 172.21.15.55:0/2398855119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c1a2a40 con 0x7ff60c102560 2024-04-26T21:50:17.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff60a7fc640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff60c101bc0 0x7ff60c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:17.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff60b7fe640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c102560 0x7ff60c1a23d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff60400d950 tx=0x7ff60400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:17.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff603fff640 1 -- 172.21.15.55:0/2398855119 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604017030 con 0x7ff60c102560 2024-04-26T21:50:17.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff603fff640 1 -- 172.21.15.55:0/2398855119 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff604004020 con 0x7ff60c102560 2024-04-26T21:50:17.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c1a2d30 con 0x7ff60c102560 2024-04-26T21:50:17.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff603fff640 1 -- 172.21.15.55:0/2398855119 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604003b50 con 0x7ff60c102560 2024-04-26T21:50:17.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.273+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c1a31d0 con 0x7ff60c102560 2024-04-26T21:50:17.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.274+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c10c240 con 0x7ff60c102560 2024-04-26T21:50:17.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.275+0000 7ff603fff640 1 -- 172.21.15.55:0/2398855119 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff60401a070 con 0x7ff60c102560 2024-04-26T21:50:17.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.276+0000 7ff603fff640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:17.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.276+0000 7ff60affd640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:17.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.276+0000 7ff603fff640 1 -- 172.21.15.55:0/2398855119 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6040cf6f0 con 0x7ff60c102560 2024-04-26T21:50:17.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.277+0000 7ff60affd640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc000f80 tx=0x7ff5fc02f870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:17.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.281+0000 7ff603fff640 1 -- 172.21.15.55:0/2398855119 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff604021070 con 0x7ff60c102560 2024-04-26T21:50:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.466+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff60c0635d0 con 0x7ff60c102560 2024-04-26T21:50:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.467+0000 7ff603fff640 1 -- 172.21.15.55:0/2398855119 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7ff60409c570 con 0x7ff60c102560 2024-04-26T21:50:17.467 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:50:17.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5f80a3bb0 msgr2=0x7ff5f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc000f80 tx=0x7ff5fc02f870 comp rx=0 tx=0).stop 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c102560 msgr2=0x7ff60c1a23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c102560 0x7ff60c1a23d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff60400d950 tx=0x7ff60400de10 comp rx=0 tx=0).stop 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 shutdown_connections 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5f80a3bb0 0x7ff5f80a6070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff60c102560 0x7ff60c1a23d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff60c101bc0 0x7ff60c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 --2- 172.21.15.55:0/2398855119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c101260 0x7ff60c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 >> 172.21.15.55:0/2398855119 conn(0x7ff60c0fcde0 msgr2=0x7ff60c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 shutdown_connections 2024-04-26T21:50:17.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.469+0000 7ff61172b640 1 -- 172.21.15.55:0/2398855119 wait complete. 2024-04-26T21:50:17.481 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:17.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 -- 172.21.15.55:0/3230659788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec103440 msgr2=0x7f03ec109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 --2- 172.21.15.55:0/3230659788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec103440 0x7f03ec109cd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009ca0 tx=0x7f03e402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 -- 172.21.15.55:0/3230659788 shutdown_connections 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 --2- 172.21.15.55:0/3230659788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec103440 0x7f03ec109cd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 --2- 172.21.15.55:0/3230659788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03ec102a80 0x7f03ec102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 --2- 172.21.15.55:0/3230659788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03ec101890 0x7f03ec101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 -- 172.21.15.55:0/3230659788 >> 172.21.15.55:0/3230659788 conn(0x7f03ec0fce20 msgr2=0x7f03ec0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 -- 172.21.15.55:0/3230659788 shutdown_connections 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 -- 172.21.15.55:0/3230659788 wait complete. 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.582+0000 7f03f2d39640 1 Processor -- start 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f2d39640 1 -- start start 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f2d39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03ec101890 0x7f03ec198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f2d39640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec102a80 0x7f03ec199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:17.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f2d39640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03ec103440 0x7f03ec1a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f2d39640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec10d410 con 0x7f03ec102a80 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f2d39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec10d290 con 0x7f03ec101890 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f2d39640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec10d590 con 0x7f03ec103440 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec102a80 0x7f03ec199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec102a80 0x7f03ec199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58346/0 (socket says 172.21.15.55:58346) 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 -- 172.21.15.55:0/2081846165 learned_addr learned my addr 172.21.15.55:0/2081846165 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f12d6640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03ec103440 0x7f03ec1a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03f0ad5640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03ec101890 0x7f03ec198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 -- 172.21.15.55:0/2081846165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03ec103440 msgr2=0x7f03ec1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03ec103440 0x7f03ec1a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 -- 172.21.15.55:0/2081846165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03ec101890 msgr2=0x7f03ec198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03ec101890 0x7f03ec198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 -- 172.21.15.55:0/2081846165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec1a0790 con 0x7f03ec102a80 2024-04-26T21:50:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03ebfff640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec102a80 0x7f03ec199150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00e9f0 tx=0x7f03dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.583+0000 7f03e9ffb640 1 -- 172.21.15.55:0/2081846165 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00cd80 con 0x7f03ec102a80 2024-04-26T21:50:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.584+0000 7f03e9ffb640 1 -- 172.21.15.55:0/2081846165 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03dc0054d0 con 0x7f03ec102a80 2024-04-26T21:50:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.584+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ec1a0a80 con 0x7f03ec102a80 2024-04-26T21:50:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.584+0000 7f03e9ffb640 1 -- 172.21.15.55:0/2081846165 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc010690 con 0x7f03ec102a80 2024-04-26T21:50:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.584+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ec1a0fd0 con 0x7f03ec102a80 2024-04-26T21:50:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.585+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ec1a1260 con 0x7f03ec102a80 2024-04-26T21:50:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.586+0000 7f03e9ffb640 1 -- 172.21.15.55:0/2081846165 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f03dc004e90 con 0x7f03ec102a80 2024-04-26T21:50:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.586+0000 7f03e9ffb640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f03d80a3c30 0x7f03d80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.586+0000 7f03f0ad5640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f03d80a3c30 0x7f03d80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.586+0000 7f03e9ffb640 1 -- 172.21.15.55:0/2081846165 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f03dc0d02f0 con 0x7f03ec102a80 2024-04-26T21:50:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.587+0000 7f03f0ad5640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f03d80a3c30 0x7f03d80a60f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f03ec1028e0 tx=0x7f03d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:17.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.591+0000 7f03e9ffb640 1 -- 172.21.15.55:0/2081846165 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f03dc09d170 con 0x7f03ec102a80 2024-04-26T21:50:17.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.820+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03ec1a1430 con 0x7f03ec102a80 2024-04-26T21:50:17.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.821+0000 7f03e9ffb640 1 -- 172.21.15.55:0/2081846165 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f03dc09ca70 con 0x7f03ec102a80 2024-04-26T21:50:17.821 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:17.821 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T21:50:17.821 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 1 2024-04-26T21:50:17.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.822+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f03d80a3c30 msgr2=0x7f03d80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.822+0000 7f03f2d39640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f03d80a3c30 0x7f03d80a60f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f03ec1028e0 tx=0x7f03d4005f90 comp rx=0 tx=0).stop 2024-04-26T21:50:17.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.822+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec102a80 msgr2=0x7f03ec199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:17.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.822+0000 7f03f2d39640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec102a80 0x7f03ec199150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00e9f0 tx=0x7f03dc00edf0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.823+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 shutdown_connections 2024-04-26T21:50:17.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.823+0000 7f03f2d39640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f03d80a3c30 0x7f03d80a60f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.823+0000 7f03f2d39640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03ec103440 0x7f03ec1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.823+0000 7f03f2d39640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03ec102a80 0x7f03ec199150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.823+0000 7f03f2d39640 1 --2- 172.21.15.55:0/2081846165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f03ec101890 0x7f03ec198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:17.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.823+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 >> 172.21.15.55:0/2081846165 conn(0x7f03ec0fce20 msgr2=0x7f03ec06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:17.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.823+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 shutdown_connections 2024-04-26T21:50:17.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:17.823+0000 7f03f2d39640 1 -- 172.21.15.55:0/2081846165 wait complete. 2024-04-26T21:50:17.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:17 smithi165 ceph-mon[33296]: Deploying daemon mds.l on smithi165 2024-04-26T21:50:17.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:17 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1775895220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:17.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:17 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2398855119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:17 smithi064 ceph-mon[32765]: Deploying daemon mds.l on smithi165 2024-04-26T21:50:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:17 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1775895220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:17 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2398855119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:18.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:17 smithi055 ceph-mon[29221]: Deploying daemon mds.l on smithi165 2024-04-26T21:50:18.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:17 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1775895220' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:18.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:17 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2398855119' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:18.978 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:18 smithi165 ceph-mon[33296]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:18.978 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:18 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2081846165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:18.978 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:18 smithi165 systemd[1]: Starting Ceph mds.l for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:18 smithi064 ceph-mon[32765]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:18 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2081846165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:19.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:18 smithi055 ceph-mon[29221]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:19.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:18 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2081846165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:19.259 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:18 smithi165 podman[62949]: 2024-04-26 21:50:18.880432748 +0000 UTC m=+0.018227326 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:19.259 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:18 smithi165 podman[62949]: 2024-04-26 21:50:18.998237845 +0000 UTC m=+0.136032410 container create 1ab915fa5ffd25a47428a2565eb150a9a9585d1cd39e6525632a2c2b0e53a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:19.259 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:19 smithi165 podman[62949]: 2024-04-26 21:50:19.2594154 +0000 UTC m=+0.397209965 container init 1ab915fa5ffd25a47428a2565eb150a9a9585d1cd39e6525632a2c2b0e53a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T21:50:19.577 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:19 smithi165 podman[62949]: 2024-04-26 21:50:19.262409321 +0000 UTC m=+0.400203887 container start 1ab915fa5ffd25a47428a2565eb150a9a9585d1cd39e6525632a2c2b0e53a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:50:19.577 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:19 smithi165 ceph-mds[62969]: -- 172.21.15.165:0/1285863484 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628356ada40 con 0x562835681000 2024-04-26T21:50:19.577 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l[62965]: starting mds.l at 2024-04-26T21:50:19.577 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:19 smithi165 bash[62949]: 1ab915fa5ffd25a47428a2565eb150a9a9585d1cd39e6525632a2c2b0e53a787 2024-04-26T21:50:19.577 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 21:50:19 smithi165 systemd[1]: Started Ceph mds.l for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:19.835 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:19.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 -- 172.21.15.55:0/1537182645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4103420 msgr2=0x7f6fb4109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:19.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/1537182645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4103420 0x7f6fb4109cb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009ca0 tx=0x7f6fac02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:19.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 -- 172.21.15.55:0/1537182645 shutdown_connections 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/1537182645 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4103420 0x7f6fb4109cb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/1537182645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb4102a60 0x7f6fb4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/1537182645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fb4101870 0x7f6fb4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 -- 172.21.15.55:0/1537182645 >> 172.21.15.55:0/1537182645 conn(0x7f6fb40fcde0 msgr2=0x7f6fb40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 -- 172.21.15.55:0/1537182645 shutdown_connections 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.950+0000 7f6fbb22a640 1 -- 172.21.15.55:0/1537182645 wait complete. 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fbb22a640 1 Processor -- start 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fbb22a640 1 -- start start 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fbb22a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4101870 0x7f6fb4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fbb22a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fb4102a60 0x7f6fb4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fbb22a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb4103420 0x7f6fb41a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fbb22a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb410d3a0 con 0x7f6fb4101870 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fbb22a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb410d220 con 0x7f6fb4103420 2024-04-26T21:50:19.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fbb22a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb410d520 con 0x7f6fb4102a60 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fb8fc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4101870 0x7f6fb4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fb8fc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4101870 0x7f6fb4198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58362/0 (socket says 172.21.15.55:58362) 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fb8fc6640 1 -- 172.21.15.55:0/2655637850 learned_addr learned my addr 172.21.15.55:0/2655637850 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fb97c7640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb4103420 0x7f6fb41a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.951+0000 7f6fb3fff640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fb4102a60 0x7f6fb4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb8fc6640 1 -- 172.21.15.55:0/2655637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fb4102a60 msgr2=0x7f6fb4199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb8fc6640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fb4102a60 0x7f6fb4199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb8fc6640 1 -- 172.21.15.55:0/2655637850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb4103420 msgr2=0x7f6fb41a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb8fc6640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb4103420 0x7f6fb41a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb8fc6640 1 -- 172.21.15.55:0/2655637850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb41a07a0 con 0x7f6fb4101870 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb3fff640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fb4102a60 0x7f6fb4199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb8fc6640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4101870 0x7f6fb4198bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800e9c0 tx=0x7f6fa800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:19.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb1ffb640 1 -- 172.21.15.55:0/2655637850 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800cd80 con 0x7f6fb4101870 2024-04-26T21:50:19.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb1ffb640 1 -- 172.21.15.55:0/2655637850 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa80035f0 con 0x7f6fb4101870 2024-04-26T21:50:19.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb41a0a90 con 0x7f6fb4101870 2024-04-26T21:50:19.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fb1ffb640 1 -- 172.21.15.55:0/2655637850 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8010690 con 0x7f6fb4101870 2024-04-26T21:50:19.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.952+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb41a0f90 con 0x7f6fb4101870 2024-04-26T21:50:19.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.953+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb41a1220 con 0x7f6fb4101870 2024-04-26T21:50:19.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.954+0000 7f6fb1ffb640 1 -- 172.21.15.55:0/2655637850 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6fa8004000 con 0x7f6fb4101870 2024-04-26T21:50:19.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.955+0000 7f6fb1ffb640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6fa40a3bb0 0x7f6fa40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:19.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.955+0000 7f6fb3fff640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6fa40a3bb0 0x7f6fa40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:19.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.955+0000 7f6fb1ffb640 1 -- 172.21.15.55:0/2655637850 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6fa80d0500 con 0x7f6fb4101870 2024-04-26T21:50:19.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.956+0000 7f6fb3fff640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6fa40a3bb0 0x7f6fa40a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6fb419a090 tx=0x7f6fa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:19.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:19.960+0000 7f6fb1ffb640 1 -- 172.21.15.55:0/2655637850 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6fa809d380 con 0x7f6fb4101870 2024-04-26T21:50:20.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.185+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fb41a13f0 con 0x7f6fb4101870 2024-04-26T21:50:20.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.185+0000 7f6fb1ffb640 1 -- 172.21.15.55:0/2655637850 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f6fa809cc80 con 0x7f6fb4101870 2024-04-26T21:50:20.186 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:20.186 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T21:50:20.186 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 2 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6fa40a3bb0 msgr2=0x7f6fa40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6fa40a3bb0 0x7f6fa40a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6fb419a090 tx=0x7f6fa0005f90 comp rx=0 tx=0).stop 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4101870 msgr2=0x7f6fb4198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4101870 0x7f6fb4198bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800e9c0 tx=0x7f6fa800ee80 comp rx=0 tx=0).stop 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 shutdown_connections 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6fa40a3bb0 0x7f6fa40a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fb4103420 0x7f6fb41a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fb4102a60 0x7f6fb4199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 --2- 172.21.15.55:0/2655637850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fb4101870 0x7f6fb4198bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 >> 172.21.15.55:0/2655637850 conn(0x7f6fb40fcde0 msgr2=0x7f6fb40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 shutdown_connections 2024-04-26T21:50:20.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:20.187+0000 7f6fbb22a640 1 -- 172.21.15.55:0/2655637850 wait complete. 2024-04-26T21:50:20.466 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.165:6832/2896142285,v1:172.21.15.165:6833/2896142285] up:boot 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: fsmap 1 up:standby 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: Deploying daemon mds.k on smithi064 2024-04-26T21:50:20.467 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:20 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2655637850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.165:6832/2896142285,v1:172.21.15.165:6833/2896142285] up:boot 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: fsmap 1 up:standby 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: Deploying daemon mds.k on smithi064 2024-04-26T21:50:20.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:20 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2655637850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:20.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.165:6832/2896142285,v1:172.21.15.165:6833/2896142285] up:boot 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: fsmap 1 up:standby 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: Deploying daemon mds.k on smithi064 2024-04-26T21:50:20.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:20 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2655637850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:22.007 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:21 smithi064 systemd[1]: Starting Ceph mds.k for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:22.199 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:22.268 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:22 smithi064 podman[62159]: 2024-04-26 21:50:21.913305438 +0000 UTC m=+0.018446765 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:22.269 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:22 smithi064 podman[62159]: 2024-04-26 21:50:22.03249959 +0000 UTC m=+0.137640896 container create a6f81ec610861b06f5eba4d3e61f262f14779fbdd009bbf2d7eb4a954250e583 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:50:22.269 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:22 smithi064 podman[62159]: 2024-04-26 21:50:22.26882908 +0000 UTC m=+0.373970390 container init a6f81ec610861b06f5eba4d3e61f262f14779fbdd009bbf2d7eb4a954250e583 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T21:50:22.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 -- 172.21.15.55:0/3484062855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0105d40 msgr2=0x7f2ee0108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:22.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 --2- 172.21.15.55:0/3484062855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0105d40 0x7f2ee0108130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009900 tx=0x7f2ed402a3f0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 -- 172.21.15.55:0/3484062855 shutdown_connections 2024-04-26T21:50:22.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 --2- 172.21.15.55:0/3484062855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee0108670 0x7f2ee010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 --2- 172.21.15.55:0/3484062855 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0105d40 0x7f2ee0108130 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 --2- 172.21.15.55:0/3484062855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0103410 0x7f2ee0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 -- 172.21.15.55:0/3484062855 >> 172.21.15.55:0/3484062855 conn(0x7f2ee00fce20 msgr2=0x7f2ee00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 -- 172.21.15.55:0/3484062855 shutdown_connections 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.314+0000 7f2ee5790640 1 -- 172.21.15.55:0/3484062855 wait complete. 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2ee5790640 1 Processor -- start 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2ee5790640 1 -- start start 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2ee5790640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0103410 0x7f2ee0102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2ee5790640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee0105d40 0x7f2ee01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2ee5790640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0108670 0x7f2ee0101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2ee5790640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee010e4c0 con 0x7f2ee0108670 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2ee5790640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee010e340 con 0x7f2ee0103410 2024-04-26T21:50:22.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2ee5790640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee010e640 con 0x7f2ee0105d40 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.315+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0108670 0x7f2ee0101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0108670 0x7f2ee0101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37172/0 (socket says 172.21.15.55:37172) 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edf7fe640 1 -- 172.21.15.55:0/1264463476 learned_addr learned my addr 172.21.15.55:0/1264463476 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edeffd640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0103410 0x7f2ee0102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2ede7fc640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee0105d40 0x7f2ee01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edf7fe640 1 -- 172.21.15.55:0/1264463476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee0105d40 msgr2=0x7f2ee01013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edf7fe640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee0105d40 0x7f2ee01013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edf7fe640 1 -- 172.21.15.55:0/1264463476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0103410 msgr2=0x7f2ee0102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edf7fe640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0103410 0x7f2ee0102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edf7fe640 1 -- 172.21.15.55:0/1264463476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee0102190 con 0x7f2ee0108670 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2ede7fc640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee0105d40 0x7f2ee01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2edf7fe640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0108670 0x7f2ee0101900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800cad0 tx=0x7f2ed800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:22.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2ec7fff640 1 -- 172.21.15.55:0/1264463476 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed800ae20 con 0x7f2ee0108670 2024-04-26T21:50:22.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee01a2d30 con 0x7f2ee0108670 2024-04-26T21:50:22.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2ec7fff640 1 -- 172.21.15.55:0/1264463476 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ed8005020 con 0x7f2ee0108670 2024-04-26T21:50:22.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2ec7fff640 1 -- 172.21.15.55:0/1264463476 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed800f6b0 con 0x7f2ee0108670 2024-04-26T21:50:22.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.316+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee01a31a0 con 0x7f2ee0108670 2024-04-26T21:50:22.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.317+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee00fe910 con 0x7f2ee0108670 2024-04-26T21:50:22.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.319+0000 7f2ec7fff640 1 -- 172.21.15.55:0/1264463476 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2ed80051d0 con 0x7f2ee0108670 2024-04-26T21:50:22.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.319+0000 7f2ec7fff640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ec80a3bb0 0x7f2ec80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:22.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.320+0000 7f2edeffd640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ec80a3bb0 0x7f2ec80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:22.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.320+0000 7f2ec7fff640 1 -- 172.21.15.55:0/1264463476 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2ed80cffe0 con 0x7f2ee0108670 2024-04-26T21:50:22.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.320+0000 7f2edeffd640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ec80a3bb0 0x7f2ec80a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00095f0 tx=0x7f2ed0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:22.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.324+0000 7f2ec7fff640 1 -- 172.21.15.55:0/1264463476 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ed8014030 con 0x7f2ee0108670 2024-04-26T21:50:22.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.560+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ee002dd40 con 0x7f2ee0108670 2024-04-26T21:50:22.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.561+0000 7f2ec7fff640 1 -- 172.21.15.55:0/1264463476 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f2ed809ce60 con 0x7f2ee0108670 2024-04-26T21:50:22.561 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:22.562 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:50:22.562 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 3 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ec80a3bb0 msgr2=0x7f2ec80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ec80a3bb0 0x7f2ec80a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00095f0 tx=0x7f2ed0005f90 comp rx=0 tx=0).stop 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0108670 msgr2=0x7f2ee0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0108670 0x7f2ee0101900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800cad0 tx=0x7f2ed800cf90 comp rx=0 tx=0).stop 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 shutdown_connections 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2ec80a3bb0 0x7f2ec80a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ee0108670 0x7f2ee0101900 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ee0105d40 0x7f2ee01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 --2- 172.21.15.55:0/1264463476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ee0103410 0x7f2ee0102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.563+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 >> 172.21.15.55:0/1264463476 conn(0x7f2ee00fce20 msgr2=0x7f2ee0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.564+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 shutdown_connections 2024-04-26T21:50:22.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:22.564+0000 7f2ee5790640 1 -- 172.21.15.55:0/1264463476 wait complete. 2024-04-26T21:50:22.574 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:22 smithi055 ceph-mon[29221]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:22.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:22 smithi064 ceph-mon[32765]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:22.757 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:22 smithi064 podman[62159]: 2024-04-26 21:50:22.271714515 +0000 UTC m=+0.376855823 container start a6f81ec610861b06f5eba4d3e61f262f14779fbdd009bbf2d7eb4a954250e583 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:22.757 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:22 smithi064 ceph-mds[62179]: -- 172.21.15.64:0/3875245006 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df9b565a40 con 0x55df9b538c00 2024-04-26T21:50:22.758 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:22 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k[62175]: starting mds.k at 2024-04-26T21:50:22.758 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:22 smithi064 bash[62159]: a6f81ec610861b06f5eba4d3e61f262f14779fbdd009bbf2d7eb4a954250e583 2024-04-26T21:50:22.758 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 21:50:22 smithi064 systemd[1]: Started Ceph mds.k for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:22.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:22 smithi165 ceph-mon[33296]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:23.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6832/689132199,v1:172.21.15.64:6833/689132199] up:boot 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: fsmap 2 up:standby 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: Deploying daemon mds.j on smithi055 2024-04-26T21:50:23.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:23 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1264463476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6832/689132199,v1:172.21.15.64:6833/689132199] up:boot 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: fsmap 2 up:standby 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: Deploying daemon mds.j on smithi055 2024-04-26T21:50:23.712 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:23 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1264463476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:23.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6832/689132199,v1:172.21.15.64:6833/689132199] up:boot 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: fsmap 2 up:standby 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: Deploying daemon mds.j on smithi055 2024-04-26T21:50:23.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:23 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1264463476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:24.576 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:24.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:24 smithi055 ceph-mon[29221]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:24.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 -- 172.21.15.55:0/2482245097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c1086d0 msgr2=0x7f7c2c10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:24.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 --2- 172.21.15.55:0/2482245097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c1086d0 0x7f7c2c10aac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009ca0 tx=0x7f7c2402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 -- 172.21.15.55:0/2482245097 shutdown_connections 2024-04-26T21:50:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 --2- 172.21.15.55:0/2482245097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c1086d0 0x7f7c2c10aac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 --2- 172.21.15.55:0/2482245097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c2c105da0 0x7f7c2c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 --2- 172.21.15.55:0/2482245097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c2c103470 0x7f7c2c105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 -- 172.21.15.55:0/2482245097 >> 172.21.15.55:0/2482245097 conn(0x7f7c2c0fcde0 msgr2=0x7f7c2c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 -- 172.21.15.55:0/2482245097 shutdown_connections 2024-04-26T21:50:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.691+0000 7f7c326b0640 1 -- 172.21.15.55:0/2482245097 wait complete. 2024-04-26T21:50:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c326b0640 1 Processor -- start 2024-04-26T21:50:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c326b0640 1 -- start start 2024-04-26T21:50:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c326b0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c2c103470 0x7f7c2c196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c326b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c105da0 0x7f7c2c196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c326b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c2c1086d0 0x7f7c2c1a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c326b0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10e4a0 con 0x7f7c2c105da0 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c326b0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10e320 con 0x7f7c2c1086d0 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c326b0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c10e620 con 0x7f7c2c103470 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c2b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c105da0 0x7f7c2c196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c2b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c105da0 0x7f7c2c196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37190/0 (socket says 172.21.15.55:37190) 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c2b7fe640 1 -- 172.21.15.55:0/4287428020 learned_addr learned my addr 172.21.15.55:0/4287428020 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c30c4d640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c2c1086d0 0x7f7c2c1a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.692+0000 7f7c2bfff640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c2c103470 0x7f7c2c196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c2b7fe640 1 -- 172.21.15.55:0/4287428020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c2c103470 msgr2=0x7f7c2c196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c2b7fe640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c2c103470 0x7f7c2c196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c2b7fe640 1 -- 172.21.15.55:0/4287428020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c2c1086d0 msgr2=0x7f7c2c1a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c2b7fe640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c2c1086d0 0x7f7c2c1a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c2b7fe640 1 -- 172.21.15.55:0/4287428020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c1a6d60 con 0x7f7c2c105da0 2024-04-26T21:50:24.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c30c4d640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c2c1086d0 0x7f7c2c1a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:24.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c2b7fe640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c105da0 0x7f7c2c196e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c005fd0 tx=0x7f7c1c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:24.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c297fa640 1 -- 172.21.15.55:0/4287428020 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c017070 con 0x7f7c2c105da0 2024-04-26T21:50:24.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c297fa640 1 -- 172.21.15.55:0/4287428020 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c1c009c30 con 0x7f7c2c105da0 2024-04-26T21:50:24.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2c1a6ff0 con 0x7f7c2c105da0 2024-04-26T21:50:24.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c297fa640 1 -- 172.21.15.55:0/4287428020 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c003d30 con 0x7f7c2c105da0 2024-04-26T21:50:24.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.693+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2c1a7560 con 0x7f7c2c105da0 2024-04-26T21:50:24.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.695+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2c069750 con 0x7f7c2c105da0 2024-04-26T21:50:24.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.695+0000 7f7c297fa640 1 -- 172.21.15.55:0/4287428020 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7c1c003ed0 con 0x7f7c2c105da0 2024-04-26T21:50:24.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.696+0000 7f7c297fa640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c140a3c30 0x7f7c140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:24.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.696+0000 7f7c2bfff640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c140a3c30 0x7f7c140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:24.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.696+0000 7f7c297fa640 1 -- 172.21.15.55:0/4287428020 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7c1c0cfa80 con 0x7f7c2c105da0 2024-04-26T21:50:24.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.697+0000 7f7c2bfff640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c140a3c30 0x7f7c140a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009bd0 tx=0x7f7c20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:24.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.701+0000 7f7c297fa640 1 -- 172.21.15.55:0/4287428020 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7c1c020030 con 0x7f7c2c105da0 2024-04-26T21:50:24.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:24 smithi064 ceph-mon[32765]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:24.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:24 smithi165 ceph-mon[33296]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.938+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c2c02db70 con 0x7f7c2c105da0 2024-04-26T21:50:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.939+0000 7f7c297fa640 1 -- 172.21.15.55:0/4287428020 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f7c2c02db70 con 0x7f7c2c105da0 2024-04-26T21:50:24.940 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:24.940 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:50:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 3 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c140a3c30 msgr2=0x7f7c140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c140a3c30 0x7f7c140a60f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009bd0 tx=0x7f7c20005f90 comp rx=0 tx=0).stop 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c105da0 msgr2=0x7f7c2c196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c105da0 0x7f7c2c196e60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c005fd0 tx=0x7f7c1c00dde0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 shutdown_connections 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7c140a3c30 0x7f7c140a60f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c2c1086d0 0x7f7c2c1a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7c2c105da0 0x7f7c2c196e60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 --2- 172.21.15.55:0/4287428020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7c2c103470 0x7f7c2c196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 >> 172.21.15.55:0/4287428020 conn(0x7f7c2c0fcde0 msgr2=0x7f7c2c102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:24.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 shutdown_connections 2024-04-26T21:50:24.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:24.941+0000 7f7c326b0640 1 -- 172.21.15.55:0/4287428020 wait complete. 2024-04-26T21:50:25.436 INFO:journalctl@ceph.mds.j.smithi055.stdout:Apr 26 21:50:25 smithi055 systemd[1]: Started Ceph mds.j for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:25.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:25 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4287428020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:25.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:25.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:25.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:25 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:25.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:25 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4287428020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:25.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:25.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:25.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:25 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:25.934 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:25 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4287428020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:25.934 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.934 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.934 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:25.934 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:25.934 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:25.934 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:25 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:26.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:26 smithi055 ceph-mon[29221]: Deploying daemon mds.d on smithi055 2024-04-26T21:50:26.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:26 smithi055 ceph-mon[29221]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:26.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:26 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.55:6834/1727764470,v1:172.21.15.55:6835/1727764470] up:boot 2024-04-26T21:50:26.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:26 smithi055 ceph-mon[29221]: fsmap 3 up:standby 2024-04-26T21:50:26.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:50:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:26 smithi064 ceph-mon[32765]: Deploying daemon mds.d on smithi055 2024-04-26T21:50:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:26 smithi064 ceph-mon[32765]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:26 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.55:6834/1727764470,v1:172.21.15.55:6835/1727764470] up:boot 2024-04-26T21:50:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:26 smithi064 ceph-mon[32765]: fsmap 3 up:standby 2024-04-26T21:50:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:50:26.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:26 smithi165 ceph-mon[33296]: Deploying daemon mds.d on smithi055 2024-04-26T21:50:26.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:26 smithi165 ceph-mon[33296]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:26 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.55:6834/1727764470,v1:172.21.15.55:6835/1727764470] up:boot 2024-04-26T21:50:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:26 smithi165 ceph-mon[33296]: fsmap 3 up:standby 2024-04-26T21:50:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T21:50:26.954 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:27.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- 172.21.15.55:0/2009221302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918101870 msgr2=0x7f3918101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:27.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 --2- 172.21.15.55:0/2009221302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918101870 0x7f3918101c90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f390c009900 tx=0x7f390c02a410 comp rx=0 tx=0).stop 2024-04-26T21:50:27.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- 172.21.15.55:0/2009221302 shutdown_connections 2024-04-26T21:50:27.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 --2- 172.21.15.55:0/2009221302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3918103420 0x7f3918109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 --2- 172.21.15.55:0/2009221302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3918102a60 0x7f3918102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 --2- 172.21.15.55:0/2009221302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918101870 0x7f3918101c90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- 172.21.15.55:0/2009221302 >> 172.21.15.55:0/2009221302 conn(0x7f39180fcde0 msgr2=0x7f39180ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- 172.21.15.55:0/2009221302 shutdown_connections 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- 172.21.15.55:0/2009221302 wait complete. 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 Processor -- start 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- start start 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3918101870 0x7f391819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3918102a60 0x7f391819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918103420 0x7f39181a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d370 con 0x7f3918103420 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d1f0 con 0x7f3918102a60 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.068+0000 7f391efa4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f391810d4f0 con 0x7f3918101870 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918103420 0x7f39181a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918103420 0x7f39181a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37200/0 (socket says 172.21.15.55:37200) 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 -- 172.21.15.55:0/1985894799 learned_addr learned my addr 172.21.15.55:0/1985894799 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391cd40640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3918101870 0x7f391819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f3917fff640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3918102a60 0x7f391819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 -- 172.21.15.55:0/1985894799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3918101870 msgr2=0x7f391819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3918101870 0x7f391819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 -- 172.21.15.55:0/1985894799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3918102a60 msgr2=0x7f391819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3918102a60 0x7f391819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 -- 172.21.15.55:0/1985894799 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39181a2a30 con 0x7f3918103420 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391d541640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918103420 0x7f39181a23c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f391000eaa0 tx=0x7f391000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f3915ffb640 1 -- 172.21.15.55:0/1985894799 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391000cdb0 con 0x7f3918103420 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f3915ffb640 1 -- 172.21.15.55:0/1985894799 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3910005020 con 0x7f3918103420 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39181a2d20 con 0x7f3918103420 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.069+0000 7f3915ffb640 1 -- 172.21.15.55:0/1985894799 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39100116d0 con 0x7f3918103420 2024-04-26T21:50:27.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.070+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39181a3210 con 0x7f3918103420 2024-04-26T21:50:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.070+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39181a34a0 con 0x7f3918103420 2024-04-26T21:50:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.071+0000 7f3915ffb640 1 -- 172.21.15.55:0/1985894799 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f39100051d0 con 0x7f3918103420 2024-04-26T21:50:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.072+0000 7f3915ffb640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39080a3c30 0x7f39080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.072+0000 7f391cd40640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39080a3c30 0x7f39080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.072+0000 7f3915ffb640 1 -- 172.21.15.55:0/1985894799 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f39100d0190 con 0x7f3918103420 2024-04-26T21:50:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.073+0000 7f391cd40640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39080a3c30 0x7f39080a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.077+0000 7f3915ffb640 1 -- 172.21.15.55:0/1985894799 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3910016030 con 0x7f3918103420 2024-04-26T21:50:27.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.331+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39181a38c0 con 0x7f3918103420 2024-04-26T21:50:27.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.332+0000 7f3915ffb640 1 -- 172.21.15.55:0/1985894799 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f391009d010 con 0x7f3918103420 2024-04-26T21:50:27.333 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:27.333 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:50:27.334 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 4 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39080a3c30 msgr2=0x7f39080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39080a3c30 0x7f39080a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c0314a0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918103420 msgr2=0x7f39181a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918103420 0x7f39181a23c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f391000eaa0 tx=0x7f391000ef60 comp rx=0 tx=0).stop 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 shutdown_connections 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39080a3c30 0x7f39080a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3918103420 0x7f39181a23c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3918102a60 0x7f391819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 --2- 172.21.15.55:0/1985894799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3918101870 0x7f391819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.334+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 >> 172.21.15.55:0/1985894799 conn(0x7f39180fcde0 msgr2=0x7f3918069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.335+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 shutdown_connections 2024-04-26T21:50:27.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:27.335+0000 7f391efa4640 1 -- 172.21.15.55:0/1985894799 wait complete. 2024-04-26T21:50:27.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:27 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1985894799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:27.687 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:27 smithi055 systemd[1]: Starting Ceph mds.d for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:27.749 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:27 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1985894799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:27.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:27 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1985894799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:28.138 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:27 smithi055 podman[67312]: 2024-04-26 21:50:27.684783616 +0000 UTC m=+0.018732178 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:28.139 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:27 smithi055 podman[67312]: 2024-04-26 21:50:27.885522768 +0000 UTC m=+0.219471315 container create 757b72f2bc210bb52074e88f5efa2f264e99b7b2223c6d956122b4f1863ede68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:50:28.139 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:28 smithi055 podman[67312]: 2024-04-26 21:50:28.138748097 +0000 UTC m=+0.472696656 container init 757b72f2bc210bb52074e88f5efa2f264e99b7b2223c6d956122b4f1863ede68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T21:50:28.436 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:28 smithi055 podman[67312]: 2024-04-26 21:50:28.141630254 +0000 UTC m=+0.475578810 container start 757b72f2bc210bb52074e88f5efa2f264e99b7b2223c6d956122b4f1863ede68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:50:28.437 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-mds[67332]: -- 172.21.15.55:0/2974345044 <== mon.0 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bb9fa1a40 con 0x555bb9f75000 2024-04-26T21:50:28.437 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d[67328]: starting mds.d at 2024-04-26T21:50:28.437 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:28 smithi055 bash[67312]: 757b72f2bc210bb52074e88f5efa2f264e99b7b2223c6d956122b4f1863ede68 2024-04-26T21:50:28.437 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 21:50:28 smithi055 systemd[1]: Started Ceph mds.d for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:28 smithi064 ceph-mon[32765]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:28 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:28 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:28 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:28 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:28 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:28 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:28 smithi165 ceph-mon[33296]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:28 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:28 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:28 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:28 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:28 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:28.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:28 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:28.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-mon[29221]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:28.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:28.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:28.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:28.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:28 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:29.348 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 -- 172.21.15.55:0/1781692655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 msgr2=0x7fc5d010aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 --2- 172.21.15.55:0/1781692655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 0x7fc5d010aac0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009ca0 tx=0x7fc5c802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 -- 172.21.15.55:0/1781692655 shutdown_connections 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 --2- 172.21.15.55:0/1781692655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 0x7fc5d010aac0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 --2- 172.21.15.55:0/1781692655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d0105d40 0x7fc5d0108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 --2- 172.21.15.55:0/1781692655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5d0103410 0x7fc5d0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 -- 172.21.15.55:0/1781692655 >> 172.21.15.55:0/1781692655 conn(0x7fc5d00fce20 msgr2=0x7fc5d00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 -- 172.21.15.55:0/1781692655 shutdown_connections 2024-04-26T21:50:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.466+0000 7fc5d7823640 1 -- 172.21.15.55:0/1781692655 wait complete. 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d7823640 1 Processor -- start 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d7823640 1 -- start start 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d7823640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d0103410 0x7fc5d0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d7823640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5d0105d40 0x7fc5d01990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d7823640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 0x7fc5d01a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d7823640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d010e4c0 con 0x7fc5d0108670 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d7823640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d010e340 con 0x7fc5d0103410 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d7823640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d010e640 con 0x7fc5d0105d40 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d5dc0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 0x7fc5d01a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d55bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d0103410 0x7fc5d0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d55bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d0103410 0x7fc5d0198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37918/0 (socket says 172.21.15.55:37918) 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d5dc0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 0x7fc5d01a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37216/0 (socket says 172.21.15.55:37216) 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d55bf640 1 -- 172.21.15.55:0/643642407 learned_addr learned my addr 172.21.15.55:0/643642407 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.467+0000 7fc5d4dbe640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5d0105d40 0x7fc5d01990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d55bf640 1 -- 172.21.15.55:0/643642407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5d0105d40 msgr2=0x7fc5d01990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d55bf640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5d0105d40 0x7fc5d01990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d55bf640 1 -- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 msgr2=0x7fc5d01a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d55bf640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 0x7fc5d01a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d55bf640 1 -- 172.21.15.55:0/643642407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d01a07b0 con 0x7fc5d0103410 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d5dc0640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 0x7fc5d01a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d55bf640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d0103410 0x7fc5d0198b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00d950 tx=0x7fc5cc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5c67fc640 1 -- 172.21.15.55:0/643642407 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc017030 con 0x7fc5d0103410 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d01a0aa0 con 0x7fc5d0103410 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5c67fc640 1 -- 172.21.15.55:0/643642407 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc004020 con 0x7fc5d0103410 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5c67fc640 1 -- 172.21.15.55:0/643642407 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0047a0 con 0x7fc5d0103410 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.468+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d01a0ff0 con 0x7fc5d0103410 2024-04-26T21:50:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.469+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d00fe910 con 0x7fc5d0103410 2024-04-26T21:50:29.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.470+0000 7fc5c67fc640 1 -- 172.21.15.55:0/643642407 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc5cc009a30 con 0x7fc5d0103410 2024-04-26T21:50:29.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.471+0000 7fc5c67fc640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:29.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.471+0000 7fc5d4dbe640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:29.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.471+0000 7fc5c67fc640 1 -- 172.21.15.55:0/643642407 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc5cc0cfb30 con 0x7fc5d0103410 2024-04-26T21:50:29.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.471+0000 7fc5d4dbe640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc5d019a0a0 tx=0x7fc5c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:29.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.475+0000 7fc5c67fc640 1 -- 172.21.15.55:0/643642407 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc5cc09c9b0 con 0x7fc5d0103410 2024-04-26T21:50:29.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:29 smithi055 ceph-mon[29221]: Deploying daemon mds.e on smithi064 2024-04-26T21:50:29.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:29 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.55:6836/3725383900,v1:172.21.15.55:6837/3725383900] up:boot 2024-04-26T21:50:29.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:29 smithi055 ceph-mon[29221]: fsmap 4 up:standby 2024-04-26T21:50:29.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:29 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:50:29.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.708+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5d002dad0 con 0x7fc5d0103410 2024-04-26T21:50:29.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.709+0000 7fc5c67fc640 1 -- 172.21.15.55:0/643642407 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3868 (secure 0 0 0) 0x7fc5cc096e20 con 0x7fc5d0103410 2024-04-26T21:50:29.710 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:29.710 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T21:50:29.710 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 5 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5bc0a3bb0 msgr2=0x7fc5bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc5d019a0a0 tx=0x7fc5c0005f90 comp rx=0 tx=0).stop 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d0103410 msgr2=0x7fc5d0198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d0103410 0x7fc5d0198b90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00d950 tx=0x7fc5cc00de10 comp rx=0 tx=0).stop 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 shutdown_connections 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5bc0a3bb0 0x7fc5bc0a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5d0108670 0x7fc5d01a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5d0105d40 0x7fc5d01990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 --2- 172.21.15.55:0/643642407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d0103410 0x7fc5d0198b90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 >> 172.21.15.55:0/643642407 conn(0x7fc5d00fce20 msgr2=0x7fc5d0103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:29.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 shutdown_connections 2024-04-26T21:50:29.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:29.711+0000 7fc5d7823640 1 -- 172.21.15.55:0/643642407 wait complete. 2024-04-26T21:50:29.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:29 smithi165 ceph-mon[33296]: Deploying daemon mds.e on smithi064 2024-04-26T21:50:29.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:29 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.55:6836/3725383900,v1:172.21.15.55:6837/3725383900] up:boot 2024-04-26T21:50:29.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:29 smithi165 ceph-mon[33296]: fsmap 4 up:standby 2024-04-26T21:50:29.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:29 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:50:30.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:29 smithi064 ceph-mon[32765]: Deploying daemon mds.e on smithi064 2024-04-26T21:50:30.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:29 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.55:6836/3725383900,v1:172.21.15.55:6837/3725383900] up:boot 2024-04-26T21:50:30.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:29 smithi064 ceph-mon[32765]: fsmap 4 up:standby 2024-04-26T21:50:30.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:29 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T21:50:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:30 smithi064 ceph-mon[32765]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:30 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/643642407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:30.757 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:30 smithi064 systemd[1]: Starting Ceph mds.e for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:30.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:30 smithi165 ceph-mon[33296]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:30.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:30 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/643642407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:30.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:30 smithi055 ceph-mon[29221]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:30.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:30 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/643642407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:31.121 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:30 smithi064 podman[62481]: 2024-04-26 21:50:30.744485503 +0000 UTC m=+0.018433718 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:31.121 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:30 smithi064 podman[62481]: 2024-04-26 21:50:30.866320902 +0000 UTC m=+0.140269105 container create 318e305602cae9061ab5cc5d96c2af622953769b2f0c88d5dfd727d0a8f9b072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T21:50:31.122 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:31 smithi064 podman[62481]: 2024-04-26 21:50:31.121766011 +0000 UTC m=+0.395714211 container init 318e305602cae9061ab5cc5d96c2af622953769b2f0c88d5dfd727d0a8f9b072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T21:50:31.507 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:31 smithi064 podman[62481]: 2024-04-26 21:50:31.124638949 +0000 UTC m=+0.398587150 container start 318e305602cae9061ab5cc5d96c2af622953769b2f0c88d5dfd727d0a8f9b072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:31.507 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:31 smithi064 ceph-mds[62501]: -- 172.21.15.64:0/1190051413 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593496cba40 con 0x55934969e800 2024-04-26T21:50:31.507 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:31 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e[62497]: starting mds.e at 2024-04-26T21:50:31.507 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:31 smithi064 bash[62481]: 318e305602cae9061ab5cc5d96c2af622953769b2f0c88d5dfd727d0a8f9b072 2024-04-26T21:50:31.508 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 21:50:31 smithi064 systemd[1]: Started Ceph mds.e for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:31.723 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:31.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 -- 172.21.15.55:0/3216495952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 msgr2=0x7fb64010af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:31.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 --2- 172.21.15.55:0/3216495952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 0x7fb64010af30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb638009cd0 tx=0x7fb63802a8e0 comp rx=0 tx=0).stop 2024-04-26T21:50:31.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 -- 172.21.15.55:0/3216495952 shutdown_connections 2024-04-26T21:50:31.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 --2- 172.21.15.55:0/3216495952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 0x7fb64010af30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:31.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 --2- 172.21.15.55:0/3216495952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb640103ce0 0x7fb640104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:31.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 --2- 172.21.15.55:0/3216495952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb640100c70 0x7fb640101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:31.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 -- 172.21.15.55:0/3216495952 >> 172.21.15.55:0/3216495952 conn(0x7fb6400fc5a0 msgr2=0x7fb6400fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 -- 172.21.15.55:0/3216495952 shutdown_connections 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.838+0000 7fb6481f0640 1 -- 172.21.15.55:0/3216495952 wait complete. 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb6481f0640 1 Processor -- start 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb6481f0640 1 -- start start 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb6481f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb640100c70 0x7fb640198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb6481f0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb640103ce0 0x7fb6401990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb6481f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 0x7fb6401a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb6481f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb64010e4c0 con 0x7fb6401046a0 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb6481f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb64010e340 con 0x7fb640100c70 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb6481f0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb64010e640 con 0x7fb640103ce0 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb645f8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb640100c70 0x7fb640198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb64678d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 0x7fb6401a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:31.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb645f8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb640100c70 0x7fb640198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:56276/0 (socket says 172.21.15.55:56276) 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb645f8c640 1 -- 172.21.15.55:0/1896580604 learned_addr learned my addr 172.21.15.55:0/1896580604 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb64578b640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb640103ce0 0x7fb6401990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb64678d640 1 -- 172.21.15.55:0/1896580604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb640103ce0 msgr2=0x7fb6401990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb64678d640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb640103ce0 0x7fb6401990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb64678d640 1 -- 172.21.15.55:0/1896580604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb640100c70 msgr2=0x7fb640198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb64678d640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb640100c70 0x7fb640198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb64678d640 1 -- 172.21.15.55:0/1896580604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6401a07c0 con 0x7fb6401046a0 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.839+0000 7fb645f8c640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb640100c70 0x7fb640198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.840+0000 7fb64678d640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 0x7fb6401a0150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb638031780 tx=0x7fb638003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.840+0000 7fb636ffd640 1 -- 172.21.15.55:0/1896580604 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638004040 con 0x7fb6401046a0 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.840+0000 7fb636ffd640 1 -- 172.21.15.55:0/1896580604 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb638004690 con 0x7fb6401046a0 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.840+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6401a0a50 con 0x7fb6401046a0 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.840+0000 7fb636ffd640 1 -- 172.21.15.55:0/1896580604 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638030ea0 con 0x7fb6401046a0 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.840+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6401a0f40 con 0x7fb6401046a0 2024-04-26T21:50:31.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.841+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb640109e90 con 0x7fb6401046a0 2024-04-26T21:50:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.842+0000 7fb636ffd640 1 -- 172.21.15.55:0/1896580604 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb638030630 con 0x7fb6401046a0 2024-04-26T21:50:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.843+0000 7fb636ffd640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb62c0a3bb0 0x7fb62c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.843+0000 7fb636ffd640 1 -- 172.21.15.55:0/1896580604 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb63802e030 con 0x7fb6401046a0 2024-04-26T21:50:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.843+0000 7fb645f8c640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb62c0a3bb0 0x7fb62c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:31.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.843+0000 7fb645f8c640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb62c0a3bb0 0x7fb62c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005fd0 tx=0x7fb63c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:31.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:31.846+0000 7fb636ffd640 1 -- 172.21.15.55:0/1896580604 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb6380be090 con 0x7fb6401046a0 2024-04-26T21:50:32.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.077+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb64002dc80 con 0x7fb6401046a0 2024-04-26T21:50:32.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.078+0000 7fb636ffd640 1 -- 172.21.15.55:0/1896580604 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fb6380bd990 con 0x7fb6401046a0 2024-04-26T21:50:32.079 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:32.079 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:50:32.079 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 6 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb62c0a3bb0 msgr2=0x7fb62c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb62c0a3bb0 0x7fb62c0a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb63c005fd0 tx=0x7fb63c00b410 comp rx=0 tx=0).stop 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 msgr2=0x7fb6401a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 0x7fb6401a0150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb638031780 tx=0x7fb638003b80 comp rx=0 tx=0).stop 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 shutdown_connections 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb62c0a3bb0 0x7fb62c0a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb6401046a0 0x7fb6401a0150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb640103ce0 0x7fb6401990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 --2- 172.21.15.55:0/1896580604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb640100c70 0x7fb640198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 >> 172.21.15.55:0/1896580604 conn(0x7fb6400fc5a0 msgr2=0x7fb640107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 shutdown_connections 2024-04-26T21:50:32.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:32.080+0000 7fb6481f0640 1 -- 172.21.15.55:0/1896580604 wait complete. 2024-04-26T21:50:32.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: Deploying daemon mds.f on smithi165 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6834/2328208855,v1:172.21.15.64:6835/2328208855] up:boot 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: fsmap 5 up:standby 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:50:32.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:32 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1896580604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:32.549 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.549 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.549 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.549 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:32.549 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:32.549 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:32.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: Deploying daemon mds.f on smithi165 2024-04-26T21:50:32.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:32.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6834/2328208855,v1:172.21.15.64:6835/2328208855] up:boot 2024-04-26T21:50:32.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: fsmap 5 up:standby 2024-04-26T21:50:32.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:50:32.550 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:32 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1896580604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: Deploying daemon mds.f on smithi165 2024-04-26T21:50:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:32.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6834/2328208855,v1:172.21.15.64:6835/2328208855] up:boot 2024-04-26T21:50:32.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: fsmap 5 up:standby 2024-04-26T21:50:32.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T21:50:32.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:32 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1896580604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:33.577 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:33 smithi165 systemd[1]: Starting Ceph mds.f for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:33.997 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:33 smithi165 podman[63272]: 2024-04-26 21:50:33.59915008 +0000 UTC m=+0.018208567 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:33.997 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:33 smithi165 podman[63272]: 2024-04-26 21:50:33.719009081 +0000 UTC m=+0.138067553 container create 5fe755a3e5039f885a46bc3cd3893a25336e9cb538f7c3b942b4b72296c62e23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T21:50:33.997 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:33 smithi165 podman[63272]: 2024-04-26 21:50:33.996489549 +0000 UTC m=+0.415548022 container init 5fe755a3e5039f885a46bc3cd3893a25336e9cb538f7c3b942b4b72296c62e23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:50:34.093 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 -- 172.21.15.55:0/527401777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994104550 msgr2=0x7f799410ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 --2- 172.21.15.55:0/527401777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994104550 0x7f799410ade0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f798c009ca0 tx=0x7f798c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 -- 172.21.15.55:0/527401777 shutdown_connections 2024-04-26T21:50:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 --2- 172.21.15.55:0/527401777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994104550 0x7f799410ade0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 --2- 172.21.15.55:0/527401777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7994103b90 0x7f7994104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 --2- 172.21.15.55:0/527401777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79941029a0 0x7f7994102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 -- 172.21.15.55:0/527401777 >> 172.21.15.55:0/527401777 conn(0x7f79940fdf30 msgr2=0x7f7994100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 -- 172.21.15.55:0/527401777 shutdown_connections 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 -- 172.21.15.55:0/527401777 wait complete. 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.213+0000 7f79993b6640 1 Processor -- start 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79993b6640 1 -- start start 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79993b6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79941029a0 0x7f7994198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79993b6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994103b90 0x7f7994199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79993b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7994104550 0x7f79941a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79993b6640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f799410e4b0 con 0x7f7994103b90 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79993b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f799410e330 con 0x7f7994104550 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79993b6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f799410e630 con 0x7f79941029a0 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79927fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994103b90 0x7f7994199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79927fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994103b90 0x7f7994199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34726/0 (socket says 172.21.15.55:34726) 2024-04-26T21:50:34.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79927fc640 1 -- 172.21.15.55:0/3311324241 learned_addr learned my addr 172.21.15.55:0/3311324241 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f7992ffd640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79941029a0 0x7f7994198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79937fe640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7994104550 0x7f79941a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f7992ffd640 1 -- 172.21.15.55:0/3311324241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7994104550 msgr2=0x7f79941a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f7992ffd640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7994104550 0x7f79941a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f7992ffd640 1 -- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994103b90 msgr2=0x7f7994199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f7992ffd640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994103b90 0x7f7994199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f7992ffd640 1 -- 172.21.15.55:0/3311324241 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79941a06e0 con 0x7f79941029a0 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.214+0000 7f79927fc640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994103b90 0x7f7994199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.215+0000 7f7992ffd640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79941029a0 0x7f7994198ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f798400e9c0 tx=0x7f798400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.215+0000 7f797bfff640 1 -- 172.21.15.55:0/3311324241 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798400ccc0 con 0x7f79941029a0 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.215+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79941a09d0 con 0x7f79941029a0 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.215+0000 7f797bfff640 1 -- 172.21.15.55:0/3311324241 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79840035f0 con 0x7f79941029a0 2024-04-26T21:50:34.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.215+0000 7f797bfff640 1 -- 172.21.15.55:0/3311324241 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798400ce60 con 0x7f79941029a0 2024-04-26T21:50:34.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.215+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79941a0f70 con 0x7f79941029a0 2024-04-26T21:50:34.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.216+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7994069ba0 con 0x7f79941029a0 2024-04-26T21:50:34.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.217+0000 7f797bfff640 1 -- 172.21.15.55:0/3311324241 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7984004000 con 0x7f79941029a0 2024-04-26T21:50:34.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.218+0000 7f797bfff640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f797c0a3bb0 0x7f797c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:34.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.218+0000 7f79927fc640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f797c0a3bb0 0x7f797c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:34.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.218+0000 7f797bfff640 1 -- 172.21.15.55:0/3311324241 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f79840d05f0 con 0x7f79941029a0 2024-04-26T21:50:34.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.218+0000 7f79927fc640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f797c0a3bb0 0x7f797c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7994199fd0 tx=0x7f798800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:34.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.221+0000 7f797bfff640 1 -- 172.21.15.55:0/3311324241 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f798409d3d0 con 0x7f79941029a0 2024-04-26T21:50:34.326 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:33 smithi165 podman[63272]: 2024-04-26 21:50:33.999733949 +0000 UTC m=+0.418792425 container start 5fe755a3e5039f885a46bc3cd3893a25336e9cb538f7c3b942b4b72296c62e23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-26T21:50:34.327 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:34 smithi165 ceph-mds[63291]: -- 172.21.15.165:0/2829280240 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c9797ba40 con 0x562c9794e800 2024-04-26T21:50:34.327 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:34 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f[63287]: starting mds.f at 2024-04-26T21:50:34.327 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:34 smithi165 bash[63272]: 5fe755a3e5039f885a46bc3cd3893a25336e9cb538f7c3b942b4b72296c62e23 2024-04-26T21:50:34.327 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 21:50:34 smithi165 systemd[1]: Started Ceph mds.f for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:34.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.451+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7994109c60 con 0x7f79941029a0 2024-04-26T21:50:34.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.452+0000 7f797bfff640 1 -- 172.21.15.55:0/3311324241 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f798409ccd0 con 0x7f79941029a0 2024-04-26T21:50:34.452 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:34.453 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:50:34.453 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 7 2024-04-26T21:50:34.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.454+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f797c0a3bb0 msgr2=0x7f797c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.454+0000 7f79993b6640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f797c0a3bb0 0x7f797c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7994199fd0 tx=0x7f798800a380 comp rx=0 tx=0).stop 2024-04-26T21:50:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.454+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79941029a0 msgr2=0x7f7994198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.454+0000 7f79993b6640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79941029a0 0x7f7994198ac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f798400e9c0 tx=0x7f798400ee80 comp rx=0 tx=0).stop 2024-04-26T21:50:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.455+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 shutdown_connections 2024-04-26T21:50:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.455+0000 7f79993b6640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f797c0a3bb0 0x7f797c0a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.455+0000 7f79993b6640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7994104550 0x7f79941a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.455+0000 7f79993b6640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7994103b90 0x7f7994199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.455+0000 7f79993b6640 1 --2- 172.21.15.55:0/3311324241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79941029a0 0x7f7994198ac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.455+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 >> 172.21.15.55:0/3311324241 conn(0x7f79940fdf30 msgr2=0x7f7994106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.455+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 shutdown_connections 2024-04-26T21:50:34.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:34.455+0000 7f79993b6640 1 -- 172.21.15.55:0/3311324241 wait complete. 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: Deploying daemon mds.c on smithi165 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.165:6834/125466680,v1:172.21.15.165:6835/125466680] up:boot 2024-04-26T21:50:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: fsmap 6 up:standby 2024-04-26T21:50:35.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:50:35.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:35 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3311324241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:35.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: Deploying daemon mds.c on smithi165 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.165:6834/125466680,v1:172.21.15.165:6835/125466680] up:boot 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: fsmap 6 up:standby 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:50:35.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:35 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3311324241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: Deploying daemon mds.c on smithi165 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.165:6834/125466680,v1:172.21.15.165:6835/125466680] up:boot 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: fsmap 6 up:standby 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T21:50:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:35 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3311324241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:36.467 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:36.535 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 systemd[1]: Starting Ceph mds.c for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:36.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 -- 172.21.15.55:0/3403957611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544102e90 msgr2=0x7f95441032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 --2- 172.21.15.55:0/3403957611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544102e90 0x7f95441032b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9534009930 tx=0x7f953402a420 comp rx=0 tx=0).stop 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 -- 172.21.15.55:0/3403957611 shutdown_connections 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 --2- 172.21.15.55:0/3403957611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95441024f0 0x7f9544109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 --2- 172.21.15.55:0/3403957611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9544101b30 0x7f9544101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 --2- 172.21.15.55:0/3403957611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544102e90 0x7f95441032b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 -- 172.21.15.55:0/3403957611 >> 172.21.15.55:0/3403957611 conn(0x7f95440fc5a0 msgr2=0x7f95440fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 -- 172.21.15.55:0/3403957611 shutdown_connections 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.583+0000 7f95497f0640 1 -- 172.21.15.55:0/3403957611 wait complete. 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95497f0640 1 Processor -- start 2024-04-26T21:50:36.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95497f0640 1 -- start start 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95497f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544101b30 0x7f9544101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95497f0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95441024f0 0x7f954419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95497f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9544102e90 0x7f954419ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95497f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f954410d3d0 con 0x7f9544101b30 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95497f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f954410d250 con 0x7f9544102e90 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95497f0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f954410d550 con 0x7f95441024f0 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544101b30 0x7f9544101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544101b30 0x7f9544101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34744/0 (socket says 172.21.15.55:34744) 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 -- 172.21.15.55:0/1904130668 learned_addr learned my addr 172.21.15.55:0/1904130668 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f95437fe640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9544102e90 0x7f954419ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f953bfff640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95441024f0 0x7f954419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 -- 172.21.15.55:0/1904130668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95441024f0 msgr2=0x7f954419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95441024f0 0x7f954419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 -- 172.21.15.55:0/1904130668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9544102e90 msgr2=0x7f954419ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9544102e90 0x7f954419ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 -- 172.21.15.55:0/1904130668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95441a0880 con 0x7f9544101b30 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.584+0000 7f9542ffd640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544101b30 0x7f9544101590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9534009230 tx=0x7f95340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.585+0000 7f9540ff9640 1 -- 172.21.15.55:0/1904130668 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953403f030 con 0x7f9544101b30 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.585+0000 7f9540ff9640 1 -- 172.21.15.55:0/1904130668 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9534003cb0 con 0x7f9544101b30 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.585+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95441a0ab0 con 0x7f9544101b30 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.585+0000 7f9540ff9640 1 -- 172.21.15.55:0/1904130668 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534035070 con 0x7f9544101b30 2024-04-26T21:50:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.585+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95441a0fa0 con 0x7f9544101b30 2024-04-26T21:50:36.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.586+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9544108db0 con 0x7f9544101b30 2024-04-26T21:50:36.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.587+0000 7f9540ff9640 1 -- 172.21.15.55:0/1904130668 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9534042030 con 0x7f9544101b30 2024-04-26T21:50:36.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.587+0000 7f9540ff9640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95300a3bb0 0x7f95300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:36.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.588+0000 7f953bfff640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95300a3bb0 0x7f95300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:36.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.588+0000 7f9540ff9640 1 -- 172.21.15.55:0/1904130668 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f95340f0550 con 0x7f9544101b30 2024-04-26T21:50:36.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.588+0000 7f953bfff640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95300a3bb0 0x7f95300a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9544101300 tx=0x7f952c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:36.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.592+0000 7f9540ff9640 1 -- 172.21.15.55:0/1904130668 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f95340bcce0 con 0x7f9544101b30 2024-04-26T21:50:36.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.818+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9544063560 con 0x7f9544101b30 2024-04-26T21:50:36.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.818+0000 7f9540ff9640 1 -- 172.21.15.55:0/1904130668 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f95340c0fe0 con 0x7f9544101b30 2024-04-26T21:50:36.819 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:36.819 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T21:50:36.819 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 7 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95300a3bb0 msgr2=0x7f95300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95300a3bb0 0x7f95300a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9544101300 tx=0x7f952c00b390 comp rx=0 tx=0).stop 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544101b30 msgr2=0x7f9544101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544101b30 0x7f9544101590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9534009230 tx=0x7f95340042f0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 shutdown_connections 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95300a3bb0 0x7f95300a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9544102e90 0x7f954419ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95441024f0 0x7f954419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 --2- 172.21.15.55:0/1904130668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9544101b30 0x7f9544101590 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 >> 172.21.15.55:0/1904130668 conn(0x7f95440fc5a0 msgr2=0x7f9544105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:36.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 shutdown_connections 2024-04-26T21:50:36.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:36.820+0000 7f95497f0640 1 -- 172.21.15.55:0/1904130668 wait complete. 2024-04-26T21:50:36.826 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 podman[63582]: 2024-04-26 21:50:36.437772626 +0000 UTC m=+0.018003632 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:36.826 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 podman[63582]: 2024-04-26 21:50:36.557223103 +0000 UTC m=+0.137454096 container create deb82a91f5eb408ae91782593706760101dad61f4e8c78c8e969200595ce7ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True) 2024-04-26T21:50:37.076 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 podman[63582]: 2024-04-26 21:50:36.82670378 +0000 UTC m=+0.406934773 container init deb82a91f5eb408ae91782593706760101dad61f4e8c78c8e969200595ce7ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T21:50:37.077 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 podman[63582]: 2024-04-26 21:50:36.829354998 +0000 UTC m=+0.409585997 container start deb82a91f5eb408ae91782593706760101dad61f4e8c78c8e969200595ce7ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T21:50:37.077 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 ceph-mds[63601]: -- 172.21.15.165:0/2271105031 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624abfcba40 con 0x5624abf9e800 2024-04-26T21:50:37.077 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c[63597]: starting mds.c at 2024-04-26T21:50:37.077 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 bash[63582]: deb82a91f5eb408ae91782593706760101dad61f4e8c78c8e969200595ce7ca6 2024-04-26T21:50:37.077 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 21:50:36 smithi165 systemd[1]: Started Ceph mds.c for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:37.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:37 smithi055 ceph-mon[29221]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:37 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1904130668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:37 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:37 smithi064 ceph-mon[32765]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:37 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1904130668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:37.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:37 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:37.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:37 smithi165 ceph-mon[33296]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:37 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1904130668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:37.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:37 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:38.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:38 smithi064 ceph-mon[32765]: Deploying daemon mds.h on smithi064 2024-04-26T21:50:38.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:38 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.165:6836/2954631237,v1:172.21.15.165:6837/2954631237] up:boot 2024-04-26T21:50:38.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:38 smithi064 ceph-mon[32765]: fsmap 7 up:standby 2024-04-26T21:50:38.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:50:38.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:38 smithi055 ceph-mon[29221]: Deploying daemon mds.h on smithi064 2024-04-26T21:50:38.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:38 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.165:6836/2954631237,v1:172.21.15.165:6837/2954631237] up:boot 2024-04-26T21:50:38.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:38 smithi055 ceph-mon[29221]: fsmap 7 up:standby 2024-04-26T21:50:38.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:50:38.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:38 smithi165 ceph-mon[33296]: Deploying daemon mds.h on smithi064 2024-04-26T21:50:38.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:38 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.165:6836/2954631237,v1:172.21.15.165:6837/2954631237] up:boot 2024-04-26T21:50:38.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:38 smithi165 ceph-mon[33296]: fsmap 7 up:standby 2024-04-26T21:50:38.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T21:50:38.832 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 -- 172.21.15.55:0/2592923807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 msgr2=0x7fe0a010aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/2592923807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 0x7fe0a010aac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe098009ca0 tx=0x7fe09802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 -- 172.21.15.55:0/2592923807 shutdown_connections 2024-04-26T21:50:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/2592923807 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 0x7fe0a010aac0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/2592923807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a0105d40 0x7fe0a0108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/2592923807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0a0103410 0x7fe0a0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 -- 172.21.15.55:0/2592923807 >> 172.21.15.55:0/2592923807 conn(0x7fe0a00fce20 msgr2=0x7fe0a00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 -- 172.21.15.55:0/2592923807 shutdown_connections 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.947+0000 7fe0a5bac640 1 -- 172.21.15.55:0/2592923807 wait complete. 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe0a5bac640 1 Processor -- start 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe0a5bac640 1 -- start start 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe0a5bac640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a0103410 0x7fe0a0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe0a5bac640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0a0105d40 0x7fe0a01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe0a5bac640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 0x7fe0a01a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe0a5bac640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a010e4e0 con 0x7fe0a0108670 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe0a5bac640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a010e360 con 0x7fe0a0103410 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe0a5bac640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a010e660 con 0x7fe0a0105d40 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 0x7fe0a01a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 0x7fe0a01a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34766/0 (socket says 172.21.15.55:34766) 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09ffff640 1 -- 172.21.15.55:0/1225278095 learned_addr learned my addr 172.21.15.55:0/1225278095 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:38.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09f7fe640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a0103410 0x7fe0a0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe097fff640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0a0105d40 0x7fe0a01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09ffff640 1 -- 172.21.15.55:0/1225278095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0a0105d40 msgr2=0x7fe0a01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09ffff640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0a0105d40 0x7fe0a01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09ffff640 1 -- 172.21.15.55:0/1225278095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a0103410 msgr2=0x7fe0a0198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09ffff640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a0103410 0x7fe0a0198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09ffff640 1 -- 172.21.15.55:0/1225278095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a01a07d0 con 0x7fe0a0108670 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe09f7fe640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a0103410 0x7fe0a0198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.948+0000 7fe097fff640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0a0105d40 0x7fe0a01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.949+0000 7fe09ffff640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 0x7fe0a01a0160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0980097b0 tx=0x7fe098031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.949+0000 7fe09d7fa640 1 -- 172.21.15.55:0/1225278095 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098003bf0 con 0x7fe0a0108670 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.949+0000 7fe09d7fa640 1 -- 172.21.15.55:0/1225278095 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe098003d90 con 0x7fe0a0108670 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.949+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a01a0a60 con 0x7fe0a0108670 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.949+0000 7fe09d7fa640 1 -- 172.21.15.55:0/1225278095 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09802fc70 con 0x7fe0a0108670 2024-04-26T21:50:38.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.949+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a01a0fb0 con 0x7fe0a0108670 2024-04-26T21:50:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.950+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a00fe910 con 0x7fe0a0108670 2024-04-26T21:50:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.951+0000 7fe09d7fa640 1 -- 172.21.15.55:0/1225278095 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe098004040 con 0x7fe0a0108670 2024-04-26T21:50:38.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.952+0000 7fe09d7fa640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe08c0a3bb0 0x7fe08c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:38.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.952+0000 7fe09f7fe640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe08c0a3bb0 0x7fe08c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:38.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.952+0000 7fe09d7fa640 1 -- 172.21.15.55:0/1225278095 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe0980f1880 con 0x7fe0a0108670 2024-04-26T21:50:38.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.952+0000 7fe09f7fe640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe08c0a3bb0 0x7fe08c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0900058c0 tx=0x7fe09000a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:38.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:38.955+0000 7fe09d7fa640 1 -- 172.21.15.55:0/1225278095 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe0980be660 con 0x7fe0a0108670 2024-04-26T21:50:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.180+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0a002dc90 con 0x7fe0a0108670 2024-04-26T21:50:39.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.181+0000 7fe09d7fa640 1 -- 172.21.15.55:0/1225278095 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7fe0980bdf60 con 0x7fe0a0108670 2024-04-26T21:50:39.182 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:39.182 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T21:50:39.182 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe08c0a3bb0 msgr2=0x7fe08c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe08c0a3bb0 0x7fe08c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0900058c0 tx=0x7fe09000a250 comp rx=0 tx=0).stop 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 msgr2=0x7fe0a01a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 0x7fe0a01a0160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0980097b0 tx=0x7fe098031840 comp rx=0 tx=0).stop 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 shutdown_connections 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe08c0a3bb0 0x7fe08c0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0a0108670 0x7fe0a01a0160 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0a0105d40 0x7fe0a01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 --2- 172.21.15.55:0/1225278095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a0103410 0x7fe0a0198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 >> 172.21.15.55:0/1225278095 conn(0x7fe0a00fce20 msgr2=0x7fe0a0103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 shutdown_connections 2024-04-26T21:50:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:39.183+0000 7fe0a5bac640 1 -- 172.21.15.55:0/1225278095 wait complete. 2024-04-26T21:50:39.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:39 smithi055 ceph-mon[29221]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:39.505 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:39 smithi064 ceph-mon[32765]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:39.505 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 systemd[1]: Starting Ceph mds.h for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:39.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:39 smithi165 ceph-mon[33296]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:39.757 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 podman[62790]: 2024-04-26 21:50:39.4080328 +0000 UTC m=+0.018279783 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:39.757 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 podman[62790]: 2024-04-26 21:50:39.52185629 +0000 UTC m=+0.132103258 container create d2f9a90bf3b0821a478dd9d2aabcf0543709cc5a072ca8d733858e6ff5af7bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T21:50:40.172 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 podman[62790]: 2024-04-26 21:50:39.759250861 +0000 UTC m=+0.369497840 container init d2f9a90bf3b0821a478dd9d2aabcf0543709cc5a072ca8d733858e6ff5af7bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T21:50:40.173 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 podman[62790]: 2024-04-26 21:50:39.762143452 +0000 UTC m=+0.372390422 container start d2f9a90bf3b0821a478dd9d2aabcf0543709cc5a072ca8d733858e6ff5af7bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T21:50:40.173 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 ceph-mds[62809]: -- 172.21.15.64:0/2328097341 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1f3d09a40 con 0x55e1f3cdcc00 2024-04-26T21:50:40.173 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h[62805]: starting mds.h at 2024-04-26T21:50:40.173 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 bash[62790]: d2f9a90bf3b0821a478dd9d2aabcf0543709cc5a072ca8d733858e6ff5af7bcb 2024-04-26T21:50:40.173 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 21:50:39 smithi064 systemd[1]: Started Ceph mds.h for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:40.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:40 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1225278095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:40.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:40 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1225278095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:40.575 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:40 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1225278095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:40.575 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.575 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.575 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:40.575 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:40.575 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:40.575 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:41.195 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:41.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 -- 172.21.15.55:0/388375048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930104550 msgr2=0x7f093010ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:41.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 --2- 172.21.15.55:0/388375048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930104550 0x7f093010ade0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0928009ca0 tx=0x7f092802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 -- 172.21.15.55:0/388375048 shutdown_connections 2024-04-26T21:50:41.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 --2- 172.21.15.55:0/388375048 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930104550 0x7f093010ade0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 --2- 172.21.15.55:0/388375048 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0930103b90 0x7f0930104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 --2- 172.21.15.55:0/388375048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09301029a0 0x7f0930102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 -- 172.21.15.55:0/388375048 >> 172.21.15.55:0/388375048 conn(0x7f09300fdf30 msgr2=0x7f0930100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 -- 172.21.15.55:0/388375048 shutdown_connections 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.309+0000 7f0937fc1640 1 -- 172.21.15.55:0/388375048 wait complete. 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0937fc1640 1 Processor -- start 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0937fc1640 1 -- start start 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0937fc1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09301029a0 0x7f0930198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0937fc1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0930103b90 0x7f09301990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0937fc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0930104550 0x7f09301a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0937fc1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f093010e4d0 con 0x7f09301029a0 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0937fc1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f093010e350 con 0x7f0930104550 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0937fc1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f093010e650 con 0x7f0930103b90 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0935d5d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09301029a0 0x7f0930198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0935d5d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09301029a0 0x7f0930198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39616/0 (socket says 172.21.15.55:39616) 2024-04-26T21:50:41.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f0935d5d640 1 -- 172.21.15.55:0/1118723397 learned_addr learned my addr 172.21.15.55:0/1118723397 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.310+0000 7f093555c640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0930103b90 0x7f09301990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f093655e640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0930104550 0x7f09301a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0935d5d640 1 -- 172.21.15.55:0/1118723397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0930103b90 msgr2=0x7f09301990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0935d5d640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0930103b90 0x7f09301990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0935d5d640 1 -- 172.21.15.55:0/1118723397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0930104550 msgr2=0x7f09301a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0935d5d640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0930104550 0x7f09301a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0935d5d640 1 -- 172.21.15.55:0/1118723397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09301a07a0 con 0x7f09301029a0 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0935d5d640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09301029a0 0x7f0930198b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f092c00c970 tx=0x7f092c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0926ffd640 1 -- 172.21.15.55:0/1118723397 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c004220 con 0x7f09301029a0 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0926ffd640 1 -- 172.21.15.55:0/1118723397 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f092c0043c0 con 0x7f09301029a0 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09301a0a90 con 0x7f09301029a0 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0926ffd640 1 -- 172.21.15.55:0/1118723397 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c00f450 con 0x7f09301029a0 2024-04-26T21:50:41.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.311+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09301a0fe0 con 0x7f09301029a0 2024-04-26T21:50:41.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.312+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0930109d30 con 0x7f09301029a0 2024-04-26T21:50:41.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.313+0000 7f0926ffd640 1 -- 172.21.15.55:0/1118723397 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f092c003610 con 0x7f09301029a0 2024-04-26T21:50:41.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.314+0000 7f0926ffd640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f091c0a3bb0 0x7f091c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:41.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.314+0000 7f093555c640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f091c0a3bb0 0x7f091c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:41.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.314+0000 7f0926ffd640 1 -- 172.21.15.55:0/1118723397 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f092c0d0080 con 0x7f09301029a0 2024-04-26T21:50:41.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.315+0000 7f093555c640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f091c0a3bb0 0x7f091c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f093019a090 tx=0x7f0920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:41.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.318+0000 7f0926ffd640 1 -- 172.21.15.55:0/1118723397 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f092c014030 con 0x7f09301029a0 2024-04-26T21:50:41.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:41 smithi055 ceph-mon[29221]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:41.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:41 smithi055 ceph-mon[29221]: Deploying daemon mds.i on smithi165 2024-04-26T21:50:41.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:41 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] up:boot 2024-04-26T21:50:41.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:41 smithi055 ceph-mon[29221]: fsmap 8 up:standby 2024-04-26T21:50:41.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:41 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:50:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:41 smithi064 ceph-mon[32765]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:41 smithi064 ceph-mon[32765]: Deploying daemon mds.i on smithi165 2024-04-26T21:50:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:41 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] up:boot 2024-04-26T21:50:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:41 smithi064 ceph-mon[32765]: fsmap 8 up:standby 2024-04-26T21:50:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:41 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:50:41.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.545+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f093002dad0 con 0x7f09301029a0 2024-04-26T21:50:41.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.545+0000 7f0926ffd640 1 -- 172.21.15.55:0/1118723397 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f092c09c8b0 con 0x7f09301029a0 2024-04-26T21:50:41.546 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:50:41.546 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 9 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f091c0a3bb0 msgr2=0x7f091c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f091c0a3bb0 0x7f091c0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f093019a090 tx=0x7f0920005f90 comp rx=0 tx=0).stop 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09301029a0 msgr2=0x7f0930198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09301029a0 0x7f0930198b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f092c00c970 tx=0x7f092c00ce30 comp rx=0 tx=0).stop 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 shutdown_connections 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f091c0a3bb0 0x7f091c0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0930104550 0x7f09301a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0930103b90 0x7f09301990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 --2- 172.21.15.55:0/1118723397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09301029a0 0x7f0930198b80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 >> 172.21.15.55:0/1118723397 conn(0x7f09300fdf30 msgr2=0x7f0930106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.547+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 shutdown_connections 2024-04-26T21:50:41.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:41.548+0000 7f0937fc1640 1 -- 172.21.15.55:0/1118723397 wait complete. 2024-04-26T21:50:41.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:41 smithi165 ceph-mon[33296]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:41 smithi165 ceph-mon[33296]: Deploying daemon mds.i on smithi165 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:41 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] up:boot 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:41 smithi165 ceph-mon[33296]: fsmap 8 up:standby 2024-04-26T21:50:41.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:41 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T21:50:42.183 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 systemd[1]: Starting Ceph mds.i for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:42.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:42 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1118723397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:42.447 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:42 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1118723397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:42.448 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 podman[63889]: 2024-04-26 21:50:42.178444938 +0000 UTC m=+0.018579361 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:42.448 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 podman[63889]: 2024-04-26 21:50:42.295213091 +0000 UTC m=+0.135347513 container create c86513383e0e7ba9a4c43d6a44e3a2ef867dbbf8df53ae487391aabf856d3faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T21:50:42.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:42 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1118723397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:42.827 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 podman[63889]: 2024-04-26 21:50:42.556862949 +0000 UTC m=+0.396997354 container init c86513383e0e7ba9a4c43d6a44e3a2ef867dbbf8df53ae487391aabf856d3faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T21:50:42.827 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 podman[63889]: 2024-04-26 21:50:42.559470094 +0000 UTC m=+0.399604500 container start c86513383e0e7ba9a4c43d6a44e3a2ef867dbbf8df53ae487391aabf856d3faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T21:50:42.827 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 ceph-mds[63907]: -- 172.21.15.165:0/1296997090 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad460bba40 con 0x55ad4608ec00 2024-04-26T21:50:42.827 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i[63903]: starting mds.i at 2024-04-26T21:50:42.827 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 bash[63889]: c86513383e0e7ba9a4c43d6a44e3a2ef867dbbf8df53ae487391aabf856d3faf 2024-04-26T21:50:42.827 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 21:50:42 smithi165 systemd[1]: Started Ceph mds.i for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:43.456 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:43 smithi055 ceph-mon[29221]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:43.457 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.457 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.457 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.457 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:43.457 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:43.457 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:43 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:43.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:43 smithi064 ceph-mon[32765]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:43.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:43.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:43.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:43 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:43.559 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:43 smithi165 ceph-mon[33296]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:43.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:43.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:43.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:43 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:43.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 -- 172.21.15.55:0/3707600163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474103410 msgr2=0x7f1474105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 --2- 172.21.15.55:0/3707600163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474103410 0x7f1474105800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1464009be0 tx=0x7f146402a6c0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 -- 172.21.15.55:0/3707600163 shutdown_connections 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 --2- 172.21.15.55:0/3707600163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1474108670 0x7f147410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 --2- 172.21.15.55:0/3707600163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1474105d40 0x7f1474108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 --2- 172.21.15.55:0/3707600163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474103410 0x7f1474105800 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 -- 172.21.15.55:0/3707600163 >> 172.21.15.55:0/3707600163 conn(0x7f14740fce20 msgr2=0x7f14740ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 -- 172.21.15.55:0/3707600163 shutdown_connections 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.668+0000 7f1478b46640 1 -- 172.21.15.55:0/3707600163 wait complete. 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1478b46640 1 Processor -- start 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1478b46640 1 -- start start 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1478b46640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1474103410 0x7f1474198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1478b46640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474105d40 0x7f14741991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1478b46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1474108670 0x7f14741a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1478b46640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f147410e4c0 con 0x7f1474105d40 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1478b46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f147410e340 con 0x7f1474108670 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1478b46640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f147410e640 con 0x7f1474103410 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474105d40 0x7f14741991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474105d40 0x7f14741991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39620/0 (socket says 172.21.15.55:39620) 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 -- 172.21.15.55:0/3627379346 learned_addr learned my addr 172.21.15.55:0/3627379346 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1472d9d640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1474108670 0x7f14741a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f147259c640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1474103410 0x7f1474198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 -- 172.21.15.55:0/3627379346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1474103410 msgr2=0x7f1474198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1474103410 0x7f1474198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 -- 172.21.15.55:0/3627379346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1474108670 msgr2=0x7f14741a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1474108670 0x7f14741a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 -- 172.21.15.55:0/3627379346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14741a0870 con 0x7f1474105d40 2024-04-26T21:50:43.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.669+0000 7f1471d9b640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474105d40 0x7f14741991a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f145c00e9a0 tx=0x7f145c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:43.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.670+0000 7f146b7fe640 1 -- 172.21.15.55:0/3627379346 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c00cd80 con 0x7f1474105d40 2024-04-26T21:50:43.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.670+0000 7f146b7fe640 1 -- 172.21.15.55:0/3627379346 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145c0035f0 con 0x7f1474105d40 2024-04-26T21:50:43.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.670+0000 7f146b7fe640 1 -- 172.21.15.55:0/3627379346 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c010690 con 0x7f1474105d40 2024-04-26T21:50:43.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.670+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14741a0b60 con 0x7f1474105d40 2024-04-26T21:50:43.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.670+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741a10d0 con 0x7f1474105d40 2024-04-26T21:50:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.671+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14740fe910 con 0x7f1474105d40 2024-04-26T21:50:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.672+0000 7f146b7fe640 1 -- 172.21.15.55:0/3627379346 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f145c004000 con 0x7f1474105d40 2024-04-26T21:50:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.672+0000 7f146b7fe640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f14600a3c30 0x7f14600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.672+0000 7f146b7fe640 1 -- 172.21.15.55:0/3627379346 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f145c0d01c0 con 0x7f1474105d40 2024-04-26T21:50:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.672+0000 7f147259c640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f14600a3c30 0x7f14600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.673+0000 7f147259c640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f14600a3c30 0x7f14600a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f146402acb0 tx=0x7f1464005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.677+0000 7f146b7fe640 1 -- 172.21.15.55:0/3627379346 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f145c0042f0 con 0x7f1474105d40 2024-04-26T21:50:43.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.908+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f147402dad0 con 0x7f1474105d40 2024-04-26T21:50:43.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.909+0000 7f146b7fe640 1 -- 172.21.15.55:0/3627379346 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f145c09d040 con 0x7f1474105d40 2024-04-26T21:50:43.909 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:43.909 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:50:43.910 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 10 2024-04-26T21:50:43.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f14600a3c30 msgr2=0x7f14600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f14600a3c30 0x7f14600a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f146402acb0 tx=0x7f1464005fd0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474105d40 msgr2=0x7f14741991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:43.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474105d40 0x7f14741991a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f145c00e9a0 tx=0x7f145c00ee60 comp rx=0 tx=0).stop 2024-04-26T21:50:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 shutdown_connections 2024-04-26T21:50:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f14600a3c30 0x7f14600a60f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1474108670 0x7f14741a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1474105d40 0x7f14741991a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 --2- 172.21.15.55:0/3627379346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1474103410 0x7f1474198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 >> 172.21.15.55:0/3627379346 conn(0x7f14740fce20 msgr2=0x7f14741052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 shutdown_connections 2024-04-26T21:50:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:43.911+0000 7f1478b46640 1 -- 172.21.15.55:0/3627379346 wait complete. 2024-04-26T21:50:44.460 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:44 smithi055 ceph-mon[29221]: Deploying daemon mds.g on smithi055 2024-04-26T21:50:44.460 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:44 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] up:boot 2024-04-26T21:50:44.460 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:44 smithi055 ceph-mon[29221]: fsmap 9 up:standby 2024-04-26T21:50:44.460 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:44 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:50:44.461 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:44 smithi055 ceph-mon[29221]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:44.461 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:44 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3627379346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:44 smithi064 ceph-mon[32765]: Deploying daemon mds.g on smithi055 2024-04-26T21:50:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:44 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] up:boot 2024-04-26T21:50:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:44 smithi064 ceph-mon[32765]: fsmap 9 up:standby 2024-04-26T21:50:44.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:44 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:50:44.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:44 smithi064 ceph-mon[32765]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:44.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:44 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3627379346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:44.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:44 smithi165 ceph-mon[33296]: Deploying daemon mds.g on smithi055 2024-04-26T21:50:44.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:44 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] up:boot 2024-04-26T21:50:44.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:44 smithi165 ceph-mon[33296]: fsmap 9 up:standby 2024-04-26T21:50:44.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:44 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T21:50:44.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:44 smithi165 ceph-mon[33296]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:44.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:44 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3627379346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:45.924 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 -- 172.21.15.55:0/1793261328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850102a60 msgr2=0x7f4850102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 --2- 172.21.15.55:0/1793261328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850102a60 0x7f4850102ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4840009900 tx=0x7f484002a410 comp rx=0 tx=0).stop 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 -- 172.21.15.55:0/1793261328 shutdown_connections 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 --2- 172.21.15.55:0/1793261328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4850103420 0x7f4850109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 --2- 172.21.15.55:0/1793261328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850102a60 0x7f4850102ee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 --2- 172.21.15.55:0/1793261328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4850101870 0x7f4850101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 -- 172.21.15.55:0/1793261328 >> 172.21.15.55:0/1793261328 conn(0x7f48500fcde0 msgr2=0x7f48500ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 -- 172.21.15.55:0/1793261328 shutdown_connections 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.030+0000 7f485583f640 1 -- 172.21.15.55:0/1793261328 wait complete. 2024-04-26T21:50:46.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f485583f640 1 Processor -- start 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f485583f640 1 -- start start 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f485583f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850101870 0x7f4850105800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f485583f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4850102a60 0x7f4850103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f485583f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4850103420 0x7f4850104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f485583f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f485010d380 con 0x7f4850101870 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f485583f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f485010d200 con 0x7f4850103420 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f485583f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f485010d500 con 0x7f4850102a60 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f484effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850101870 0x7f4850105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f484effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850101870 0x7f4850105800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39672/0 (socket says 172.21.15.55:39672) 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f484effd640 1 -- 172.21.15.55:0/4272850862 learned_addr learned my addr 172.21.15.55:0/4272850862 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f484f7fe640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4850103420 0x7f4850104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f484effd640 1 -- 172.21.15.55:0/4272850862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4850102a60 msgr2=0x7f4850103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f484e7fc640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4850102a60 0x7f4850103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f484effd640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4850102a60 0x7f4850103e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.031+0000 7f484effd640 1 -- 172.21.15.55:0/4272850862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4850103420 msgr2=0x7f4850104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f484effd640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4850103420 0x7f4850104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f484effd640 1 -- 172.21.15.55:0/4272850862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4850104c10 con 0x7f4850101870 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f484f7fe640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4850103420 0x7f4850104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f484effd640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850101870 0x7f4850105800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f484400d770 tx=0x7f484400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f485483d640 1 -- 172.21.15.55:0/4272850862 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844017070 con 0x7f4850101870 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f485483d640 1 -- 172.21.15.55:0/4272850862 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4844009c30 con 0x7f4850101870 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48501a2d50 con 0x7f4850101870 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f485483d640 1 -- 172.21.15.55:0/4272850862 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844003cc0 con 0x7f4850101870 2024-04-26T21:50:46.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.032+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48501a3190 con 0x7f4850101870 2024-04-26T21:50:46.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.033+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48501a3420 con 0x7f4850101870 2024-04-26T21:50:46.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.034+0000 7f485483d640 1 -- 172.21.15.55:0/4272850862 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4844003e60 con 0x7f4850101870 2024-04-26T21:50:46.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.035+0000 7f485483d640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48380a3bb0 0x7f48380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:46.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.035+0000 7f484e7fc640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48380a3bb0 0x7f48380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:46.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.035+0000 7f485483d640 1 -- 172.21.15.55:0/4272850862 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f48440cfe80 con 0x7f4850101870 2024-04-26T21:50:46.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.035+0000 7f484e7fc640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48380a3bb0 0x7f48380a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f484002a930 tx=0x7f4840000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:46.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.041+0000 7f485483d640 1 -- 172.21.15.55:0/4272850862 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4844009de0 con 0x7f4850101870 2024-04-26T21:50:46.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.280+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48501a3710 con 0x7f4850101870 2024-04-26T21:50:46.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.281+0000 7f485483d640 1 -- 172.21.15.55:0/4272850862 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f484409cc60 con 0x7f4850101870 2024-04-26T21:50:46.282 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:46.282 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:50:46.282 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 10 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48380a3bb0 msgr2=0x7f48380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48380a3bb0 0x7f48380a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f484002a930 tx=0x7f4840000bc0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850101870 msgr2=0x7f4850105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850101870 0x7f4850105800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f484400d770 tx=0x7f484400dc30 comp rx=0 tx=0).stop 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 shutdown_connections 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48380a3bb0 0x7f48380a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4850103420 0x7f4850104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4850102a60 0x7f4850103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 --2- 172.21.15.55:0/4272850862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4850101870 0x7f4850105800 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.283+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 >> 172.21.15.55:0/4272850862 conn(0x7f48500fcde0 msgr2=0x7f4850069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.284+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 shutdown_connections 2024-04-26T21:50:46.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:46.284+0000 7f485583f640 1 -- 172.21.15.55:0/4272850862 wait complete. 2024-04-26T21:50:46.943 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:46.943 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:46.943 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: Deploying daemon mds.a on smithi055 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4272850862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.55:6838/2681477087,v1:172.21.15.55:6839/2681477087] up:boot 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: fsmap 10 up:standby 2024-04-26T21:50:46.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:46 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: Deploying daemon mds.a on smithi055 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4272850862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.55:6838/2681477087,v1:172.21.15.55:6839/2681477087] up:boot 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: fsmap 10 up:standby 2024-04-26T21:50:47.018 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:46 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:50:47.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:47.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:47.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:47.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:47.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:47.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:47.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:47.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: Deploying daemon mds.a on smithi055 2024-04-26T21:50:47.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4272850862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:47.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.55:6838/2681477087,v1:172.21.15.55:6839/2681477087] up:boot 2024-04-26T21:50:47.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: fsmap 10 up:standby 2024-04-26T21:50:47.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:46 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T21:50:47.937 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:47 smithi055 systemd[1]: Starting Ceph mds.a for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:48.219 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:47 smithi055 podman[68281]: 2024-04-26 21:50:47.858249843 +0000 UTC m=+0.018590728 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:48.219 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:47 smithi055 podman[68281]: 2024-04-26 21:50:47.965498391 +0000 UTC m=+0.125839244 container create b8a6de870765e91c067df02f0dc18bff31ce1e9890fc803d6f774e1cd74232cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T21:50:48.219 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:48 smithi055 podman[68281]: 2024-04-26 21:50:48.193064223 +0000 UTC m=+0.353405086 container init b8a6de870765e91c067df02f0dc18bff31ce1e9890fc803d6f774e1cd74232cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T21:50:48.219 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:48 smithi055 podman[68281]: 2024-04-26 21:50:48.1957096 +0000 UTC m=+0.356050454 container start b8a6de870765e91c067df02f0dc18bff31ce1e9890fc803d6f774e1cd74232cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T21:50:48.296 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:48.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 -- 172.21.15.55:0/2661721944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0103420 msgr2=0x7f86c0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 --2- 172.21.15.55:0/2661721944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0103420 0x7f86c0109cb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009ca0 tx=0x7f86b802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 -- 172.21.15.55:0/2661721944 shutdown_connections 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 --2- 172.21.15.55:0/2661721944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0103420 0x7f86c0109cb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 --2- 172.21.15.55:0/2661721944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c0102a60 0x7f86c0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 --2- 172.21.15.55:0/2661721944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c0101870 0x7f86c0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 -- 172.21.15.55:0/2661721944 >> 172.21.15.55:0/2661721944 conn(0x7f86c00fcde0 msgr2=0x7f86c00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 -- 172.21.15.55:0/2661721944 shutdown_connections 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.409+0000 7f86c866d640 1 -- 172.21.15.55:0/2661721944 wait complete. 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c866d640 1 Processor -- start 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c866d640 1 -- start start 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c866d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c0101870 0x7f86c0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c866d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0102a60 0x7f86c0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c866d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c0103420 0x7f86c019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c866d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010d400 con 0x7f86c0102a60 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c866d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010d280 con 0x7f86c0103420 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c866d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010d580 con 0x7f86c0101870 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c5c08640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0102a60 0x7f86c0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c5c08640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0102a60 0x7f86c0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39724/0 (socket says 172.21.15.55:39724) 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c5c08640 1 -- 172.21.15.55:0/1194168965 learned_addr learned my addr 172.21.15.55:0/1194168965 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c6c0a640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c0103420 0x7f86c019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c6409640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c0101870 0x7f86c0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c5c08640 1 -- 172.21.15.55:0/1194168965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c0101870 msgr2=0x7f86c0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c5c08640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c0101870 0x7f86c0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c5c08640 1 -- 172.21.15.55:0/1194168965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c0103420 msgr2=0x7f86c019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c5c08640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c0103420 0x7f86c019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c5c08640 1 -- 172.21.15.55:0/1194168965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c019f6f0 con 0x7f86c0102a60 2024-04-26T21:50:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c6409640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c0101870 0x7f86c0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.410+0000 7f86c6c0a640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c0103420 0x7f86c019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.411+0000 7f86c5c08640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0102a60 0x7f86c0198010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00b4a0 tx=0x7f86bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.411+0000 7f86b77fe640 1 -- 172.21.15.55:0/1194168965 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc014070 con 0x7f86c0102a60 2024-04-26T21:50:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.411+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c019f9e0 con 0x7f86c0102a60 2024-04-26T21:50:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.411+0000 7f86b77fe640 1 -- 172.21.15.55:0/1194168965 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86bc010ac0 con 0x7f86c0102a60 2024-04-26T21:50:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.411+0000 7f86b77fe640 1 -- 172.21.15.55:0/1194168965 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc017c30 con 0x7f86c0102a60 2024-04-26T21:50:48.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.411+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c019ff80 con 0x7f86c0102a60 2024-04-26T21:50:48.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.412+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c0108be0 con 0x7f86c0102a60 2024-04-26T21:50:48.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.413+0000 7f86b77fe640 1 -- 172.21.15.55:0/1194168965 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f86bc017070 con 0x7f86c0102a60 2024-04-26T21:50:48.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.414+0000 7f86b77fe640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:48.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.414+0000 7f86c6409640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:48.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.414+0000 7f86b77fe640 1 -- 172.21.15.55:0/1194168965 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f86bc0cf710 con 0x7f86c0102a60 2024-04-26T21:50:48.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.414+0000 7f86c6409640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f86b0005d00 tx=0x7f86b0005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:48.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.417+0000 7f86b77fe640 1 -- 172.21.15.55:0/1194168965 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f86bc09c570 con 0x7f86c0102a60 2024-04-26T21:50:48.600 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-mds[68300]: -- 172.21.15.55:0/962729100 <== mon.0 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edd57d1a40 con 0x55edd57a4800 2024-04-26T21:50:48.600 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a[68296]: starting mds.a at 2024-04-26T21:50:48.600 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:48 smithi055 bash[68281]: b8a6de870765e91c067df02f0dc18bff31ce1e9890fc803d6f774e1cd74232cb 2024-04-26T21:50:48.600 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 21:50:48 smithi055 systemd[1]: Started Ceph mds.a for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:48.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.653+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86c002dad0 con 0x7f86c0102a60 2024-04-26T21:50:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.653+0000 7f86b77fe640 1 -- 172.21.15.55:0/1194168965 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f86bc09be70 con 0x7f86c0102a60 2024-04-26T21:50:48.654 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:48.654 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:50:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 12 2024-04-26T21:50:48.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86ac0a3bb0 msgr2=0x7f86ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f86b0005d00 tx=0x7f86b0005ef0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0102a60 msgr2=0x7f86c0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:48.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0102a60 0x7f86c0198010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00b4a0 tx=0x7f86bc00b960 comp rx=0 tx=0).stop 2024-04-26T21:50:48.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 shutdown_connections 2024-04-26T21:50:48.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86ac0a3bb0 0x7f86ac0a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86c0103420 0x7f86c019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86c0102a60 0x7f86c0198010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 --2- 172.21.15.55:0/1194168965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c0101870 0x7f86c0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:48.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.655+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 >> 172.21.15.55:0/1194168965 conn(0x7f86c00fcde0 msgr2=0x7f86c0105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:48.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.656+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 shutdown_connections 2024-04-26T21:50:48.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:48.656+0000 7f86c866d640 1 -- 172.21.15.55:0/1194168965 wait complete. 2024-04-26T21:50:48.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-mon[29221]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:48.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:48.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:48 smithi064 ceph-mon[32765]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:49.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:49.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:49.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:49.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:49.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:48 smithi165 ceph-mon[33296]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:49.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:49.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:49.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:49.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T21:50:49.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T21:50:49.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:49.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:49 smithi055 ceph-mon[29221]: Deploying daemon mds.b on smithi064 2024-04-26T21:50:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:49 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.55:6840/1689426633,v1:172.21.15.55:6841/1689426633] up:boot 2024-04-26T21:50:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:49 smithi055 ceph-mon[29221]: fsmap 11 up:standby 2024-04-26T21:50:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:50:49.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:49 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1194168965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:49 smithi064 ceph-mon[32765]: Deploying daemon mds.b on smithi064 2024-04-26T21:50:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:49 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.55:6840/1689426633,v1:172.21.15.55:6841/1689426633] up:boot 2024-04-26T21:50:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:49 smithi064 ceph-mon[32765]: fsmap 11 up:standby 2024-04-26T21:50:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:50:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:49 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1194168965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:50.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:49 smithi165 ceph-mon[33296]: Deploying daemon mds.b on smithi064 2024-04-26T21:50:50.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:49 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.55:6840/1689426633,v1:172.21.15.55:6841/1689426633] up:boot 2024-04-26T21:50:50.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:49 smithi165 ceph-mon[33296]: fsmap 11 up:standby 2024-04-26T21:50:50.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T21:50:50.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:49 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1194168965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:50.668 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:50.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:50 smithi064 ceph-mon[32765]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:50.757 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:50 smithi064 systemd[1]: Starting Ceph mds.b for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 -- 172.21.15.55:0/2803550038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 msgr2=0x7fb16410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 --2- 172.21.15.55:0/2803550038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 0x7fb16410aac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009ca0 tx=0x7fb15c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 -- 172.21.15.55:0/2803550038 shutdown_connections 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 --2- 172.21.15.55:0/2803550038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 0x7fb16410aac0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 --2- 172.21.15.55:0/2803550038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb164105d40 0x7fb164108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 --2- 172.21.15.55:0/2803550038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb164103410 0x7fb164105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 -- 172.21.15.55:0/2803550038 >> 172.21.15.55:0/2803550038 conn(0x7fb1640fce20 msgr2=0x7fb1640ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 -- 172.21.15.55:0/2803550038 shutdown_connections 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.783+0000 7fb16955f640 1 -- 172.21.15.55:0/2803550038 wait complete. 2024-04-26T21:50:50.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb16955f640 1 Processor -- start 2024-04-26T21:50:50.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb16955f640 1 -- start start 2024-04-26T21:50:50.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb16955f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb164103410 0x7fb164198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb16955f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb164105d40 0x7fb1641990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb16955f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 0x7fb1641a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb16955f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410e4c0 con 0x7fb164108670 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb16955f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410e340 con 0x7fb164103410 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb16955f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb16410e640 con 0x7fb164105d40 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb1637fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 0x7fb1641a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb162ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb164103410 0x7fb164198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb162ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb164103410 0x7fb164198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49098/0 (socket says 172.21.15.55:49098) 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb1637fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 0x7fb1641a0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45158/0 (socket says 172.21.15.55:45158) 2024-04-26T21:50:50.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb162ffd640 1 -- 172.21.15.55:0/2522033016 learned_addr learned my addr 172.21.15.55:0/2522033016 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb1627fc640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb164105d40 0x7fb1641990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb162ffd640 1 -- 172.21.15.55:0/2522033016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb164105d40 msgr2=0x7fb1641990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb162ffd640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb164105d40 0x7fb1641990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb162ffd640 1 -- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 msgr2=0x7fb1641a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb162ffd640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 0x7fb1641a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb162ffd640 1 -- 172.21.15.55:0/2522033016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1641a0780 con 0x7fb164103410 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb1627fc640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb164105d40 0x7fb1641990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.784+0000 7fb1637fe640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 0x7fb1641a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.785+0000 7fb162ffd640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb164103410 0x7fb164198b60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d950 tx=0x7fb15400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.785+0000 7fb14bfff640 1 -- 172.21.15.55:0/2522033016 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154017070 con 0x7fb164103410 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.785+0000 7fb14bfff640 1 -- 172.21.15.55:0/2522033016 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb154009c30 con 0x7fb164103410 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.785+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1641a0a70 con 0x7fb164103410 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.785+0000 7fb14bfff640 1 -- 172.21.15.55:0/2522033016 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15400f620 con 0x7fb164103410 2024-04-26T21:50:50.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.785+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1641a0fc0 con 0x7fb164103410 2024-04-26T21:50:50.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.786+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1640fe910 con 0x7fb164103410 2024-04-26T21:50:50.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.787+0000 7fb14bfff640 1 -- 172.21.15.55:0/2522033016 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb154005030 con 0x7fb164103410 2024-04-26T21:50:50.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.788+0000 7fb14bfff640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb14c0a3bb0 0x7fb14c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:50.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.788+0000 7fb1627fc640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb14c0a3bb0 0x7fb14c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.788+0000 7fb14bfff640 1 -- 172.21.15.55:0/2522033016 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb1540cfc30 con 0x7fb164103410 2024-04-26T21:50:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.788+0000 7fb1627fc640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb14c0a3bb0 0x7fb14c0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb16419a070 tx=0x7fb15800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:50.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:50.791+0000 7fb14bfff640 1 -- 172.21.15.55:0/2522033016 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb15409ca10 con 0x7fb164103410 2024-04-26T21:50:50.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:50 smithi055 ceph-mon[29221]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.016+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb16402dad0 con 0x7fb164103410 2024-04-26T21:50:51.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.017+0000 7fb14bfff640 1 -- 172.21.15.55:0/2522033016 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7fb154009de0 con 0x7fb164103410 2024-04-26T21:50:51.017 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:51.017 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T21:50:51.018 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 12 2024-04-26T21:50:51.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb14c0a3bb0 msgr2=0x7fb14c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:51.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb14c0a3bb0 0x7fb14c0a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb16419a070 tx=0x7fb15800b5f0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb164103410 msgr2=0x7fb164198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb164103410 0x7fb164198b60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d950 tx=0x7fb15400de10 comp rx=0 tx=0).stop 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 shutdown_connections 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb14c0a3bb0 0x7fb14c0a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb164108670 0x7fb1641a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb164105d40 0x7fb1641990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 --2- 172.21.15.55:0/2522033016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb164103410 0x7fb164198b60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 >> 172.21.15.55:0/2522033016 conn(0x7fb1640fce20 msgr2=0x7fb164103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 shutdown_connections 2024-04-26T21:50:51.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:51.019+0000 7fb16955f640 1 -- 172.21.15.55:0/2522033016 wait complete. 2024-04-26T21:50:51.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:50 smithi165 ceph-mon[33296]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:51.130 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:50 smithi064 podman[63100]: 2024-04-26 21:50:50.756751823 +0000 UTC m=+0.018585130 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T21:50:51.130 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:50 smithi064 podman[63100]: 2024-04-26 21:50:50.860818899 +0000 UTC m=+0.122652198 container create 5568b9c16c3dc924c4bc38ffa578fcadc4641c0777cf387a00860b44084760e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T21:50:51.130 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:51 smithi064 podman[63100]: 2024-04-26 21:50:51.071690413 +0000 UTC m=+0.333523714 container init 5568b9c16c3dc924c4bc38ffa578fcadc4641c0777cf387a00860b44084760e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T21:50:51.130 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:51 smithi064 podman[63100]: 2024-04-26 21:50:51.074342793 +0000 UTC m=+0.336176094 container start 5568b9c16c3dc924c4bc38ffa578fcadc4641c0777cf387a00860b44084760e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T21:50:51.130 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-mds[63119]: -- 172.21.15.64:0/3639848264 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a24f4d9a40 con 0x55a24f4ac800 2024-04-26T21:50:51.130 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: starting mds.b at 2024-04-26T21:50:51.130 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:51 smithi064 bash[63100]: 5568b9c16c3dc924c4bc38ffa578fcadc4641c0777cf387a00860b44084760e7 2024-04-26T21:50:51.507 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:50:51 smithi064 systemd[1]: Started Ceph mds.b for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T21:50:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:51 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2522033016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2522033016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:52.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:51 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2522033016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:52.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:52.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:52.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:52 smithi165 ceph-mon[33296]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:52.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:52 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:boot 2024-04-26T21:50:52.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:52 smithi165 ceph-mon[33296]: fsmap 12 up:standby 2024-04-26T21:50:52.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:50:52.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:52 smithi055 ceph-mon[29221]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:52.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:52 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:boot 2024-04-26T21:50:52.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:52 smithi055 ceph-mon[29221]: fsmap 12 up:standby 2024-04-26T21:50:52.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:50:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:52 smithi064 ceph-mon[32765]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:52 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:boot 2024-04-26T21:50:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:52 smithi064 ceph-mon[32765]: fsmap 12 up:standby 2024-04-26T21:50:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T21:50:53.031 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:50:53.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 -- 172.21.15.55:0/936661652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c101870 msgr2=0x7fab7c101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 --2- 172.21.15.55:0/936661652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c101870 0x7fab7c101c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab70009900 tx=0x7fab7002a410 comp rx=0 tx=0).stop 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 -- 172.21.15.55:0/936661652 shutdown_connections 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 --2- 172.21.15.55:0/936661652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab7c103420 0x7fab7c109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 --2- 172.21.15.55:0/936661652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab7c102a60 0x7fab7c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 --2- 172.21.15.55:0/936661652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c101870 0x7fab7c101c90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 -- 172.21.15.55:0/936661652 >> 172.21.15.55:0/936661652 conn(0x7fab7c0fcde0 msgr2=0x7fab7c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 -- 172.21.15.55:0/936661652 shutdown_connections 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 -- 172.21.15.55:0/936661652 wait complete. 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 Processor -- start 2024-04-26T21:50:53.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.144+0000 7fab82319640 1 -- start start 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab82319640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab7c101870 0x7fab7c06a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab82319640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab7c102a60 0x7fab7c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab82319640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c103420 0x7fab7c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab82319640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10d370 con 0x7fab7c103420 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab82319640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10d1f0 con 0x7fab7c101870 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab82319640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10d4f0 con 0x7fab7c102a60 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c103420 0x7fab7c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c103420 0x7fab7c19fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45178/0 (socket says 172.21.15.55:45178) 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 -- 172.21.15.55:0/3417906568 learned_addr learned my addr 172.21.15.55:0/3417906568 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab7b7fe640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab7c102a60 0x7fab7c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab7bfff640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab7c101870 0x7fab7c06a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 -- 172.21.15.55:0/3417906568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab7c102a60 msgr2=0x7fab7c19d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab7c102a60 0x7fab7c19d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 -- 172.21.15.55:0/3417906568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab7c101870 msgr2=0x7fab7c06a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab7c101870 0x7fab7c06a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 -- 172.21.15.55:0/3417906568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c1a07f0 con 0x7fab7c103420 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.145+0000 7fab808b6640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c103420 0x7fab7c19fee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fab7400cce0 tx=0x7fab7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.146+0000 7fab797fa640 1 -- 172.21.15.55:0/3417906568 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7400f450 con 0x7fab7c103420 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.146+0000 7fab797fa640 1 -- 172.21.15.55:0/3417906568 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab7400faa0 con 0x7fab7c103420 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.146+0000 7fab797fa640 1 -- 172.21.15.55:0/3417906568 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74018b70 con 0x7fab7c103420 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.146+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c1a0a80 con 0x7fab7c103420 2024-04-26T21:50:53.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.146+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a0fd0 con 0x7fab7c103420 2024-04-26T21:50:53.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.147+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7c108c10 con 0x7fab7c103420 2024-04-26T21:50:53.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.149+0000 7fab797fa640 1 -- 172.21.15.55:0/3417906568 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fab74018d10 con 0x7fab7c103420 2024-04-26T21:50:53.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.149+0000 7fab797fa640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab640a3c30 0x7fab640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.149+0000 7fab7bfff640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab640a3c30 0x7fab640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.149+0000 7fab797fa640 1 -- 172.21.15.55:0/3417906568 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fab740d0330 con 0x7fab7c103420 2024-04-26T21:50:53.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.150+0000 7fab7bfff640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab640a3c30 0x7fab640a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab70009230 tx=0x7fab700314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:53.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.152+0000 7fab797fa640 1 -- 172.21.15.55:0/3417906568 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fab74014030 con 0x7fab7c103420 2024-04-26T21:50:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.384+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab7c02dad0 con 0x7fab7c103420 2024-04-26T21:50:53.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.385+0000 7fab797fa640 1 -- 172.21.15.55:0/3417906568 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7fab7409d2c0 con 0x7fab7c103420 2024-04-26T21:50:53.386 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:53.386 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T21:50:53.386 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 13 2024-04-26T21:50:53.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab640a3c30 msgr2=0x7fab640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab640a3c30 0x7fab640a60f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab70009230 tx=0x7fab700314a0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c103420 msgr2=0x7fab7c19fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c103420 0x7fab7c19fee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fab7400cce0 tx=0x7fab7400a620 comp rx=0 tx=0).stop 2024-04-26T21:50:53.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 shutdown_connections 2024-04-26T21:50:53.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab640a3c30 0x7fab640a60f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab7c103420 0x7fab7c19fee0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab7c102a60 0x7fab7c19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 --2- 172.21.15.55:0/3417906568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab7c101870 0x7fab7c06a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 >> 172.21.15.55:0/3417906568 conn(0x7fab7c0fcde0 msgr2=0x7fab7c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:53.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 shutdown_connections 2024-04-26T21:50:53.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.388+0000 7fab82319640 1 -- 172.21.15.55:0/3417906568 wait complete. 2024-04-26T21:50:53.402 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:50:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 -- 172.21.15.55:0/956972331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 msgr2=0x7f105c102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 --2- 172.21.15.55:0/956972331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 0x7f105c102da0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1050009900 tx=0x7f105002a410 comp rx=0 tx=0).stop 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 -- 172.21.15.55:0/956972331 shutdown_connections 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 --2- 172.21.15.55:0/956972331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f105c104530 0x7f105c10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 --2- 172.21.15.55:0/956972331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f105c103b70 0x7f105c103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 --2- 172.21.15.55:0/956972331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 0x7f105c102da0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 -- 172.21.15.55:0/956972331 >> 172.21.15.55:0/956972331 conn(0x7f105c0fdf10 msgr2=0x7f105c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 -- 172.21.15.55:0/956972331 shutdown_connections 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.505+0000 7f1062d12640 1 -- 172.21.15.55:0/956972331 wait complete. 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1062d12640 1 Processor -- start 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1062d12640 1 -- start start 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1062d12640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 0x7f105c0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1062d12640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f105c103b70 0x7f105c074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1062d12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f105c104530 0x7f105c075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1062d12640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f105c10e4b0 con 0x7f105c102980 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1062d12640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f105c10e330 con 0x7f105c104530 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1062d12640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f105c10e630 con 0x7f105c103b70 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1060aae640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 0x7f105c0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1060aae640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 0x7f105c0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45186/0 (socket says 172.21.15.55:45186) 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f1060aae640 1 -- 172.21.15.55:0/3153477260 learned_addr learned my addr 172.21.15.55:0/3153477260 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f10612af640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f105c104530 0x7f105c075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.506+0000 7f105bfff640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f105c103b70 0x7f105c074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1060aae640 1 -- 172.21.15.55:0/3153477260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f105c103b70 msgr2=0x7f105c074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1060aae640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f105c103b70 0x7f105c074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1060aae640 1 -- 172.21.15.55:0/3153477260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f105c104530 msgr2=0x7f105c075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1060aae640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f105c104530 0x7f105c075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1060aae640 1 -- 172.21.15.55:0/3153477260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105c075af0 con 0x7f105c102980 2024-04-26T21:50:53.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1060aae640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 0x7f105c0766e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1050006010 tx=0x7f10500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:53.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1059ffb640 1 -- 172.21.15.55:0/3153477260 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105003f030 con 0x7f105c102980 2024-04-26T21:50:53.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105c1a2c90 con 0x7f105c102980 2024-04-26T21:50:53.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1059ffb640 1 -- 172.21.15.55:0/3153477260 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1050031910 con 0x7f105c102980 2024-04-26T21:50:53.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1059ffb640 1 -- 172.21.15.55:0/3153477260 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050035070 con 0x7f105c102980 2024-04-26T21:50:53.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.507+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105c1a3190 con 0x7f105c102980 2024-04-26T21:50:53.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.508+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105c109dc0 con 0x7f105c102980 2024-04-26T21:50:53.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.509+0000 7f1059ffb640 1 -- 172.21.15.55:0/3153477260 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1050042030 con 0x7f105c102980 2024-04-26T21:50:53.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.510+0000 7f1059ffb640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f104c0a3bb0 0x7f104c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.510+0000 7f1059ffb640 1 -- 172.21.15.55:0/3153477260 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f10500f04f0 con 0x7f105c102980 2024-04-26T21:50:53.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.510+0000 7f105bfff640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f104c0a3bb0 0x7f104c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.511+0000 7f105bfff640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f104c0a3bb0 0x7f104c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f105c076450 tx=0x7f1048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:53.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.515+0000 7f1059ffb640 1 -- 172.21.15.55:0/3153477260 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1050003e60 con 0x7f105c102980 2024-04-26T21:50:53.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.702+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f105c063530 con 0x7f105c102980 2024-04-26T21:50:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.703+0000 7f1059ffb640 1 -- 172.21.15.55:0/3153477260 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18422 (secure 0 0 0) 0x7f10500bcc80 con 0x7f105c102980 2024-04-26T21:50:53.703 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:53.703 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":81,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:50:06.966268+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f104c0a3bb0 msgr2=0x7f104c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f104c0a3bb0 0x7f104c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f105c076450 tx=0x7f1048005f90 comp rx=0 tx=0).stop 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 msgr2=0x7f105c0766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 0x7f105c0766e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1050006010 tx=0x7f10500042f0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 shutdown_connections 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f104c0a3bb0 0x7f104c0a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f105c104530 0x7f105c075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f105c103b70 0x7f105c074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 --2- 172.21.15.55:0/3153477260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f105c102980 0x7f105c0766e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 >> 172.21.15.55:0/3153477260 conn(0x7f105c0fdf10 msgr2=0x7f105c106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:53.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 shutdown_connections 2024-04-26T21:50:53.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.705+0000 7f1062d12640 1 -- 172.21.15.55:0/3153477260 wait complete. 2024-04-26T21:50:53.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:50:53.718 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 -- 172.21.15.55:0/2103172001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8101870 msgr2=0x7f63a8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 --2- 172.21.15.55:0/2103172001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8101870 0x7f63a8101c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009900 tx=0x7f63a402a410 comp rx=0 tx=0).stop 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 -- 172.21.15.55:0/2103172001 shutdown_connections 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 --2- 172.21.15.55:0/2103172001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103420 0x7f63a8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 --2- 172.21.15.55:0/2103172001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8102a60 0x7f63a8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 --2- 172.21.15.55:0/2103172001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8101870 0x7f63a8101c90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 -- 172.21.15.55:0/2103172001 >> 172.21.15.55:0/2103172001 conn(0x7f63a80fcde0 msgr2=0x7f63a80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 -- 172.21.15.55:0/2103172001 shutdown_connections 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.823+0000 7f63b0020640 1 -- 172.21.15.55:0/2103172001 wait complete. 2024-04-26T21:50:53.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63b0020640 1 Processor -- start 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63b0020640 1 -- start start 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63b0020640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8101870 0x7f63a8105800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63b0020640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8102a60 0x7f63a8103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63b0020640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8103420 0x7f63a8104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63b0020640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810d3c0 con 0x7f63a8103420 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63b0020640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810d240 con 0x7f63a8101870 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63b0020640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810d540 con 0x7f63a8102a60 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63ae5bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8103420 0x7f63a8104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63ae5bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8103420 0x7f63a8104380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45208/0 (socket says 172.21.15.55:45208) 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63addbc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8101870 0x7f63a8105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63ae5bd640 1 -- 172.21.15.55:0/4013009644 learned_addr learned my addr 172.21.15.55:0/4013009644 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63ad5bb640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8102a60 0x7f63a8103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63ae5bd640 1 -- 172.21.15.55:0/4013009644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8102a60 msgr2=0x7f63a8103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63ae5bd640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8102a60 0x7f63a8103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63ae5bd640 1 -- 172.21.15.55:0/4013009644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8101870 msgr2=0x7f63a8105800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.824+0000 7f63ae5bd640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8101870 0x7f63a8105800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.825+0000 7f63ae5bd640 1 -- 172.21.15.55:0/4013009644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a8104c10 con 0x7f63a8103420 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.825+0000 7f63ae5bd640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8103420 0x7f63a8104380 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009fd0 tx=0x7f63a000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.825+0000 7f639effd640 1 -- 172.21.15.55:0/4013009644 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0017070 con 0x7f63a8103420 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.825+0000 7f639effd640 1 -- 172.21.15.55:0/4013009644 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63a0003e20 con 0x7f63a8103420 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.825+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a81a2d90 con 0x7f63a8103420 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.825+0000 7f639effd640 1 -- 172.21.15.55:0/4013009644 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a00106c0 con 0x7f63a8103420 2024-04-26T21:50:53.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.825+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81a3220 con 0x7f63a8103420 2024-04-26T21:50:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.826+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a81a3510 con 0x7f63a8103420 2024-04-26T21:50:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.827+0000 7f639effd640 1 -- 172.21.15.55:0/4013009644 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f63a0004140 con 0x7f63a8103420 2024-04-26T21:50:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.828+0000 7f639effd640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f63940a3c30 0x7f63940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.828+0000 7f63addbc640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f63940a3c30 0x7f63940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.828+0000 7f639effd640 1 -- 172.21.15.55:0/4013009644 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f63a00cfac0 con 0x7f63a8103420 2024-04-26T21:50:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.829+0000 7f63addbc640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f63940a3c30 0x7f63940a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005ab0 tx=0x7f63a4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:53.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:53.832+0000 7f639effd640 1 -- 172.21.15.55:0/4013009644 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f63a00047e0 con 0x7f63a8103420 2024-04-26T21:50:54.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.044+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f63a81a3800 con 0x7f63a8103420 2024-04-26T21:50:54.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.045+0000 7f639effd640 1 -- 172.21.15.55:0/4013009644 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f63a009c2d0 con 0x7f63a8103420 2024-04-26T21:50:54.045 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f63940a3c30 msgr2=0x7f63940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f63940a3c30 0x7f63940a60f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63a4005ab0 tx=0x7f63a4005a20 comp rx=0 tx=0).stop 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8103420 msgr2=0x7f63a8104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8103420 0x7f63a8104380 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009fd0 tx=0x7f63a000b260 comp rx=0 tx=0).stop 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 shutdown_connections 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f63940a3c30 0x7f63940a60f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63a8103420 0x7f63a8104380 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8102a60 0x7f63a8103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 --2- 172.21.15.55:0/4013009644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8101870 0x7f63a8105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 >> 172.21.15.55:0/4013009644 conn(0x7f63a80fcde0 msgr2=0x7f63a8069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:54.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 shutdown_connections 2024-04-26T21:50:54.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.047+0000 7f63b0020640 1 -- 172.21.15.55:0/4013009644 wait complete. 2024-04-26T21:50:54.059 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- 172.21.15.55:0/546465233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d81034b0 msgr2=0x7f66d81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 --2- 172.21.15.55:0/546465233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d81034b0 0x7f66d81058a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009900 tx=0x7f66d002a410 comp rx=0 tx=0).stop 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- 172.21.15.55:0/546465233 shutdown_connections 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 --2- 172.21.15.55:0/546465233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d8108710 0x7f66d810ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 --2- 172.21.15.55:0/546465233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d8105de0 0x7f66d81081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 --2- 172.21.15.55:0/546465233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d81034b0 0x7f66d81058a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- 172.21.15.55:0/546465233 >> 172.21.15.55:0/546465233 conn(0x7f66d80fcde0 msgr2=0x7f66d80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- 172.21.15.55:0/546465233 shutdown_connections 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- 172.21.15.55:0/546465233 wait complete. 2024-04-26T21:50:54.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 Processor -- start 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- start start 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d81034b0 0x7f66d8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d8105de0 0x7f66d8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d8108710 0x7f66d81a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810e4d0 con 0x7f66d8108710 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810e350 con 0x7f66d81034b0 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.161+0000 7f66debbf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66d810e650 con 0x7f66d8105de0 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d8108710 0x7f66d81a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d8108710 0x7f66d81a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45232/0 (socket says 172.21.15.55:45232) 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dc95b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d81034b0 0x7f66d8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 -- 172.21.15.55:0/3492838602 learned_addr learned my addr 172.21.15.55:0/3492838602 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66d7fff640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d8105de0 0x7f66d8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:54.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 -- 172.21.15.55:0/3492838602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d8105de0 msgr2=0x7f66d8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d8105de0 0x7f66d8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 -- 172.21.15.55:0/3492838602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d81034b0 msgr2=0x7f66d8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d81034b0 0x7f66d8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 -- 172.21.15.55:0/3492838602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d81a07f0 con 0x7f66d8108710 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66d7fff640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d8105de0 0x7f66d8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dc95b640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d81034b0 0x7f66d8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66dd15c640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d8108710 0x7f66d81a0180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66c000e970 tx=0x7f66c000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66d5ffb640 1 -- 172.21.15.55:0/3492838602 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c000ccc0 con 0x7f66d8108710 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66d5ffb640 1 -- 172.21.15.55:0/3492838602 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c000ce60 con 0x7f66d8108710 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d81a0ae0 con 0x7f66d8108710 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66d5ffb640 1 -- 172.21.15.55:0/3492838602 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0010430 con 0x7f66d8108710 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.162+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d81a1030 con 0x7f66d8108710 2024-04-26T21:50:54.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.163+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d80fe8d0 con 0x7f66d8108710 2024-04-26T21:50:54.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.164+0000 7f66d5ffb640 1 -- 172.21.15.55:0/3492838602 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f66c0004ff0 con 0x7f66d8108710 2024-04-26T21:50:54.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.165+0000 7f66d5ffb640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66c40a3c30 0x7f66c40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:54.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.165+0000 7f66d5ffb640 1 -- 172.21.15.55:0/3492838602 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f66c00d0380 con 0x7f66d8108710 2024-04-26T21:50:54.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.165+0000 7f66dc95b640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66c40a3c30 0x7f66c40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:54.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.166+0000 7f66dc95b640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66c40a3c30 0x7f66c40a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66d0000c00 tx=0x7f66d00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:54.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.170+0000 7f66d5ffb640 1 -- 172.21.15.55:0/3492838602 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f66c009d200 con 0x7f66d8108710 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3417906568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3153477260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:54 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4013009644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.362+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f66d802dd20 con 0x7f66d8108710 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.362+0000 7f66d5ffb640 1 -- 172.21.15.55:0/3492838602 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f66c009cb00 con 0x7f66d8108710 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:50:54.363 INFO:teuthology.orchestra.run.smithi055.stdout:] 2024-04-26T21:50:54.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66c40a3c30 msgr2=0x7f66c40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66c40a3c30 0x7f66c40a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66d0000c00 tx=0x7f66d00314a0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d8108710 msgr2=0x7f66d81a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d8108710 0x7f66d81a0180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66c000e970 tx=0x7f66c000ed70 comp rx=0 tx=0).stop 2024-04-26T21:50:54.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 shutdown_connections 2024-04-26T21:50:54.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66c40a3c30 0x7f66c40a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66d8108710 0x7f66d81a0180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66d8105de0 0x7f66d8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 --2- 172.21.15.55:0/3492838602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66d81034b0 0x7f66d8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 >> 172.21.15.55:0/3492838602 conn(0x7f66d80fcde0 msgr2=0x7f66d8103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:54.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 shutdown_connections 2024-04-26T21:50:54.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.365+0000 7f66debbf640 1 -- 172.21.15.55:0/3492838602 wait complete. 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3417906568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3153477260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:54 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4013009644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:54.378 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T21:50:54.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T21:50:54.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.382 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:54.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:54.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:54.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3417906568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3153477260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:54.383 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:54 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4013009644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:50:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 -- 172.21.15.55:0/4103344308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250103440 msgr2=0x7f2250109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 --2- 172.21.15.55:0/4103344308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250103440 0x7f2250109cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2248009ca0 tx=0x7f224802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 -- 172.21.15.55:0/4103344308 shutdown_connections 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 --2- 172.21.15.55:0/4103344308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250103440 0x7f2250109cd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 --2- 172.21.15.55:0/4103344308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250102a80 0x7f2250102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 --2- 172.21.15.55:0/4103344308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2250101890 0x7f2250101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 -- 172.21.15.55:0/4103344308 >> 172.21.15.55:0/4103344308 conn(0x7f22500fce20 msgr2=0x7f22500ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 -- 172.21.15.55:0/4103344308 shutdown_connections 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 -- 172.21.15.55:0/4103344308 wait complete. 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.485+0000 7f2255445640 1 Processor -- start 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2255445640 1 -- start start 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2255445640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2250101890 0x7f2250198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2255445640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250102a80 0x7f2250199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2255445640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250103440 0x7f22501a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2255445640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f225010d3a0 con 0x7f2250102a80 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2255445640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f225010d220 con 0x7f2250103440 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2255445640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f225010d520 con 0x7f2250101890 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250102a80 0x7f2250199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224effd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2250101890 0x7f2250198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250102a80 0x7f2250199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45260/0 (socket says 172.21.15.55:45260) 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 -- 172.21.15.55:0/3461393857 learned_addr learned my addr 172.21.15.55:0/3461393857 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224f7fe640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250103440 0x7f22501a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 -- 172.21.15.55:0/3461393857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2250101890 msgr2=0x7f2250198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2250101890 0x7f2250198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 -- 172.21.15.55:0/3461393857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250103440 msgr2=0x7f22501a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250103440 0x7f22501a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 -- 172.21.15.55:0/3461393857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22501a07f0 con 0x7f2250102a80 2024-04-26T21:50:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224effd640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2250101890 0x7f2250198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:50:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f224e7fc640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250102a80 0x7f2250199110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f224400d950 tx=0x7f224400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2237fff640 1 -- 172.21.15.55:0/3461393857 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244017030 con 0x7f2250102a80 2024-04-26T21:50:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22501a0ae0 con 0x7f2250102a80 2024-04-26T21:50:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2237fff640 1 -- 172.21.15.55:0/3461393857 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2244004020 con 0x7f2250102a80 2024-04-26T21:50:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.486+0000 7f2237fff640 1 -- 172.21.15.55:0/3461393857 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440047a0 con 0x7f2250102a80 2024-04-26T21:50:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.487+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22501a1080 con 0x7f2250102a80 2024-04-26T21:50:54.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.488+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22501a12b0 con 0x7f2250102a80 2024-04-26T21:50:54.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.489+0000 7f2237fff640 1 -- 172.21.15.55:0/3461393857 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2244009a30 con 0x7f2250102a80 2024-04-26T21:50:54.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.489+0000 7f2237fff640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f22380a3bb0 0x7f22380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:54.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.489+0000 7f224effd640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f22380a3bb0 0x7f22380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:54.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.489+0000 7f2237fff640 1 -- 172.21.15.55:0/3461393857 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f22440cffd0 con 0x7f2250102a80 2024-04-26T21:50:54.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.490+0000 7f224effd640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f22380a3bb0 0x7f22380a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22501028e0 tx=0x7f2240005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:54.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.493+0000 7f2237fff640 1 -- 172.21.15.55:0/3461393857 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f224409ce50 con 0x7f2250102a80 2024-04-26T21:50:54.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:54.680+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f22501a1480 con 0x7f2250102a80 2024-04-26T21:50:55.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.343+0000 7f2237fff640 1 -- 172.21.15.55:0/3461393857 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f224409c750 con 0x7f2250102a80 2024-04-26T21:50:55.343 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f22380a3bb0 msgr2=0x7f22380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f22380a3bb0 0x7f22380a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22501028e0 tx=0x7f2240005f90 comp rx=0 tx=0).stop 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250102a80 msgr2=0x7f2250199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250102a80 0x7f2250199110 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f224400d950 tx=0x7f224400de10 comp rx=0 tx=0).stop 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 shutdown_connections 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f22380a3bb0 0x7f22380a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2250103440 0x7f22501a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2250102a80 0x7f2250199110 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 --2- 172.21.15.55:0/3461393857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2250101890 0x7f2250198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 >> 172.21.15.55:0/3461393857 conn(0x7f22500fce20 msgr2=0x7f22500fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:55.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 shutdown_connections 2024-04-26T21:50:55.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.346+0000 7f2255445640 1 -- 172.21.15.55:0/3461393857 wait complete. 2024-04-26T21:50:55.359 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:55 smithi055 ceph-mon[29221]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:55.360 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:55 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:55.360 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3492838602' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:50:55.360 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3461393857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:50:55.361 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T21:50:55.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 -- 172.21.15.55:0/2536670575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750101850 msgr2=0x7f0750101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 --2- 172.21.15.55:0/2536670575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750101850 0x7f0750101c70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f074802a410 comp rx=0 tx=0).stop 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 -- 172.21.15.55:0/2536670575 shutdown_connections 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 --2- 172.21.15.55:0/2536670575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0750103400 0x7f0750109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 --2- 172.21.15.55:0/2536670575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0750102a40 0x7f0750102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 --2- 172.21.15.55:0/2536670575 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750101850 0x7f0750101c70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 -- 172.21.15.55:0/2536670575 >> 172.21.15.55:0/2536670575 conn(0x7f07500fcdc0 msgr2=0x7f07500ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 -- 172.21.15.55:0/2536670575 shutdown_connections 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 -- 172.21.15.55:0/2536670575 wait complete. 2024-04-26T21:50:55.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 Processor -- start 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 -- start start 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0750101850 0x7f0750198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.471+0000 7f07579f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0750102a40 0x7f07501990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07579f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750103400 0x7f07501a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07579f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f075010e4a0 con 0x7f0750103400 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07579f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f075010e320 con 0x7f0750102a40 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07579f0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f075010e620 con 0x7f0750101850 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750103400 0x7f07501a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750103400 0x7f07501a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45278/0 (socket says 172.21.15.55:45278) 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 -- 172.21.15.55:0/3991569163 learned_addr learned my addr 172.21.15.55:0/3991569163 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f075578c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0750101850 0x7f0750198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:55.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0754f8b640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0750102a40 0x7f07501990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 -- 172.21.15.55:0/3991569163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0750101850 msgr2=0x7f0750198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0750101850 0x7f0750198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 -- 172.21.15.55:0/3991569163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0750102a40 msgr2=0x7f07501990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0750102a40 0x7f07501990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 -- 172.21.15.55:0/3991569163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07501a07d0 con 0x7f0750103400 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f0755f8d640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750103400 0x7f07501a0160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f074c00eb10 tx=0x7f074c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07467fc640 1 -- 172.21.15.55:0/3991569163 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c004b20 con 0x7f0750103400 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07467fc640 1 -- 172.21.15.55:0/3991569163 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c004cc0 con 0x7f0750103400 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07467fc640 1 -- 172.21.15.55:0/3991569163 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c010670 con 0x7f0750103400 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07501a0ac0 con 0x7f0750103400 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.472+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501a1010 con 0x7f0750103400 2024-04-26T21:50:55.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.473+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0750108c40 con 0x7f0750103400 2024-04-26T21:50:55.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.475+0000 7f07467fc640 1 -- 172.21.15.55:0/3991569163 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f074c005000 con 0x7f0750103400 2024-04-26T21:50:55.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.475+0000 7f07467fc640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f073c0a3bb0 0x7f073c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:55.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.475+0000 7f07467fc640 1 -- 172.21.15.55:0/3991569163 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f074c0d0800 con 0x7f0750103400 2024-04-26T21:50:55.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.475+0000 7f075578c640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f073c0a3bb0 0x7f073c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:55.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.476+0000 7f075578c640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f073c0a3bb0 0x7f073c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0748009230 tx=0x7f07480314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:55.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.479+0000 7f07467fc640 1 -- 172.21.15.55:0/3991569163 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f074c054060 con 0x7f0750103400 2024-04-26T21:50:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:55 smithi064 ceph-mon[32765]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:55 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:55 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3492838602' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:50:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:55 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3461393857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:50:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:55 smithi165 ceph-mon[33296]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:55 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:55 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3492838602' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T21:50:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:55 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3461393857' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T21:50:55.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:55.666+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f075002dad0 con 0x7f0750103400 2024-04-26T21:50:56.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.345+0000 7f07467fc640 1 -- 172.21.15.55:0/3991569163 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f074c09d510 con 0x7f0750103400 2024-04-26T21:50:56.346 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.349+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f073c0a3bb0 msgr2=0x7f073c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.349+0000 7f07579f0640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f073c0a3bb0 0x7f073c0a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0748009230 tx=0x7f07480314a0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750103400 msgr2=0x7f07501a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750103400 0x7f07501a0160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f074c00eb10 tx=0x7f074c00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 shutdown_connections 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f073c0a3bb0 0x7f073c0a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0750103400 0x7f07501a0160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0750102a40 0x7f07501990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 --2- 172.21.15.55:0/3991569163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0750101850 0x7f0750198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 >> 172.21.15.55:0/3991569163 conn(0x7f07500fcdc0 msgr2=0x7f0750105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 shutdown_connections 2024-04-26T21:50:56.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.350+0000 7f07579f0640 1 -- 172.21.15.55:0/3991569163 wait complete. 2024-04-26T21:50:56.372 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3461393857' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:50:56.373 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:56 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3991569163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:50:56.374 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T21:50:56.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- 172.21.15.55:0/3246651356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82241086d0 msgr2=0x7f822410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:56.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 --2- 172.21.15.55:0/3246651356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82241086d0 0x7f822410aac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f821c009ca0 tx=0x7f821c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- 172.21.15.55:0/3246651356 shutdown_connections 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 --2- 172.21.15.55:0/3246651356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82241086d0 0x7f822410aac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 --2- 172.21.15.55:0/3246651356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8224105da0 0x7f8224108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 --2- 172.21.15.55:0/3246651356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8224103470 0x7f8224105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- 172.21.15.55:0/3246651356 >> 172.21.15.55:0/3246651356 conn(0x7f82240fce20 msgr2=0x7f82240ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- 172.21.15.55:0/3246651356 shutdown_connections 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- 172.21.15.55:0/3246651356 wait complete. 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 Processor -- start 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- start start 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8224103470 0x7f8224102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8224105da0 0x7f8224101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82241086d0 0x7f8224101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f822410e530 con 0x7f8224105da0 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f822410e3b0 con 0x7f8224103470 2024-04-26T21:50:56.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.484+0000 7f822b10a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f822410e6b0 con 0x7f82241086d0 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8224105da0 0x7f8224101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8224105da0 0x7f8224101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45298/0 (socket says 172.21.15.55:45298) 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 -- 172.21.15.55:0/3518663896 learned_addr learned my addr 172.21.15.55:0/3518663896 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f82296a7640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82241086d0 0x7f8224101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8228ea6640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8224103470 0x7f8224102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 -- 172.21.15.55:0/3518663896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82241086d0 msgr2=0x7f8224101960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82241086d0 0x7f8224101960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 -- 172.21.15.55:0/3518663896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8224103470 msgr2=0x7f8224102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8224103470 0x7f8224102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 -- 172.21.15.55:0/3518663896 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82241021f0 con 0x7f8224105da0 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f82296a7640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82241086d0 0x7f8224101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8228ea6640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8224103470 0x7f8224102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:56.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8223fff640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8224105da0 0x7f8224101420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f821000d770 tx=0x7f821000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:56.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8221ffb640 1 -- 172.21.15.55:0/3518663896 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210017070 con 0x7f8224105da0 2024-04-26T21:50:56.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8221ffb640 1 -- 172.21.15.55:0/3518663896 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8210009c30 con 0x7f8224105da0 2024-04-26T21:50:56.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82241a2d50 con 0x7f8224105da0 2024-04-26T21:50:56.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f8221ffb640 1 -- 172.21.15.55:0/3518663896 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821000f620 con 0x7f8224105da0 2024-04-26T21:50:56.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.485+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82241a32e0 con 0x7f8224105da0 2024-04-26T21:50:56.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.486+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82240fe910 con 0x7f8224105da0 2024-04-26T21:50:56.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.488+0000 7f8221ffb640 1 -- 172.21.15.55:0/3518663896 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f821001a070 con 0x7f8224105da0 2024-04-26T21:50:56.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.488+0000 7f8221ffb640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82140a3c30 0x7f82140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:56.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.488+0000 7f8228ea6640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82140a3c30 0x7f82140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:56.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.488+0000 7f8221ffb640 1 -- 172.21.15.55:0/3518663896 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f82100cfda0 con 0x7f8224105da0 2024-04-26T21:50:56.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.489+0000 7f8228ea6640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82140a3c30 0x7f82140a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8218006fd0 tx=0x7f8218008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:56.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.492+0000 7f8221ffb640 1 -- 172.21.15.55:0/3518663896 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8210009de0 con 0x7f8224105da0 2024-04-26T21:50:56.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:56.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:56.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3461393857' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:50:56.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:50:56.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:56 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3991569163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3461393857' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T21:50:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T21:50:56.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:56 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3991569163' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T21:50:56.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:56.718+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8224063640 con 0x7f8224105da0 2024-04-26T21:50:57.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.348+0000 7f8221ffb640 1 -- 172.21.15.55:0/3518663896 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8210096e20 con 0x7f8224105da0 2024-04-26T21:50:57.348 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T21:50:57.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82140a3c30 msgr2=0x7f82140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82140a3c30 0x7f82140a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8218006fd0 tx=0x7f8218008040 comp rx=0 tx=0).stop 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8224105da0 msgr2=0x7f8224101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8224105da0 0x7f8224101420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f821000d770 tx=0x7f821000dc30 comp rx=0 tx=0).stop 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 shutdown_connections 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82140a3c30 0x7f82140a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82241086d0 0x7f8224101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8224105da0 0x7f8224101420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 --2- 172.21.15.55:0/3518663896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8224103470 0x7f8224102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 >> 172.21.15.55:0/3518663896 conn(0x7f82240fce20 msgr2=0x7f8224103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 shutdown_connections 2024-04-26T21:50:57.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.350+0000 7f822b10a640 1 -- 172.21.15.55:0/3518663896 wait complete. 2024-04-26T21:50:57.364 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:57 smithi055 ceph-mon[29221]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:57.364 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:57 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3991569163' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:50:57.364 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:57 smithi055 ceph-mon[29221]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:50:57.364 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:57 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3518663896' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:50:57.364 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:57 smithi055 ceph-mon[29221]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:50:57.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:50:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 -- 172.21.15.55:0/1848233298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5568105da0 msgr2=0x7f5568108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1848233298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5568105da0 0x7f5568108190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f555c009900 tx=0x7f555c02a410 comp rx=0 tx=0).stop 2024-04-26T21:50:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 -- 172.21.15.55:0/1848233298 shutdown_connections 2024-04-26T21:50:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1848233298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55681086d0 0x7f556810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1848233298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5568105da0 0x7f5568108190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1848233298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5568103470 0x7f5568105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 -- 172.21.15.55:0/1848233298 >> 172.21.15.55:0/1848233298 conn(0x7f55680fce20 msgr2=0x7f55680ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 -- 172.21.15.55:0/1848233298 shutdown_connections 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.468+0000 7f556dbb4640 1 -- 172.21.15.55:0/1848233298 wait complete. 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f556dbb4640 1 Processor -- start 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f556dbb4640 1 -- start start 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f556dbb4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5568103470 0x7f5568198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f556dbb4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5568105da0 0x7f55681991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f556dbb4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55681086d0 0x7f55681a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f556dbb4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f556810e530 con 0x7f55681086d0 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f556dbb4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f556810e3b0 con 0x7f5568103470 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f556dbb4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f556810e6b0 con 0x7f5568105da0 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5567fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55681086d0 0x7f55681a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5567fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55681086d0 0x7f55681a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45316/0 (socket says 172.21.15.55:45316) 2024-04-26T21:50:57.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5567fff640 1 -- 172.21.15.55:0/1871045690 learned_addr learned my addr 172.21.15.55:0/1871045690 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f55677fe640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5568103470 0x7f5568198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5566ffd640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5568105da0 0x7f55681991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5567fff640 1 -- 172.21.15.55:0/1871045690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5568105da0 msgr2=0x7f55681991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5567fff640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5568105da0 0x7f55681991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5567fff640 1 -- 172.21.15.55:0/1871045690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5568103470 msgr2=0x7f5568198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5567fff640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5568103470 0x7f5568198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.469+0000 7f5567fff640 1 -- 172.21.15.55:0/1871045690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55681a08c0 con 0x7f55681086d0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.470+0000 7f5567fff640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55681086d0 0x7f55681a0220 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f556000b970 tx=0x7f556000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.470+0000 7f5564ff9640 1 -- 172.21.15.55:0/1871045690 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560019070 con 0x7f55681086d0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.470+0000 7f5564ff9640 1 -- 172.21.15.55:0/1871045690 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5560014440 con 0x7f55681086d0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.470+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55681a0bb0 con 0x7f55681086d0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.470+0000 7f5564ff9640 1 -- 172.21.15.55:0/1871045690 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600075d0 con 0x7f55681086d0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.470+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a1150 con 0x7f55681086d0 2024-04-26T21:50:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.471+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55680fe910 con 0x7f55681086d0 2024-04-26T21:50:57.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.472+0000 7f5564ff9640 1 -- 172.21.15.55:0/1871045690 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f556001c070 con 0x7f55681086d0 2024-04-26T21:50:57.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.472+0000 7f5564ff9640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f55500a3c30 0x7f55500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.473+0000 7f55677fe640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f55500a3c30 0x7f55500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.473+0000 7f5564ff9640 1 -- 172.21.15.55:0/1871045690 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f55600cf880 con 0x7f55681086d0 2024-04-26T21:50:57.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.473+0000 7f55677fe640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f55500a3c30 0x7f55500a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5558005fd0 tx=0x7f5558005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:57.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.477+0000 7f5564ff9640 1 -- 172.21.15.55:0/1871045690 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f556009c3b0 con 0x7f55681086d0 2024-04-26T21:50:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:57 smithi064 ceph-mon[32765]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:57 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3991569163' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:50:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:57 smithi064 ceph-mon[32765]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:50:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:57 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3518663896' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:50:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:57 smithi064 ceph-mon[32765]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:50:57.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:57 smithi165 ceph-mon[33296]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:57.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:57 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3991569163' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T21:50:57.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:57 smithi165 ceph-mon[33296]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T21:50:57.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:57 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3518663896' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T21:50:57.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:57 smithi165 ceph-mon[33296]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T21:50:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.664+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f556802dd20 con 0x7f55681086d0 2024-04-26T21:50:57.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.666+0000 7f5564ff9640 1 -- 172.21.15.55:0/1871045690 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22198 (secure 0 0 0) 0x7f556009bcb0 con 0x7f55681086d0 2024-04-26T21:50:57.667 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:57.667 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":84,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:50:56.719987+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f55500a3c30 msgr2=0x7f55500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f55500a3c30 0x7f55500a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5558005fd0 tx=0x7f5558005e20 comp rx=0 tx=0).stop 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55681086d0 msgr2=0x7f55681a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55681086d0 0x7f55681a0220 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f556000b970 tx=0x7f556000be30 comp rx=0 tx=0).stop 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 shutdown_connections 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f55500a3c30 0x7f55500a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55681086d0 0x7f55681a0220 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5568105da0 0x7f55681991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 --2- 172.21.15.55:0/1871045690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5568103470 0x7f5568198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 >> 172.21.15.55:0/1871045690 conn(0x7f55680fce20 msgr2=0x7f5568103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:57.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 shutdown_connections 2024-04-26T21:50:57.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.668+0000 7f556dbb4640 1 -- 172.21.15.55:0/1871045690 wait complete. 2024-04-26T21:50:57.680 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:50:57.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.781+0000 7fc05a22a640 1 -- 172.21.15.55:0/611656795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 msgr2=0x7fc054108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.781+0000 7fc05a22a640 1 --2- 172.21.15.55:0/611656795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 0x7fc054108190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc048009900 tx=0x7fc04802a410 comp rx=0 tx=0).stop 2024-04-26T21:50:57.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 -- 172.21.15.55:0/611656795 shutdown_connections 2024-04-26T21:50:57.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 --2- 172.21.15.55:0/611656795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0541086d0 0x7fc05410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 --2- 172.21.15.55:0/611656795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 0x7fc054108190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 --2- 172.21.15.55:0/611656795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc054103470 0x7fc054105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 -- 172.21.15.55:0/611656795 >> 172.21.15.55:0/611656795 conn(0x7fc0540fce20 msgr2=0x7fc0540ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 -- 172.21.15.55:0/611656795 shutdown_connections 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 -- 172.21.15.55:0/611656795 wait complete. 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 Processor -- start 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 -- start start 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc054103470 0x7fc054198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 0x7fc0541992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0541086d0 0x7fc0541a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc05410e590 con 0x7fc054105da0 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc05410e410 con 0x7fc054103470 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.782+0000 7fc05a22a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc05410e710 con 0x7fc0541086d0 2024-04-26T21:50:57.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 0x7fc0541992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 0x7fc0541992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45332/0 (socket says 172.21.15.55:45332) 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 -- 172.21.15.55:0/2464335735 learned_addr learned my addr 172.21.15.55:0/2464335735 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc0537fe640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc054103470 0x7fc054198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc053fff640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0541086d0 0x7fc0541a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 -- 172.21.15.55:0/2464335735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0541086d0 msgr2=0x7fc0541a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0541086d0 0x7fc0541a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 -- 172.21.15.55:0/2464335735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc054103470 msgr2=0x7fc054198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc054103470 0x7fc054198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 -- 172.21.15.55:0/2464335735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0541a0980 con 0x7fc054105da0 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc0537fe640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc054103470 0x7fc054198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc053fff640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0541086d0 0x7fc0541a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc052ffd640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 0x7fc0541992a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc048006010 tx=0x7fc048004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:57.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc050ff9640 1 -- 172.21.15.55:0/2464335735 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04803f030 con 0x7fc054105da0 2024-04-26T21:50:57.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc050ff9640 1 -- 172.21.15.55:0/2464335735 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc048003cb0 con 0x7fc054105da0 2024-04-26T21:50:57.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0541a0c10 con 0x7fc054105da0 2024-04-26T21:50:57.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc050ff9640 1 -- 172.21.15.55:0/2464335735 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048035070 con 0x7fc054105da0 2024-04-26T21:50:57.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.783+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0541a1100 con 0x7fc054105da0 2024-04-26T21:50:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.784+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0540fe910 con 0x7fc054105da0 2024-04-26T21:50:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.785+0000 7fc050ff9640 1 -- 172.21.15.55:0/2464335735 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc048042030 con 0x7fc054105da0 2024-04-26T21:50:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.786+0000 7fc050ff9640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc03c0a3bb0 0x7fc03c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.786+0000 7fc0537fe640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc03c0a3bb0 0x7fc03c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.786+0000 7fc050ff9640 1 -- 172.21.15.55:0/2464335735 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fc0480f0af0 con 0x7fc054105da0 2024-04-26T21:50:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.786+0000 7fc0537fe640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc03c0a3bb0 0x7fc03c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc044006fd0 tx=0x7fc044008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:57.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.790+0000 7fc050ff9640 1 -- 172.21.15.55:0/2464335735 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc048003e60 con 0x7fc054105da0 2024-04-26T21:50:57.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.978+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0540635d0 con 0x7fc054105da0 2024-04-26T21:50:57.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.980+0000 7fc050ff9640 1 -- 172.21.15.55:0/2464335735 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22198 (secure 0 0 0) 0x7fc0480bd580 con 0x7fc054105da0 2024-04-26T21:50:57.980 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:50:57.981 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":84,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:50:56.719987+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:50:57.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc03c0a3bb0 msgr2=0x7fc03c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc03c0a3bb0 0x7fc03c0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc044006fd0 tx=0x7fc044008040 comp rx=0 tx=0).stop 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 msgr2=0x7fc0541992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 0x7fc0541992a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc048006010 tx=0x7fc048004430 comp rx=0 tx=0).stop 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 shutdown_connections 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc03c0a3bb0 0x7fc03c0a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0541086d0 0x7fc0541a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc054105da0 0x7fc0541992a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 --2- 172.21.15.55:0/2464335735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc054103470 0x7fc054198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 >> 172.21.15.55:0/2464335735 conn(0x7fc0540fce20 msgr2=0x7fc054103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 shutdown_connections 2024-04-26T21:50:57.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:57.982+0000 7fc05a22a640 1 -- 172.21.15.55:0/2464335735 wait complete. 2024-04-26T21:50:57.994 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T21:50:58.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 -- 172.21.15.55:0/17907225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900102a80 msgr2=0x7f4900102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 --2- 172.21.15.55:0/17907225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900102a80 0x7f4900102f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009900 tx=0x7f48f002a410 comp rx=0 tx=0).stop 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 -- 172.21.15.55:0/17907225 shutdown_connections 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 --2- 172.21.15.55:0/17907225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4900103440 0x7f4900109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 --2- 172.21.15.55:0/17907225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900102a80 0x7f4900102f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 --2- 172.21.15.55:0/17907225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4900101890 0x7f4900101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 -- 172.21.15.55:0/17907225 >> 172.21.15.55:0/17907225 conn(0x7f49000fce20 msgr2=0x7f49000ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 -- 172.21.15.55:0/17907225 shutdown_connections 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 -- 172.21.15.55:0/17907225 wait complete. 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.098+0000 7f490799e640 1 Processor -- start 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f490799e640 1 -- start start 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f490799e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4900101890 0x7f4900198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f490799e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4900102a80 0x7f49001991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:58.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f490799e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900103440 0x7f49001a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f490799e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f490010e520 con 0x7f4900103440 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f490799e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f490010e3a0 con 0x7f4900101890 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f490799e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f490010e6a0 con 0x7f4900102a80 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900103440 0x7f49001a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900103440 0x7f49001a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45352/0 (socket says 172.21.15.55:45352) 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 -- 172.21.15.55:0/3635420878 learned_addr learned my addr 172.21.15.55:0/3635420878 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f490573a640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4900101890 0x7f4900198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4904f39640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4900102a80 0x7f49001991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 -- 172.21.15.55:0/3635420878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4900102a80 msgr2=0x7f49001991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4900102a80 0x7f49001991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 -- 172.21.15.55:0/3635420878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4900101890 msgr2=0x7f4900198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4900101890 0x7f4900198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 -- 172.21.15.55:0/3635420878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49001a0880 con 0x7f4900103440 2024-04-26T21:50:58.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4904f39640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4900102a80 0x7f49001991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:58.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f4905f3b640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900103440 0x7f49001a01e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f48f800b970 tx=0x7f48f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:58.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.099+0000 7f48f67fc640 1 -- 172.21.15.55:0/3635420878 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8019070 con 0x7f4900103440 2024-04-26T21:50:58.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.100+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49001a0b70 con 0x7f4900103440 2024-04-26T21:50:58.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.100+0000 7f48f67fc640 1 -- 172.21.15.55:0/3635420878 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48f8014440 con 0x7f4900103440 2024-04-26T21:50:58.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.100+0000 7f48f67fc640 1 -- 172.21.15.55:0/3635420878 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80075d0 con 0x7f4900103440 2024-04-26T21:50:58.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.100+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49001a1070 con 0x7f4900103440 2024-04-26T21:50:58.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.100+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4900108c20 con 0x7f4900103440 2024-04-26T21:50:58.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.102+0000 7f48f67fc640 1 -- 172.21.15.55:0/3635420878 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f48f801c070 con 0x7f4900103440 2024-04-26T21:50:58.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.102+0000 7f48f67fc640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:58.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.102+0000 7f490573a640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:58.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.102+0000 7f48f67fc640 1 -- 172.21.15.55:0/3635420878 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f48f80cfeb0 con 0x7f4900103440 2024-04-26T21:50:58.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.103+0000 7f490573a640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005fd0 tx=0x7f48fc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:58.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.106+0000 7f48f67fc640 1 -- 172.21.15.55:0/3635420878 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f48f80ff060 con 0x7f4900103440 2024-04-26T21:50:58.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.333+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f490002dad0 con 0x7f4900103440 2024-04-26T21:50:58.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.352+0000 7f48f67fc640 1 -- 172.21.15.55:0/3635420878 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f48f809c9e0 con 0x7f4900103440 2024-04-26T21:50:58.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48ec0a3bb0 msgr2=0x7f48ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:58.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f48fc005fd0 tx=0x7f48fc005850 comp rx=0 tx=0).stop 2024-04-26T21:50:58.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900103440 msgr2=0x7f49001a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900103440 0x7f49001a01e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f48f800b970 tx=0x7f48f800be30 comp rx=0 tx=0).stop 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 shutdown_connections 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f48ec0a3bb0 0x7f48ec0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4900103440 0x7f49001a01e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4900102a80 0x7f49001991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 --2- 172.21.15.55:0/3635420878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4900101890 0x7f4900198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 >> 172.21.15.55:0/3635420878 conn(0x7f49000fce20 msgr2=0x7f4900105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 shutdown_connections 2024-04-26T21:50:58.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.354+0000 7f490799e640 1 -- 172.21.15.55:0/3635420878 wait complete. 2024-04-26T21:50:58.365 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:58 smithi055 ceph-mon[29221]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:50:58.365 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:58 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3518663896' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:50:58.365 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:58 smithi055 ceph-mon[29221]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:50:58.365 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:58 smithi055 ceph-mon[29221]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:50:58.365 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:58 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1871045690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:58.366 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:58 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2464335735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:58.366 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-26T21:50:58.367 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.470+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/721001773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0102a80 msgr2=0x7f0ca0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.470+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/721001773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0102a80 0x7f0ca0102f00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009900 tx=0x7f0c9402a410 comp rx=0 tx=0).stop 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.470+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/721001773 shutdown_connections 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.470+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/721001773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ca0103440 0x7f0ca0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.470+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/721001773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0102a80 0x7f0ca0102f00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.470+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/721001773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca0101890 0x7f0ca0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.470+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/721001773 >> 172.21.15.55:0/721001773 conn(0x7f0ca00fce20 msgr2=0x7f0ca00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/721001773 shutdown_connections 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/721001773 wait complete. 2024-04-26T21:50:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 Processor -- start 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 -- start start 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0101890 0x7f0ca0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca0102a80 0x7f0ca0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ca0103440 0x7f0ca019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca010d3a0 con 0x7f0ca0101890 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca010d220 con 0x7f0ca0102a80 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.471+0000 7f0ca5a2a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca010d520 con 0x7f0ca0103440 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0101890 0x7f0ca0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0101890 0x7f0ca0197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45360/0 (socket says 172.21.15.55:45360) 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 -- 172.21.15.55:0/3379886980 learned_addr learned my addr 172.21.15.55:0/3379886980 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9f7fe640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ca0103440 0x7f0ca019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9e7fc640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca0102a80 0x7f0ca0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 -- 172.21.15.55:0/3379886980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ca0103440 msgr2=0x7f0ca019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ca0103440 0x7f0ca019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 -- 172.21.15.55:0/3379886980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca0102a80 msgr2=0x7f0ca0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca0102a80 0x7f0ca0197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 -- 172.21.15.55:0/3379886980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca019f6a0 con 0x7f0ca0101890 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9f7fe640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ca0103440 0x7f0ca019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0c9effd640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0101890 0x7f0ca0197a80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000d950 tx=0x7f0c9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0ca4a28640 1 -- 172.21.15.55:0/3379886980 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90017020 con 0x7f0ca0101890 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0ca4a28640 1 -- 172.21.15.55:0/3379886980 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c90009c30 con 0x7f0ca0101890 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca019f990 con 0x7f0ca0101890 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0ca4a28640 1 -- 172.21.15.55:0/3379886980 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000f6b0 con 0x7f0ca0101890 2024-04-26T21:50:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.472+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca019fee0 con 0x7f0ca0101890 2024-04-26T21:50:58.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.473+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca0108c20 con 0x7f0ca0101890 2024-04-26T21:50:58.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.474+0000 7f0ca4a28640 1 -- 172.21.15.55:0/3379886980 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c9001a020 con 0x7f0ca0101890 2024-04-26T21:50:58.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.475+0000 7f0ca4a28640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0c880a3bb0 0x7f0c880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:58.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.475+0000 7f0c9e7fc640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0c880a3bb0 0x7f0c880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:58.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.475+0000 7f0ca4a28640 1 -- 172.21.15.55:0/3379886980 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0c900d0030 con 0x7f0ca0101890 2024-04-26T21:50:58.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.476+0000 7f0c9e7fc640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0c880a3bb0 0x7f0c880a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0198f90 tx=0x7f0c940314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:58.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.480+0000 7f0ca4a28640 1 -- 172.21.15.55:0/3379886980 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0c90009de0 con 0x7f0ca0101890 2024-04-26T21:50:58.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:58 smithi064 ceph-mon[32765]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:50:58.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:58 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3518663896' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:50:58.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:58 smithi064 ceph-mon[32765]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:50:58.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:58 smithi064 ceph-mon[32765]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:50:58.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:58 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1871045690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:58.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:58 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2464335735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:58.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:58 smithi165 ceph-mon[33296]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T21:50:58.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:58 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3518663896' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T21:50:58.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:58 smithi165 ceph-mon[33296]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T21:50:58.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:58 smithi165 ceph-mon[33296]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T21:50:58.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:58 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1871045690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:58.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:58 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2464335735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:50:58.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:58.709+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f0ca002dd40 con 0x7f0ca0101890 2024-04-26T21:50:59.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.272+0000 7f0ca4a28640 1 -- 172.21.15.55:0/3379886980 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f0c9009cb60 con 0x7f0ca0101890 2024-04-26T21:50:59.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0c880a3bb0 msgr2=0x7f0c880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:59.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0c880a3bb0 0x7f0c880a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0198f90 tx=0x7f0c940314a0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0101890 msgr2=0x7f0ca0197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0101890 0x7f0ca0197a80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000d950 tx=0x7f0c9000de10 comp rx=0 tx=0).stop 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 shutdown_connections 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0c880a3bb0 0x7f0c880a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ca0103440 0x7f0ca019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ca0102a80 0x7f0ca0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 --2- 172.21.15.55:0/3379886980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca0101890 0x7f0ca0197a80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 >> 172.21.15.55:0/3379886980 conn(0x7f0ca00fce20 msgr2=0x7f0ca0105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 shutdown_connections 2024-04-26T21:50:59.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.275+0000 7f0ca5a2a640 1 -- 172.21.15.55:0/3379886980 wait complete. 2024-04-26T21:50:59.297 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-26T21:50:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 -- 172.21.15.55:0/2109032078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 msgr2=0x7f765c10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 --2- 172.21.15.55:0/2109032078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 0x7f765c10aac0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7654009ca0 tx=0x7f765402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 -- 172.21.15.55:0/2109032078 shutdown_connections 2024-04-26T21:50:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 --2- 172.21.15.55:0/2109032078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 0x7f765c10aac0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 --2- 172.21.15.55:0/2109032078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765c105d40 0x7f765c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 --2- 172.21.15.55:0/2109032078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f765c103410 0x7f765c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 -- 172.21.15.55:0/2109032078 >> 172.21.15.55:0/2109032078 conn(0x7f765c0fce20 msgr2=0x7f765c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:50:59.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 -- 172.21.15.55:0/2109032078 shutdown_connections 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.403+0000 7f7660bda640 1 -- 172.21.15.55:0/2109032078 wait complete. 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7660bda640 1 Processor -- start 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7660bda640 1 -- start start 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7660bda640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f765c103410 0x7f765c102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7660bda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765c105d40 0x7f765c1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7660bda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 0x7f765c101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7660bda640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f765c10e530 con 0x7f765c108670 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7660bda640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f765c10e3b0 con 0x7f765c105d40 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7660bda640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f765c10e6b0 con 0x7f765c103410 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f765ad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 0x7f765c101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f765ad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 0x7f765c101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45382/0 (socket says 172.21.15.55:45382) 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f765ad9d640 1 -- 172.21.15.55:0/1036742015 learned_addr learned my addr 172.21.15.55:0/1036742015 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:50:59.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f765a59c640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f765c103410 0x7f765c102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.404+0000 7f7653fff640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765c105d40 0x7f765c1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f765ad9d640 1 -- 172.21.15.55:0/1036742015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f765c103410 msgr2=0x7f765c102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f765ad9d640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f765c103410 0x7f765c102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f765ad9d640 1 -- 172.21.15.55:0/1036742015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765c105d40 msgr2=0x7f765c1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f765ad9d640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765c105d40 0x7f765c1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f765ad9d640 1 -- 172.21.15.55:0/1036742015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765c102190 con 0x7f765c108670 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f7653fff640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765c105d40 0x7f765c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f765ad9d640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 0x7f765c101900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7654033040 tx=0x7f7654003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f76537fe640 1 -- 172.21.15.55:0/1036742015 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76540317c0 con 0x7f765c108670 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f76537fe640 1 -- 172.21.15.55:0/1036742015 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76540044f0 con 0x7f765c108670 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765c1a2cd0 con 0x7f765c108670 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f76537fe640 1 -- 172.21.15.55:0/1036742015 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765402fd00 con 0x7f765c108670 2024-04-26T21:50:59.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.405+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765c1a31d0 con 0x7f765c108670 2024-04-26T21:50:59.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.406+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f765c0fe910 con 0x7f765c108670 2024-04-26T21:50:59.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.407+0000 7f76537fe640 1 -- 172.21.15.55:0/1036742015 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7654004040 con 0x7f765c108670 2024-04-26T21:50:59.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.408+0000 7f76537fe640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76480a3bb0 0x7f76480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:50:59.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.408+0000 7f765a59c640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76480a3bb0 0x7f76480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:50:59.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.408+0000 7f76537fe640 1 -- 172.21.15.55:0/1036742015 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f76540f1aa0 con 0x7f765c108670 2024-04-26T21:50:59.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.408+0000 7f765a59c640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76480a3bb0 0x7f76480a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f764c005fd0 tx=0x7f764c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:50:59.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.412+0000 7f76537fe640 1 -- 172.21.15.55:0/1036742015 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f76540be530 con 0x7f765c108670 2024-04-26T21:50:59.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:59 smithi055 ceph-mon[29221]: pgmap v288: 97 pgs: 17 creating+peering, 73 unknown, 7 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:59.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:59 smithi055 ceph-mon[29221]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:50:59.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3635420878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:50:59.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3635420878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:50:59.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:59 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T21:50:59.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:59 smithi055 ceph-mon[29221]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:50:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:50:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3379886980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T21:50:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:59 smithi064 ceph-mon[32765]: pgmap v288: 97 pgs: 17 creating+peering, 73 unknown, 7 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:59 smithi064 ceph-mon[32765]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:50:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3635420878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:50:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3635420878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:50:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:59 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T21:50:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:59 smithi064 ceph-mon[32765]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:50:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:50:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3379886980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T21:50:59.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:59 smithi165 ceph-mon[33296]: pgmap v288: 97 pgs: 17 creating+peering, 73 unknown, 7 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T21:50:59.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:59 smithi165 ceph-mon[33296]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T21:50:59.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3635420878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T21:50:59.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3635420878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T21:50:59.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:59 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T21:50:59.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:59 smithi165 ceph-mon[33296]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T21:50:59.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:50:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3379886980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T21:50:59.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:50:59.638+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f765c02dad0 con 0x7f765c108670 2024-04-26T21:51:00.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.285+0000 7f76537fe640 1 -- 172.21.15.55:0/1036742015 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f76540bde30 con 0x7f765c108670 2024-04-26T21:51:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76480a3bb0 msgr2=0x7f76480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76480a3bb0 0x7f76480a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f764c005fd0 tx=0x7f764c00b390 comp rx=0 tx=0).stop 2024-04-26T21:51:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 msgr2=0x7f765c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 0x7f765c101900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7654033040 tx=0x7f7654003ba0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 shutdown_connections 2024-04-26T21:51:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f76480a3bb0 0x7f76480a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f765c108670 0x7f765c101900 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f765c105d40 0x7f765c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 --2- 172.21.15.55:0/1036742015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f765c103410 0x7f765c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 >> 172.21.15.55:0/1036742015 conn(0x7f765c0fce20 msgr2=0x7f765c103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 shutdown_connections 2024-04-26T21:51:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.288+0000 7f7660bda640 1 -- 172.21.15.55:0/1036742015 wait complete. 2024-04-26T21:51:00.305 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-26T21:51:00.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 -- 172.21.15.55:0/3967887498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0768f0 msgr2=0x7f998c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:00.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 --2- 172.21.15.55:0/3967887498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0768f0 0x7f998c074d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9974009900 tx=0x7f997402a480 comp rx=0 tx=0).stop 2024-04-26T21:51:00.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 -- 172.21.15.55:0/3967887498 shutdown_connections 2024-04-26T21:51:00.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 --2- 172.21.15.55:0/3967887498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998c10a840 0x7f998c10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 --2- 172.21.15.55:0/3967887498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f998c075310 0x7f998c075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 --2- 172.21.15.55:0/3967887498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c0768f0 0x7f998c074d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 -- 172.21.15.55:0/3967887498 >> 172.21.15.55:0/3967887498 conn(0x7f998c0fdf00 msgr2=0x7f998c100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:00.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 -- 172.21.15.55:0/3967887498 shutdown_connections 2024-04-26T21:51:00.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.415+0000 7f99929bd640 1 -- 172.21.15.55:0/3967887498 wait complete. 2024-04-26T21:51:00.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f99929bd640 1 Processor -- start 2024-04-26T21:51:00.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f99929bd640 1 -- start start 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f99929bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c075310 0x7f998c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f99929bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998c0768f0 0x7f998c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f99929bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f998c10a840 0x7f998c1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f99929bd640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1106c0 con 0x7f998c075310 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f99929bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f998c110540 con 0x7f998c10a840 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f99929bd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f998c110840 con 0x7f998c0768f0 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f998bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c075310 0x7f998c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f998bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c075310 0x7f998c19adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45406/0 (socket says 172.21.15.55:45406) 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f998bfff640 1 -- 172.21.15.55:0/580355547 learned_addr learned my addr 172.21.15.55:0/580355547 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f9990f5a640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f998c10a840 0x7f998c1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.416+0000 7f998b7fe640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998c0768f0 0x7f998c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f998bfff640 1 -- 172.21.15.55:0/580355547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998c0768f0 msgr2=0x7f998c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:00.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f998bfff640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998c0768f0 0x7f998c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f998bfff640 1 -- 172.21.15.55:0/580355547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f998c10a840 msgr2=0x7f998c1a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f998bfff640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f998c10a840 0x7f998c1a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f998bfff640 1 -- 172.21.15.55:0/580355547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c1a29d0 con 0x7f998c075310 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f998bfff640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c075310 0x7f998c19adc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9974005940 tx=0x7f9974031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f99897fa640 1 -- 172.21.15.55:0/580355547 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997403f030 con 0x7f998c075310 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f99897fa640 1 -- 172.21.15.55:0/580355547 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9974003aa0 con 0x7f998c075310 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c1a2c60 con 0x7f998c075310 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f99897fa640 1 -- 172.21.15.55:0/580355547 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974035070 con 0x7f998c075310 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.417+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c1a3150 con 0x7f998c075310 2024-04-26T21:51:00.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.418+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998c109600 con 0x7f998c075310 2024-04-26T21:51:00.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.419+0000 7f99897fa640 1 -- 172.21.15.55:0/580355547 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9974042030 con 0x7f998c075310 2024-04-26T21:51:00.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.420+0000 7f99897fa640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99780a3bb0 0x7f99780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:00.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.420+0000 7f998b7fe640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99780a3bb0 0x7f99780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:00.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.420+0000 7f99897fa640 1 -- 172.21.15.55:0/580355547 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f99740f08d0 con 0x7f998c075310 2024-04-26T21:51:00.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.421+0000 7f998b7fe640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99780a3bb0 0x7f99780a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f997c009bd0 tx=0x7f997c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:00.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.425+0000 7f99897fa640 1 -- 172.21.15.55:0/580355547 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f99740bcdb0 con 0x7f998c075310 2024-04-26T21:51:00.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:00 smithi055 ceph-mon[29221]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:51:00.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:00 smithi055 ceph-mon[29221]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:51:00.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:00 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3379886980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T21:51:00.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:00 smithi055 ceph-mon[29221]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:51:00.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:00 smithi055 ceph-mon[29221]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:51:00.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:00 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1036742015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:51:00.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:00 smithi165 ceph-mon[33296]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:51:00.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:00 smithi165 ceph-mon[33296]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:51:00.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:00 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3379886980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T21:51:00.478 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:00 smithi165 ceph-mon[33296]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:51:00.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:00 smithi165 ceph-mon[33296]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:51:00.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:00 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1036742015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:51:00.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:00 smithi064 ceph-mon[32765]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T21:51:00.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:00 smithi064 ceph-mon[32765]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T21:51:00.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:00 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3379886980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T21:51:00.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:00 smithi064 ceph-mon[32765]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-26T21:51:00.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:00 smithi064 ceph-mon[32765]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T21:51:00.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:00 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1036742015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T21:51:00.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:00.651+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f998c02dd20 con 0x7f998c075310 2024-04-26T21:51:01.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.294+0000 7f99897fa640 1 -- 172.21.15.55:0/580355547 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7f99740c10b0 con 0x7f998c075310 2024-04-26T21:51:01.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99780a3bb0 msgr2=0x7f99780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99780a3bb0 0x7f99780a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f997c009bd0 tx=0x7f997c005f90 comp rx=0 tx=0).stop 2024-04-26T21:51:01.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c075310 msgr2=0x7f998c19adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c075310 0x7f998c19adc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9974005940 tx=0x7f9974031a50 comp rx=0 tx=0).stop 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 shutdown_connections 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99780a3bb0 0x7f99780a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f998c10a840 0x7f998c1a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f998c0768f0 0x7f998c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 --2- 172.21.15.55:0/580355547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f998c075310 0x7f998c19adc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 >> 172.21.15.55:0/580355547 conn(0x7f998c0fdf00 msgr2=0x7f998c1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 shutdown_connections 2024-04-26T21:51:01.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.297+0000 7f99929bd640 1 -- 172.21.15.55:0/580355547 wait complete. 2024-04-26T21:51:01.311 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-26T21:51:01.312 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 -- 172.21.15.55:0/1109682887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 msgr2=0x7fca5c10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 --2- 172.21.15.55:0/1109682887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 0x7fca5c10aac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fca54009ca0 tx=0x7fca5402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 -- 172.21.15.55:0/1109682887 shutdown_connections 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 --2- 172.21.15.55:0/1109682887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 0x7fca5c10aac0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 --2- 172.21.15.55:0/1109682887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca5c105d40 0x7fca5c108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 --2- 172.21.15.55:0/1109682887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca5c103410 0x7fca5c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 -- 172.21.15.55:0/1109682887 >> 172.21.15.55:0/1109682887 conn(0x7fca5c0fce00 msgr2=0x7fca5c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 -- 172.21.15.55:0/1109682887 shutdown_connections 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.422+0000 7fca640da640 1 -- 172.21.15.55:0/1109682887 wait complete. 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca640da640 1 Processor -- start 2024-04-26T21:51:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca640da640 1 -- start start 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca640da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca5c103410 0x7fca5c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca640da640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca5c105d40 0x7fca5c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca640da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 0x7fca5c1a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca640da640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c10e4e0 con 0x7fca5c108670 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca640da640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c10e360 con 0x7fca5c103410 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca640da640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c10e660 con 0x7fca5c105d40 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca62677640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 0x7fca5c1a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca62677640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 0x7fca5c1a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45326/0 (socket says 172.21.15.55:45326) 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca62677640 1 -- 172.21.15.55:0/2597135947 learned_addr learned my addr 172.21.15.55:0/2597135947 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca61e76640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca5c103410 0x7fca5c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca62677640 1 -- 172.21.15.55:0/2597135947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca5c105d40 msgr2=0x7fca5c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca61675640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca5c105d40 0x7fca5c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.423+0000 7fca62677640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca5c105d40 0x7fca5c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca62677640 1 -- 172.21.15.55:0/2597135947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca5c103410 msgr2=0x7fca5c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca62677640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca5c103410 0x7fca5c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca62677640 1 -- 172.21.15.55:0/2597135947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c1a07d0 con 0x7fca5c108670 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca61675640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca5c105d40 0x7fca5c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca61e76640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca5c103410 0x7fca5c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca62677640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 0x7fca5c1a0160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fca540097b0 tx=0x7fca54031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca52ffd640 1 -- 172.21.15.55:0/2597135947 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54003bf0 con 0x7fca5c108670 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca52ffd640 1 -- 172.21.15.55:0/2597135947 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca54003d90 con 0x7fca5c108670 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c1a0a60 con 0x7fca5c108670 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca52ffd640 1 -- 172.21.15.55:0/2597135947 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5402fc70 con 0x7fca5c108670 2024-04-26T21:51:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.424+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c1a1000 con 0x7fca5c108670 2024-04-26T21:51:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.425+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5c0fe8f0 con 0x7fca5c108670 2024-04-26T21:51:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.426+0000 7fca52ffd640 1 -- 172.21.15.55:0/2597135947 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fca54004040 con 0x7fca5c108670 2024-04-26T21:51:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.427+0000 7fca52ffd640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca480a3c30 0x7fca480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.427+0000 7fca61e76640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca480a3c30 0x7fca480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.427+0000 7fca52ffd640 1 -- 172.21.15.55:0/2597135947 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fca540f19c0 con 0x7fca5c108670 2024-04-26T21:51:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.427+0000 7fca61e76640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca480a3c30 0x7fca480a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fca58009fc0 tx=0x7fca58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:01.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.431+0000 7fca52ffd640 1 -- 172.21.15.55:0/2597135947 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fca540f3e20 con 0x7fca5c108670 2024-04-26T21:51:01.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: pgmap v290: 97 pgs: 17 creating+peering, 27 unknown, 53 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-04-26T21:51:01.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1036742015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:51:01.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] up:active 2024-04-26T21:51:01.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:51:01.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:51:01.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: Cluster is now healthy 2024-04-26T21:51:01.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:51:01.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-26T21:51:01.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T21:51:01.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/580355547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:51:01.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: pgmap v290: 97 pgs: 17 creating+peering, 27 unknown, 53 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-04-26T21:51:01.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1036742015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:51:01.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] up:active 2024-04-26T21:51:01.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:51:01.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:51:01.495 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: Cluster is now healthy 2024-04-26T21:51:01.496 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:51:01.496 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-26T21:51:01.496 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T21:51:01.496 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:01 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/580355547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: pgmap v290: 97 pgs: 17 creating+peering, 27 unknown, 53 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 4 op/s 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1036742015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] up:active 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: Cluster is now healthy 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-26T21:51:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/580355547' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T21:51:01.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.646+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fca5c062d70 con 0x7fca480a3c30 2024-04-26T21:51:01.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.669+0000 7fca52ffd640 1 -- 172.21.15.55:0/2597135947 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca5c062d70 con 0x7fca480a3c30 2024-04-26T21:51:01.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.671+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca480a3c30 msgr2=0x7fca480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.671+0000 7fca640da640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca480a3c30 0x7fca480a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fca58009fc0 tx=0x7fca58005f90 comp rx=0 tx=0).stop 2024-04-26T21:51:01.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.671+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 msgr2=0x7fca5c1a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.671+0000 7fca640da640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 0x7fca5c1a0160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fca540097b0 tx=0x7fca54031840 comp rx=0 tx=0).stop 2024-04-26T21:51:01.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.672+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 shutdown_connections 2024-04-26T21:51:01.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.672+0000 7fca640da640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca480a3c30 0x7fca480a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.672+0000 7fca640da640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca5c108670 0x7fca5c1a0160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.672+0000 7fca640da640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca5c105d40 0x7fca5c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.672+0000 7fca640da640 1 --2- 172.21.15.55:0/2597135947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca5c103410 0x7fca5c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.672+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 >> 172.21.15.55:0/2597135947 conn(0x7fca5c0fce00 msgr2=0x7fca5c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:01.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.672+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 shutdown_connections 2024-04-26T21:51:01.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.672+0000 7fca640da640 1 -- 172.21.15.55:0/2597135947 wait complete. 2024-04-26T21:51:01.683 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T21:51:01.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 -- 172.21.15.55:0/2943217998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4108670 msgr2=0x7f36b410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 --2- 172.21.15.55:0/2943217998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4108670 0x7f36b410aac0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009900 tx=0x7f36ac02a410 comp rx=0 tx=0).stop 2024-04-26T21:51:01.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 -- 172.21.15.55:0/2943217998 shutdown_connections 2024-04-26T21:51:01.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 --2- 172.21.15.55:0/2943217998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4108670 0x7f36b410aac0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 --2- 172.21.15.55:0/2943217998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36b4105d40 0x7f36b4108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 --2- 172.21.15.55:0/2943217998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36b4103410 0x7f36b4105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 -- 172.21.15.55:0/2943217998 >> 172.21.15.55:0/2943217998 conn(0x7f36b40fce20 msgr2=0x7f36b40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 -- 172.21.15.55:0/2943217998 shutdown_connections 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 -- 172.21.15.55:0/2943217998 wait complete. 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.789+0000 7f36b98c8640 1 Processor -- start 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b98c8640 1 -- start start 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b98c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4103410 0x7f36b4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b98c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36b4105d40 0x7f36b41990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b98c8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36b4108670 0x7f36b41a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b98c8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f36b410e4c0 con 0x7f36b4103410 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b98c8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f36b410e340 con 0x7f36b4105d40 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b98c8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f36b410e640 con 0x7f36b4108670 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4103410 0x7f36b4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4103410 0x7f36b4198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45362/0 (socket says 172.21.15.55:45362) 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 -- 172.21.15.55:0/290723605 learned_addr learned my addr 172.21.15.55:0/290723605 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b37fe640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36b4108670 0x7f36b41a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b27fc640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36b4105d40 0x7f36b41990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 -- 172.21.15.55:0/290723605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36b4108670 msgr2=0x7f36b41a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36b4108670 0x7f36b41a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 -- 172.21.15.55:0/290723605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36b4105d40 msgr2=0x7f36b41990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36b4105d40 0x7f36b41990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 -- 172.21.15.55:0/290723605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b41a07b0 con 0x7f36b4103410 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.790+0000 7f36b2ffd640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4103410 0x7f36b4198b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f36a400db80 tx=0x7f36a400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.791+0000 7f36b88c6640 1 -- 172.21.15.55:0/290723605 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4017070 con 0x7f36b4103410 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.791+0000 7f36b88c6640 1 -- 172.21.15.55:0/290723605 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a4003b70 con 0x7f36b4103410 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.791+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b41a0aa0 con 0x7f36b4103410 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.791+0000 7f36b88c6640 1 -- 172.21.15.55:0/290723605 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40106c0 con 0x7f36b4103410 2024-04-26T21:51:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.791+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b41a0ff0 con 0x7f36b4103410 2024-04-26T21:51:01.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.792+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b4069ba0 con 0x7f36b4103410 2024-04-26T21:51:01.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.793+0000 7f36b88c6640 1 -- 172.21.15.55:0/290723605 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f36a4004020 con 0x7f36b4103410 2024-04-26T21:51:01.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.793+0000 7f36b88c6640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36a00a3bb0 0x7f36a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.794+0000 7f36b27fc640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36a00a3bb0 0x7f36a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.794+0000 7f36b88c6640 1 -- 172.21.15.55:0/290723605 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f36a40d02a0 con 0x7f36b4103410 2024-04-26T21:51:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.794+0000 7f36b27fc640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36a00a3bb0 0x7f36a00a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f369c0095f0 tx=0x7f369c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:01.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:01.798+0000 7f36b88c6640 1 -- 172.21.15.55:0/290723605 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f36a409cdd0 con 0x7f36b4103410 2024-04-26T21:51:01.936 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:51:01.651+0000 7f80d0570700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:51:01.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:51:01.651+0000 7f80d0570700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:51:01.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:51:01.651+0000 7f80d0570700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:51:01.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:51:01.651+0000 7f80d0570700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:51:01.937 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 21:51:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T21:51:01.651+0000 7f80d0570700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:51:02.007 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: 2024-04-26T21:51:01.620+0000 7f515bb7f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T21:51:02.007 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: 2024-04-26T21:51:01.620+0000 7f515bb7f700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T21:51:02.007 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 21:51:01 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: 2024-04-26T21:51:01.620+0000 7f515bb7f700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T21:51:02.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.013+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f36b4062e00 con 0x7f36a00a3bb0 2024-04-26T21:51:02.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.037+0000 7f36b88c6640 1 -- 172.21.15.55:0/290723605 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f36b4062e00 con 0x7f36a00a3bb0 2024-04-26T21:51:02.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36a00a3bb0 msgr2=0x7f36a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36a00a3bb0 0x7f36a00a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f369c0095f0 tx=0x7f369c005f90 comp rx=0 tx=0).stop 2024-04-26T21:51:02.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4103410 msgr2=0x7f36b4198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4103410 0x7f36b4198b90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f36a400db80 tx=0x7f36a400b630 comp rx=0 tx=0).stop 2024-04-26T21:51:02.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 shutdown_connections 2024-04-26T21:51:02.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36a00a3bb0 0x7f36a00a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36b4108670 0x7f36b41a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36b4105d40 0x7f36b41990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 --2- 172.21.15.55:0/290723605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36b4103410 0x7f36b4198b90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 >> 172.21.15.55:0/290723605 conn(0x7f36b40fce20 msgr2=0x7f36b4103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:02.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.039+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 shutdown_connections 2024-04-26T21:51:02.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.040+0000 7f36b98c8640 1 -- 172.21.15.55:0/290723605 wait complete. 2024-04-26T21:51:02.051 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 -- 172.21.15.55:0/3958965691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 msgr2=0x7f9e4c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 --2- 172.21.15.55:0/3958965691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 0x7f9e4c102ef0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009900 tx=0x7f9e3c02a410 comp rx=0 tx=0).stop 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 -- 172.21.15.55:0/3958965691 shutdown_connections 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 --2- 172.21.15.55:0/3958965691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c103430 0x7f9e4c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 --2- 172.21.15.55:0/3958965691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 0x7f9e4c102ef0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 --2- 172.21.15.55:0/3958965691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4c101880 0x7f9e4c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 -- 172.21.15.55:0/3958965691 >> 172.21.15.55:0/3958965691 conn(0x7f9e4c0fcdf0 msgr2=0x7f9e4c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 -- 172.21.15.55:0/3958965691 shutdown_connections 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.157+0000 7f9e5387b640 1 -- 172.21.15.55:0/3958965691 wait complete. 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e5387b640 1 Processor -- start 2024-04-26T21:51:02.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e5387b640 1 -- start start 2024-04-26T21:51:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e5387b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4c101880 0x7f9e4c10c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e5387b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 0x7f9e4c10a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e5387b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c103430 0x7f9e4c10ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e5387b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c10e4d0 con 0x7f9e4c102a70 2024-04-26T21:51:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e5387b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c10e350 con 0x7f9e4c101880 2024-04-26T21:51:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e5387b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c10e650 con 0x7f9e4c103430 2024-04-26T21:51:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e50e16640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 0x7f9e4c10a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e50e16640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 0x7f9e4c10a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45370/0 (socket says 172.21.15.55:45370) 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e50e16640 1 -- 172.21.15.55:0/992770494 learned_addr learned my addr 172.21.15.55:0/992770494 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e51617640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4c101880 0x7f9e4c10c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e51e18640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c103430 0x7f9e4c10ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e50e16640 1 -- 172.21.15.55:0/992770494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c103430 msgr2=0x7f9e4c10ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.158+0000 7f9e50e16640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c103430 0x7f9e4c10ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e50e16640 1 -- 172.21.15.55:0/992770494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4c101880 msgr2=0x7f9e4c10c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e50e16640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4c101880 0x7f9e4c10c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e50e16640 1 -- 172.21.15.55:0/992770494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c10b610 con 0x7f9e4c102a70 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e50e16640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 0x7f9e4c10a840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005940 tx=0x7f9e3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e427fc640 1 -- 172.21.15.55:0/992770494 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c03f030 con 0x7f9e4c102a70 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e427fc640 1 -- 172.21.15.55:0/992770494 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e3c003cb0 con 0x7f9e4c102a70 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c1a2c70 con 0x7f9e4c102a70 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e427fc640 1 -- 172.21.15.55:0/992770494 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c035070 con 0x7f9e4c102a70 2024-04-26T21:51:02.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.159+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c1a31c0 con 0x7f9e4c102a70 2024-04-26T21:51:02.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.160+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4c108c70 con 0x7f9e4c102a70 2024-04-26T21:51:02.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.161+0000 7f9e427fc640 1 -- 172.21.15.55:0/992770494 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e3c042030 con 0x7f9e4c102a70 2024-04-26T21:51:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.162+0000 7f9e427fc640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e380a3bb0 0x7f9e380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.162+0000 7f9e51617640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e380a3bb0 0x7f9e380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.162+0000 7f9e427fc640 1 -- 172.21.15.55:0/992770494 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9e3c0f0700 con 0x7f9e4c102a70 2024-04-26T21:51:02.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.162+0000 7f9e51617640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e380a3bb0 0x7f9e380a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c1028d0 tx=0x7f9e48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:02.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.166+0000 7f9e427fc640 1 -- 172.21.15.55:0/992770494 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9e3c003e60 con 0x7f9e4c102a70 2024-04-26T21:51:02.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.383+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9e4c062e00 con 0x7f9e380a3bb0 2024-04-26T21:51:02.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.388+0000 7f9e427fc640 1 -- 172.21.15.55:0/992770494 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9e4c062e00 con 0x7f9e380a3bb0 2024-04-26T21:51:02.389 INFO:teuthology.orchestra.run.smithi055.stdout:/volumes/qa/sv_0/27ba293d-e59e-4146-a50b-6524382a757a 2024-04-26T21:51:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.390+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e380a3bb0 msgr2=0x7f9e380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.390+0000 7f9e5387b640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e380a3bb0 0x7f9e380a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c1028d0 tx=0x7f9e48005f90 comp rx=0 tx=0).stop 2024-04-26T21:51:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.390+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 msgr2=0x7f9e4c10a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.390+0000 7f9e5387b640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 0x7f9e4c10a840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005940 tx=0x7f9e3c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.391+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 shutdown_connections 2024-04-26T21:51:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.391+0000 7f9e5387b640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9e380a3bb0 0x7f9e380a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.391+0000 7f9e5387b640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c103430 0x7f9e4c10ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.391+0000 7f9e5387b640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e4c102a70 0x7f9e4c10a840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.391+0000 7f9e5387b640 1 --2- 172.21.15.55:0/992770494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e4c101880 0x7f9e4c10c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.391+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 >> 172.21.15.55:0/992770494 conn(0x7f9e4c0fcdf0 msgr2=0x7f9e4c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:02.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.391+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 shutdown_connections 2024-04-26T21:51:02.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.391+0000 7f9e5387b640 1 -- 172.21.15.55:0/992770494 wait complete. 2024-04-26T21:51:02.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:02 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/580355547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:51:02.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:02 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] up:active 2024-04-26T21:51:02.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:02 smithi055 ceph-mon[29221]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T21:51:02.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:02 smithi055 ceph-mon[29221]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:51:02.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:02 smithi055 ceph-mon[29221]: pgmap v291: 97 pgs: 11 creating+peering, 86 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 7 op/s 2024-04-26T21:51:02.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:02 smithi055 ceph-mon[29221]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:02.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:02 smithi055 ceph-mon[29221]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:02.402 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T21:51:02.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 -- 172.21.15.55:0/1713868212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94101890 msgr2=0x7f8a94101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 --2- 172.21.15.55:0/1713868212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94101890 0x7f8a94101cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009900 tx=0x7f8a8402a410 comp rx=0 tx=0).stop 2024-04-26T21:51:02.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 -- 172.21.15.55:0/1713868212 shutdown_connections 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 --2- 172.21.15.55:0/1713868212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a94103440 0x7f8a94109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 --2- 172.21.15.55:0/1713868212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a94102a80 0x7f8a94102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 --2- 172.21.15.55:0/1713868212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94101890 0x7f8a94101cb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 -- 172.21.15.55:0/1713868212 >> 172.21.15.55:0/1713868212 conn(0x7f8a940fce20 msgr2=0x7f8a940ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 -- 172.21.15.55:0/1713868212 shutdown_connections 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 -- 172.21.15.55:0/1713868212 wait complete. 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.509+0000 7f8a99683640 1 Processor -- start 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a99683640 1 -- start start 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a99683640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a94101890 0x7f8a94198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a99683640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a94102a80 0x7f8a941991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a99683640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94103440 0x7f8a941a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a99683640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9410d400 con 0x7f8a94103440 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a99683640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9410d280 con 0x7f8a94102a80 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a99683640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9410d580 con 0x7f8a94101890 2024-04-26T21:51:02.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a937fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94103440 0x7f8a941a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a94101890 0x7f8a94198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a927fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a94102a80 0x7f8a941991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a94101890 0x7f8a94198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:43306/0 (socket says 172.21.15.55:43306) 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a92ffd640 1 -- 172.21.15.55:0/4008070585 learned_addr learned my addr 172.21.15.55:0/4008070585 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a927fc640 1 -- 172.21.15.55:0/4008070585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a94101890 msgr2=0x7f8a94198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a927fc640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a94101890 0x7f8a94198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a927fc640 1 -- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94103440 msgr2=0x7f8a941a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a927fc640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94103440 0x7f8a941a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a927fc640 1 -- 172.21.15.55:0/4008070585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a941a0840 con 0x7f8a94102a80 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a937fe640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94103440 0x7f8a941a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.510+0000 7f8a927fc640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a94102a80 0x7f8a941991c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800e9f0 tx=0x7f8a8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.511+0000 7f8a7bfff640 1 -- 172.21.15.55:0/4008070585 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800ccc0 con 0x7f8a94102a80 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.511+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a941a0b30 con 0x7f8a94102a80 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.511+0000 7f8a7bfff640 1 -- 172.21.15.55:0/4008070585 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a88005020 con 0x7f8a94102a80 2024-04-26T21:51:02.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.511+0000 7f8a7bfff640 1 -- 172.21.15.55:0/4008070585 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88004a00 con 0x7f8a94102a80 2024-04-26T21:51:02.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.511+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a941a1080 con 0x7f8a94102a80 2024-04-26T21:51:02.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.512+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a941a12b0 con 0x7f8a94102a80 2024-04-26T21:51:02.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.513+0000 7f8a7bfff640 1 -- 172.21.15.55:0/4008070585 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8a88004ba0 con 0x7f8a94102a80 2024-04-26T21:51:02.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.513+0000 7f8a7bfff640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.513+0000 7f8a92ffd640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.514+0000 7f8a7bfff640 1 -- 172.21.15.55:0/4008070585 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8a880d04d0 con 0x7f8a94102a80 2024-04-26T21:51:02.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.514+0000 7f8a92ffd640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a84006010 tx=0x7f8a84005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:02.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.517+0000 7f8a7bfff640 1 -- 172.21.15.55:0/4008070585 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8a8809cf60 con 0x7f8a94102a80 2024-04-26T21:51:02.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:02 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/580355547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:51:02.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:02 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] up:active 2024-04-26T21:51:02.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:02 smithi165 ceph-mon[33296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T21:51:02.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:02 smithi165 ceph-mon[33296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:51:02.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:02 smithi165 ceph-mon[33296]: pgmap v291: 97 pgs: 11 creating+peering, 86 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 7 op/s 2024-04-26T21:51:02.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:02 smithi165 ceph-mon[33296]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:02.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:02 smithi165 ceph-mon[33296]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:02.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.731+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8a9406b680 con 0x7f8a7c0a3bb0 2024-04-26T21:51:02.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.754+0000 7f8a7bfff640 1 -- 172.21.15.55:0/4008070585 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a9406b680 con 0x7f8a7c0a3bb0 2024-04-26T21:51:02.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a7c0a3bb0 msgr2=0x7f8a7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8a84006010 tx=0x7f8a84005a20 comp rx=0 tx=0).stop 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a94102a80 msgr2=0x7f8a941991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a94102a80 0x7f8a941991c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800e9f0 tx=0x7f8a8800eeb0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 shutdown_connections 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a7c0a3bb0 0x7f8a7c0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a94103440 0x7f8a941a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a94102a80 0x7f8a941991c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 --2- 172.21.15.55:0/4008070585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a94101890 0x7f8a94198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 >> 172.21.15.55:0/4008070585 conn(0x7f8a940fce20 msgr2=0x7f8a9406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 shutdown_connections 2024-04-26T21:51:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.756+0000 7f8a99683640 1 -- 172.21.15.55:0/4008070585 wait complete. 2024-04-26T21:51:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:02 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/580355547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T21:51:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:02 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] up:active 2024-04-26T21:51:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:02 smithi064 ceph-mon[32765]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-26T21:51:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:02 smithi064 ceph-mon[32765]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:51:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:02 smithi064 ceph-mon[32765]: pgmap v291: 97 pgs: 11 creating+peering, 86 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s wr, 7 op/s 2024-04-26T21:51:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:02 smithi064 ceph-mon[32765]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:02.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:02 smithi064 ceph-mon[32765]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:02.767 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 -- 172.21.15.55:0/3406735772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788105d40 msgr2=0x7fb788108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 --2- 172.21.15.55:0/3406735772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788105d40 0x7fb788108130 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb778009900 tx=0x7fb77802a410 comp rx=0 tx=0).stop 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 -- 172.21.15.55:0/3406735772 shutdown_connections 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 --2- 172.21.15.55:0/3406735772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb788108670 0x7fb78810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 --2- 172.21.15.55:0/3406735772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788105d40 0x7fb788108130 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 --2- 172.21.15.55:0/3406735772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb788103410 0x7fb788105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 -- 172.21.15.55:0/3406735772 >> 172.21.15.55:0/3406735772 conn(0x7fb7880fce20 msgr2=0x7fb7880ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 -- 172.21.15.55:0/3406735772 shutdown_connections 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.873+0000 7fb790524640 1 -- 172.21.15.55:0/3406735772 wait complete. 2024-04-26T21:51:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb790524640 1 Processor -- start 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb790524640 1 -- start start 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb790524640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788103410 0x7fb78819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb790524640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb788105d40 0x7fb78819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb790524640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb788108670 0x7fb7881a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb790524640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb78810e4c0 con 0x7fb788103410 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb790524640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb78810e340 con 0x7fb788105d40 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb790524640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb78810e640 con 0x7fb788108670 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78e2c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788103410 0x7fb78819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78e2c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788103410 0x7fb78819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45402/0 (socket says 172.21.15.55:45402) 2024-04-26T21:51:02.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78e2c0640 1 -- 172.21.15.55:0/695240772 learned_addr learned my addr 172.21.15.55:0/695240772 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78dabf640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb788105d40 0x7fb78819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78eac1640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb788108670 0x7fb7881a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78e2c0640 1 -- 172.21.15.55:0/695240772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb788108670 msgr2=0x7fb7881a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78e2c0640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb788108670 0x7fb7881a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78e2c0640 1 -- 172.21.15.55:0/695240772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb788105d40 msgr2=0x7fb78819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78e2c0640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb788105d40 0x7fb78819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78e2c0640 1 -- 172.21.15.55:0/695240772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7881a2a60 con 0x7fb788103410 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.874+0000 7fb78eac1640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb788108670 0x7fb7881a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.875+0000 7fb78e2c0640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788103410 0x7fb78819ae40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb78400e9f0 tx=0x7fb78400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.875+0000 7fb77f7fe640 1 -- 172.21.15.55:0/695240772 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78400cd80 con 0x7fb788103410 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.875+0000 7fb77f7fe640 1 -- 172.21.15.55:0/695240772 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7840108e0 con 0x7fb788103410 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.875+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7881a2d50 con 0x7fb788103410 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.875+0000 7fb77f7fe640 1 -- 172.21.15.55:0/695240772 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784018980 con 0x7fb788103410 2024-04-26T21:51:02.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.875+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7881a31c0 con 0x7fb788103410 2024-04-26T21:51:02.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.876+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78810a1c0 con 0x7fb788103410 2024-04-26T21:51:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.877+0000 7fb77f7fe640 1 -- 172.21.15.55:0/695240772 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb784010430 con 0x7fb788103410 2024-04-26T21:51:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.878+0000 7fb77f7fe640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7740a3bb0 0x7fb7740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.878+0000 7fb78dabf640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7740a3bb0 0x7fb7740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.878+0000 7fb77f7fe640 1 -- 172.21.15.55:0/695240772 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb7840d06b0 con 0x7fb788103410 2024-04-26T21:51:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.878+0000 7fb78dabf640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7740a3bb0 0x7fb7740a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb778009230 tx=0x7fb7780314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:02.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:02.882+0000 7fb77f7fe640 1 -- 172.21.15.55:0/695240772 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb78409d140 con 0x7fb788103410 2024-04-26T21:51:03.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.103+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb788062e00 con 0x7fb7740a3bb0 2024-04-26T21:51:03.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.110+0000 7fb77f7fe640 1 -- 172.21.15.55:0/695240772 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb788062e00 con 0x7fb7740a3bb0 2024-04-26T21:51:03.110 INFO:teuthology.orchestra.run.smithi055.stdout:/volumes/qa/sv_1/ac2348ca-3ce8-43e6-9f97-0188cfba123e 2024-04-26T21:51:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7740a3bb0 msgr2=0x7fb7740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7740a3bb0 0x7fb7740a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb778009230 tx=0x7fb7780314a0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788103410 msgr2=0x7fb78819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788103410 0x7fb78819ae40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb78400e9f0 tx=0x7fb78400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 shutdown_connections 2024-04-26T21:51:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7740a3bb0 0x7fb7740a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb788108670 0x7fb7881a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb788105d40 0x7fb78819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 --2- 172.21.15.55:0/695240772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb788103410 0x7fb78819ae40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 >> 172.21.15.55:0/695240772 conn(0x7fb7880fce20 msgr2=0x7fb788102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 shutdown_connections 2024-04-26T21:51:03.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.112+0000 7fb790524640 1 -- 172.21.15.55:0/695240772 wait complete. 2024-04-26T21:51:03.123 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:51:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2978703670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934102a80 msgr2=0x7fc934102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2978703670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934102a80 0x7fc934102f00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92402a410 comp rx=0 tx=0).stop 2024-04-26T21:51:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2978703670 shutdown_connections 2024-04-26T21:51:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2978703670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc934103440 0x7fc934109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2978703670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934102a80 0x7fc934102f00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2978703670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc934101890 0x7fc934101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2978703670 >> 172.21.15.55:0/2978703670 conn(0x7fc9340fce20 msgr2=0x7fc9340ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2978703670 shutdown_connections 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2978703670 wait complete. 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 Processor -- start 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- start start 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934101890 0x7fc93406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc934102a80 0x7fc93419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc934103440 0x7fc93419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410d3a0 con 0x7fc934101890 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410d220 con 0x7fc934103440 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.225+0000 7fc93b6f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc93410d520 con 0x7fc934102a80 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934101890 0x7fc93406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934101890 0x7fc93406bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45426/0 (socket says 172.21.15.55:45426) 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 -- 172.21.15.55:0/2061150727 learned_addr learned my addr 172.21.15.55:0/2061150727 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc939c8e640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc934103440 0x7fc93419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc938c8c640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc934102a80 0x7fc93419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 -- 172.21.15.55:0/2061150727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc934102a80 msgr2=0x7fc93419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc934102a80 0x7fc93419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 -- 172.21.15.55:0/2061150727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc934103440 msgr2=0x7fc93419ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc934103440 0x7fc93419ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 -- 172.21.15.55:0/2061150727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9341a0800 con 0x7fc934101890 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93948d640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934101890 0x7fc93406bad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc93000e9c0 tx=0x7fc93000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc92a7fc640 1 -- 172.21.15.55:0/2061150727 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93000cde0 con 0x7fc934101890 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc92a7fc640 1 -- 172.21.15.55:0/2061150727 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9300108e0 con 0x7fc934101890 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc92a7fc640 1 -- 172.21.15.55:0/2061150727 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930018960 con 0x7fc934101890 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9341a0af0 con 0x7fc934101890 2024-04-26T21:51:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.226+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341a1060 con 0x7fc934101890 2024-04-26T21:51:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.227+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc934108d40 con 0x7fc934101890 2024-04-26T21:51:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.228+0000 7fc92a7fc640 1 -- 172.21.15.55:0/2061150727 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc93001f020 con 0x7fc934101890 2024-04-26T21:51:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.229+0000 7fc92a7fc640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9200a3c30 0x7fc9200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.229+0000 7fc938c8c640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9200a3c30 0x7fc9200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.229+0000 7fc92a7fc640 1 -- 172.21.15.55:0/2061150727 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc9300d05d0 con 0x7fc934101890 2024-04-26T21:51:03.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.229+0000 7fc938c8c640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9200a3c30 0x7fc9200a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc93406b840 tx=0x7fc9240314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.234+0000 7fc92a7fc640 1 -- 172.21.15.55:0/2061150727 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc930054060 con 0x7fc934101890 2024-04-26T21:51:03.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:03 smithi055 ceph-mon[29221]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:03 smithi055 ceph-mon[29221]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:03 smithi055 ceph-mon[29221]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.460+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9340635d0 con 0x7fc934101890 2024-04-26T21:51:03.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.461+0000 7fc92a7fc640 1 -- 172.21.15.55:0/2061150727 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11863 (secure 0 0 0) 0x7fc93009d060 con 0x7fc934101890 2024-04-26T21:51:03.461 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:51:03.461 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:51:01.298886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:51:03.462 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 20 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9200a3c30 msgr2=0x7fc9200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9200a3c30 0x7fc9200a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc93406b840 tx=0x7fc9240314a0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934101890 msgr2=0x7fc93406bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934101890 0x7fc93406bad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc93000e9c0 tx=0x7fc93000ee80 comp rx=0 tx=0).stop 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 shutdown_connections 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9200a3c30 0x7fc9200a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc934103440 0x7fc93419ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc934102a80 0x7fc93419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 --2- 172.21.15.55:0/2061150727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc934101890 0x7fc93406bad0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 >> 172.21.15.55:0/2061150727 conn(0x7fc9340fce20 msgr2=0x7fc934105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 shutdown_connections 2024-04-26T21:51:03.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.463+0000 7fc93b6f1640 1 -- 172.21.15.55:0/2061150727 wait complete. 2024-04-26T21:51:03.475 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:51:03.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:03 smithi064 ceph-mon[32765]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:03 smithi064 ceph-mon[32765]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:03 smithi064 ceph-mon[32765]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 -- 172.21.15.55:0/355938710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a8101b50 msgr2=0x7f31a8101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 --2- 172.21.15.55:0/355938710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a8101b50 0x7f31a8101fb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f319c009900 tx=0x7f319c02a410 comp rx=0 tx=0).stop 2024-04-26T21:51:03.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 -- 172.21.15.55:0/355938710 shutdown_connections 2024-04-26T21:51:03.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 --2- 172.21.15.55:0/355938710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a81024f0 0x7f31a8102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 --2- 172.21.15.55:0/355938710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a8101b50 0x7f31a8101fb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 --2- 172.21.15.55:0/355938710 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a81011f0 0x7f31a8101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 -- 172.21.15.55:0/355938710 >> 172.21.15.55:0/355938710 conn(0x7f31a80fce10 msgr2=0x7f31a80ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 -- 172.21.15.55:0/355938710 shutdown_connections 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 -- 172.21.15.55:0/355938710 wait complete. 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 Processor -- start 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 -- start start 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a81011f0 0x7f31a819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a8101b50 0x7f31a819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.576+0000 7f31afdda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a81024f0 0x7f31a81a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31afdda640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f31a8110670 con 0x7f31a81011f0 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31afdda640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31a81104f0 con 0x7f31a81024f0 2024-04-26T21:51:03.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31afdda640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f31a81107f0 con 0x7f31a8101b50 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a81011f0 0x7f31a819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a81011f0 0x7f31a819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45436/0 (socket says 172.21.15.55:45436) 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 -- 172.21.15.55:0/4022683302 learned_addr learned my addr 172.21.15.55:0/4022683302 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31ae377640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a81024f0 0x7f31a81a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31ad375640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a8101b50 0x7f31a819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 -- 172.21.15.55:0/4022683302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a8101b50 msgr2=0x7f31a819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a8101b50 0x7f31a819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 -- 172.21.15.55:0/4022683302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a81024f0 msgr2=0x7f31a81a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a81024f0 0x7f31a81a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 -- 172.21.15.55:0/4022683302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a81a2960 con 0x7f31a81011f0 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31adb76640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a81011f0 0x7f31a819ad40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f31a000e9f0 tx=0x7f31a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:03.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31a6ffd640 1 -- 172.21.15.55:0/4022683302 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a000cd80 con 0x7f31a81011f0 2024-04-26T21:51:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31a6ffd640 1 -- 172.21.15.55:0/4022683302 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a00108e0 con 0x7f31a81011f0 2024-04-26T21:51:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31a6ffd640 1 -- 172.21.15.55:0/4022683302 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0018980 con 0x7f31a81011f0 2024-04-26T21:51:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a81a2c50 con 0x7f31a81011f0 2024-04-26T21:51:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.577+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a81a3150 con 0x7f31a81011f0 2024-04-26T21:51:03.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.578+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a810c140 con 0x7f31a81011f0 2024-04-26T21:51:03.583 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:03 smithi165 ceph-mon[33296]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.583 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:03 smithi165 ceph-mon[33296]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.583 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:03 smithi165 ceph-mon[33296]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:51:03.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.580+0000 7f31a6ffd640 1 -- 172.21.15.55:0/4022683302 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31a0010430 con 0x7f31a81011f0 2024-04-26T21:51:03.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.580+0000 7f31a6ffd640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31900a3c30 0x7f31900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.580+0000 7f31a6ffd640 1 -- 172.21.15.55:0/4022683302 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f31a00d05c0 con 0x7f31a81011f0 2024-04-26T21:51:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.580+0000 7f31ad375640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31900a3c30 0x7f31900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.581+0000 7f31ad375640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31900a3c30 0x7f31900a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f31a819c250 tx=0x7f319c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:03.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.585+0000 7f31a6ffd640 1 -- 172.21.15.55:0/4022683302 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f31a009d0f0 con 0x7f31a81011f0 2024-04-26T21:51:03.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.773+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31a802dd80 con 0x7f31a81011f0 2024-04-26T21:51:03.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.775+0000 7f31a6ffd640 1 -- 172.21.15.55:0/4022683302 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22185 (secure 0 0 0) 0x7f31a009c9f0 con 0x7f31a81011f0 2024-04-26T21:51:03.775 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:51:03.775 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":85,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:50:58.120222+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:51:03.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.776+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31900a3c30 msgr2=0x7f31900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31900a3c30 0x7f31900a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f31a819c250 tx=0x7f319c0314a0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a81011f0 msgr2=0x7f31a819ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a81011f0 0x7f31a819ad40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f31a000e9f0 tx=0x7f31a000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 shutdown_connections 2024-04-26T21:51:03.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31900a3c30 0x7f31900a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31a81024f0 0x7f31a81a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31a8101b50 0x7f31a819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 --2- 172.21.15.55:0/4022683302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f31a81011f0 0x7f31a819ad40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 >> 172.21.15.55:0/4022683302 conn(0x7f31a80fce10 msgr2=0x7f31a80fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:03.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 shutdown_connections 2024-04-26T21:51:03.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.777+0000 7f31afdda640 1 -- 172.21.15.55:0/4022683302 wait complete. 2024-04-26T21:51:03.789 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T21:51:03.789 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:51:03.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 -- 172.21.15.55:0/3663494384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 msgr2=0x7fd348108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/3663494384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 0x7fd348108190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd338009900 tx=0x7fd33802a410 comp rx=0 tx=0).stop 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 -- 172.21.15.55:0/3663494384 shutdown_connections 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/3663494384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3481086d0 0x7fd34810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/3663494384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 0x7fd348108190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/3663494384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd348103470 0x7fd348105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 -- 172.21.15.55:0/3663494384 >> 172.21.15.55:0/3663494384 conn(0x7fd3480fcde0 msgr2=0x7fd3480ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 -- 172.21.15.55:0/3663494384 shutdown_connections 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.890+0000 7fd34c9e2640 1 -- 172.21.15.55:0/3663494384 wait complete. 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd34c9e2640 1 Processor -- start 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd34c9e2640 1 -- start start 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd34c9e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd348103470 0x7fd348198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd34c9e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 0x7fd3481991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd34c9e2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3481086d0 0x7fd3481a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd34c9e2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd34810e4f0 con 0x7fd348105da0 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd34c9e2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd34810e370 con 0x7fd348103470 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd34c9e2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd34810e670 con 0x7fd3481086d0 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd345d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 0x7fd3481991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd345d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 0x7fd3481991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45454/0 (socket says 172.21.15.55:45454) 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.891+0000 7fd345d9b640 1 -- 172.21.15.55:0/247387584 learned_addr learned my addr 172.21.15.55:0/247387584 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd345d9b640 1 -- 172.21.15.55:0/247387584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3481086d0 msgr2=0x7fd3481a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd345d9b640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3481086d0 0x7fd3481a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd345d9b640 1 -- 172.21.15.55:0/247387584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd348103470 msgr2=0x7fd348198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd345d9b640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd348103470 0x7fd348198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd345d9b640 1 -- 172.21.15.55:0/247387584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3481a0890 con 0x7fd348105da0 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd345d9b640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 0x7fd3481991c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd338006010 tx=0x7fd3380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd33f7fe640 1 -- 172.21.15.55:0/247387584 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33803f030 con 0x7fd348105da0 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd33f7fe640 1 -- 172.21.15.55:0/247387584 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd338003cb0 con 0x7fd348105da0 2024-04-26T21:51:03.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3481a0b20 con 0x7fd348105da0 2024-04-26T21:51:03.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd33f7fe640 1 -- 172.21.15.55:0/247387584 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338035070 con 0x7fd348105da0 2024-04-26T21:51:03.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.892+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3481a1010 con 0x7fd348105da0 2024-04-26T21:51:03.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.893+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3480fe8d0 con 0x7fd348105da0 2024-04-26T21:51:03.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.894+0000 7fd33f7fe640 1 -- 172.21.15.55:0/247387584 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd338042030 con 0x7fd348105da0 2024-04-26T21:51:03.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.895+0000 7fd33f7fe640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd3340a3bb0 0x7fd3340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:03.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.895+0000 7fd34659c640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd3340a3bb0 0x7fd3340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:03.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.895+0000 7fd33f7fe640 1 -- 172.21.15.55:0/247387584 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd3380f0ab0 con 0x7fd348105da0 2024-04-26T21:51:03.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.895+0000 7fd34659c640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd3340a3bb0 0x7fd3340a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd34819a150 tx=0x7fd330005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:03.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:03.900+0000 7fd33f7fe640 1 -- 172.21.15.55:0/247387584 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd3380bd540 con 0x7fd348105da0 2024-04-26T21:51:04.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.082+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd348062d70 con 0x7fd3340a3bb0 2024-04-26T21:51:04.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.090+0000 7fd33f7fe640 1 -- 172.21.15.55:0/247387584 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258898 (secure 0 0 0) 0x7fd348062d70 con 0x7fd3340a3bb0 2024-04-26T21:51:04.094 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:51:04.095 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.093+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd3340a3bb0 msgr2=0x7fd3340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.093+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd3340a3bb0 0x7fd3340a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd34819a150 tx=0x7fd330005ed0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.093+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 msgr2=0x7fd3481991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.093+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 0x7fd3481991c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd338006010 tx=0x7fd3380042f0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.094+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 shutdown_connections 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.094+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd3340a3bb0 0x7fd3340a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.094+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3481086d0 0x7fd3481a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.094+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd348105da0 0x7fd3481991c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.094+0000 7fd34c9e2640 1 --2- 172.21.15.55:0/247387584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd348103470 0x7fd348198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.094+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 >> 172.21.15.55:0/247387584 conn(0x7fd3480fcde0 msgr2=0x7fd348103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.094+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 shutdown_connections 2024-04-26T21:51:04.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.094+0000 7fd34c9e2640 1 -- 172.21.15.55:0/247387584 wait complete. 2024-04-26T21:51:04.105 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-04-26T21:51:03.342584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593584,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":133,"num_read_kb":116,"num_write":271,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351848,"kb_used_data":10988,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589976,"statfs":{"total":1151940427776,"available":1151580135424,"internally_reserved":0,"allocated":11251712,"data_stored":7030309,"data_compressed":109407,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3216,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.995087"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482830+0000","last_change":"2024-04-26T21:50:57.139486+0000","last_active":"2024-04-26T21:50:58.482830+0000","last_peered":"2024-04-26T21:50:58.482830+0000","last_clean":"2024-04-26T21:50:58.482830+0000","last_became_active":"2024-04-26T21:50:57.139133+0000","last_became_peered":"2024-04-26T21:50:57.139133+0000","last_unstale":"2024-04-26T21:50:58.482830+0000","last_undegraded":"2024-04-26T21:50:58.482830+0000","last_fullsized":"2024-04-26T21:50:58.482830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:33.211627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481810+0000","last_change":"2024-04-26T21:50:57.138519+0000","last_active":"2024-04-26T21:50:58.481810+0000","last_peered":"2024-04-26T21:50:58.481810+0000","last_clean":"2024-04-26T21:50:58.481810+0000","last_became_active":"2024-04-26T21:50:57.138131+0000","last_became_peered":"2024-04-26T21:50:57.138131+0000","last_unstale":"2024-04-26T21:50:58.481810+0000","last_undegraded":"2024-04-26T21:50:58.481810+0000","last_fullsized":"2024-04-26T21:50:58.481810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:34.349694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481810+0000","last_change":"2024-04-26T21:50:57.146584+0000","last_active":"2024-04-26T21:50:58.481810+0000","last_peered":"2024-04-26T21:50:58.481810+0000","last_clean":"2024-04-26T21:50:58.481810+0000","last_became_active":"2024-04-26T21:50:57.146262+0000","last_became_peered":"2024-04-26T21:50:57.146262+0000","last_unstale":"2024-04-26T21:50:58.481810+0000","last_undegraded":"2024-04-26T21:50:58.481810+0000","last_fullsized":"2024-04-26T21:50:58.481810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:49.946882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483645+0000","last_change":"2024-04-26T21:50:57.144526+0000","last_active":"2024-04-26T21:50:58.483645+0000","last_peered":"2024-04-26T21:50:58.483645+0000","last_clean":"2024-04-26T21:50:58.483645+0000","last_became_active":"2024-04-26T21:50:57.144052+0000","last_became_peered":"2024-04-26T21:50:57.144052+0000","last_unstale":"2024-04-26T21:50:58.483645+0000","last_undegraded":"2024-04-26T21:50:58.483645+0000","last_fullsized":"2024-04-26T21:50:58.483645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:16.499874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359621+0000","last_change":"2024-04-26T21:50:57.136292+0000","last_active":"2024-04-26T21:50:58.359621+0000","last_peered":"2024-04-26T21:50:58.359621+0000","last_clean":"2024-04-26T21:50:58.359621+0000","last_became_active":"2024-04-26T21:50:57.135936+0000","last_became_peered":"2024-04-26T21:50:57.135936+0000","last_unstale":"2024-04-26T21:50:58.359621+0000","last_undegraded":"2024-04-26T21:50:58.359621+0000","last_fullsized":"2024-04-26T21:50:58.359621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:35.415440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.135948+0000","last_change":"2024-04-26T21:50:57.147764+0000","last_active":"2024-04-26T21:50:58.135948+0000","last_peered":"2024-04-26T21:50:58.135948+0000","last_clean":"2024-04-26T21:50:58.135948+0000","last_became_active":"2024-04-26T21:50:57.147034+0000","last_became_peered":"2024-04-26T21:50:57.147034+0000","last_unstale":"2024-04-26T21:50:58.135948+0000","last_undegraded":"2024-04-26T21:50:58.135948+0000","last_fullsized":"2024-04-26T21:50:58.135948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:44.833213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482822+0000","last_change":"2024-04-26T21:50:57.144813+0000","last_active":"2024-04-26T21:50:58.482822+0000","last_peered":"2024-04-26T21:50:58.482822+0000","last_clean":"2024-04-26T21:50:58.482822+0000","last_became_active":"2024-04-26T21:50:57.144442+0000","last_became_peered":"2024-04-26T21:50:57.144442+0000","last_unstale":"2024-04-26T21:50:58.482822+0000","last_undegraded":"2024-04-26T21:50:58.482822+0000","last_fullsized":"2024-04-26T21:50:58.482822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:59.032263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359358+0000","last_change":"2024-04-26T21:50:57.145958+0000","last_active":"2024-04-26T21:50:58.359358+0000","last_peered":"2024-04-26T21:50:58.359358+0000","last_clean":"2024-04-26T21:50:58.359358+0000","last_became_active":"2024-04-26T21:50:57.145623+0000","last_became_peered":"2024-04-26T21:50:57.145623+0000","last_unstale":"2024-04-26T21:50:58.359358+0000","last_undegraded":"2024-04-26T21:50:58.359358+0000","last_fullsized":"2024-04-26T21:50:58.359358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.716079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483712+0000","last_change":"2024-04-26T21:50:57.132957+0000","last_active":"2024-04-26T21:50:58.483712+0000","last_peered":"2024-04-26T21:50:58.483712+0000","last_clean":"2024-04-26T21:50:58.483712+0000","last_became_active":"2024-04-26T21:50:57.132606+0000","last_became_peered":"2024-04-26T21:50:57.132606+0000","last_unstale":"2024-04-26T21:50:58.483712+0000","last_undegraded":"2024-04-26T21:50:58.483712+0000","last_fullsized":"2024-04-26T21:50:58.483712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:00.994370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136350+0000","last_change":"2024-04-26T21:50:57.150598+0000","last_active":"2024-04-26T21:50:58.136350+0000","last_peered":"2024-04-26T21:50:58.136350+0000","last_clean":"2024-04-26T21:50:58.136350+0000","last_became_active":"2024-04-26T21:50:57.149697+0000","last_became_peered":"2024-04-26T21:50:57.149697+0000","last_unstale":"2024-04-26T21:50:58.136350+0000","last_undegraded":"2024-04-26T21:50:58.136350+0000","last_fullsized":"2024-04-26T21:50:58.136350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.621137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483258+0000","last_change":"2024-04-26T21:50:57.147264+0000","last_active":"2024-04-26T21:50:58.483258+0000","last_peered":"2024-04-26T21:50:58.483258+0000","last_clean":"2024-04-26T21:50:58.483258+0000","last_became_active":"2024-04-26T21:50:57.146888+0000","last_became_peered":"2024-04-26T21:50:57.146888+0000","last_unstale":"2024-04-26T21:50:58.483258+0000","last_undegraded":"2024-04-26T21:50:58.483258+0000","last_fullsized":"2024-04-26T21:50:58.483258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:49.584327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483424+0000","last_change":"2024-04-26T21:50:57.142005+0000","last_active":"2024-04-26T21:50:58.483424+0000","last_peered":"2024-04-26T21:50:58.483424+0000","last_clean":"2024-04-26T21:50:58.483424+0000","last_became_active":"2024-04-26T21:50:57.141605+0000","last_became_peered":"2024-04-26T21:50:57.141605+0000","last_unstale":"2024-04-26T21:50:58.483424+0000","last_undegraded":"2024-04-26T21:50:58.483424+0000","last_fullsized":"2024-04-26T21:50:58.483424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:43.485295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136333+0000","last_change":"2024-04-26T21:50:57.138049+0000","last_active":"2024-04-26T21:50:58.136333+0000","last_peered":"2024-04-26T21:50:58.136333+0000","last_clean":"2024-04-26T21:50:58.136333+0000","last_became_active":"2024-04-26T21:50:57.137717+0000","last_became_peered":"2024-04-26T21:50:57.137717+0000","last_unstale":"2024-04-26T21:50:58.136333+0000","last_undegraded":"2024-04-26T21:50:58.136333+0000","last_fullsized":"2024-04-26T21:50:58.136333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:28.918224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481605+0000","last_change":"2024-04-26T21:50:57.139183+0000","last_active":"2024-04-26T21:50:58.481605+0000","last_peered":"2024-04-26T21:50:58.481605+0000","last_clean":"2024-04-26T21:50:58.481605+0000","last_became_active":"2024-04-26T21:50:57.138759+0000","last_became_peered":"2024-04-26T21:50:57.138759+0000","last_unstale":"2024-04-26T21:50:58.481605+0000","last_undegraded":"2024-04-26T21:50:58.481605+0000","last_fullsized":"2024-04-26T21:50:58.481605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:12.003122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481632+0000","last_change":"2024-04-26T21:50:57.143110+0000","last_active":"2024-04-26T21:50:58.481632+0000","last_peered":"2024-04-26T21:50:58.481632+0000","last_clean":"2024-04-26T21:50:58.481632+0000","last_became_active":"2024-04-26T21:50:57.142754+0000","last_became_peered":"2024-04-26T21:50:57.142754+0000","last_unstale":"2024-04-26T21:50:58.481632+0000","last_undegraded":"2024-04-26T21:50:58.481632+0000","last_fullsized":"2024-04-26T21:50:58.481632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:08.321000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482284+0000","last_change":"2024-04-26T21:50:57.146514+0000","last_active":"2024-04-26T21:50:58.482284+0000","last_peered":"2024-04-26T21:50:58.482284+0000","last_clean":"2024-04-26T21:50:58.482284+0000","last_became_active":"2024-04-26T21:50:57.146046+0000","last_became_peered":"2024-04-26T21:50:57.146046+0000","last_unstale":"2024-04-26T21:50:58.482284+0000","last_undegraded":"2024-04-26T21:50:58.482284+0000","last_fullsized":"2024-04-26T21:50:58.482284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:21.130220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483000+0000","last_change":"2024-04-26T21:50:57.140492+0000","last_active":"2024-04-26T21:50:58.483000+0000","last_peered":"2024-04-26T21:50:58.483000+0000","last_clean":"2024-04-26T21:50:58.483000+0000","last_became_active":"2024-04-26T21:50:57.140147+0000","last_became_peered":"2024-04-26T21:50:57.140147+0000","last_unstale":"2024-04-26T21:50:58.483000+0000","last_undegraded":"2024-04-26T21:50:58.483000+0000","last_fullsized":"2024-04-26T21:50:58.483000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:21.679202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.357924+0000","last_change":"2024-04-26T21:50:57.140992+0000","last_active":"2024-04-26T21:50:58.357924+0000","last_peered":"2024-04-26T21:50:58.357924+0000","last_clean":"2024-04-26T21:50:58.357924+0000","last_became_active":"2024-04-26T21:50:57.140644+0000","last_became_peered":"2024-04-26T21:50:57.140644+0000","last_unstale":"2024-04-26T21:50:58.357924+0000","last_undegraded":"2024-04-26T21:50:58.357924+0000","last_fullsized":"2024-04-26T21:50:58.357924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131570+0000","last_change":"2024-04-26T21:50:57.141242+0000","last_active":"2024-04-26T21:50:58.131570+0000","last_peered":"2024-04-26T21:50:58.131570+0000","last_clean":"2024-04-26T21:50:58.131570+0000","last_became_active":"2024-04-26T21:50:57.140909+0000","last_became_peered":"2024-04-26T21:50:57.140909+0000","last_unstale":"2024-04-26T21:50:58.131570+0000","last_undegraded":"2024-04-26T21:50:58.131570+0000","last_fullsized":"2024-04-26T21:50:58.131570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:21.996984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.357655+0000","last_change":"2024-04-26T21:50:56.371876+0000","last_active":"2024-04-26T21:50:58.357655+0000","last_peered":"2024-04-26T21:50:58.357655+0000","last_clean":"2024-04-26T21:50:58.357655+0000","last_became_active":"2024-04-26T21:50:56.371509+0000","last_became_peered":"2024-04-26T21:50:56.371509+0000","last_unstale":"2024-04-26T21:50:58.357655+0000","last_undegraded":"2024-04-26T21:50:58.357655+0000","last_fullsized":"2024-04-26T21:50:58.357655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:50.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359780+0000","last_change":"2024-04-26T21:50:57.145654+0000","last_active":"2024-04-26T21:50:58.359780+0000","last_peered":"2024-04-26T21:50:58.359780+0000","last_clean":"2024-04-26T21:50:58.359780+0000","last_became_active":"2024-04-26T21:50:57.145355+0000","last_became_peered":"2024-04-26T21:50:57.145355+0000","last_unstale":"2024-04-26T21:50:58.359780+0000","last_undegraded":"2024-04-26T21:50:58.359780+0000","last_fullsized":"2024-04-26T21:50:58.359780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:51.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359873+0000","last_change":"2024-04-26T21:50:56.362758+0000","last_active":"2024-04-26T21:50:58.359873+0000","last_peered":"2024-04-26T21:50:58.359873+0000","last_clean":"2024-04-26T21:50:58.359873+0000","last_became_active":"2024-04-26T21:50:56.361338+0000","last_became_peered":"2024-04-26T21:50:56.361338+0000","last_unstale":"2024-04-26T21:50:58.359873+0000","last_undegraded":"2024-04-26T21:50:58.359873+0000","last_fullsized":"2024-04-26T21:50:58.359873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:44.603426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482048+0000","last_change":"2024-04-26T21:50:57.150701+0000","last_active":"2024-04-26T21:50:58.482048+0000","last_peered":"2024-04-26T21:50:58.482048+0000","last_clean":"2024-04-26T21:50:58.482048+0000","last_became_active":"2024-04-26T21:50:57.150388+0000","last_became_peered":"2024-04-26T21:50:57.150388+0000","last_unstale":"2024-04-26T21:50:58.482048+0000","last_undegraded":"2024-04-26T21:50:58.482048+0000","last_fullsized":"2024-04-26T21:50:58.482048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:43:07.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.304006+0000","last_change":"2024-04-26T21:50:56.371379+0000","last_active":"2024-04-26T21:51:00.304006+0000","last_peered":"2024-04-26T21:51:00.304006+0000","last_clean":"2024-04-26T21:51:00.304006+0000","last_became_active":"2024-04-26T21:50:56.371025+0000","last_became_peered":"2024-04-26T21:50:56.371025+0000","last_unstale":"2024-04-26T21:51:00.304006+0000","last_undegraded":"2024-04-26T21:51:00.304006+0000","last_fullsized":"2024-04-26T21:51:00.304006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:00.633393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483023+0000","last_change":"2024-04-26T21:50:57.144777+0000","last_active":"2024-04-26T21:50:58.483023+0000","last_peered":"2024-04-26T21:50:58.483023+0000","last_clean":"2024-04-26T21:50:58.483023+0000","last_became_active":"2024-04-26T21:50:57.144431+0000","last_became_peered":"2024-04-26T21:50:57.144431+0000","last_unstale":"2024-04-26T21:50:58.483023+0000","last_undegraded":"2024-04-26T21:50:58.483023+0000","last_fullsized":"2024-04-26T21:50:58.483023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:42.164240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359838+0000","last_change":"2024-04-26T21:50:56.360843+0000","last_active":"2024-04-26T21:50:58.359838+0000","last_peered":"2024-04-26T21:50:58.359838+0000","last_clean":"2024-04-26T21:50:58.359838+0000","last_became_active":"2024-04-26T21:50:56.360440+0000","last_became_peered":"2024-04-26T21:50:56.360440+0000","last_unstale":"2024-04-26T21:50:58.359838+0000","last_undegraded":"2024-04-26T21:50:58.359838+0000","last_fullsized":"2024-04-26T21:50:58.359838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:47.585306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482223+0000","last_change":"2024-04-26T21:50:57.146449+0000","last_active":"2024-04-26T21:50:58.482223+0000","last_peered":"2024-04-26T21:50:58.482223+0000","last_clean":"2024-04-26T21:50:58.482223+0000","last_became_active":"2024-04-26T21:50:57.146128+0000","last_became_peered":"2024-04-26T21:50:57.146128+0000","last_unstale":"2024-04-26T21:50:58.482223+0000","last_undegraded":"2024-04-26T21:50:58.482223+0000","last_fullsized":"2024-04-26T21:50:58.482223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:09.165868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481908+0000","last_change":"2024-04-26T21:50:56.371455+0000","last_active":"2024-04-26T21:50:58.481908+0000","last_peered":"2024-04-26T21:50:58.481908+0000","last_clean":"2024-04-26T21:50:58.481908+0000","last_became_active":"2024-04-26T21:50:56.371096+0000","last_became_peered":"2024-04-26T21:50:56.371096+0000","last_unstale":"2024-04-26T21:50:58.481908+0000","last_undegraded":"2024-04-26T21:50:58.481908+0000","last_fullsized":"2024-04-26T21:50:58.481908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:37.167965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359634+0000","last_change":"2024-04-26T21:50:57.138817+0000","last_active":"2024-04-26T21:50:58.359634+0000","last_peered":"2024-04-26T21:50:58.359634+0000","last_clean":"2024-04-26T21:50:58.359634+0000","last_became_active":"2024-04-26T21:50:57.138489+0000","last_became_peered":"2024-04-26T21:50:57.138489+0000","last_unstale":"2024-04-26T21:50:58.359634+0000","last_undegraded":"2024-04-26T21:50:58.359634+0000","last_fullsized":"2024-04-26T21:50:58.359634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:56.072063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.135787+0000","last_change":"2024-04-26T21:50:56.366990+0000","last_active":"2024-04-26T21:50:58.135787+0000","last_peered":"2024-04-26T21:50:58.135787+0000","last_clean":"2024-04-26T21:50:58.135787+0000","last_became_active":"2024-04-26T21:50:56.366608+0000","last_became_peered":"2024-04-26T21:50:56.366608+0000","last_unstale":"2024-04-26T21:50:58.135787+0000","last_undegraded":"2024-04-26T21:50:58.135787+0000","last_fullsized":"2024-04-26T21:50:58.135787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:19.951038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482126+0000","last_change":"2024-04-26T21:50:57.147601+0000","last_active":"2024-04-26T21:50:58.482126+0000","last_peered":"2024-04-26T21:50:58.482126+0000","last_clean":"2024-04-26T21:50:58.482126+0000","last_became_active":"2024-04-26T21:50:57.147197+0000","last_became_peered":"2024-04-26T21:50:57.147197+0000","last_unstale":"2024-04-26T21:50:58.482126+0000","last_undegraded":"2024-04-26T21:50:58.482126+0000","last_fullsized":"2024-04-26T21:50:58.482126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:32:47.898167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.305012+0000","last_change":"2024-04-26T21:50:56.362272+0000","last_active":"2024-04-26T21:51:00.305012+0000","last_peered":"2024-04-26T21:51:00.305012+0000","last_clean":"2024-04-26T21:51:00.305012+0000","last_became_active":"2024-04-26T21:50:56.361941+0000","last_became_peered":"2024-04-26T21:50:56.361941+0000","last_unstale":"2024-04-26T21:51:00.305012+0000","last_undegraded":"2024-04-26T21:51:00.305012+0000","last_fullsized":"2024-04-26T21:51:00.305012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359467+0000","last_change":"2024-04-26T21:50:57.135566+0000","last_active":"2024-04-26T21:50:58.359467+0000","last_peered":"2024-04-26T21:50:58.359467+0000","last_clean":"2024-04-26T21:50:58.359467+0000","last_became_active":"2024-04-26T21:50:57.135231+0000","last_became_peered":"2024-04-26T21:50:57.135231+0000","last_unstale":"2024-04-26T21:50:58.359467+0000","last_undegraded":"2024-04-26T21:50:58.359467+0000","last_fullsized":"2024-04-26T21:50:58.359467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:09.381936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481543+0000","last_change":"2024-04-26T21:50:56.372420+0000","last_active":"2024-04-26T21:50:58.481543+0000","last_peered":"2024-04-26T21:50:58.481543+0000","last_clean":"2024-04-26T21:50:58.481543+0000","last_became_active":"2024-04-26T21:50:56.372096+0000","last_became_peered":"2024-04-26T21:50:56.372096+0000","last_unstale":"2024-04-26T21:50:58.481543+0000","last_undegraded":"2024-04-26T21:50:58.481543+0000","last_fullsized":"2024-04-26T21:50:58.481543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:03.515804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.357660+0000","last_change":"2024-04-26T21:50:57.140071+0000","last_active":"2024-04-26T21:50:58.357660+0000","last_peered":"2024-04-26T21:50:58.357660+0000","last_clean":"2024-04-26T21:50:58.357660+0000","last_became_active":"2024-04-26T21:50:57.139693+0000","last_became_peered":"2024-04-26T21:50:57.139693+0000","last_unstale":"2024-04-26T21:50:58.357660+0000","last_undegraded":"2024-04-26T21:50:58.357660+0000","last_fullsized":"2024-04-26T21:50:58.357660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:17.963966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136115+0000","last_change":"2024-04-26T21:50:56.363748+0000","last_active":"2024-04-26T21:50:58.136115+0000","last_peered":"2024-04-26T21:50:58.136115+0000","last_clean":"2024-04-26T21:50:58.136115+0000","last_became_active":"2024-04-26T21:50:56.363328+0000","last_became_peered":"2024-04-26T21:50:56.363328+0000","last_unstale":"2024-04-26T21:50:58.136115+0000","last_undegraded":"2024-04-26T21:50:58.136115+0000","last_fullsized":"2024-04-26T21:50:58.136115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:24.450981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483095+0000","last_change":"2024-04-26T21:50:57.143064+0000","last_active":"2024-04-26T21:50:58.483095+0000","last_peered":"2024-04-26T21:50:58.483095+0000","last_clean":"2024-04-26T21:50:58.483095+0000","last_became_active":"2024-04-26T21:50:57.142715+0000","last_became_peered":"2024-04-26T21:50:57.142715+0000","last_unstale":"2024-04-26T21:50:58.483095+0000","last_undegraded":"2024-04-26T21:50:58.483095+0000","last_fullsized":"2024-04-26T21:50:58.483095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:35.901855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.286208+0000","last_change":"2024-04-26T21:50:56.369898+0000","last_active":"2024-04-26T21:50:59.286208+0000","last_peered":"2024-04-26T21:50:59.286208+0000","last_clean":"2024-04-26T21:50:59.286208+0000","last_became_active":"2024-04-26T21:50:56.369349+0000","last_became_peered":"2024-04-26T21:50:56.369349+0000","last_unstale":"2024-04-26T21:50:59.286208+0000","last_undegraded":"2024-04-26T21:50:59.286208+0000","last_fullsized":"2024-04-26T21:50:59.286208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:41.378193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482077+0000","last_change":"2024-04-26T21:50:57.141417+0000","last_active":"2024-04-26T21:50:58.482077+0000","last_peered":"2024-04-26T21:50:58.482077+0000","last_clean":"2024-04-26T21:50:58.482077+0000","last_became_active":"2024-04-26T21:50:57.141091+0000","last_became_peered":"2024-04-26T21:50:57.141091+0000","last_unstale":"2024-04-26T21:50:58.482077+0000","last_undegraded":"2024-04-26T21:50:58.482077+0000","last_fullsized":"2024-04-26T21:50:58.482077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:45.357827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482229+0000","last_change":"2024-04-26T21:50:56.364439+0000","last_active":"2024-04-26T21:50:58.482229+0000","last_peered":"2024-04-26T21:50:58.482229+0000","last_clean":"2024-04-26T21:50:58.482229+0000","last_became_active":"2024-04-26T21:50:56.364090+0000","last_became_peered":"2024-04-26T21:50:56.364090+0000","last_unstale":"2024-04-26T21:50:58.482229+0000","last_undegraded":"2024-04-26T21:50:58.482229+0000","last_fullsized":"2024-04-26T21:50:58.482229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:43.562189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483474+0000","last_change":"2024-04-26T21:50:57.145976+0000","last_active":"2024-04-26T21:50:58.483474+0000","last_peered":"2024-04-26T21:50:58.483474+0000","last_clean":"2024-04-26T21:50:58.483474+0000","last_became_active":"2024-04-26T21:50:57.145620+0000","last_became_peered":"2024-04-26T21:50:57.145620+0000","last_unstale":"2024-04-26T21:50:58.483474+0000","last_undegraded":"2024-04-26T21:50:58.483474+0000","last_fullsized":"2024-04-26T21:50:58.483474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:26.351362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136186+0000","last_change":"2024-04-26T21:50:56.365786+0000","last_active":"2024-04-26T21:50:58.136186+0000","last_peered":"2024-04-26T21:50:58.136186+0000","last_clean":"2024-04-26T21:50:58.136186+0000","last_became_active":"2024-04-26T21:50:56.365410+0000","last_became_peered":"2024-04-26T21:50:56.365410+0000","last_unstale":"2024-04-26T21:50:58.136186+0000","last_undegraded":"2024-04-26T21:50:58.136186+0000","last_fullsized":"2024-04-26T21:50:58.136186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:01.321661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359892+0000","last_change":"2024-04-26T21:50:57.137333+0000","last_active":"2024-04-26T21:50:58.359892+0000","last_peered":"2024-04-26T21:50:58.359892+0000","last_clean":"2024-04-26T21:50:58.359892+0000","last_became_active":"2024-04-26T21:50:57.136983+0000","last_became_peered":"2024-04-26T21:50:57.136983+0000","last_unstale":"2024-04-26T21:50:58.359892+0000","last_undegraded":"2024-04-26T21:50:58.359892+0000","last_fullsized":"2024-04-26T21:50:58.359892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:18.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.303964+0000","last_change":"2024-04-26T21:50:56.358340+0000","last_active":"2024-04-26T21:51:00.303964+0000","last_peered":"2024-04-26T21:51:00.303964+0000","last_clean":"2024-04-26T21:51:00.303964+0000","last_became_active":"2024-04-26T21:50:56.357972+0000","last_became_peered":"2024-04-26T21:50:56.357972+0000","last_unstale":"2024-04-26T21:51:00.303964+0000","last_undegraded":"2024-04-26T21:51:00.303964+0000","last_fullsized":"2024-04-26T21:51:00.303964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:52.256528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136077+0000","last_change":"2024-04-26T21:50:57.137027+0000","last_active":"2024-04-26T21:50:58.136077+0000","last_peered":"2024-04-26T21:50:58.136077+0000","last_clean":"2024-04-26T21:50:58.136077+0000","last_became_active":"2024-04-26T21:50:57.136400+0000","last_became_peered":"2024-04-26T21:50:57.136400+0000","last_unstale":"2024-04-26T21:50:58.136077+0000","last_undegraded":"2024-04-26T21:50:58.136077+0000","last_fullsized":"2024-04-26T21:50:58.136077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:45.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481952+0000","last_change":"2024-04-26T21:50:56.369784+0000","last_active":"2024-04-26T21:50:58.481952+0000","last_peered":"2024-04-26T21:50:58.481952+0000","last_clean":"2024-04-26T21:50:58.481952+0000","last_became_active":"2024-04-26T21:50:56.369093+0000","last_became_peered":"2024-04-26T21:50:56.369093+0000","last_unstale":"2024-04-26T21:50:58.481952+0000","last_undegraded":"2024-04-26T21:50:58.481952+0000","last_fullsized":"2024-04-26T21:50:58.481952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:27.595146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482905+0000","last_change":"2024-04-26T21:50:57.141404+0000","last_active":"2024-04-26T21:50:58.482905+0000","last_peered":"2024-04-26T21:50:58.482905+0000","last_clean":"2024-04-26T21:50:58.482905+0000","last_became_active":"2024-04-26T21:50:57.140973+0000","last_became_peered":"2024-04-26T21:50:57.140973+0000","last_unstale":"2024-04-26T21:50:58.482905+0000","last_undegraded":"2024-04-26T21:50:58.482905+0000","last_fullsized":"2024-04-26T21:50:58.482905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:09.405982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482145+0000","last_change":"2024-04-26T21:50:56.370727+0000","last_active":"2024-04-26T21:50:58.482145+0000","last_peered":"2024-04-26T21:50:58.482145+0000","last_clean":"2024-04-26T21:50:58.482145+0000","last_became_active":"2024-04-26T21:50:56.370373+0000","last_became_peered":"2024-04-26T21:50:56.370373+0000","last_unstale":"2024-04-26T21:50:58.482145+0000","last_undegraded":"2024-04-26T21:50:58.482145+0000","last_fullsized":"2024-04-26T21:50:58.482145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:12.689699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482233+0000","last_change":"2024-04-26T21:50:57.145771+0000","last_active":"2024-04-26T21:50:58.482233+0000","last_peered":"2024-04-26T21:50:58.482233+0000","last_clean":"2024-04-26T21:50:58.482233+0000","last_became_active":"2024-04-26T21:50:57.145424+0000","last_became_peered":"2024-04-26T21:50:57.145424+0000","last_unstale":"2024-04-26T21:50:58.482233+0000","last_undegraded":"2024-04-26T21:50:58.482233+0000","last_fullsized":"2024-04-26T21:50:58.482233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:10.485045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.306053+0000","last_change":"2024-04-26T21:50:56.365996+0000","last_active":"2024-04-26T21:51:00.306053+0000","last_peered":"2024-04-26T21:51:00.306053+0000","last_clean":"2024-04-26T21:51:00.306053+0000","last_became_active":"2024-04-26T21:50:56.365653+0000","last_became_peered":"2024-04-26T21:50:56.365653+0000","last_unstale":"2024-04-26T21:51:00.306053+0000","last_undegraded":"2024-04-26T21:51:00.306053+0000","last_fullsized":"2024-04-26T21:51:00.306053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:11.514305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481663+0000","last_change":"2024-04-26T21:50:57.147536+0000","last_active":"2024-04-26T21:50:58.481663+0000","last_peered":"2024-04-26T21:50:58.481663+0000","last_clean":"2024-04-26T21:50:58.481663+0000","last_became_active":"2024-04-26T21:50:57.147071+0000","last_became_peered":"2024-04-26T21:50:57.147071+0000","last_unstale":"2024-04-26T21:50:58.481663+0000","last_undegraded":"2024-04-26T21:50:58.481663+0000","last_fullsized":"2024-04-26T21:50:58.481663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:56.231509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:01.299882+0000","last_change":"2024-04-26T21:50:56.358270+0000","last_active":"2024-04-26T21:51:01.299882+0000","last_peered":"2024-04-26T21:51:01.299882+0000","last_clean":"2024-04-26T21:51:01.299882+0000","last_became_active":"2024-04-26T21:50:56.357909+0000","last_became_peered":"2024-04-26T21:50:56.357909+0000","last_unstale":"2024-04-26T21:51:01.299882+0000","last_undegraded":"2024-04-26T21:51:01.299882+0000","last_fullsized":"2024-04-26T21:51:01.299882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:45.627352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482419+0000","last_change":"2024-04-26T21:50:57.142181+0000","last_active":"2024-04-26T21:50:58.482419+0000","last_peered":"2024-04-26T21:50:58.482419+0000","last_clean":"2024-04-26T21:50:58.482419+0000","last_became_active":"2024-04-26T21:50:57.141837+0000","last_became_peered":"2024-04-26T21:50:57.141837+0000","last_unstale":"2024-04-26T21:50:58.482419+0000","last_undegraded":"2024-04-26T21:50:58.482419+0000","last_fullsized":"2024-04-26T21:50:58.482419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:42.150522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.357622+0000","last_change":"2024-04-26T21:50:56.360120+0000","last_active":"2024-04-26T21:50:58.357622+0000","last_peered":"2024-04-26T21:50:58.357622+0000","last_clean":"2024-04-26T21:50:58.357622+0000","last_became_active":"2024-04-26T21:50:56.359728+0000","last_became_peered":"2024-04-26T21:50:56.359728+0000","last_unstale":"2024-04-26T21:50:58.357622+0000","last_undegraded":"2024-04-26T21:50:58.357622+0000","last_fullsized":"2024-04-26T21:50:58.357622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:19.995617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136261+0000","last_change":"2024-04-26T21:50:57.146682+0000","last_active":"2024-04-26T21:50:58.136261+0000","last_peered":"2024-04-26T21:50:58.136261+0000","last_clean":"2024-04-26T21:50:58.136261+0000","last_became_active":"2024-04-26T21:50:57.146329+0000","last_became_peered":"2024-04-26T21:50:57.146329+0000","last_unstale":"2024-04-26T21:50:58.136261+0000","last_undegraded":"2024-04-26T21:50:58.136261+0000","last_fullsized":"2024-04-26T21:50:58.136261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:41.035526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482691+0000","last_change":"2024-04-26T21:50:56.370533+0000","last_active":"2024-04-26T21:50:58.482691+0000","last_peered":"2024-04-26T21:50:58.482691+0000","last_clean":"2024-04-26T21:50:58.482691+0000","last_became_active":"2024-04-26T21:50:56.370029+0000","last_became_peered":"2024-04-26T21:50:56.370029+0000","last_unstale":"2024-04-26T21:50:58.482691+0000","last_undegraded":"2024-04-26T21:50:58.482691+0000","last_fullsized":"2024-04-26T21:50:58.482691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.005405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136075+0000","last_change":"2024-04-26T21:50:57.142420+0000","last_active":"2024-04-26T21:50:58.136075+0000","last_peered":"2024-04-26T21:50:58.136075+0000","last_clean":"2024-04-26T21:50:58.136075+0000","last_became_active":"2024-04-26T21:50:57.142023+0000","last_became_peered":"2024-04-26T21:50:57.142023+0000","last_unstale":"2024-04-26T21:50:58.136075+0000","last_undegraded":"2024-04-26T21:50:58.136075+0000","last_fullsized":"2024-04-26T21:50:58.136075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:31.567136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481756+0000","last_change":"2024-04-26T21:50:57.139924+0000","last_active":"2024-04-26T21:50:58.481756+0000","last_peered":"2024-04-26T21:50:58.481756+0000","last_clean":"2024-04-26T21:50:58.481756+0000","last_became_active":"2024-04-26T21:50:57.139561+0000","last_became_peered":"2024-04-26T21:50:57.139561+0000","last_unstale":"2024-04-26T21:50:58.481756+0000","last_undegraded":"2024-04-26T21:50:58.481756+0000","last_fullsized":"2024-04-26T21:50:58.481756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:52.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.305663+0000","last_change":"2024-04-26T21:50:56.365256+0000","last_active":"2024-04-26T21:51:00.305663+0000","last_peered":"2024-04-26T21:51:00.305663+0000","last_clean":"2024-04-26T21:51:00.305663+0000","last_became_active":"2024-04-26T21:50:56.364516+0000","last_became_peered":"2024-04-26T21:50:56.364516+0000","last_unstale":"2024-04-26T21:51:00.305663+0000","last_undegraded":"2024-04-26T21:51:00.305663+0000","last_fullsized":"2024-04-26T21:51:00.305663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:13.550129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359682+0000","last_change":"2024-04-26T21:50:57.138177+0000","last_active":"2024-04-26T21:50:58.359682+0000","last_peered":"2024-04-26T21:50:58.359682+0000","last_clean":"2024-04-26T21:50:58.359682+0000","last_became_active":"2024-04-26T21:50:57.137725+0000","last_became_peered":"2024-04-26T21:50:57.137725+0000","last_unstale":"2024-04-26T21:50:58.359682+0000","last_undegraded":"2024-04-26T21:50:58.359682+0000","last_fullsized":"2024-04-26T21:50:58.359682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:14.113935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.285879+0000","last_change":"2024-04-26T21:50:56.358280+0000","last_active":"2024-04-26T21:50:59.285879+0000","last_peered":"2024-04-26T21:50:59.285879+0000","last_clean":"2024-04-26T21:50:59.285879+0000","last_became_active":"2024-04-26T21:50:56.357854+0000","last_became_peered":"2024-04-26T21:50:56.357854+0000","last_unstale":"2024-04-26T21:50:59.285879+0000","last_undegraded":"2024-04-26T21:50:59.285879+0000","last_fullsized":"2024-04-26T21:50:59.285879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.499124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482404+0000","last_change":"2024-04-26T21:49:42.292617+0000","last_active":"2024-04-26T21:50:58.482404+0000","last_peered":"2024-04-26T21:50:58.482404+0000","last_clean":"2024-04-26T21:50:58.482404+0000","last_became_active":"2024-04-26T21:49:42.292160+0000","last_became_peered":"2024-04-26T21:49:42.292160+0000","last_unstale":"2024-04-26T21:50:58.482404+0000","last_undegraded":"2024-04-26T21:50:58.482404+0000","last_fullsized":"2024-04-26T21:50:58.482404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_clean_scrub_stamp":"2024-04-26T21:46:13.787665+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:24.663652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136070+0000","last_change":"2024-04-26T21:50:57.145641+0000","last_active":"2024-04-26T21:50:58.136070+0000","last_peered":"2024-04-26T21:50:58.136070+0000","last_clean":"2024-04-26T21:50:58.136070+0000","last_became_active":"2024-04-26T21:50:57.145304+0000","last_became_peered":"2024-04-26T21:50:57.145304+0000","last_unstale":"2024-04-26T21:50:58.136070+0000","last_undegraded":"2024-04-26T21:50:58.136070+0000","last_fullsized":"2024-04-26T21:50:58.136070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:11.389728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.303377+0000","last_change":"2024-04-26T21:50:56.367451+0000","last_active":"2024-04-26T21:51:00.303377+0000","last_peered":"2024-04-26T21:51:00.303377+0000","last_clean":"2024-04-26T21:51:00.303377+0000","last_became_active":"2024-04-26T21:50:56.367054+0000","last_became_peered":"2024-04-26T21:50:56.367054+0000","last_unstale":"2024-04-26T21:51:00.303377+0000","last_undegraded":"2024-04-26T21:51:00.303377+0000","last_fullsized":"2024-04-26T21:51:00.303377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:04.761499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.282168+0000","last_change":"2024-04-26T21:50:56.364074+0000","last_active":"2024-04-26T21:50:59.282168+0000","last_peered":"2024-04-26T21:50:59.282168+0000","last_clean":"2024-04-26T21:50:59.282168+0000","last_became_active":"2024-04-26T21:50:56.363688+0000","last_became_peered":"2024-04-26T21:50:56.363688+0000","last_unstale":"2024-04-26T21:50:59.282168+0000","last_undegraded":"2024-04-26T21:50:59.282168+0000","last_fullsized":"2024-04-26T21:50:59.282168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:35.526125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136395+0000","last_change":"2024-04-26T21:50:57.136811+0000","last_active":"2024-04-26T21:50:58.136395+0000","last_peered":"2024-04-26T21:50:58.136395+0000","last_clean":"2024-04-26T21:50:58.136395+0000","last_became_active":"2024-04-26T21:50:57.136252+0000","last_became_peered":"2024-04-26T21:50:57.136252+0000","last_unstale":"2024-04-26T21:50:58.136395+0000","last_undegraded":"2024-04-26T21:50:58.136395+0000","last_fullsized":"2024-04-26T21:50:58.136395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:01.061283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359169+0000","last_change":"2024-04-26T21:50:56.364317+0000","last_active":"2024-04-26T21:50:58.359169+0000","last_peered":"2024-04-26T21:50:58.359169+0000","last_clean":"2024-04-26T21:50:58.359169+0000","last_became_active":"2024-04-26T21:50:56.363882+0000","last_became_peered":"2024-04-26T21:50:56.363882+0000","last_unstale":"2024-04-26T21:50:58.359169+0000","last_undegraded":"2024-04-26T21:50:58.359169+0000","last_fullsized":"2024-04-26T21:50:58.359169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:12.994293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359964+0000","last_change":"2024-04-26T21:50:57.144856+0000","last_active":"2024-04-26T21:50:58.359964+0000","last_peered":"2024-04-26T21:50:58.359964+0000","last_clean":"2024-04-26T21:50:58.359964+0000","last_became_active":"2024-04-26T21:50:57.143717+0000","last_became_peered":"2024-04-26T21:50:57.143717+0000","last_unstale":"2024-04-26T21:50:58.359964+0000","last_undegraded":"2024-04-26T21:50:58.359964+0000","last_fullsized":"2024-04-26T21:50:58.359964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:53.049243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.285053+0000","last_change":"2024-04-26T21:50:56.370284+0000","last_active":"2024-04-26T21:50:59.285053+0000","last_peered":"2024-04-26T21:50:59.285053+0000","last_clean":"2024-04-26T21:50:59.285053+0000","last_became_active":"2024-04-26T21:50:56.369738+0000","last_became_peered":"2024-04-26T21:50:56.369738+0000","last_unstale":"2024-04-26T21:50:59.285053+0000","last_undegraded":"2024-04-26T21:50:59.285053+0000","last_fullsized":"2024-04-26T21:50:59.285053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:23.390553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481955+0000","last_change":"2024-04-26T21:50:57.142687+0000","last_active":"2024-04-26T21:50:58.481955+0000","last_peered":"2024-04-26T21:50:58.481955+0000","last_clean":"2024-04-26T21:50:58.481955+0000","last_became_active":"2024-04-26T21:50:57.142365+0000","last_became_peered":"2024-04-26T21:50:57.142365+0000","last_unstale":"2024-04-26T21:50:58.481955+0000","last_undegraded":"2024-04-26T21:50:58.481955+0000","last_fullsized":"2024-04-26T21:50:58.481955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:14.261741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482415+0000","last_change":"2024-04-26T21:50:56.364163+0000","last_active":"2024-04-26T21:50:58.482415+0000","last_peered":"2024-04-26T21:50:58.482415+0000","last_clean":"2024-04-26T21:50:58.482415+0000","last_became_active":"2024-04-26T21:50:56.363729+0000","last_became_peered":"2024-04-26T21:50:56.363729+0000","last_unstale":"2024-04-26T21:50:58.482415+0000","last_undegraded":"2024-04-26T21:50:58.482415+0000","last_fullsized":"2024-04-26T21:50:58.482415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:54.620884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359539+0000","last_change":"2024-04-26T21:50:57.142692+0000","last_active":"2024-04-26T21:50:58.359539+0000","last_peered":"2024-04-26T21:50:58.359539+0000","last_clean":"2024-04-26T21:50:58.359539+0000","last_became_active":"2024-04-26T21:50:57.142137+0000","last_became_peered":"2024-04-26T21:50:57.142137+0000","last_unstale":"2024-04-26T21:50:58.359539+0000","last_undegraded":"2024-04-26T21:50:58.359539+0000","last_fullsized":"2024-04-26T21:50:58.359539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.996485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359411+0000","last_change":"2024-04-26T21:50:56.366374+0000","last_active":"2024-04-26T21:50:58.359411+0000","last_peered":"2024-04-26T21:50:58.359411+0000","last_clean":"2024-04-26T21:50:58.359411+0000","last_became_active":"2024-04-26T21:50:56.366014+0000","last_became_peered":"2024-04-26T21:50:56.366014+0000","last_unstale":"2024-04-26T21:50:58.359411+0000","last_undegraded":"2024-04-26T21:50:58.359411+0000","last_fullsized":"2024-04-26T21:50:58.359411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:41.867792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482482+0000","last_change":"2024-04-26T21:50:57.142833+0000","last_active":"2024-04-26T21:50:58.482482+0000","last_peered":"2024-04-26T21:50:58.482482+0000","last_clean":"2024-04-26T21:50:58.482482+0000","last_became_active":"2024-04-26T21:50:57.142061+0000","last_became_peered":"2024-04-26T21:50:57.142061+0000","last_unstale":"2024-04-26T21:50:58.482482+0000","last_undegraded":"2024-04-26T21:50:58.482482+0000","last_fullsized":"2024-04-26T21:50:58.482482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:33.023159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131553+0000","last_change":"2024-04-26T21:50:56.363172+0000","last_active":"2024-04-26T21:50:58.131553+0000","last_peered":"2024-04-26T21:50:58.131553+0000","last_clean":"2024-04-26T21:50:58.131553+0000","last_became_active":"2024-04-26T21:50:56.362782+0000","last_became_peered":"2024-04-26T21:50:56.362782+0000","last_unstale":"2024-04-26T21:50:58.131553+0000","last_undegraded":"2024-04-26T21:50:58.131553+0000","last_fullsized":"2024-04-26T21:50:58.131553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:37.183497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359658+0000","last_change":"2024-04-26T21:50:57.142088+0000","last_active":"2024-04-26T21:50:58.359658+0000","last_peered":"2024-04-26T21:50:58.359658+0000","last_clean":"2024-04-26T21:50:58.359658+0000","last_became_active":"2024-04-26T21:50:57.141685+0000","last_became_peered":"2024-04-26T21:50:57.141685+0000","last_unstale":"2024-04-26T21:50:58.359658+0000","last_undegraded":"2024-04-26T21:50:58.359658+0000","last_fullsized":"2024-04-26T21:50:58.359658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:24.033787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.291885+0000","last_change":"2024-04-26T21:50:56.371717+0000","last_active":"2024-04-26T21:50:59.291885+0000","last_peered":"2024-04-26T21:50:59.291885+0000","last_clean":"2024-04-26T21:50:59.291885+0000","last_became_active":"2024-04-26T21:50:56.370930+0000","last_became_peered":"2024-04-26T21:50:56.370930+0000","last_unstale":"2024-04-26T21:50:59.291885+0000","last_undegraded":"2024-04-26T21:50:59.291885+0000","last_fullsized":"2024-04-26T21:50:59.291885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.977392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482853+0000","last_change":"2024-04-26T21:50:57.139671+0000","last_active":"2024-04-26T21:50:58.482853+0000","last_peered":"2024-04-26T21:50:58.482853+0000","last_clean":"2024-04-26T21:50:58.482853+0000","last_became_active":"2024-04-26T21:50:57.139245+0000","last_became_peered":"2024-04-26T21:50:57.139245+0000","last_unstale":"2024-04-26T21:50:58.482853+0000","last_undegraded":"2024-04-26T21:50:58.482853+0000","last_fullsized":"2024-04-26T21:50:58.482853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:26.903905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.306177+0000","last_change":"2024-04-26T21:50:56.369836+0000","last_active":"2024-04-26T21:51:00.306177+0000","last_peered":"2024-04-26T21:51:00.306177+0000","last_clean":"2024-04-26T21:51:00.306177+0000","last_became_active":"2024-04-26T21:50:56.369522+0000","last_became_peered":"2024-04-26T21:50:56.369522+0000","last_unstale":"2024-04-26T21:51:00.306177+0000","last_undegraded":"2024-04-26T21:51:00.306177+0000","last_fullsized":"2024-04-26T21:51:00.306177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:40.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483494+0000","last_change":"2024-04-26T21:50:57.138551+0000","last_active":"2024-04-26T21:50:58.483494+0000","last_peered":"2024-04-26T21:50:58.483494+0000","last_clean":"2024-04-26T21:50:58.483494+0000","last_became_active":"2024-04-26T21:50:57.138187+0000","last_became_peered":"2024-04-26T21:50:57.138187+0000","last_unstale":"2024-04-26T21:50:58.483494+0000","last_undegraded":"2024-04-26T21:50:58.483494+0000","last_fullsized":"2024-04-26T21:50:58.483494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:46.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482101+0000","last_change":"2024-04-26T21:50:56.362009+0000","last_active":"2024-04-26T21:50:58.482101+0000","last_peered":"2024-04-26T21:50:58.482101+0000","last_clean":"2024-04-26T21:50:58.482101+0000","last_became_active":"2024-04-26T21:50:56.361646+0000","last_became_peered":"2024-04-26T21:50:56.361646+0000","last_unstale":"2024-04-26T21:50:58.482101+0000","last_undegraded":"2024-04-26T21:50:58.482101+0000","last_fullsized":"2024-04-26T21:50:58.482101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:25.316195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482479+0000","last_change":"2024-04-26T21:50:57.139983+0000","last_active":"2024-04-26T21:50:58.482479+0000","last_peered":"2024-04-26T21:50:58.482479+0000","last_clean":"2024-04-26T21:50:58.482479+0000","last_became_active":"2024-04-26T21:50:57.138426+0000","last_became_peered":"2024-04-26T21:50:57.138426+0000","last_unstale":"2024-04-26T21:50:58.482479+0000","last_undegraded":"2024-04-26T21:50:58.482479+0000","last_fullsized":"2024-04-26T21:50:58.482479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:53.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136164+0000","last_change":"2024-04-26T21:50:56.369527+0000","last_active":"2024-04-26T21:50:58.136164+0000","last_peered":"2024-04-26T21:50:58.136164+0000","last_clean":"2024-04-26T21:50:58.136164+0000","last_became_active":"2024-04-26T21:50:56.369168+0000","last_became_peered":"2024-04-26T21:50:56.369168+0000","last_unstale":"2024-04-26T21:50:58.136164+0000","last_undegraded":"2024-04-26T21:50:58.136164+0000","last_fullsized":"2024-04-26T21:50:58.136164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:38.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483267+0000","last_change":"2024-04-26T21:50:57.137957+0000","last_active":"2024-04-26T21:50:58.483267+0000","last_peered":"2024-04-26T21:50:58.483267+0000","last_clean":"2024-04-26T21:50:58.483267+0000","last_became_active":"2024-04-26T21:50:57.137595+0000","last_became_peered":"2024-04-26T21:50:57.137595+0000","last_unstale":"2024-04-26T21:50:58.483267+0000","last_undegraded":"2024-04-26T21:50:58.483267+0000","last_fullsized":"2024-04-26T21:50:58.483267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:09.355937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482430+0000","last_change":"2024-04-26T21:50:57.143098+0000","last_active":"2024-04-26T21:50:58.482430+0000","last_peered":"2024-04-26T21:50:58.482430+0000","last_clean":"2024-04-26T21:50:58.482430+0000","last_became_active":"2024-04-26T21:50:57.142524+0000","last_became_peered":"2024-04-26T21:50:57.142524+0000","last_unstale":"2024-04-26T21:50:58.482430+0000","last_undegraded":"2024-04-26T21:50:58.482430+0000","last_fullsized":"2024-04-26T21:50:58.482430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:33.333382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131345+0000","last_change":"2024-04-26T21:50:57.143123+0000","last_active":"2024-04-26T21:50:58.131345+0000","last_peered":"2024-04-26T21:50:58.131345+0000","last_clean":"2024-04-26T21:50:58.131345+0000","last_became_active":"2024-04-26T21:50:57.142805+0000","last_became_peered":"2024-04-26T21:50:57.142805+0000","last_unstale":"2024-04-26T21:50:58.131345+0000","last_undegraded":"2024-04-26T21:50:58.131345+0000","last_fullsized":"2024-04-26T21:50:58.131345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:19.330202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131371+0000","last_change":"2024-04-26T21:50:57.140268+0000","last_active":"2024-04-26T21:50:58.131371+0000","last_peered":"2024-04-26T21:50:58.131371+0000","last_clean":"2024-04-26T21:50:58.131371+0000","last_became_active":"2024-04-26T21:50:57.139927+0000","last_became_peered":"2024-04-26T21:50:57.139927+0000","last_unstale":"2024-04-26T21:50:58.131371+0000","last_undegraded":"2024-04-26T21:50:58.131371+0000","last_fullsized":"2024-04-26T21:50:58.131371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:01.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481790+0000","last_change":"2024-04-26T21:50:57.143564+0000","last_active":"2024-04-26T21:50:58.481790+0000","last_peered":"2024-04-26T21:50:58.481790+0000","last_clean":"2024-04-26T21:50:58.481790+0000","last_became_active":"2024-04-26T21:50:57.142740+0000","last_became_peered":"2024-04-26T21:50:57.142740+0000","last_unstale":"2024-04-26T21:50:58.481790+0000","last_undegraded":"2024-04-26T21:50:58.481790+0000","last_fullsized":"2024-04-26T21:50:58.481790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:58.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136433+0000","last_change":"2024-04-26T21:50:57.146883+0000","last_active":"2024-04-26T21:50:58.136433+0000","last_peered":"2024-04-26T21:50:58.136433+0000","last_clean":"2024-04-26T21:50:58.136433+0000","last_became_active":"2024-04-26T21:50:57.146540+0000","last_became_peered":"2024-04-26T21:50:57.146540+0000","last_unstale":"2024-04-26T21:50:58.136433+0000","last_undegraded":"2024-04-26T21:50:58.136433+0000","last_fullsized":"2024-04-26T21:50:58.136433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.787618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482576+0000","last_change":"2024-04-26T21:50:57.136938+0000","last_active":"2024-04-26T21:50:58.482576+0000","last_peered":"2024-04-26T21:50:58.482576+0000","last_clean":"2024-04-26T21:50:58.482576+0000","last_became_active":"2024-04-26T21:50:57.136474+0000","last_became_peered":"2024-04-26T21:50:57.136474+0000","last_unstale":"2024-04-26T21:50:58.482576+0000","last_undegraded":"2024-04-26T21:50:58.482576+0000","last_fullsized":"2024-04-26T21:50:58.482576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:42.534342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131777+0000","last_change":"2024-04-26T21:50:57.150958+0000","last_active":"2024-04-26T21:50:58.131777+0000","last_peered":"2024-04-26T21:50:58.131777+0000","last_clean":"2024-04-26T21:50:58.131777+0000","last_became_active":"2024-04-26T21:50:57.150593+0000","last_became_peered":"2024-04-26T21:50:57.150593+0000","last_unstale":"2024-04-26T21:50:58.131777+0000","last_undegraded":"2024-04-26T21:50:58.131777+0000","last_fullsized":"2024-04-26T21:50:58.131777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:26.583983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359926+0000","last_change":"2024-04-26T21:50:57.147766+0000","last_active":"2024-04-26T21:50:58.359926+0000","last_peered":"2024-04-26T21:50:58.359926+0000","last_clean":"2024-04-26T21:50:58.359926+0000","last_became_active":"2024-04-26T21:50:57.147442+0000","last_became_peered":"2024-04-26T21:50:57.147442+0000","last_unstale":"2024-04-26T21:50:58.359926+0000","last_undegraded":"2024-04-26T21:50:58.359926+0000","last_fullsized":"2024-04-26T21:50:58.359926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.489117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481685+0000","last_change":"2024-04-26T21:50:57.138357+0000","last_active":"2024-04-26T21:50:58.481685+0000","last_peered":"2024-04-26T21:50:58.481685+0000","last_clean":"2024-04-26T21:50:58.481685+0000","last_became_active":"2024-04-26T21:50:57.135570+0000","last_became_peered":"2024-04-26T21:50:57.135570+0000","last_unstale":"2024-04-26T21:50:58.481685+0000","last_undegraded":"2024-04-26T21:50:58.481685+0000","last_fullsized":"2024-04-26T21:50:58.481685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:17.206623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359485+0000","last_change":"2024-04-26T21:50:57.142804+0000","last_active":"2024-04-26T21:50:58.359485+0000","last_peered":"2024-04-26T21:50:58.359485+0000","last_clean":"2024-04-26T21:50:58.359485+0000","last_became_active":"2024-04-26T21:50:57.141902+0000","last_became_peered":"2024-04-26T21:50:57.141902+0000","last_unstale":"2024-04-26T21:50:58.359485+0000","last_undegraded":"2024-04-26T21:50:58.359485+0000","last_fullsized":"2024-04-26T21:50:58.359485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:37.590730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482798+0000","last_change":"2024-04-26T21:50:57.138637+0000","last_active":"2024-04-26T21:50:58.482798+0000","last_peered":"2024-04-26T21:50:58.482798+0000","last_clean":"2024-04-26T21:50:58.482798+0000","last_became_active":"2024-04-26T21:50:57.138295+0000","last_became_peered":"2024-04-26T21:50:57.138295+0000","last_unstale":"2024-04-26T21:50:58.482798+0000","last_undegraded":"2024-04-26T21:50:58.482798+0000","last_fullsized":"2024-04-26T21:50:58.482798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:19.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136322+0000","last_change":"2024-04-26T21:50:57.137377+0000","last_active":"2024-04-26T21:50:58.136322+0000","last_peered":"2024-04-26T21:50:58.136322+0000","last_clean":"2024-04-26T21:50:58.136322+0000","last_became_active":"2024-04-26T21:50:57.136893+0000","last_became_peered":"2024-04-26T21:50:57.136893+0000","last_unstale":"2024-04-26T21:50:58.136322+0000","last_undegraded":"2024-04-26T21:50:58.136322+0000","last_fullsized":"2024-04-26T21:50:58.136322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:44.257438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482573+0000","last_change":"2024-04-26T21:50:57.141439+0000","last_active":"2024-04-26T21:50:58.482573+0000","last_peered":"2024-04-26T21:50:58.482573+0000","last_clean":"2024-04-26T21:50:58.482573+0000","last_became_active":"2024-04-26T21:50:57.141060+0000","last_became_peered":"2024-04-26T21:50:57.141060+0000","last_unstale":"2024-04-26T21:50:58.482573+0000","last_undegraded":"2024-04-26T21:50:58.482573+0000","last_fullsized":"2024-04-26T21:50:58.482573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:27.174348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250381,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3216,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":48531,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":1022125,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":1030681,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":431563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":460766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":430055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":1020501,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":39543,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:51:04.110 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:51:04.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 -- 172.21.15.55:0/3101467423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 msgr2=0x7f8e58102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:04.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/3101467423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 0x7f8e58102ee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009900 tx=0x7f8e4c02a410 comp rx=0 tx=0).stop 2024-04-26T21:51:04.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 -- 172.21.15.55:0/3101467423 shutdown_connections 2024-04-26T21:51:04.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/3101467423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e58103420 0x7f8e58109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/3101467423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 0x7f8e58102ee0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/3101467423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e58101870 0x7f8e58101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 -- 172.21.15.55:0/3101467423 >> 172.21.15.55:0/3101467423 conn(0x7f8e580fcde0 msgr2=0x7f8e580ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 -- 172.21.15.55:0/3101467423 shutdown_connections 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.248+0000 7f8e5cef8640 1 -- 172.21.15.55:0/3101467423 wait complete. 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e5cef8640 1 Processor -- start 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e5cef8640 1 -- start start 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e5cef8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e58101870 0x7f8e5819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e5cef8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 0x7f8e5819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e5cef8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e58103420 0x7f8e581a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e5cef8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5810d380 con 0x7f8e58102a60 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e5cef8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5810d200 con 0x7f8e58101870 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e5cef8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5810d500 con 0x7f8e58103420 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e55d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 0x7f8e5819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e55d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 0x7f8e5819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45472/0 (socket says 172.21.15.55:45472) 2024-04-26T21:51:04.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.249+0000 7f8e55d9b640 1 -- 172.21.15.55:0/2360588281 learned_addr learned my addr 172.21.15.55:0/2360588281 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e5659c640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e58101870 0x7f8e5819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e56d9d640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e58103420 0x7f8e581a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e55d9b640 1 -- 172.21.15.55:0/2360588281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e58103420 msgr2=0x7f8e581a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e55d9b640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e58103420 0x7f8e581a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e55d9b640 1 -- 172.21.15.55:0/2360588281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e58101870 msgr2=0x7f8e5819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e55d9b640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e58101870 0x7f8e5819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e55d9b640 1 -- 172.21.15.55:0/2360588281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e581a29f0 con 0x7f8e58102a60 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e55d9b640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 0x7f8e5819b310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c005940 tx=0x7f8e4c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e477fe640 1 -- 172.21.15.55:0/2360588281 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c03f030 con 0x7f8e58102a60 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e477fe640 1 -- 172.21.15.55:0/2360588281 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e4c003aa0 con 0x7f8e58102a60 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e477fe640 1 -- 172.21.15.55:0/2360588281 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c035070 con 0x7f8e58102a60 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e581a2c80 con 0x7f8e58102a60 2024-04-26T21:51:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.250+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581a3170 con 0x7f8e58102a60 2024-04-26T21:51:04.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.251+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e581a3340 con 0x7f8e58102a60 2024-04-26T21:51:04.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.253+0000 7f8e477fe640 1 -- 172.21.15.55:0/2360588281 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e4c042030 con 0x7f8e58102a60 2024-04-26T21:51:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.254+0000 7f8e477fe640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.254+0000 7f8e5659c640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.254+0000 7f8e477fe640 1 -- 172.21.15.55:0/2360588281 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8e4c0f0f70 con 0x7f8e58102a60 2024-04-26T21:51:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.254+0000 7f8e5659c640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e400a3bb0 0x7f8e400a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e48005fd0 tx=0x7f8e4800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:04.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.257+0000 7f8e477fe640 1 -- 172.21.15.55:0/2360588281 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8e4c0bda00 con 0x7f8e58102a60 2024-04-26T21:51:04.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.333+0000 7f8e477fe640 1 -- 172.21.15.55:0/2360588281 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8e4c0b9970 con 0x7f8e58102a60 2024-04-26T21:51:04.390 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:04 smithi055 ceph-mon[29221]: pgmap v292: 97 pgs: 97 active+clean; 580 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-26T21:51:04.390 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:04 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2061150727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:51:04.390 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:04 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4022683302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:04.391 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:04 smithi055 ceph-mon[29221]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:51:04.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.447+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e5806a220 con 0x7f8e400a3bb0 2024-04-26T21:51:04.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.455+0000 7f8e477fe640 1 -- 172.21.15.55:0/2360588281 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258898 (secure 0 0 0) 0x7f8e5806a220 con 0x7f8e400a3bb0 2024-04-26T21:51:04.458 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:51:04.459 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e400a3bb0 msgr2=0x7f8e400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e400a3bb0 0x7f8e400a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8e48005fd0 tx=0x7f8e4800b410 comp rx=0 tx=0).stop 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 msgr2=0x7f8e5819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 0x7f8e5819b310 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c005940 tx=0x7f8e4c031a50 comp rx=0 tx=0).stop 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 shutdown_connections 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e58103420 0x7f8e581a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e58102a60 0x7f8e5819b310 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 --2- 172.21.15.55:0/2360588281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e58101870 0x7f8e5819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.458+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 >> 172.21.15.55:0/2360588281 conn(0x7f8e580fcde0 msgr2=0x7f8e58069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.459+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 shutdown_connections 2024-04-26T21:51:04.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:04.459+0000 7f8e5cef8640 1 -- 172.21.15.55:0/2360588281 wait complete. 2024-04-26T21:51:04.470 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-04-26T21:51:03.342584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593584,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":133,"num_read_kb":116,"num_write":271,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351848,"kb_used_data":10988,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589976,"statfs":{"total":1151940427776,"available":1151580135424,"internally_reserved":0,"allocated":11251712,"data_stored":7030309,"data_compressed":109407,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3216,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.995087"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482830+0000","last_change":"2024-04-26T21:50:57.139486+0000","last_active":"2024-04-26T21:50:58.482830+0000","last_peered":"2024-04-26T21:50:58.482830+0000","last_clean":"2024-04-26T21:50:58.482830+0000","last_became_active":"2024-04-26T21:50:57.139133+0000","last_became_peered":"2024-04-26T21:50:57.139133+0000","last_unstale":"2024-04-26T21:50:58.482830+0000","last_undegraded":"2024-04-26T21:50:58.482830+0000","last_fullsized":"2024-04-26T21:50:58.482830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:33.211627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481810+0000","last_change":"2024-04-26T21:50:57.138519+0000","last_active":"2024-04-26T21:50:58.481810+0000","last_peered":"2024-04-26T21:50:58.481810+0000","last_clean":"2024-04-26T21:50:58.481810+0000","last_became_active":"2024-04-26T21:50:57.138131+0000","last_became_peered":"2024-04-26T21:50:57.138131+0000","last_unstale":"2024-04-26T21:50:58.481810+0000","last_undegraded":"2024-04-26T21:50:58.481810+0000","last_fullsized":"2024-04-26T21:50:58.481810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:34.349694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481810+0000","last_change":"2024-04-26T21:50:57.146584+0000","last_active":"2024-04-26T21:50:58.481810+0000","last_peered":"2024-04-26T21:50:58.481810+0000","last_clean":"2024-04-26T21:50:58.481810+0000","last_became_active":"2024-04-26T21:50:57.146262+0000","last_became_peered":"2024-04-26T21:50:57.146262+0000","last_unstale":"2024-04-26T21:50:58.481810+0000","last_undegraded":"2024-04-26T21:50:58.481810+0000","last_fullsized":"2024-04-26T21:50:58.481810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:49.946882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483645+0000","last_change":"2024-04-26T21:50:57.144526+0000","last_active":"2024-04-26T21:50:58.483645+0000","last_peered":"2024-04-26T21:50:58.483645+0000","last_clean":"2024-04-26T21:50:58.483645+0000","last_became_active":"2024-04-26T21:50:57.144052+0000","last_became_peered":"2024-04-26T21:50:57.144052+0000","last_unstale":"2024-04-26T21:50:58.483645+0000","last_undegraded":"2024-04-26T21:50:58.483645+0000","last_fullsized":"2024-04-26T21:50:58.483645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:28:16.499874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359621+0000","last_change":"2024-04-26T21:50:57.136292+0000","last_active":"2024-04-26T21:50:58.359621+0000","last_peered":"2024-04-26T21:50:58.359621+0000","last_clean":"2024-04-26T21:50:58.359621+0000","last_became_active":"2024-04-26T21:50:57.135936+0000","last_became_peered":"2024-04-26T21:50:57.135936+0000","last_unstale":"2024-04-26T21:50:58.359621+0000","last_undegraded":"2024-04-26T21:50:58.359621+0000","last_fullsized":"2024-04-26T21:50:58.359621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:35.415440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.135948+0000","last_change":"2024-04-26T21:50:57.147764+0000","last_active":"2024-04-26T21:50:58.135948+0000","last_peered":"2024-04-26T21:50:58.135948+0000","last_clean":"2024-04-26T21:50:58.135948+0000","last_became_active":"2024-04-26T21:50:57.147034+0000","last_became_peered":"2024-04-26T21:50:57.147034+0000","last_unstale":"2024-04-26T21:50:58.135948+0000","last_undegraded":"2024-04-26T21:50:58.135948+0000","last_fullsized":"2024-04-26T21:50:58.135948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:44.833213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482822+0000","last_change":"2024-04-26T21:50:57.144813+0000","last_active":"2024-04-26T21:50:58.482822+0000","last_peered":"2024-04-26T21:50:58.482822+0000","last_clean":"2024-04-26T21:50:58.482822+0000","last_became_active":"2024-04-26T21:50:57.144442+0000","last_became_peered":"2024-04-26T21:50:57.144442+0000","last_unstale":"2024-04-26T21:50:58.482822+0000","last_undegraded":"2024-04-26T21:50:58.482822+0000","last_fullsized":"2024-04-26T21:50:58.482822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:59.032263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359358+0000","last_change":"2024-04-26T21:50:57.145958+0000","last_active":"2024-04-26T21:50:58.359358+0000","last_peered":"2024-04-26T21:50:58.359358+0000","last_clean":"2024-04-26T21:50:58.359358+0000","last_became_active":"2024-04-26T21:50:57.145623+0000","last_became_peered":"2024-04-26T21:50:57.145623+0000","last_unstale":"2024-04-26T21:50:58.359358+0000","last_undegraded":"2024-04-26T21:50:58.359358+0000","last_fullsized":"2024-04-26T21:50:58.359358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.716079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483712+0000","last_change":"2024-04-26T21:50:57.132957+0000","last_active":"2024-04-26T21:50:58.483712+0000","last_peered":"2024-04-26T21:50:58.483712+0000","last_clean":"2024-04-26T21:50:58.483712+0000","last_became_active":"2024-04-26T21:50:57.132606+0000","last_became_peered":"2024-04-26T21:50:57.132606+0000","last_unstale":"2024-04-26T21:50:58.483712+0000","last_undegraded":"2024-04-26T21:50:58.483712+0000","last_fullsized":"2024-04-26T21:50:58.483712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:00.994370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136350+0000","last_change":"2024-04-26T21:50:57.150598+0000","last_active":"2024-04-26T21:50:58.136350+0000","last_peered":"2024-04-26T21:50:58.136350+0000","last_clean":"2024-04-26T21:50:58.136350+0000","last_became_active":"2024-04-26T21:50:57.149697+0000","last_became_peered":"2024-04-26T21:50:57.149697+0000","last_unstale":"2024-04-26T21:50:58.136350+0000","last_undegraded":"2024-04-26T21:50:58.136350+0000","last_fullsized":"2024-04-26T21:50:58.136350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.621137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483258+0000","last_change":"2024-04-26T21:50:57.147264+0000","last_active":"2024-04-26T21:50:58.483258+0000","last_peered":"2024-04-26T21:50:58.483258+0000","last_clean":"2024-04-26T21:50:58.483258+0000","last_became_active":"2024-04-26T21:50:57.146888+0000","last_became_peered":"2024-04-26T21:50:57.146888+0000","last_unstale":"2024-04-26T21:50:58.483258+0000","last_undegraded":"2024-04-26T21:50:58.483258+0000","last_fullsized":"2024-04-26T21:50:58.483258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:49.584327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483424+0000","last_change":"2024-04-26T21:50:57.142005+0000","last_active":"2024-04-26T21:50:58.483424+0000","last_peered":"2024-04-26T21:50:58.483424+0000","last_clean":"2024-04-26T21:50:58.483424+0000","last_became_active":"2024-04-26T21:50:57.141605+0000","last_became_peered":"2024-04-26T21:50:57.141605+0000","last_unstale":"2024-04-26T21:50:58.483424+0000","last_undegraded":"2024-04-26T21:50:58.483424+0000","last_fullsized":"2024-04-26T21:50:58.483424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:43.485295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136333+0000","last_change":"2024-04-26T21:50:57.138049+0000","last_active":"2024-04-26T21:50:58.136333+0000","last_peered":"2024-04-26T21:50:58.136333+0000","last_clean":"2024-04-26T21:50:58.136333+0000","last_became_active":"2024-04-26T21:50:57.137717+0000","last_became_peered":"2024-04-26T21:50:57.137717+0000","last_unstale":"2024-04-26T21:50:58.136333+0000","last_undegraded":"2024-04-26T21:50:58.136333+0000","last_fullsized":"2024-04-26T21:50:58.136333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:28.918224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481605+0000","last_change":"2024-04-26T21:50:57.139183+0000","last_active":"2024-04-26T21:50:58.481605+0000","last_peered":"2024-04-26T21:50:58.481605+0000","last_clean":"2024-04-26T21:50:58.481605+0000","last_became_active":"2024-04-26T21:50:57.138759+0000","last_became_peered":"2024-04-26T21:50:57.138759+0000","last_unstale":"2024-04-26T21:50:58.481605+0000","last_undegraded":"2024-04-26T21:50:58.481605+0000","last_fullsized":"2024-04-26T21:50:58.481605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:12.003122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481632+0000","last_change":"2024-04-26T21:50:57.143110+0000","last_active":"2024-04-26T21:50:58.481632+0000","last_peered":"2024-04-26T21:50:58.481632+0000","last_clean":"2024-04-26T21:50:58.481632+0000","last_became_active":"2024-04-26T21:50:57.142754+0000","last_became_peered":"2024-04-26T21:50:57.142754+0000","last_unstale":"2024-04-26T21:50:58.481632+0000","last_undegraded":"2024-04-26T21:50:58.481632+0000","last_fullsized":"2024-04-26T21:50:58.481632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:08.321000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482284+0000","last_change":"2024-04-26T21:50:57.146514+0000","last_active":"2024-04-26T21:50:58.482284+0000","last_peered":"2024-04-26T21:50:58.482284+0000","last_clean":"2024-04-26T21:50:58.482284+0000","last_became_active":"2024-04-26T21:50:57.146046+0000","last_became_peered":"2024-04-26T21:50:57.146046+0000","last_unstale":"2024-04-26T21:50:58.482284+0000","last_undegraded":"2024-04-26T21:50:58.482284+0000","last_fullsized":"2024-04-26T21:50:58.482284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:21.130220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483000+0000","last_change":"2024-04-26T21:50:57.140492+0000","last_active":"2024-04-26T21:50:58.483000+0000","last_peered":"2024-04-26T21:50:58.483000+0000","last_clean":"2024-04-26T21:50:58.483000+0000","last_became_active":"2024-04-26T21:50:57.140147+0000","last_became_peered":"2024-04-26T21:50:57.140147+0000","last_unstale":"2024-04-26T21:50:58.483000+0000","last_undegraded":"2024-04-26T21:50:58.483000+0000","last_fullsized":"2024-04-26T21:50:58.483000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:21.679202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.357924+0000","last_change":"2024-04-26T21:50:57.140992+0000","last_active":"2024-04-26T21:50:58.357924+0000","last_peered":"2024-04-26T21:50:58.357924+0000","last_clean":"2024-04-26T21:50:58.357924+0000","last_became_active":"2024-04-26T21:50:57.140644+0000","last_became_peered":"2024-04-26T21:50:57.140644+0000","last_unstale":"2024-04-26T21:50:58.357924+0000","last_undegraded":"2024-04-26T21:50:58.357924+0000","last_fullsized":"2024-04-26T21:50:58.357924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131570+0000","last_change":"2024-04-26T21:50:57.141242+0000","last_active":"2024-04-26T21:50:58.131570+0000","last_peered":"2024-04-26T21:50:58.131570+0000","last_clean":"2024-04-26T21:50:58.131570+0000","last_became_active":"2024-04-26T21:50:57.140909+0000","last_became_peered":"2024-04-26T21:50:57.140909+0000","last_unstale":"2024-04-26T21:50:58.131570+0000","last_undegraded":"2024-04-26T21:50:58.131570+0000","last_fullsized":"2024-04-26T21:50:58.131570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:21.996984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.357655+0000","last_change":"2024-04-26T21:50:56.371876+0000","last_active":"2024-04-26T21:50:58.357655+0000","last_peered":"2024-04-26T21:50:58.357655+0000","last_clean":"2024-04-26T21:50:58.357655+0000","last_became_active":"2024-04-26T21:50:56.371509+0000","last_became_peered":"2024-04-26T21:50:56.371509+0000","last_unstale":"2024-04-26T21:50:58.357655+0000","last_undegraded":"2024-04-26T21:50:58.357655+0000","last_fullsized":"2024-04-26T21:50:58.357655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:50.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359780+0000","last_change":"2024-04-26T21:50:57.145654+0000","last_active":"2024-04-26T21:50:58.359780+0000","last_peered":"2024-04-26T21:50:58.359780+0000","last_clean":"2024-04-26T21:50:58.359780+0000","last_became_active":"2024-04-26T21:50:57.145355+0000","last_became_peered":"2024-04-26T21:50:57.145355+0000","last_unstale":"2024-04-26T21:50:58.359780+0000","last_undegraded":"2024-04-26T21:50:58.359780+0000","last_fullsized":"2024-04-26T21:50:58.359780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:51.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359873+0000","last_change":"2024-04-26T21:50:56.362758+0000","last_active":"2024-04-26T21:50:58.359873+0000","last_peered":"2024-04-26T21:50:58.359873+0000","last_clean":"2024-04-26T21:50:58.359873+0000","last_became_active":"2024-04-26T21:50:56.361338+0000","last_became_peered":"2024-04-26T21:50:56.361338+0000","last_unstale":"2024-04-26T21:50:58.359873+0000","last_undegraded":"2024-04-26T21:50:58.359873+0000","last_fullsized":"2024-04-26T21:50:58.359873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:44.603426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482048+0000","last_change":"2024-04-26T21:50:57.150701+0000","last_active":"2024-04-26T21:50:58.482048+0000","last_peered":"2024-04-26T21:50:58.482048+0000","last_clean":"2024-04-26T21:50:58.482048+0000","last_became_active":"2024-04-26T21:50:57.150388+0000","last_became_peered":"2024-04-26T21:50:57.150388+0000","last_unstale":"2024-04-26T21:50:58.482048+0000","last_undegraded":"2024-04-26T21:50:58.482048+0000","last_fullsized":"2024-04-26T21:50:58.482048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:43:07.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.304006+0000","last_change":"2024-04-26T21:50:56.371379+0000","last_active":"2024-04-26T21:51:00.304006+0000","last_peered":"2024-04-26T21:51:00.304006+0000","last_clean":"2024-04-26T21:51:00.304006+0000","last_became_active":"2024-04-26T21:50:56.371025+0000","last_became_peered":"2024-04-26T21:50:56.371025+0000","last_unstale":"2024-04-26T21:51:00.304006+0000","last_undegraded":"2024-04-26T21:51:00.304006+0000","last_fullsized":"2024-04-26T21:51:00.304006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:00.633393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483023+0000","last_change":"2024-04-26T21:50:57.144777+0000","last_active":"2024-04-26T21:50:58.483023+0000","last_peered":"2024-04-26T21:50:58.483023+0000","last_clean":"2024-04-26T21:50:58.483023+0000","last_became_active":"2024-04-26T21:50:57.144431+0000","last_became_peered":"2024-04-26T21:50:57.144431+0000","last_unstale":"2024-04-26T21:50:58.483023+0000","last_undegraded":"2024-04-26T21:50:58.483023+0000","last_fullsized":"2024-04-26T21:50:58.483023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:42.164240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359838+0000","last_change":"2024-04-26T21:50:56.360843+0000","last_active":"2024-04-26T21:50:58.359838+0000","last_peered":"2024-04-26T21:50:58.359838+0000","last_clean":"2024-04-26T21:50:58.359838+0000","last_became_active":"2024-04-26T21:50:56.360440+0000","last_became_peered":"2024-04-26T21:50:56.360440+0000","last_unstale":"2024-04-26T21:50:58.359838+0000","last_undegraded":"2024-04-26T21:50:58.359838+0000","last_fullsized":"2024-04-26T21:50:58.359838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:47.585306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482223+0000","last_change":"2024-04-26T21:50:57.146449+0000","last_active":"2024-04-26T21:50:58.482223+0000","last_peered":"2024-04-26T21:50:58.482223+0000","last_clean":"2024-04-26T21:50:58.482223+0000","last_became_active":"2024-04-26T21:50:57.146128+0000","last_became_peered":"2024-04-26T21:50:57.146128+0000","last_unstale":"2024-04-26T21:50:58.482223+0000","last_undegraded":"2024-04-26T21:50:58.482223+0000","last_fullsized":"2024-04-26T21:50:58.482223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:09.165868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481908+0000","last_change":"2024-04-26T21:50:56.371455+0000","last_active":"2024-04-26T21:50:58.481908+0000","last_peered":"2024-04-26T21:50:58.481908+0000","last_clean":"2024-04-26T21:50:58.481908+0000","last_became_active":"2024-04-26T21:50:56.371096+0000","last_became_peered":"2024-04-26T21:50:56.371096+0000","last_unstale":"2024-04-26T21:50:58.481908+0000","last_undegraded":"2024-04-26T21:50:58.481908+0000","last_fullsized":"2024-04-26T21:50:58.481908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:37.167965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359634+0000","last_change":"2024-04-26T21:50:57.138817+0000","last_active":"2024-04-26T21:50:58.359634+0000","last_peered":"2024-04-26T21:50:58.359634+0000","last_clean":"2024-04-26T21:50:58.359634+0000","last_became_active":"2024-04-26T21:50:57.138489+0000","last_became_peered":"2024-04-26T21:50:57.138489+0000","last_unstale":"2024-04-26T21:50:58.359634+0000","last_undegraded":"2024-04-26T21:50:58.359634+0000","last_fullsized":"2024-04-26T21:50:58.359634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:56.072063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.135787+0000","last_change":"2024-04-26T21:50:56.366990+0000","last_active":"2024-04-26T21:50:58.135787+0000","last_peered":"2024-04-26T21:50:58.135787+0000","last_clean":"2024-04-26T21:50:58.135787+0000","last_became_active":"2024-04-26T21:50:56.366608+0000","last_became_peered":"2024-04-26T21:50:56.366608+0000","last_unstale":"2024-04-26T21:50:58.135787+0000","last_undegraded":"2024-04-26T21:50:58.135787+0000","last_fullsized":"2024-04-26T21:50:58.135787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:19.951038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482126+0000","last_change":"2024-04-26T21:50:57.147601+0000","last_active":"2024-04-26T21:50:58.482126+0000","last_peered":"2024-04-26T21:50:58.482126+0000","last_clean":"2024-04-26T21:50:58.482126+0000","last_became_active":"2024-04-26T21:50:57.147197+0000","last_became_peered":"2024-04-26T21:50:57.147197+0000","last_unstale":"2024-04-26T21:50:58.482126+0000","last_undegraded":"2024-04-26T21:50:58.482126+0000","last_fullsized":"2024-04-26T21:50:58.482126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:32:47.898167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.305012+0000","last_change":"2024-04-26T21:50:56.362272+0000","last_active":"2024-04-26T21:51:00.305012+0000","last_peered":"2024-04-26T21:51:00.305012+0000","last_clean":"2024-04-26T21:51:00.305012+0000","last_became_active":"2024-04-26T21:50:56.361941+0000","last_became_peered":"2024-04-26T21:50:56.361941+0000","last_unstale":"2024-04-26T21:51:00.305012+0000","last_undegraded":"2024-04-26T21:51:00.305012+0000","last_fullsized":"2024-04-26T21:51:00.305012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359467+0000","last_change":"2024-04-26T21:50:57.135566+0000","last_active":"2024-04-26T21:50:58.359467+0000","last_peered":"2024-04-26T21:50:58.359467+0000","last_clean":"2024-04-26T21:50:58.359467+0000","last_became_active":"2024-04-26T21:50:57.135231+0000","last_became_peered":"2024-04-26T21:50:57.135231+0000","last_unstale":"2024-04-26T21:50:58.359467+0000","last_undegraded":"2024-04-26T21:50:58.359467+0000","last_fullsized":"2024-04-26T21:50:58.359467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:09.381936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481543+0000","last_change":"2024-04-26T21:50:56.372420+0000","last_active":"2024-04-26T21:50:58.481543+0000","last_peered":"2024-04-26T21:50:58.481543+0000","last_clean":"2024-04-26T21:50:58.481543+0000","last_became_active":"2024-04-26T21:50:56.372096+0000","last_became_peered":"2024-04-26T21:50:56.372096+0000","last_unstale":"2024-04-26T21:50:58.481543+0000","last_undegraded":"2024-04-26T21:50:58.481543+0000","last_fullsized":"2024-04-26T21:50:58.481543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:03.515804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.357660+0000","last_change":"2024-04-26T21:50:57.140071+0000","last_active":"2024-04-26T21:50:58.357660+0000","last_peered":"2024-04-26T21:50:58.357660+0000","last_clean":"2024-04-26T21:50:58.357660+0000","last_became_active":"2024-04-26T21:50:57.139693+0000","last_became_peered":"2024-04-26T21:50:57.139693+0000","last_unstale":"2024-04-26T21:50:58.357660+0000","last_undegraded":"2024-04-26T21:50:58.357660+0000","last_fullsized":"2024-04-26T21:50:58.357660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:17.963966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136115+0000","last_change":"2024-04-26T21:50:56.363748+0000","last_active":"2024-04-26T21:50:58.136115+0000","last_peered":"2024-04-26T21:50:58.136115+0000","last_clean":"2024-04-26T21:50:58.136115+0000","last_became_active":"2024-04-26T21:50:56.363328+0000","last_became_peered":"2024-04-26T21:50:56.363328+0000","last_unstale":"2024-04-26T21:50:58.136115+0000","last_undegraded":"2024-04-26T21:50:58.136115+0000","last_fullsized":"2024-04-26T21:50:58.136115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:24.450981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483095+0000","last_change":"2024-04-26T21:50:57.143064+0000","last_active":"2024-04-26T21:50:58.483095+0000","last_peered":"2024-04-26T21:50:58.483095+0000","last_clean":"2024-04-26T21:50:58.483095+0000","last_became_active":"2024-04-26T21:50:57.142715+0000","last_became_peered":"2024-04-26T21:50:57.142715+0000","last_unstale":"2024-04-26T21:50:58.483095+0000","last_undegraded":"2024-04-26T21:50:58.483095+0000","last_fullsized":"2024-04-26T21:50:58.483095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:35.901855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.286208+0000","last_change":"2024-04-26T21:50:56.369898+0000","last_active":"2024-04-26T21:50:59.286208+0000","last_peered":"2024-04-26T21:50:59.286208+0000","last_clean":"2024-04-26T21:50:59.286208+0000","last_became_active":"2024-04-26T21:50:56.369349+0000","last_became_peered":"2024-04-26T21:50:56.369349+0000","last_unstale":"2024-04-26T21:50:59.286208+0000","last_undegraded":"2024-04-26T21:50:59.286208+0000","last_fullsized":"2024-04-26T21:50:59.286208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:41.378193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482077+0000","last_change":"2024-04-26T21:50:57.141417+0000","last_active":"2024-04-26T21:50:58.482077+0000","last_peered":"2024-04-26T21:50:58.482077+0000","last_clean":"2024-04-26T21:50:58.482077+0000","last_became_active":"2024-04-26T21:50:57.141091+0000","last_became_peered":"2024-04-26T21:50:57.141091+0000","last_unstale":"2024-04-26T21:50:58.482077+0000","last_undegraded":"2024-04-26T21:50:58.482077+0000","last_fullsized":"2024-04-26T21:50:58.482077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:45.357827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482229+0000","last_change":"2024-04-26T21:50:56.364439+0000","last_active":"2024-04-26T21:50:58.482229+0000","last_peered":"2024-04-26T21:50:58.482229+0000","last_clean":"2024-04-26T21:50:58.482229+0000","last_became_active":"2024-04-26T21:50:56.364090+0000","last_became_peered":"2024-04-26T21:50:56.364090+0000","last_unstale":"2024-04-26T21:50:58.482229+0000","last_undegraded":"2024-04-26T21:50:58.482229+0000","last_fullsized":"2024-04-26T21:50:58.482229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:43.562189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483474+0000","last_change":"2024-04-26T21:50:57.145976+0000","last_active":"2024-04-26T21:50:58.483474+0000","last_peered":"2024-04-26T21:50:58.483474+0000","last_clean":"2024-04-26T21:50:58.483474+0000","last_became_active":"2024-04-26T21:50:57.145620+0000","last_became_peered":"2024-04-26T21:50:57.145620+0000","last_unstale":"2024-04-26T21:50:58.483474+0000","last_undegraded":"2024-04-26T21:50:58.483474+0000","last_fullsized":"2024-04-26T21:50:58.483474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:26.351362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136186+0000","last_change":"2024-04-26T21:50:56.365786+0000","last_active":"2024-04-26T21:50:58.136186+0000","last_peered":"2024-04-26T21:50:58.136186+0000","last_clean":"2024-04-26T21:50:58.136186+0000","last_became_active":"2024-04-26T21:50:56.365410+0000","last_became_peered":"2024-04-26T21:50:56.365410+0000","last_unstale":"2024-04-26T21:50:58.136186+0000","last_undegraded":"2024-04-26T21:50:58.136186+0000","last_fullsized":"2024-04-26T21:50:58.136186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:01.321661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359892+0000","last_change":"2024-04-26T21:50:57.137333+0000","last_active":"2024-04-26T21:50:58.359892+0000","last_peered":"2024-04-26T21:50:58.359892+0000","last_clean":"2024-04-26T21:50:58.359892+0000","last_became_active":"2024-04-26T21:50:57.136983+0000","last_became_peered":"2024-04-26T21:50:57.136983+0000","last_unstale":"2024-04-26T21:50:58.359892+0000","last_undegraded":"2024-04-26T21:50:58.359892+0000","last_fullsized":"2024-04-26T21:50:58.359892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:18.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.303964+0000","last_change":"2024-04-26T21:50:56.358340+0000","last_active":"2024-04-26T21:51:00.303964+0000","last_peered":"2024-04-26T21:51:00.303964+0000","last_clean":"2024-04-26T21:51:00.303964+0000","last_became_active":"2024-04-26T21:50:56.357972+0000","last_became_peered":"2024-04-26T21:50:56.357972+0000","last_unstale":"2024-04-26T21:51:00.303964+0000","last_undegraded":"2024-04-26T21:51:00.303964+0000","last_fullsized":"2024-04-26T21:51:00.303964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:52.256528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136077+0000","last_change":"2024-04-26T21:50:57.137027+0000","last_active":"2024-04-26T21:50:58.136077+0000","last_peered":"2024-04-26T21:50:58.136077+0000","last_clean":"2024-04-26T21:50:58.136077+0000","last_became_active":"2024-04-26T21:50:57.136400+0000","last_became_peered":"2024-04-26T21:50:57.136400+0000","last_unstale":"2024-04-26T21:50:58.136077+0000","last_undegraded":"2024-04-26T21:50:58.136077+0000","last_fullsized":"2024-04-26T21:50:58.136077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:45.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481952+0000","last_change":"2024-04-26T21:50:56.369784+0000","last_active":"2024-04-26T21:50:58.481952+0000","last_peered":"2024-04-26T21:50:58.481952+0000","last_clean":"2024-04-26T21:50:58.481952+0000","last_became_active":"2024-04-26T21:50:56.369093+0000","last_became_peered":"2024-04-26T21:50:56.369093+0000","last_unstale":"2024-04-26T21:50:58.481952+0000","last_undegraded":"2024-04-26T21:50:58.481952+0000","last_fullsized":"2024-04-26T21:50:58.481952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:27.595146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482905+0000","last_change":"2024-04-26T21:50:57.141404+0000","last_active":"2024-04-26T21:50:58.482905+0000","last_peered":"2024-04-26T21:50:58.482905+0000","last_clean":"2024-04-26T21:50:58.482905+0000","last_became_active":"2024-04-26T21:50:57.140973+0000","last_became_peered":"2024-04-26T21:50:57.140973+0000","last_unstale":"2024-04-26T21:50:58.482905+0000","last_undegraded":"2024-04-26T21:50:58.482905+0000","last_fullsized":"2024-04-26T21:50:58.482905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:09.405982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482145+0000","last_change":"2024-04-26T21:50:56.370727+0000","last_active":"2024-04-26T21:50:58.482145+0000","last_peered":"2024-04-26T21:50:58.482145+0000","last_clean":"2024-04-26T21:50:58.482145+0000","last_became_active":"2024-04-26T21:50:56.370373+0000","last_became_peered":"2024-04-26T21:50:56.370373+0000","last_unstale":"2024-04-26T21:50:58.482145+0000","last_undegraded":"2024-04-26T21:50:58.482145+0000","last_fullsized":"2024-04-26T21:50:58.482145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:12.689699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482233+0000","last_change":"2024-04-26T21:50:57.145771+0000","last_active":"2024-04-26T21:50:58.482233+0000","last_peered":"2024-04-26T21:50:58.482233+0000","last_clean":"2024-04-26T21:50:58.482233+0000","last_became_active":"2024-04-26T21:50:57.145424+0000","last_became_peered":"2024-04-26T21:50:57.145424+0000","last_unstale":"2024-04-26T21:50:58.482233+0000","last_undegraded":"2024-04-26T21:50:58.482233+0000","last_fullsized":"2024-04-26T21:50:58.482233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:10.485045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.306053+0000","last_change":"2024-04-26T21:50:56.365996+0000","last_active":"2024-04-26T21:51:00.306053+0000","last_peered":"2024-04-26T21:51:00.306053+0000","last_clean":"2024-04-26T21:51:00.306053+0000","last_became_active":"2024-04-26T21:50:56.365653+0000","last_became_peered":"2024-04-26T21:50:56.365653+0000","last_unstale":"2024-04-26T21:51:00.306053+0000","last_undegraded":"2024-04-26T21:51:00.306053+0000","last_fullsized":"2024-04-26T21:51:00.306053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:11.514305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481663+0000","last_change":"2024-04-26T21:50:57.147536+0000","last_active":"2024-04-26T21:50:58.481663+0000","last_peered":"2024-04-26T21:50:58.481663+0000","last_clean":"2024-04-26T21:50:58.481663+0000","last_became_active":"2024-04-26T21:50:57.147071+0000","last_became_peered":"2024-04-26T21:50:57.147071+0000","last_unstale":"2024-04-26T21:50:58.481663+0000","last_undegraded":"2024-04-26T21:50:58.481663+0000","last_fullsized":"2024-04-26T21:50:58.481663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:56.231509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:01.299882+0000","last_change":"2024-04-26T21:50:56.358270+0000","last_active":"2024-04-26T21:51:01.299882+0000","last_peered":"2024-04-26T21:51:01.299882+0000","last_clean":"2024-04-26T21:51:01.299882+0000","last_became_active":"2024-04-26T21:50:56.357909+0000","last_became_peered":"2024-04-26T21:50:56.357909+0000","last_unstale":"2024-04-26T21:51:01.299882+0000","last_undegraded":"2024-04-26T21:51:01.299882+0000","last_fullsized":"2024-04-26T21:51:01.299882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:45.627352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482419+0000","last_change":"2024-04-26T21:50:57.142181+0000","last_active":"2024-04-26T21:50:58.482419+0000","last_peered":"2024-04-26T21:50:58.482419+0000","last_clean":"2024-04-26T21:50:58.482419+0000","last_became_active":"2024-04-26T21:50:57.141837+0000","last_became_peered":"2024-04-26T21:50:57.141837+0000","last_unstale":"2024-04-26T21:50:58.482419+0000","last_undegraded":"2024-04-26T21:50:58.482419+0000","last_fullsized":"2024-04-26T21:50:58.482419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:42.150522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.357622+0000","last_change":"2024-04-26T21:50:56.360120+0000","last_active":"2024-04-26T21:50:58.357622+0000","last_peered":"2024-04-26T21:50:58.357622+0000","last_clean":"2024-04-26T21:50:58.357622+0000","last_became_active":"2024-04-26T21:50:56.359728+0000","last_became_peered":"2024-04-26T21:50:56.359728+0000","last_unstale":"2024-04-26T21:50:58.357622+0000","last_undegraded":"2024-04-26T21:50:58.357622+0000","last_fullsized":"2024-04-26T21:50:58.357622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:54:19.995617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136261+0000","last_change":"2024-04-26T21:50:57.146682+0000","last_active":"2024-04-26T21:50:58.136261+0000","last_peered":"2024-04-26T21:50:58.136261+0000","last_clean":"2024-04-26T21:50:58.136261+0000","last_became_active":"2024-04-26T21:50:57.146329+0000","last_became_peered":"2024-04-26T21:50:57.146329+0000","last_unstale":"2024-04-26T21:50:58.136261+0000","last_undegraded":"2024-04-26T21:50:58.136261+0000","last_fullsized":"2024-04-26T21:50:58.136261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:41.035526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482691+0000","last_change":"2024-04-26T21:50:56.370533+0000","last_active":"2024-04-26T21:50:58.482691+0000","last_peered":"2024-04-26T21:50:58.482691+0000","last_clean":"2024-04-26T21:50:58.482691+0000","last_became_active":"2024-04-26T21:50:56.370029+0000","last_became_peered":"2024-04-26T21:50:56.370029+0000","last_unstale":"2024-04-26T21:50:58.482691+0000","last_undegraded":"2024-04-26T21:50:58.482691+0000","last_fullsized":"2024-04-26T21:50:58.482691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.005405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136075+0000","last_change":"2024-04-26T21:50:57.142420+0000","last_active":"2024-04-26T21:50:58.136075+0000","last_peered":"2024-04-26T21:50:58.136075+0000","last_clean":"2024-04-26T21:50:58.136075+0000","last_became_active":"2024-04-26T21:50:57.142023+0000","last_became_peered":"2024-04-26T21:50:57.142023+0000","last_unstale":"2024-04-26T21:50:58.136075+0000","last_undegraded":"2024-04-26T21:50:58.136075+0000","last_fullsized":"2024-04-26T21:50:58.136075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:31.567136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481756+0000","last_change":"2024-04-26T21:50:57.139924+0000","last_active":"2024-04-26T21:50:58.481756+0000","last_peered":"2024-04-26T21:50:58.481756+0000","last_clean":"2024-04-26T21:50:58.481756+0000","last_became_active":"2024-04-26T21:50:57.139561+0000","last_became_peered":"2024-04-26T21:50:57.139561+0000","last_unstale":"2024-04-26T21:50:58.481756+0000","last_undegraded":"2024-04-26T21:50:58.481756+0000","last_fullsized":"2024-04-26T21:50:58.481756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:52.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.305663+0000","last_change":"2024-04-26T21:50:56.365256+0000","last_active":"2024-04-26T21:51:00.305663+0000","last_peered":"2024-04-26T21:51:00.305663+0000","last_clean":"2024-04-26T21:51:00.305663+0000","last_became_active":"2024-04-26T21:50:56.364516+0000","last_became_peered":"2024-04-26T21:50:56.364516+0000","last_unstale":"2024-04-26T21:51:00.305663+0000","last_undegraded":"2024-04-26T21:51:00.305663+0000","last_fullsized":"2024-04-26T21:51:00.305663+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:13.550129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359682+0000","last_change":"2024-04-26T21:50:57.138177+0000","last_active":"2024-04-26T21:50:58.359682+0000","last_peered":"2024-04-26T21:50:58.359682+0000","last_clean":"2024-04-26T21:50:58.359682+0000","last_became_active":"2024-04-26T21:50:57.137725+0000","last_became_peered":"2024-04-26T21:50:57.137725+0000","last_unstale":"2024-04-26T21:50:58.359682+0000","last_undegraded":"2024-04-26T21:50:58.359682+0000","last_fullsized":"2024-04-26T21:50:58.359682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:14.113935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.285879+0000","last_change":"2024-04-26T21:50:56.358280+0000","last_active":"2024-04-26T21:50:59.285879+0000","last_peered":"2024-04-26T21:50:59.285879+0000","last_clean":"2024-04-26T21:50:59.285879+0000","last_became_active":"2024-04-26T21:50:56.357854+0000","last_became_peered":"2024-04-26T21:50:56.357854+0000","last_unstale":"2024-04-26T21:50:59.285879+0000","last_undegraded":"2024-04-26T21:50:59.285879+0000","last_fullsized":"2024-04-26T21:50:59.285879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.499124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482404+0000","last_change":"2024-04-26T21:49:42.292617+0000","last_active":"2024-04-26T21:50:58.482404+0000","last_peered":"2024-04-26T21:50:58.482404+0000","last_clean":"2024-04-26T21:50:58.482404+0000","last_became_active":"2024-04-26T21:49:42.292160+0000","last_became_peered":"2024-04-26T21:49:42.292160+0000","last_unstale":"2024-04-26T21:50:58.482404+0000","last_undegraded":"2024-04-26T21:50:58.482404+0000","last_fullsized":"2024-04-26T21:50:58.482404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_clean_scrub_stamp":"2024-04-26T21:46:13.787665+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:24.663652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136070+0000","last_change":"2024-04-26T21:50:57.145641+0000","last_active":"2024-04-26T21:50:58.136070+0000","last_peered":"2024-04-26T21:50:58.136070+0000","last_clean":"2024-04-26T21:50:58.136070+0000","last_became_active":"2024-04-26T21:50:57.145304+0000","last_became_peered":"2024-04-26T21:50:57.145304+0000","last_unstale":"2024-04-26T21:50:58.136070+0000","last_undegraded":"2024-04-26T21:50:58.136070+0000","last_fullsized":"2024-04-26T21:50:58.136070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:11.389728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.303377+0000","last_change":"2024-04-26T21:50:56.367451+0000","last_active":"2024-04-26T21:51:00.303377+0000","last_peered":"2024-04-26T21:51:00.303377+0000","last_clean":"2024-04-26T21:51:00.303377+0000","last_became_active":"2024-04-26T21:50:56.367054+0000","last_became_peered":"2024-04-26T21:50:56.367054+0000","last_unstale":"2024-04-26T21:51:00.303377+0000","last_undegraded":"2024-04-26T21:51:00.303377+0000","last_fullsized":"2024-04-26T21:51:00.303377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:04.761499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.282168+0000","last_change":"2024-04-26T21:50:56.364074+0000","last_active":"2024-04-26T21:50:59.282168+0000","last_peered":"2024-04-26T21:50:59.282168+0000","last_clean":"2024-04-26T21:50:59.282168+0000","last_became_active":"2024-04-26T21:50:56.363688+0000","last_became_peered":"2024-04-26T21:50:56.363688+0000","last_unstale":"2024-04-26T21:50:59.282168+0000","last_undegraded":"2024-04-26T21:50:59.282168+0000","last_fullsized":"2024-04-26T21:50:59.282168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:35.526125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136395+0000","last_change":"2024-04-26T21:50:57.136811+0000","last_active":"2024-04-26T21:50:58.136395+0000","last_peered":"2024-04-26T21:50:58.136395+0000","last_clean":"2024-04-26T21:50:58.136395+0000","last_became_active":"2024-04-26T21:50:57.136252+0000","last_became_peered":"2024-04-26T21:50:57.136252+0000","last_unstale":"2024-04-26T21:50:58.136395+0000","last_undegraded":"2024-04-26T21:50:58.136395+0000","last_fullsized":"2024-04-26T21:50:58.136395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:01.061283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359169+0000","last_change":"2024-04-26T21:50:56.364317+0000","last_active":"2024-04-26T21:50:58.359169+0000","last_peered":"2024-04-26T21:50:58.359169+0000","last_clean":"2024-04-26T21:50:58.359169+0000","last_became_active":"2024-04-26T21:50:56.363882+0000","last_became_peered":"2024-04-26T21:50:56.363882+0000","last_unstale":"2024-04-26T21:50:58.359169+0000","last_undegraded":"2024-04-26T21:50:58.359169+0000","last_fullsized":"2024-04-26T21:50:58.359169+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:12.994293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359964+0000","last_change":"2024-04-26T21:50:57.144856+0000","last_active":"2024-04-26T21:50:58.359964+0000","last_peered":"2024-04-26T21:50:58.359964+0000","last_clean":"2024-04-26T21:50:58.359964+0000","last_became_active":"2024-04-26T21:50:57.143717+0000","last_became_peered":"2024-04-26T21:50:57.143717+0000","last_unstale":"2024-04-26T21:50:58.359964+0000","last_undegraded":"2024-04-26T21:50:58.359964+0000","last_fullsized":"2024-04-26T21:50:58.359964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:53.049243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.285053+0000","last_change":"2024-04-26T21:50:56.370284+0000","last_active":"2024-04-26T21:50:59.285053+0000","last_peered":"2024-04-26T21:50:59.285053+0000","last_clean":"2024-04-26T21:50:59.285053+0000","last_became_active":"2024-04-26T21:50:56.369738+0000","last_became_peered":"2024-04-26T21:50:56.369738+0000","last_unstale":"2024-04-26T21:50:59.285053+0000","last_undegraded":"2024-04-26T21:50:59.285053+0000","last_fullsized":"2024-04-26T21:50:59.285053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:23.390553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481955+0000","last_change":"2024-04-26T21:50:57.142687+0000","last_active":"2024-04-26T21:50:58.481955+0000","last_peered":"2024-04-26T21:50:58.481955+0000","last_clean":"2024-04-26T21:50:58.481955+0000","last_became_active":"2024-04-26T21:50:57.142365+0000","last_became_peered":"2024-04-26T21:50:57.142365+0000","last_unstale":"2024-04-26T21:50:58.481955+0000","last_undegraded":"2024-04-26T21:50:58.481955+0000","last_fullsized":"2024-04-26T21:50:58.481955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:14.261741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482415+0000","last_change":"2024-04-26T21:50:56.364163+0000","last_active":"2024-04-26T21:50:58.482415+0000","last_peered":"2024-04-26T21:50:58.482415+0000","last_clean":"2024-04-26T21:50:58.482415+0000","last_became_active":"2024-04-26T21:50:56.363729+0000","last_became_peered":"2024-04-26T21:50:56.363729+0000","last_unstale":"2024-04-26T21:50:58.482415+0000","last_undegraded":"2024-04-26T21:50:58.482415+0000","last_fullsized":"2024-04-26T21:50:58.482415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:54.620884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359539+0000","last_change":"2024-04-26T21:50:57.142692+0000","last_active":"2024-04-26T21:50:58.359539+0000","last_peered":"2024-04-26T21:50:58.359539+0000","last_clean":"2024-04-26T21:50:58.359539+0000","last_became_active":"2024-04-26T21:50:57.142137+0000","last_became_peered":"2024-04-26T21:50:57.142137+0000","last_unstale":"2024-04-26T21:50:58.359539+0000","last_undegraded":"2024-04-26T21:50:58.359539+0000","last_fullsized":"2024-04-26T21:50:58.359539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.996485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359411+0000","last_change":"2024-04-26T21:50:56.366374+0000","last_active":"2024-04-26T21:50:58.359411+0000","last_peered":"2024-04-26T21:50:58.359411+0000","last_clean":"2024-04-26T21:50:58.359411+0000","last_became_active":"2024-04-26T21:50:56.366014+0000","last_became_peered":"2024-04-26T21:50:56.366014+0000","last_unstale":"2024-04-26T21:50:58.359411+0000","last_undegraded":"2024-04-26T21:50:58.359411+0000","last_fullsized":"2024-04-26T21:50:58.359411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:41.867792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482482+0000","last_change":"2024-04-26T21:50:57.142833+0000","last_active":"2024-04-26T21:50:58.482482+0000","last_peered":"2024-04-26T21:50:58.482482+0000","last_clean":"2024-04-26T21:50:58.482482+0000","last_became_active":"2024-04-26T21:50:57.142061+0000","last_became_peered":"2024-04-26T21:50:57.142061+0000","last_unstale":"2024-04-26T21:50:58.482482+0000","last_undegraded":"2024-04-26T21:50:58.482482+0000","last_fullsized":"2024-04-26T21:50:58.482482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:33.023159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131553+0000","last_change":"2024-04-26T21:50:56.363172+0000","last_active":"2024-04-26T21:50:58.131553+0000","last_peered":"2024-04-26T21:50:58.131553+0000","last_clean":"2024-04-26T21:50:58.131553+0000","last_became_active":"2024-04-26T21:50:56.362782+0000","last_became_peered":"2024-04-26T21:50:56.362782+0000","last_unstale":"2024-04-26T21:50:58.131553+0000","last_undegraded":"2024-04-26T21:50:58.131553+0000","last_fullsized":"2024-04-26T21:50:58.131553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:37.183497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359658+0000","last_change":"2024-04-26T21:50:57.142088+0000","last_active":"2024-04-26T21:50:58.359658+0000","last_peered":"2024-04-26T21:50:58.359658+0000","last_clean":"2024-04-26T21:50:58.359658+0000","last_became_active":"2024-04-26T21:50:57.141685+0000","last_became_peered":"2024-04-26T21:50:57.141685+0000","last_unstale":"2024-04-26T21:50:58.359658+0000","last_undegraded":"2024-04-26T21:50:58.359658+0000","last_fullsized":"2024-04-26T21:50:58.359658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:24.033787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:59.291885+0000","last_change":"2024-04-26T21:50:56.371717+0000","last_active":"2024-04-26T21:50:59.291885+0000","last_peered":"2024-04-26T21:50:59.291885+0000","last_clean":"2024-04-26T21:50:59.291885+0000","last_became_active":"2024-04-26T21:50:56.370930+0000","last_became_peered":"2024-04-26T21:50:56.370930+0000","last_unstale":"2024-04-26T21:50:59.291885+0000","last_undegraded":"2024-04-26T21:50:59.291885+0000","last_fullsized":"2024-04-26T21:50:59.291885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.977392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482853+0000","last_change":"2024-04-26T21:50:57.139671+0000","last_active":"2024-04-26T21:50:58.482853+0000","last_peered":"2024-04-26T21:50:58.482853+0000","last_clean":"2024-04-26T21:50:58.482853+0000","last_became_active":"2024-04-26T21:50:57.139245+0000","last_became_peered":"2024-04-26T21:50:57.139245+0000","last_unstale":"2024-04-26T21:50:58.482853+0000","last_undegraded":"2024-04-26T21:50:58.482853+0000","last_fullsized":"2024-04-26T21:50:58.482853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:26.903905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:51:00.306177+0000","last_change":"2024-04-26T21:50:56.369836+0000","last_active":"2024-04-26T21:51:00.306177+0000","last_peered":"2024-04-26T21:51:00.306177+0000","last_clean":"2024-04-26T21:51:00.306177+0000","last_became_active":"2024-04-26T21:50:56.369522+0000","last_became_peered":"2024-04-26T21:50:56.369522+0000","last_unstale":"2024-04-26T21:51:00.306177+0000","last_undegraded":"2024-04-26T21:51:00.306177+0000","last_fullsized":"2024-04-26T21:51:00.306177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:40.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483494+0000","last_change":"2024-04-26T21:50:57.138551+0000","last_active":"2024-04-26T21:50:58.483494+0000","last_peered":"2024-04-26T21:50:58.483494+0000","last_clean":"2024-04-26T21:50:58.483494+0000","last_became_active":"2024-04-26T21:50:57.138187+0000","last_became_peered":"2024-04-26T21:50:57.138187+0000","last_unstale":"2024-04-26T21:50:58.483494+0000","last_undegraded":"2024-04-26T21:50:58.483494+0000","last_fullsized":"2024-04-26T21:50:58.483494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:46.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482101+0000","last_change":"2024-04-26T21:50:56.362009+0000","last_active":"2024-04-26T21:50:58.482101+0000","last_peered":"2024-04-26T21:50:58.482101+0000","last_clean":"2024-04-26T21:50:58.482101+0000","last_became_active":"2024-04-26T21:50:56.361646+0000","last_became_peered":"2024-04-26T21:50:56.361646+0000","last_unstale":"2024-04-26T21:50:58.482101+0000","last_undegraded":"2024-04-26T21:50:58.482101+0000","last_fullsized":"2024-04-26T21:50:58.482101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:25.316195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482479+0000","last_change":"2024-04-26T21:50:57.139983+0000","last_active":"2024-04-26T21:50:58.482479+0000","last_peered":"2024-04-26T21:50:58.482479+0000","last_clean":"2024-04-26T21:50:58.482479+0000","last_became_active":"2024-04-26T21:50:57.138426+0000","last_became_peered":"2024-04-26T21:50:57.138426+0000","last_unstale":"2024-04-26T21:50:58.482479+0000","last_undegraded":"2024-04-26T21:50:58.482479+0000","last_fullsized":"2024-04-26T21:50:58.482479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:53.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136164+0000","last_change":"2024-04-26T21:50:56.369527+0000","last_active":"2024-04-26T21:50:58.136164+0000","last_peered":"2024-04-26T21:50:58.136164+0000","last_clean":"2024-04-26T21:50:58.136164+0000","last_became_active":"2024-04-26T21:50:56.369168+0000","last_became_peered":"2024-04-26T21:50:56.369168+0000","last_unstale":"2024-04-26T21:50:58.136164+0000","last_undegraded":"2024-04-26T21:50:58.136164+0000","last_fullsized":"2024-04-26T21:50:58.136164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:38.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.483267+0000","last_change":"2024-04-26T21:50:57.137957+0000","last_active":"2024-04-26T21:50:58.483267+0000","last_peered":"2024-04-26T21:50:58.483267+0000","last_clean":"2024-04-26T21:50:58.483267+0000","last_became_active":"2024-04-26T21:50:57.137595+0000","last_became_peered":"2024-04-26T21:50:57.137595+0000","last_unstale":"2024-04-26T21:50:58.483267+0000","last_undegraded":"2024-04-26T21:50:58.483267+0000","last_fullsized":"2024-04-26T21:50:58.483267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:09.355937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482430+0000","last_change":"2024-04-26T21:50:57.143098+0000","last_active":"2024-04-26T21:50:58.482430+0000","last_peered":"2024-04-26T21:50:58.482430+0000","last_clean":"2024-04-26T21:50:58.482430+0000","last_became_active":"2024-04-26T21:50:57.142524+0000","last_became_peered":"2024-04-26T21:50:57.142524+0000","last_unstale":"2024-04-26T21:50:58.482430+0000","last_undegraded":"2024-04-26T21:50:58.482430+0000","last_fullsized":"2024-04-26T21:50:58.482430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:33.333382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131345+0000","last_change":"2024-04-26T21:50:57.143123+0000","last_active":"2024-04-26T21:50:58.131345+0000","last_peered":"2024-04-26T21:50:58.131345+0000","last_clean":"2024-04-26T21:50:58.131345+0000","last_became_active":"2024-04-26T21:50:57.142805+0000","last_became_peered":"2024-04-26T21:50:57.142805+0000","last_unstale":"2024-04-26T21:50:58.131345+0000","last_undegraded":"2024-04-26T21:50:58.131345+0000","last_fullsized":"2024-04-26T21:50:58.131345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:19.330202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131371+0000","last_change":"2024-04-26T21:50:57.140268+0000","last_active":"2024-04-26T21:50:58.131371+0000","last_peered":"2024-04-26T21:50:58.131371+0000","last_clean":"2024-04-26T21:50:58.131371+0000","last_became_active":"2024-04-26T21:50:57.139927+0000","last_became_peered":"2024-04-26T21:50:57.139927+0000","last_unstale":"2024-04-26T21:50:58.131371+0000","last_undegraded":"2024-04-26T21:50:58.131371+0000","last_fullsized":"2024-04-26T21:50:58.131371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:01.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481790+0000","last_change":"2024-04-26T21:50:57.143564+0000","last_active":"2024-04-26T21:50:58.481790+0000","last_peered":"2024-04-26T21:50:58.481790+0000","last_clean":"2024-04-26T21:50:58.481790+0000","last_became_active":"2024-04-26T21:50:57.142740+0000","last_became_peered":"2024-04-26T21:50:57.142740+0000","last_unstale":"2024-04-26T21:50:58.481790+0000","last_undegraded":"2024-04-26T21:50:58.481790+0000","last_fullsized":"2024-04-26T21:50:58.481790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:58.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136433+0000","last_change":"2024-04-26T21:50:57.146883+0000","last_active":"2024-04-26T21:50:58.136433+0000","last_peered":"2024-04-26T21:50:58.136433+0000","last_clean":"2024-04-26T21:50:58.136433+0000","last_became_active":"2024-04-26T21:50:57.146540+0000","last_became_peered":"2024-04-26T21:50:57.146540+0000","last_unstale":"2024-04-26T21:50:58.136433+0000","last_undegraded":"2024-04-26T21:50:58.136433+0000","last_fullsized":"2024-04-26T21:50:58.136433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.787618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482576+0000","last_change":"2024-04-26T21:50:57.136938+0000","last_active":"2024-04-26T21:50:58.482576+0000","last_peered":"2024-04-26T21:50:58.482576+0000","last_clean":"2024-04-26T21:50:58.482576+0000","last_became_active":"2024-04-26T21:50:57.136474+0000","last_became_peered":"2024-04-26T21:50:57.136474+0000","last_unstale":"2024-04-26T21:50:58.482576+0000","last_undegraded":"2024-04-26T21:50:58.482576+0000","last_fullsized":"2024-04-26T21:50:58.482576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:42.534342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.131777+0000","last_change":"2024-04-26T21:50:57.150958+0000","last_active":"2024-04-26T21:50:58.131777+0000","last_peered":"2024-04-26T21:50:58.131777+0000","last_clean":"2024-04-26T21:50:58.131777+0000","last_became_active":"2024-04-26T21:50:57.150593+0000","last_became_peered":"2024-04-26T21:50:57.150593+0000","last_unstale":"2024-04-26T21:50:58.131777+0000","last_undegraded":"2024-04-26T21:50:58.131777+0000","last_fullsized":"2024-04-26T21:50:58.131777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:26.583983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359926+0000","last_change":"2024-04-26T21:50:57.147766+0000","last_active":"2024-04-26T21:50:58.359926+0000","last_peered":"2024-04-26T21:50:58.359926+0000","last_clean":"2024-04-26T21:50:58.359926+0000","last_became_active":"2024-04-26T21:50:57.147442+0000","last_became_peered":"2024-04-26T21:50:57.147442+0000","last_unstale":"2024-04-26T21:50:58.359926+0000","last_undegraded":"2024-04-26T21:50:58.359926+0000","last_fullsized":"2024-04-26T21:50:58.359926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.489117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.481685+0000","last_change":"2024-04-26T21:50:57.138357+0000","last_active":"2024-04-26T21:50:58.481685+0000","last_peered":"2024-04-26T21:50:58.481685+0000","last_clean":"2024-04-26T21:50:58.481685+0000","last_became_active":"2024-04-26T21:50:57.135570+0000","last_became_peered":"2024-04-26T21:50:57.135570+0000","last_unstale":"2024-04-26T21:50:58.481685+0000","last_undegraded":"2024-04-26T21:50:58.481685+0000","last_fullsized":"2024-04-26T21:50:58.481685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:17.206623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.359485+0000","last_change":"2024-04-26T21:50:57.142804+0000","last_active":"2024-04-26T21:50:58.359485+0000","last_peered":"2024-04-26T21:50:58.359485+0000","last_clean":"2024-04-26T21:50:58.359485+0000","last_became_active":"2024-04-26T21:50:57.141902+0000","last_became_peered":"2024-04-26T21:50:57.141902+0000","last_unstale":"2024-04-26T21:50:58.359485+0000","last_undegraded":"2024-04-26T21:50:58.359485+0000","last_fullsized":"2024-04-26T21:50:58.359485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:37.590730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482798+0000","last_change":"2024-04-26T21:50:57.138637+0000","last_active":"2024-04-26T21:50:58.482798+0000","last_peered":"2024-04-26T21:50:58.482798+0000","last_clean":"2024-04-26T21:50:58.482798+0000","last_became_active":"2024-04-26T21:50:57.138295+0000","last_became_peered":"2024-04-26T21:50:57.138295+0000","last_unstale":"2024-04-26T21:50:58.482798+0000","last_undegraded":"2024-04-26T21:50:58.482798+0000","last_fullsized":"2024-04-26T21:50:58.482798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:19.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.136322+0000","last_change":"2024-04-26T21:50:57.137377+0000","last_active":"2024-04-26T21:50:58.136322+0000","last_peered":"2024-04-26T21:50:58.136322+0000","last_clean":"2024-04-26T21:50:58.136322+0000","last_became_active":"2024-04-26T21:50:57.136893+0000","last_became_peered":"2024-04-26T21:50:57.136893+0000","last_unstale":"2024-04-26T21:50:58.136322+0000","last_undegraded":"2024-04-26T21:50:58.136322+0000","last_fullsized":"2024-04-26T21:50:58.136322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:44.257438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-26T21:50:58.482573+0000","last_change":"2024-04-26T21:50:57.141439+0000","last_active":"2024-04-26T21:50:58.482573+0000","last_peered":"2024-04-26T21:50:58.482573+0000","last_clean":"2024-04-26T21:50:58.482573+0000","last_became_active":"2024-04-26T21:50:57.141060+0000","last_became_peered":"2024-04-26T21:50:57.141060+0000","last_unstale":"2024-04-26T21:50:58.482573+0000","last_undegraded":"2024-04-26T21:50:58.482573+0000","last_fullsized":"2024-04-26T21:50:58.482573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:27.174348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250381,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3216,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":90,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":48531,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":1022125,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":439259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":1030681,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":431563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":439999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":460766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":430055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":1020501,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":39543,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:51:04.474 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T21:51:04.475 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T21:51:04.475 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T21:51:04.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:04 smithi165 ceph-mon[33296]: pgmap v292: 97 pgs: 97 active+clean; 580 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-26T21:51:04.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:04 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2061150727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:51:04.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:04 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4022683302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:04.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:04 smithi165 ceph-mon[33296]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:51:04.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:04 smithi064 ceph-mon[32765]: pgmap v292: 97 pgs: 97 active+clean; 580 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-26T21:51:04.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:04 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2061150727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:51:04.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:04 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4022683302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:04.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:04 smithi064 ceph-mon[32765]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:51:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:05.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:05 smithi165 ceph-mon[33296]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T21:51:05.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:05 smithi165 ceph-mon[33296]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:51:05.596 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:05 smithi055 ceph-mon[29221]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T21:51:05.596 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:05 smithi055 ceph-mon[29221]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:51:05.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:05 smithi064 ceph-mon[32765]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T21:51:05.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:05 smithi064 ceph-mon[32765]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:51:06.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:06 smithi055 ceph-mon[29221]: pgmap v293: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.7 KiB/s wr, 22 op/s 2024-04-26T21:51:06.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 -- 172.21.15.55:0/4049720532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 msgr2=0x7fa6ec10a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:06.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/4049720532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 0x7fa6ec10a7f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b0d0 tx=0x7fa6e400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 -- 172.21.15.55:0/4049720532 shutdown_connections 2024-04-26T21:51:06.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/4049720532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 0x7fa6ec10a7f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/4049720532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6ec1023a0 0x7fa6ec1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/4049720532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6ec1035a0 0x7fa6ec103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 -- 172.21.15.55:0/4049720532 >> 172.21.15.55:0/4049720532 conn(0x7fa6ec0fd940 msgr2=0x7fa6ec0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 -- 172.21.15.55:0/4049720532 shutdown_connections 2024-04-26T21:51:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.436+0000 7fa6f1dda700 1 -- 172.21.15.55:0/4049720532 wait complete. 2024-04-26T21:51:06.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.437+0000 7fa6f1dda700 1 Processor -- start 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.437+0000 7fa6f1dda700 1 -- start start 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.437+0000 7fa6f1dda700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6ec1023a0 0x7fa6ec199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.437+0000 7fa6f1dda700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6ec1035a0 0x7fa6ec199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.437+0000 7fa6f1dda700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 0x7fa6ec1a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.437+0000 7fa6f1dda700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10efd0 con 0x7fa6ec103f40 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.437+0000 7fa6f1dda700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10ee50 con 0x7fa6ec1023a0 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.437+0000 7fa6f1dda700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec10f150 con 0x7fa6ec1035a0 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 0x7fa6ec1a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 0x7fa6ec1a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45486/0 (socket says 172.21.15.55:45486) 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 -- 172.21.15.55:0/2822733047 learned_addr learned my addr 172.21.15.55:0/2822733047 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6eaffd700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6ec1035a0 0x7fa6ec199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6eb7fe700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6ec1023a0 0x7fa6ec199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 -- 172.21.15.55:0/2822733047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6ec1035a0 msgr2=0x7fa6ec199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6ec1035a0 0x7fa6ec199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 -- 172.21.15.55:0/2822733047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6ec1023a0 msgr2=0x7fa6ec199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6ec1023a0 0x7fa6ec199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 -- 172.21.15.55:0/2822733047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec1a1090 con 0x7fa6ec103f40 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6ebfff700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 0x7fa6ec1a09f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400d010 tx=0x7fa6e4004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6e8ff9700 1 -- 172.21.15.55:0/2822733047 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e401f030 con 0x7fa6ec103f40 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec1a1320 con 0x7fa6ec103f40 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6e8ff9700 1 -- 172.21.15.55:0/2822733047 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6e4007b00 con 0x7fa6ec103f40 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6e8ff9700 1 -- 172.21.15.55:0/2822733047 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40047a0 con 0x7fa6ec103f40 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.438+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec1a17f0 con 0x7fa6ec103f40 2024-04-26T21:51:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.440+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec051960 con 0x7fa6ec103f40 2024-04-26T21:51:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.440+0000 7fa6e8ff9700 1 -- 172.21.15.55:0/2822733047 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa6e4024460 con 0x7fa6ec103f40 2024-04-26T21:51:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.441+0000 7fa6e8ff9700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6d40a3d80 0x7fa6d40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.441+0000 7fa6e8ff9700 1 -- 172.21.15.55:0/2822733047 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa6e400f020 con 0x7fa6ec103f40 2024-04-26T21:51:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.441+0000 7fa6eb7fe700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6d40a3d80 0x7fa6d40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:06.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.441+0000 7fa6eb7fe700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6d40a3d80 0x7fa6d40a6240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005fd0 tx=0x7fa6dc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:06.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.446+0000 7fa6e8ff9700 1 -- 172.21.15.55:0/2822733047 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa6e4029020 con 0x7fa6ec103f40 2024-04-26T21:51:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.741+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fa6ec02e560 con 0x7fa6ec103f40 2024-04-26T21:51:06.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.745+0000 7fa6e8ff9700 1 -- 172.21.15.55:0/2822733047 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fa6ec02e560 con 0x7fa6ec103f40 2024-04-26T21:51:06.745 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2024-04-26T21:51:06.746 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDKISxm52VGLBAACiWImYEJdzYYcULxgqw/lQ== 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6d40a3d80 msgr2=0x7fa6d40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6d40a3d80 0x7fa6d40a6240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005fd0 tx=0x7fa6dc005e50 comp rx=0 tx=0).stop 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 msgr2=0x7fa6ec1a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 0x7fa6ec1a09f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400d010 tx=0x7fa6e4004e40 comp rx=0 tx=0).stop 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 shutdown_connections 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6ec103f40 0x7fa6ec1a09f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6d40a3d80 0x7fa6d40a6240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6ec1023a0 0x7fa6ec199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 --2- 172.21.15.55:0/2822733047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa6ec1035a0 0x7fa6ec199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.747+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 >> 172.21.15.55:0/2822733047 conn(0x7fa6ec0fd940 msgr2=0x7fa6ec1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.748+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 shutdown_connections 2024-04-26T21:51:06.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:06.748+0000 7fa6f1dda700 1 -- 172.21.15.55:0/2822733047 wait complete. 2024-04-26T21:51:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:06 smithi064 ceph-mon[32765]: pgmap v293: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.7 KiB/s wr, 22 op/s 2024-04-26T21:51:06.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:06 smithi165 ceph-mon[33296]: pgmap v293: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 9.7 KiB/s wr, 22 op/s 2024-04-26T21:51:07.429 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:51:07.429 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T21:51:07.429 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T21:51:07.469 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T21:51:07.469 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T21:51:07.469 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T21:51:07.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:07 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2822733047' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:51:07.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:07 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2822733047' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:51:07.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:07 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2822733047' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:51:07.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:07 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2822733047' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:51:07.758 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:07.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:07 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2822733047' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T21:51:07.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:07 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2822733047' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T21:51:08.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:08 smithi055 ceph-mon[29221]: pgmap v294: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-26T21:51:08.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:08 smithi064 ceph-mon[32765]: pgmap v294: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-26T21:51:08.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:08 smithi165 ceph-mon[33296]: pgmap v294: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-26T21:51:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 -- 172.21.15.55:0/55557986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca81094c0 msgr2=0x7f9ca81098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:09.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 --2- 172.21.15.55:0/55557986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca81094c0 0x7f9ca81098a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009990 tx=0x7f9ca4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 -- 172.21.15.55:0/55557986 shutdown_connections 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 --2- 172.21.15.55:0/55557986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ca8103e30 0x7f9ca8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 --2- 172.21.15.55:0/55557986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca81094c0 0x7f9ca81098a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 --2- 172.21.15.55:0/55557986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ca8103470 0x7f9ca81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 -- 172.21.15.55:0/55557986 >> 172.21.15.55:0/55557986 conn(0x7f9ca807a580 msgr2=0x7f9ca807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 -- 172.21.15.55:0/55557986 shutdown_connections 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.499+0000 7f9cb0580700 1 -- 172.21.15.55:0/55557986 wait complete. 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cb0580700 1 Processor -- start 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cb0580700 1 -- start start 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cb0580700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ca8103470 0x7f9ca819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cb0580700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca8103e30 0x7f9ca819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:09.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cb0580700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ca81094c0 0x7f9ca8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cb0580700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca81135c0 con 0x7f9ca8103e30 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cb0580700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8113440 con 0x7f9ca81094c0 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cb0580700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8113740 con 0x7f9ca8103470 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cadb1b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca8103e30 0x7f9ca819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cadb1b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca8103e30 0x7f9ca819fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45504/0 (socket says 172.21.15.55:45504) 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.500+0000 7f9cadb1b700 1 -- 172.21.15.55:0/335934792 learned_addr learned my addr 172.21.15.55:0/335934792 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9caeb1d700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ca81094c0 0x7f9ca8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cae31c700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ca8103470 0x7f9ca819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cadb1b700 1 -- 172.21.15.55:0/335934792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ca8103470 msgr2=0x7f9ca819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cadb1b700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ca8103470 0x7f9ca819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cadb1b700 1 -- 172.21.15.55:0/335934792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ca81094c0 msgr2=0x7f9ca8199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cadb1b700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ca81094c0 0x7f9ca8199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cadb1b700 1 -- 172.21.15.55:0/335934792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca819a060 con 0x7f9ca8103e30 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9caeb1d700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ca81094c0 0x7f9ca8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:51:09.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cadb1b700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca8103e30 0x7f9ca819fc30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c920 tx=0x7f9c9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:09.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9c9f7fe700 1 -- 172.21.15.55:0/335934792 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98007b00 con 0x7f9ca8103e30 2024-04-26T21:51:09.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca819a350 con 0x7f9ca8103e30 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9c9f7fe700 1 -- 172.21.15.55:0/335934792 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c98004ce0 con 0x7f9ca8103e30 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9c9f7fe700 1 -- 172.21.15.55:0/335934792 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98005650 con 0x7f9ca8103e30 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.501+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca81a65c0 con 0x7f9ca8103e30 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.502+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca806a110 con 0x7f9ca8103e30 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.503+0000 7f9c9f7fe700 1 -- 172.21.15.55:0/335934792 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9c98007ca0 con 0x7f9ca8103e30 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.504+0000 7f9c9f7fe700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9c940a3cd0 0x7f9c940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.504+0000 7f9cae31c700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9c940a3cd0 0x7f9c940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.504+0000 7f9c9f7fe700 1 -- 172.21.15.55:0/335934792 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9c980d0470 con 0x7f9ca8103e30 2024-04-26T21:51:09.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.504+0000 7f9cae31c700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9c940a3cd0 0x7f9c940a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006010 tx=0x7f9ca401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:09.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.508+0000 7f9c9f7fe700 1 -- 172.21.15.55:0/335934792 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9c98014030 con 0x7f9ca8103e30 2024-04-26T21:51:09.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.780+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9ca802e490 con 0x7f9ca8103e30 2024-04-26T21:51:09.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.785+0000 7f9c9f7fe700 1 -- 172.21.15.55:0/335934792 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7f9c9809cda0 con 0x7f9ca8103e30 2024-04-26T21:51:09.789 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:51:09.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9c940a3cd0 msgr2=0x7f9c940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:09.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9c940a3cd0 0x7f9c940a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4006010 tx=0x7f9ca401a040 comp rx=0 tx=0).stop 2024-04-26T21:51:09.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca8103e30 msgr2=0x7f9ca819fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca8103e30 0x7f9ca819fc30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c920 tx=0x7f9c9800cce0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 shutdown_connections 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9c940a3cd0 0x7f9c940a6190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ca81094c0 0x7f9ca8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ca8103470 0x7f9ca819f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 --2- 172.21.15.55:0/335934792 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ca8103e30 0x7f9ca819fc30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 >> 172.21.15.55:0/335934792 conn(0x7f9ca807a580 msgr2=0x7f9ca8102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 shutdown_connections 2024-04-26T21:51:09.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:09.788+0000 7f9cb0580700 1 -- 172.21.15.55:0/335934792 wait complete. 2024-04-26T21:51:10.591 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:10 smithi055 ceph-mon[29221]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 18 op/s 2024-04-26T21:51:10.591 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:10 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/335934792' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:51:10.626 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":3284422558},{"type":"v1","addr":"172.21.15.55:6801","nonce":3284422558}]},"active_addr":"172.21.15.55:6801/3284422558","active_change":"2024-04-26T21:43:49.252329+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.55:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":3125130596}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":591233485}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":1425243009}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":892508231}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":3206114798}]}]} 2024-04-26T21:51:10.633 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T21:51:10.633 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T21:51:10.633 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:51:10.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:10 smithi064 ceph-mon[32765]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 18 op/s 2024-04-26T21:51:10.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:10 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/335934792' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:51:10.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:10 smithi165 ceph-mon[33296]: pgmap v295: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.2 KiB/s wr, 18 op/s 2024-04-26T21:51:10.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:10 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/335934792' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T21:51:10.891 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:12.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:12 smithi165 ceph-mon[33296]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-26T21:51:12.588 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:12 smithi055 ceph-mon[29221]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-26T21:51:12.732 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:12 smithi064 ceph-mon[32765]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.2 KiB/s wr, 16 op/s 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 -- 172.21.15.55:0/1169131851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27001087f0 msgr2=0x7f2700110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 --2- 172.21.15.55:0/1169131851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27001087f0 0x7f2700110370 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f26f800b0d0 tx=0x7f26f800b3e0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 -- 172.21.15.55:0/1169131851 shutdown_connections 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 --2- 172.21.15.55:0/1169131851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27001087f0 0x7f2700110370 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 --2- 172.21.15.55:0/1169131851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f270006bd50 0x7f270006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 --2- 172.21.15.55:0/1169131851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f270006c700 0x7f27001080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 -- 172.21.15.55:0/1169131851 >> 172.21.15.55:0/1169131851 conn(0x7f27000792d0 msgr2=0x7f27000796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 -- 172.21.15.55:0/1169131851 shutdown_connections 2024-04-26T21:51:12.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.838+0000 7f2707d33700 1 -- 172.21.15.55:0/1169131851 wait complete. 2024-04-26T21:51:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2707d33700 1 Processor -- start 2024-04-26T21:51:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2707d33700 1 -- start start 2024-04-26T21:51:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2707d33700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f270006bd50 0x7f270019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2707d33700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f270006c700 0x7f270019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2707d33700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27001087f0 0x7f27001998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2707d33700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2700113660 con 0x7f270006c700 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2707d33700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27001134e0 con 0x7f27001087f0 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2707d33700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f27001137e0 con 0x7f270006bd50 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f27052ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f270006c700 0x7f270019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f27052ce700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f270006c700 0x7f270019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45722/0 (socket says 172.21.15.55:45722) 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f27052ce700 1 -- 172.21.15.55:0/3498789348 learned_addr learned my addr 172.21.15.55:0/3498789348 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f27062d0700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27001087f0 0x7f27001998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.839+0000 7f2705acf700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f270006bd50 0x7f270019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f27052ce700 1 -- 172.21.15.55:0/3498789348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f270006bd50 msgr2=0x7f270019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f27052ce700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f270006bd50 0x7f270019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f27052ce700 1 -- 172.21.15.55:0/3498789348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27001087f0 msgr2=0x7f27001998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f27052ce700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27001087f0 0x7f27001998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f27052ce700 1 -- 172.21.15.55:0/3498789348 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270019a080 con 0x7f270006c700 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f27062d0700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27001087f0 0x7f27001998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f27052ce700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f270006c700 0x7f270019fc10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c9b0 tx=0x7f26f000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f26f6ffd700 1 -- 172.21.15.55:0/3498789348 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0007b00 con 0x7f270006c700 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270019a370 con 0x7f270006c700 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f26f6ffd700 1 -- 172.21.15.55:0/3498789348 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f0007ca0 con 0x7f270006c700 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f26f6ffd700 1 -- 172.21.15.55:0/3498789348 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0018730 con 0x7f270006c700 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.840+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27001a65b0 con 0x7f270006c700 2024-04-26T21:51:12.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.841+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270010ba30 con 0x7f270006c700 2024-04-26T21:51:12.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.843+0000 7f26f6ffd700 1 -- 172.21.15.55:0/3498789348 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f26f000f450 con 0x7f270006c700 2024-04-26T21:51:12.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.843+0000 7f26f6ffd700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26ec0a3cd0 0x7f26ec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:12.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.843+0000 7f2705acf700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26ec0a3cd0 0x7f26ec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:12.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.843+0000 7f26f6ffd700 1 -- 172.21.15.55:0/3498789348 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f26f00d0790 con 0x7f270006c700 2024-04-26T21:51:12.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.844+0000 7f2705acf700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26ec0a3cd0 0x7f26ec0a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009910 tx=0x7f26fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:12.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:12.847+0000 7f26f6ffd700 1 -- 172.21.15.55:0/3498789348 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f26f0014030 con 0x7f270006c700 2024-04-26T21:51:13.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.056+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f270019ad80 con 0x7f270006c700 2024-04-26T21:51:13.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.057+0000 7f26f6ffd700 1 -- 172.21.15.55:0/3498789348 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22185 (secure 0 0 0) 0x7f26f009d250 con 0x7f270006c700 2024-04-26T21:51:13.058 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:51:13.058 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":85,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:50:58.120222+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:51:13.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26ec0a3cd0 msgr2=0x7f26ec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:13.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26ec0a3cd0 0x7f26ec0a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009910 tx=0x7f26fc008040 comp rx=0 tx=0).stop 2024-04-26T21:51:13.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f270006c700 msgr2=0x7f270019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:13.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f270006c700 0x7f270019fc10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c9b0 tx=0x7f26f000cd70 comp rx=0 tx=0).stop 2024-04-26T21:51:13.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 shutdown_connections 2024-04-26T21:51:13.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26ec0a3cd0 0x7f26ec0a6190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:13.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27001087f0 0x7f27001998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:13.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f270006bd50 0x7f270019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:13.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 --2- 172.21.15.55:0/3498789348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f270006c700 0x7f270019fc10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:13.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 >> 172.21.15.55:0/3498789348 conn(0x7f27000792d0 msgr2=0x7f2700100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:13.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 shutdown_connections 2024-04-26T21:51:13.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:13.060+0000 7f2707d33700 1 -- 172.21.15.55:0/3498789348 wait complete. 2024-04-26T21:51:13.479 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:13 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3498789348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:13.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:13 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3498789348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:13.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:13 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3498789348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:13.895 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T21:51:13.895 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T21:51:14.147 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:14.614 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:14 smithi055 ceph-mon[29221]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-26T21:51:14.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:14 smithi064 ceph-mon[32765]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-26T21:51:14.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:14 smithi165 ceph-mon[33296]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-26T21:51:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.892+0000 7f6579231700 1 -- 172.21.15.55:0/2946038574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6574108740 msgr2=0x7f6574110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.892+0000 7f6579231700 1 --2- 172.21.15.55:0/2946038574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6574108740 0x7f6574110350 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f656c00b0d0 tx=0x7f656c00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:51:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.892+0000 7f6579231700 1 -- 172.21.15.55:0/2946038574 shutdown_connections 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.892+0000 7f6579231700 1 --2- 172.21.15.55:0/2946038574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6574108740 0x7f6574110350 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.892+0000 7f6579231700 1 --2- 172.21.15.55:0/2946038574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f657406bd90 0x7f657406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.892+0000 7f6579231700 1 --2- 172.21.15.55:0/2946038574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f657406c740 0x7f6574108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.892+0000 7f6579231700 1 -- 172.21.15.55:0/2946038574 >> 172.21.15.55:0/2946038574 conn(0x7f6574078370 msgr2=0x7f6574078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.893+0000 7f6579231700 1 -- 172.21.15.55:0/2946038574 shutdown_connections 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.893+0000 7f6579231700 1 -- 172.21.15.55:0/2946038574 wait complete. 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.893+0000 7f6579231700 1 Processor -- start 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6579231700 1 -- start start 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6579231700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f657406bd90 0x7f657419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6579231700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f657406c740 0x7f657419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6579231700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6574108740 0x7f6574199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6579231700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6574113640 con 0x7f657406bd90 2024-04-26T21:51:15.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6579231700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f65741134c0 con 0x7f657406c740 2024-04-26T21:51:15.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6579231700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f65741137c0 con 0x7f6574108740 2024-04-26T21:51:15.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6572d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f657406bd90 0x7f657419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:15.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f657259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f657406c740 0x7f657419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:15.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6572d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f657406bd90 0x7f657419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45736/0 (socket says 172.21.15.55:45736) 2024-04-26T21:51:15.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f657259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f657406c740 0x7f657419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46262/0 (socket says 172.21.15.55:46262) 2024-04-26T21:51:15.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6572d9d700 1 -- 172.21.15.55:0/3520692657 learned_addr learned my addr 172.21.15.55:0/3520692657 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:15.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f657359e700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6574108740 0x7f6574199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:15.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6572d9d700 1 -- 172.21.15.55:0/3520692657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6574108740 msgr2=0x7f6574199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:15.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6572d9d700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6574108740 0x7f6574199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:15.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6572d9d700 1 -- 172.21.15.55:0/3520692657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f657406c740 msgr2=0x7f657419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:15.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6572d9d700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f657406c740 0x7f657419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.894+0000 7f6572d9d700 1 -- 172.21.15.55:0/3520692657 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f657419a080 con 0x7f657406bd90 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.895+0000 7f6572d9d700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f657406bd90 0x7f657419f710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f656400d980 tx=0x7f656400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.895+0000 7f655bfff700 1 -- 172.21.15.55:0/3520692657 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564017070 con 0x7f657406bd90 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.895+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f657419a370 con 0x7f657406bd90 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.895+0000 7f655bfff700 1 -- 172.21.15.55:0/3520692657 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6564005020 con 0x7f657406bd90 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.895+0000 7f655bfff700 1 -- 172.21.15.55:0/3520692657 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564009b50 con 0x7f657406bd90 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.895+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65741a65e0 con 0x7f657406bd90 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.896+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657406a110 con 0x7f657406bd90 2024-04-26T21:51:15.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.897+0000 7f655bfff700 1 -- 172.21.15.55:0/3520692657 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f65640051d0 con 0x7f657406bd90 2024-04-26T21:51:15.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.898+0000 7f655bfff700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f655c0a3cd0 0x7f655c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:15.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.898+0000 7f657259c700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f655c0a3cd0 0x7f655c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:15.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.898+0000 7f655bfff700 1 -- 172.21.15.55:0/3520692657 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f65640d0420 con 0x7f657406bd90 2024-04-26T21:51:15.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.898+0000 7f657259c700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f655c0a3cd0 0x7f655c0a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6574109c70 tx=0x7f656800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:15.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:15.903+0000 7f655bfff700 1 -- 172.21.15.55:0/3520692657 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6564020020 con 0x7f657406bd90 2024-04-26T21:51:16.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.114+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f657402e490 con 0x7f657406bd90 2024-04-26T21:51:16.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.116+0000 7f655bfff700 1 -- 172.21.15.55:0/3520692657 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22185 (secure 0 0 0) 0x7f656409cd50 con 0x7f657406bd90 2024-04-26T21:51:16.117 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:51:16.117 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":85,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:50:58.120222+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f655c0a3cd0 msgr2=0x7f655c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f655c0a3cd0 0x7f655c0a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6574109c70 tx=0x7f656800b410 comp rx=0 tx=0).stop 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f657406bd90 msgr2=0x7f657419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f657406bd90 0x7f657419f710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f656400d980 tx=0x7f656400dc90 comp rx=0 tx=0).stop 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 shutdown_connections 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6574108740 0x7f6574199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f657406bd90 0x7f657419f710 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f655c0a3cd0 0x7f655c0a6190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 --2- 172.21.15.55:0/3520692657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f657406c740 0x7f657419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 >> 172.21.15.55:0/3520692657 conn(0x7f6574078370 msgr2=0x7f6574100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 shutdown_connections 2024-04-26T21:51:16.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:16.118+0000 7f6579231700 1 -- 172.21.15.55:0/3520692657 wait complete. 2024-04-26T21:51:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:16 smithi055 ceph-mon[29221]: pgmap v298: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-26T21:51:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:16 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3520692657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:16.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:16 smithi064 ceph-mon[32765]: pgmap v298: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-26T21:51:16.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:16 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3520692657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:16.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:16 smithi165 ceph-mon[33296]: pgmap v298: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-26T21:51:16.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:16 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3520692657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:51:16.934 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T21:51:16.934 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T21:51:16.935 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T21:51:16.935 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T21:51:16.935 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T21:51:16.936 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T21:51:16.936 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T21:51:16.937 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T21:51:16.937 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T21:51:16.937 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T21:51:16.938 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T21:51:16.938 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T21:51:17.662 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:17.843 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.593 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.594 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.594 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.595 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.595 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.595 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.596 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.596 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.597 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.597 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:18.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:18 smithi055 ceph-mon[29221]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T21:51:18.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:18 smithi064 ceph-mon[32765]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T21:51:18.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:18 smithi165 ceph-mon[33296]: pgmap v299: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T21:51:20.589 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:20 smithi055 ceph-mon[29221]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T21:51:20.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:20 smithi064 ceph-mon[32765]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T21:51:20.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:20 smithi165 ceph-mon[33296]: pgmap v300: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T21:51:22.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:22 smithi055 ceph-mon[29221]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T21:51:22.759 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:22 smithi064 ceph-mon[32765]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T21:51:22.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:22 smithi165 ceph-mon[33296]: pgmap v301: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T21:51:24.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:24 smithi055 ceph-mon[29221]: pgmap v302: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T21:51:24.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:24 smithi064 ceph-mon[32765]: pgmap v302: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T21:51:24.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:24 smithi165 ceph-mon[33296]: pgmap v302: 97 pgs: 97 active+clean; 627 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T21:51:26.586 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:26 smithi055 ceph-mon[29221]: pgmap v303: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T21:51:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:26 smithi064 ceph-mon[32765]: pgmap v303: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T21:51:26.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:26 smithi165 ceph-mon[33296]: pgmap v303: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-26T21:51:28.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 -- 172.21.15.55:0/2780885227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff44406bad0 msgr2=0x7ff4441102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 --2- 172.21.15.55:0/2780885227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff44406bad0 0x7ff4441102e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00b0a0 tx=0x7ff43c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 -- 172.21.15.55:0/2780885227 shutdown_connections 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 --2- 172.21.15.55:0/2780885227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff44406bad0 0x7ff4441102e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 --2- 172.21.15.55:0/2780885227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4441094a0 0x7ff444109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 --2- 172.21.15.55:0/2780885227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff44406b130 0x7ff44406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 -- 172.21.15.55:0/2780885227 >> 172.21.15.55:0/2780885227 conn(0x7ff4440792d0 msgr2=0x7ff4440796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 -- 172.21.15.55:0/2780885227 shutdown_connections 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.887+0000 7ff44c5a8700 1 -- 172.21.15.55:0/2780885227 wait complete. 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44c5a8700 1 Processor -- start 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44c5a8700 1 -- start start 2024-04-26T21:51:28.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44c5a8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff44406b130 0x7ff4441a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44c5a8700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff44406bad0 0x7ff4441a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44c5a8700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4441094a0 0x7ff44419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44c5a8700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4441135c0 con 0x7ff4441094a0 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44c5a8700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff444113440 con 0x7ff44406b130 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44c5a8700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff444113740 con 0x7ff44406bad0 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4441094a0 0x7ff44419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4441094a0 0x7ff44419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51528/0 (socket says 172.21.15.55:51528) 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 -- 172.21.15.55:0/3581846290 learned_addr learned my addr 172.21.15.55:0/3581846290 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:28.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44a344700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff44406b130 0x7ff4441a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 -- 172.21.15.55:0/3581846290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff44406bad0 msgr2=0x7ff4441a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff44406bad0 0x7ff4441a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 -- 172.21.15.55:0/3581846290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff44406b130 msgr2=0x7ff4441a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff44406b130 0x7ff4441a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 -- 172.21.15.55:0/3581846290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44419c440 con 0x7ff4441094a0 2024-04-26T21:51:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44a344700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff44406b130 0x7ff4441a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:51:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.888+0000 7ff44ab45700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4441094a0 0x7ff44419bb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00b830 tx=0x7ff43c004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:28.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.889+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c021070 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.889+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff43c003c00 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.889+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44419c6d0 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.889+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c004730 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.889+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4441a89c0 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.889+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff444063b30 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.891+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff43c003db0 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.891+0000 7ff43b7fe700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4300a3d50 0x7ff4300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.891+0000 7ff44a344700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4300a3d50 0x7ff4300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.891+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff43c00f020 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.892+0000 7ff43b7fe700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] conn(0x7ff4300ad530 0x7ff4300af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.892+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 --> [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff43c01de50 con 0x7ff4300ad530 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.892+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ff43c0d2120 con 0x7ff4441094a0 2024-04-26T21:51:28.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.892+0000 7ff449b43700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] conn(0x7ff4300ad530 0x7ff4300af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.892+0000 7ff44a344700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4300a3d50 0x7ff4300a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff440005e10 tx=0x7ff440005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.892+0000 7ff449b43700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] conn(0x7ff4300ad530 0x7ff4300af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.893+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 <== osd.9 v2:172.21.15.55:6826/3581040676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff43c01de50 con 0x7ff4300ad530 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.909+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 --> [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff444063f60 con 0x7ff4300ad530 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.909+0000 7ff43b7fe700 1 -- 172.21.15.55:0/3581846290 <== osd.9 v2:172.21.15.55:6826/3581040676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff444063f60 con 0x7ff4300ad530 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.909+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] conn(0x7ff4300ad530 msgr2=0x7ff4300af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.909+0000 7ff44c5a8700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] conn(0x7ff4300ad530 0x7ff4300af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4300a3d50 msgr2=0x7ff4300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44c5a8700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4300a3d50 0x7ff4300a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff440005e10 tx=0x7ff440005d80 comp rx=0 tx=0).stop 2024-04-26T21:51:28.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4441094a0 msgr2=0x7ff44419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:28.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44c5a8700 1 --2- 172.21.15.55:0/3581846290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4441094a0 0x7ff44419bb60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00b830 tx=0x7ff43c004d90 comp rx=0 tx=0).stop 2024-04-26T21:51:28.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44ab45700 1 -- 172.21.15.55:0/3581846290 reap_dead start 2024-04-26T21:51:28.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 shutdown_connections 2024-04-26T21:51:28.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 >> 172.21.15.55:0/3581846290 conn(0x7ff4440792d0 msgr2=0x7ff44410e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:28.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 shutdown_connections 2024-04-26T21:51:28.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:26.910+0000 7ff44c5a8700 1 -- 172.21.15.55:0/3581846290 wait complete. 2024-04-26T21:51:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:28 smithi064 ceph-mon[32765]: pgmap v304: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T21:51:28.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:28 smithi165 ceph-mon[33296]: pgmap v304: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T21:51:28.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:28 smithi055 ceph-mon[29221]: pgmap v304: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T21:51:30.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:30 smithi055 ceph-mon[29221]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T21:51:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:30 smithi064 ceph-mon[32765]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T21:51:30.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:30 smithi165 ceph-mon[33296]: pgmap v305: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T21:51:30.963 INFO:teuthology.orchestra.run.smithi055.stdout:287762808862 2024-04-26T21:51:30.963 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T21:51:31.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 -- 172.21.15.55:0/1906018422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 msgr2=0x7f11cc06bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 --2- 172.21.15.55:0/1906018422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 0x7f11cc06bf50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009990 tx=0x7f11bc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 -- 172.21.15.55:0/1906018422 shutdown_connections 2024-04-26T21:51:31.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 --2- 172.21.15.55:0/1906018422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11cc06c490 0x7f11cc110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 --2- 172.21.15.55:0/1906018422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11cc1094c0 0x7f11cc1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 --2- 172.21.15.55:0/1906018422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 0x7f11cc06bf50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 -- 172.21.15.55:0/1906018422 >> 172.21.15.55:0/1906018422 conn(0x7f11cc0792d0 msgr2=0x7f11cc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:31.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 -- 172.21.15.55:0/1906018422 shutdown_connections 2024-04-26T21:51:31.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.400+0000 7f11d3d09700 1 -- 172.21.15.55:0/1906018422 wait complete. 2024-04-26T21:51:31.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d3d09700 1 Processor -- start 2024-04-26T21:51:31.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d3d09700 1 -- start start 2024-04-26T21:51:31.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d3d09700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 0x7f11cc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d3d09700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11cc06c490 0x7f11cc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d3d09700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11cc1094c0 0x7f11cc199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d3d09700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1135b0 con 0x7f11cc06baf0 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d3d09700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc113430 con 0x7f11cc1094c0 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d3d09700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc113730 con 0x7f11cc06c490 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d1aa5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 0x7f11cc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d1aa5700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 0x7f11cc19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51460/0 (socket says 172.21.15.55:51460) 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d1aa5700 1 -- 172.21.15.55:0/1236395579 learned_addr learned my addr 172.21.15.55:0/1236395579 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.401+0000 7f11d22a6700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11cc1094c0 0x7f11cc199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d12a4700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11cc06c490 0x7f11cc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d1aa5700 1 -- 172.21.15.55:0/1236395579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11cc06c490 msgr2=0x7f11cc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d1aa5700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11cc06c490 0x7f11cc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d1aa5700 1 -- 172.21.15.55:0/1236395579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11cc1094c0 msgr2=0x7f11cc199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d1aa5700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11cc1094c0 0x7f11cc199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d1aa5700 1 -- 172.21.15.55:0/1236395579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc19a090 con 0x7f11cc06baf0 2024-04-26T21:51:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d1aa5700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 0x7f11cc19f770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f11c800d950 tx=0x7f11c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8016070 con 0x7f11cc06baf0 2024-04-26T21:51:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc19a380 con 0x7f11cc06baf0 2024-04-26T21:51:31.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c8009df0 con 0x7f11cc06baf0 2024-04-26T21:51:31.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c800f3c0 con 0x7f11cc06baf0 2024-04-26T21:51:31.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.402+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc1a6640 con 0x7f11cc06baf0 2024-04-26T21:51:31.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.403+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11cc063bb0 con 0x7f11cc06baf0 2024-04-26T21:51:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.405+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f11c8019070 con 0x7f11cc06baf0 2024-04-26T21:51:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.406+0000 7f11c2ffd700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11b80a3d50 0x7f11b80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.406+0000 7f11d12a4700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11b80a3d50 0x7f11b80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.406+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f11c80cfe90 con 0x7f11cc06baf0 2024-04-26T21:51:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.406+0000 7f11c2ffd700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] conn(0x7f11b80ad530 0x7f11b80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.406+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 --> [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11b80b0000 con 0x7f11b80ad530 2024-04-26T21:51:31.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.406+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f11c80d25e0 con 0x7f11cc06baf0 2024-04-26T21:51:31.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.406+0000 7f11d22a6700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] conn(0x7f11b80ad530 0x7f11b80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.406+0000 7f11d22a6700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] conn(0x7f11b80ad530 0x7f11b80af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:31.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.407+0000 7f11d12a4700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11b80a3d50 0x7f11b80a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f11bc006010 tx=0x7f11bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:31.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.407+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 <== osd.3 v2:172.21.15.55:6810/3700452481 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11b80b0000 con 0x7f11b80ad530 2024-04-26T21:51:31.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.424+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 --> [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11cc063fe0 con 0x7f11b80ad530 2024-04-26T21:51:31.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.424+0000 7f11c2ffd700 1 -- 172.21.15.55:0/1236395579 <== osd.3 v2:172.21.15.55:6810/3700452481 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f11cc063fe0 con 0x7f11b80ad530 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.424+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] conn(0x7f11b80ad530 msgr2=0x7f11b80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.424+0000 7f11d3d09700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] conn(0x7f11b80ad530 0x7f11b80af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11b80a3d50 msgr2=0x7f11b80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d3d09700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f11b80a3d50 0x7f11b80a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f11bc006010 tx=0x7f11bc01a040 comp rx=0 tx=0).stop 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 msgr2=0x7f11cc19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d3d09700 1 --2- 172.21.15.55:0/1236395579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11cc06baf0 0x7f11cc19f770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f11c800d950 tx=0x7f11c800dc60 comp rx=0 tx=0).stop 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d22a6700 1 -- 172.21.15.55:0/1236395579 reap_dead start 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 shutdown_connections 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 >> 172.21.15.55:0/1236395579 conn(0x7f11cc0792d0 msgr2=0x7f11cc1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 shutdown_connections 2024-04-26T21:51:31.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.425+0000 7f11d3d09700 1 -- 172.21.15.55:0/1236395579 wait complete. 2024-04-26T21:51:31.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 -- 172.21.15.55:0/1026430678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 msgr2=0x7f8bac10f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 --2- 172.21.15.55:0/1026430678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 0x7f8bac10f1c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400b0a0 tx=0x7f8ba400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 -- 172.21.15.55:0/1026430678 shutdown_connections 2024-04-26T21:51:31.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 --2- 172.21.15.55:0/1026430678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 0x7f8bac10f1c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 --2- 172.21.15.55:0/1026430678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bac108380 0x7f8bac108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 --2- 172.21.15.55:0/1026430678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bac102310 0x7f8bac102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 -- 172.21.15.55:0/1026430678 >> 172.21.15.55:0/1026430678 conn(0x7f8bac079290 msgr2=0x7f8bac0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 -- 172.21.15.55:0/1026430678 shutdown_connections 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.269+0000 7f8bb218c700 1 -- 172.21.15.55:0/1026430678 wait complete. 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bb218c700 1 Processor -- start 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bb218c700 1 -- start start 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bb218c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bac102310 0x7f8bac19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bb218c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 0x7f8bac19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bb218c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bac108380 0x7f8bac198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bb218c700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac1124a0 con 0x7f8bac102cd0 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bb218c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac112320 con 0x7f8bac108380 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bb218c700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac112620 con 0x7f8bac102310 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8baaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 0x7f8bac19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8baaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 0x7f8bac19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51440/0 (socket says 172.21.15.55:51440) 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8baaffd700 1 -- 172.21.15.55:0/3543016795 learned_addr learned my addr 172.21.15.55:0/3543016795 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:31.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.270+0000 7f8bab7fe700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bac102310 0x7f8bac19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8babfff700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bac108380 0x7f8bac198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8baaffd700 1 -- 172.21.15.55:0/3543016795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bac102310 msgr2=0x7f8bac19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8baaffd700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bac102310 0x7f8bac19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8baaffd700 1 -- 172.21.15.55:0/3543016795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bac108380 msgr2=0x7f8bac198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8baaffd700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bac108380 0x7f8bac198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8baaffd700 1 -- 172.21.15.55:0/3543016795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac198fe0 con 0x7f8bac102cd0 2024-04-26T21:51:31.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8baaffd700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 0x7f8bac19eb70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0007b50 tx=0x7f8ba0007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:31.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000e500 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba000eb50 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bac199270 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000aab0 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.271+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bac06bb60 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.272+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8bac06c110 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.273+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8ba000ac50 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.274+0000 7f8ba8ff9700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b940a3cd0 0x7f8b940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.274+0000 7f8bab7fe700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b940a3cd0 0x7f8b940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.274+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f8ba00d1db0 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.274+0000 7f8ba8ff9700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] conn(0x7f8b940ad4b0 0x7f8b940af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.274+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 --> [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b940aff80 con 0x7f8b940ad4b0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.274+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f8ba00d2160 con 0x7f8bac102cd0 2024-04-26T21:51:31.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.274+0000 7f8babfff700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] conn(0x7f8b940ad4b0 0x7f8b940af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.274+0000 7f8bab7fe700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b940a3cd0 0x7f8b940a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009be0 tx=0x7f8b9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.275+0000 7f8babfff700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] conn(0x7f8b940ad4b0 0x7f8b940af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.275+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 <== osd.6 v2:172.21.15.55:6818/4025342485 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b940aff80 con 0x7f8b940ad4b0 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.292+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 --> [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8bac06c590 con 0x7f8b940ad4b0 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8ba8ff9700 1 -- 172.21.15.55:0/3543016795 <== osd.6 v2:172.21.15.55:6818/4025342485 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8bac06c590 con 0x7f8b940ad4b0 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] conn(0x7f8b940ad4b0 msgr2=0x7f8b940af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8bb218c700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] conn(0x7f8b940ad4b0 0x7f8b940af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b940a3cd0 msgr2=0x7f8b940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8bb218c700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b940a3cd0 0x7f8b940a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009be0 tx=0x7f8b9c009260 comp rx=0 tx=0).stop 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 msgr2=0x7f8bac19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8bb218c700 1 --2- 172.21.15.55:0/3543016795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bac102cd0 0x7f8bac19eb70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0007b50 tx=0x7f8ba0007e60 comp rx=0 tx=0).stop 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8babfff700 1 -- 172.21.15.55:0/3543016795 reap_dead start 2024-04-26T21:51:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 shutdown_connections 2024-04-26T21:51:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.293+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 >> 172.21.15.55:0/3543016795 conn(0x7f8bac079290 msgr2=0x7f8bac100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.294+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 shutdown_connections 2024-04-26T21:51:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.294+0000 7f8bb218c700 1 -- 172.21.15.55:0/3543016795 wait complete. 2024-04-26T21:51:32.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.165+0000 7f703a3a0700 1 -- 172.21.15.55:0/233191738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102320 msgr2=0x7f70341027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.165+0000 7f703a3a0700 1 --2- 172.21.15.55:0/233191738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102320 0x7f70341027a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7028009990 tx=0x7f7028009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.165+0000 7f703a3a0700 1 -- 172.21.15.55:0/233191738 shutdown_connections 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.165+0000 7f703a3a0700 1 --2- 172.21.15.55:0/233191738 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7034102ce0 0x7f703410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.165+0000 7f703a3a0700 1 --2- 172.21.15.55:0/233191738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7034108390 0x7f7034108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.165+0000 7f703a3a0700 1 --2- 172.21.15.55:0/233191738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102320 0x7f70341027a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.165+0000 7f703a3a0700 1 -- 172.21.15.55:0/233191738 >> 172.21.15.55:0/233191738 conn(0x7f70340792d0 msgr2=0x7f70340796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.165+0000 7f703a3a0700 1 -- 172.21.15.55:0/233191738 shutdown_connections 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.166+0000 7f703a3a0700 1 -- 172.21.15.55:0/233191738 wait complete. 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.166+0000 7f703a3a0700 1 Processor -- start 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.166+0000 7f703a3a0700 1 -- start start 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f703a3a0700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7034102320 0x7f70341987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f703a3a0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102ce0 0x7f7034198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f703a3a0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7034108390 0x7f7034199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f703a3a0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7034112550 con 0x7f7034102ce0 2024-04-26T21:51:32.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f703a3a0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f70341123d0 con 0x7f7034108390 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f703a3a0700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70341126d0 con 0x7f7034102320 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f70337fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102ce0 0x7f7034198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f70337fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102ce0 0x7f7034198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51428/0 (socket says 172.21.15.55:51428) 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f70337fe700 1 -- 172.21.15.55:0/3894459726 learned_addr learned my addr 172.21.15.55:0/3894459726 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f7033fff700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7034102320 0x7f70341987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f703893d700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7034108390 0x7f7034199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f70337fe700 1 -- 172.21.15.55:0/3894459726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7034102320 msgr2=0x7f70341987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f70337fe700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7034102320 0x7f70341987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f70337fe700 1 -- 172.21.15.55:0/3894459726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7034108390 msgr2=0x7f7034199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f70337fe700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7034108390 0x7f7034199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f70337fe700 1 -- 172.21.15.55:0/3894459726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70341a7ff0 con 0x7f7034102ce0 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.167+0000 7f703893d700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7034108390 0x7f7034199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.168+0000 7f70337fe700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102ce0 0x7f7034198d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c00 tx=0x7f70280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:32.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.168+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028027070 con 0x7f7034102ce0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.168+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70280040e0 con 0x7f7034102ce0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.168+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70341a8280 con 0x7f7034102ce0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.168+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702800f460 con 0x7f7034102ce0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.168+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70341a8730 con 0x7f7034102ce0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.169+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7034063bb0 con 0x7f7034102ce0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.170+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f702802a070 con 0x7f7034102ce0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.171+0000 7f70317fa700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f701c0a3d50 0x7f701c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.171+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f70280d12e0 con 0x7f7034102ce0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.171+0000 7f7033fff700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f701c0a3d50 0x7f701c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.171+0000 7f70317fa700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] conn(0x7f701c0ad530 0x7f701c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.171+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 --> [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f701c0b0000 con 0x7f701c0ad530 2024-04-26T21:51:32.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.171+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f70280d8050 con 0x7f7034102ce0 2024-04-26T21:51:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.171+0000 7f7033fff700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f701c0a3d50 0x7f701c0a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7024005950 tx=0x7f702400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.171+0000 7f703893d700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] conn(0x7f701c0ad530 0x7f701c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.172+0000 7f703893d700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] conn(0x7f701c0ad530 0x7f701c0af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.173+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 <== osd.2 v2:172.21.15.165:6800/2376036424 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f701c0b0000 con 0x7f701c0ad530 2024-04-26T21:51:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.189+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 --> [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7034064060 con 0x7f701c0ad530 2024-04-26T21:51:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.189+0000 7f70317fa700 1 -- 172.21.15.55:0/3894459726 <== osd.2 v2:172.21.15.165:6800/2376036424 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7034064060 con 0x7f701c0ad530 2024-04-26T21:51:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 >> [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] conn(0x7f701c0ad530 msgr2=0x7f701c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] conn(0x7f701c0ad530 0x7f701c0af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f701c0a3d50 msgr2=0x7f701c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f701c0a3d50 0x7f701c0a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7024005950 tx=0x7f702400a300 comp rx=0 tx=0).stop 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102ce0 msgr2=0x7f7034198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 --2- 172.21.15.55:0/3894459726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034102ce0 0x7f7034198d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c00 tx=0x7f70280047b0 comp rx=0 tx=0).stop 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703893d700 1 -- 172.21.15.55:0/3894459726 reap_dead start 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 shutdown_connections 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 >> 172.21.15.55:0/3894459726 conn(0x7f70340792d0 msgr2=0x7f7034100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 shutdown_connections 2024-04-26T21:51:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:31.190+0000 7f703a3a0700 1 -- 172.21.15.55:0/3894459726 wait complete. 2024-04-26T21:51:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:32 smithi064 ceph-mon[32765]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T21:51:32.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:32 smithi165 ceph-mon[33296]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T21:51:32.897 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:32 smithi055 ceph-mon[29221]: pgmap v306: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-26T21:51:32.930 INFO:teuthology.orchestra.run.smithi055.stdout:77309411394 2024-04-26T21:51:32.931 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T21:51:33.488 INFO:teuthology.orchestra.run.smithi055.stdout:197568495663 2024-04-26T21:51:33.488 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T21:51:34.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.120+0000 7fd6e250a700 1 -- 172.21.15.55:0/4276625253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc1094e0 msgr2=0x7fd6dc1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.120+0000 7fd6e250a700 1 --2- 172.21.15.55:0/4276625253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc1094e0 0x7fd6dc1098c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009990 tx=0x7fd6cc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.120+0000 7fd6e250a700 1 -- 172.21.15.55:0/4276625253 shutdown_connections 2024-04-26T21:51:34.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.120+0000 7fd6e250a700 1 --2- 172.21.15.55:0/4276625253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc103e40 0x7fd6dc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.120+0000 7fd6e250a700 1 --2- 172.21.15.55:0/4276625253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc1094e0 0x7fd6dc1098c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.120+0000 7fd6e250a700 1 --2- 172.21.15.55:0/4276625253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc1034a0 0x7fd6dc103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.120+0000 7fd6e250a700 1 -- 172.21.15.55:0/4276625253 >> 172.21.15.55:0/4276625253 conn(0x7fd6dc07a4f0 msgr2=0x7fd6dc07a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:34.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.120+0000 7fd6e250a700 1 -- 172.21.15.55:0/4276625253 shutdown_connections 2024-04-26T21:51:34.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.121+0000 7fd6e250a700 1 -- 172.21.15.55:0/4276625253 wait complete. 2024-04-26T21:51:34.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.121+0000 7fd6e250a700 1 Processor -- start 2024-04-26T21:51:34.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.121+0000 7fd6e250a700 1 -- start start 2024-04-26T21:51:34.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e250a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc1034a0 0x7fd6dc075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:34.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e250a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc103e40 0x7fd6dc075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:34.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e250a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc1094e0 0x7fd6dc0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:34.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e250a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc1135f0 con 0x7fd6dc103e40 2024-04-26T21:51:34.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e250a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc113470 con 0x7fd6dc1094e0 2024-04-26T21:51:34.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e250a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc113770 con 0x7fd6dc1034a0 2024-04-26T21:51:34.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6db7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc103e40 0x7fd6dc075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:34.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6db7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc103e40 0x7fd6dc075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51478/0 (socket says 172.21.15.55:51478) 2024-04-26T21:51:34.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6db7fe700 1 -- 172.21.15.55:0/2632366905 learned_addr learned my addr 172.21.15.55:0/2632366905 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:34.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e0aa7700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc1094e0 0x7fd6dc0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:34.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6dbfff700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc1034a0 0x7fd6dc075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:34.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e0aa7700 1 -- 172.21.15.55:0/2632366905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc1034a0 msgr2=0x7fd6dc075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e0aa7700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc1034a0 0x7fd6dc075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e0aa7700 1 -- 172.21.15.55:0/2632366905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc103e40 msgr2=0x7fd6dc075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e0aa7700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc103e40 0x7fd6dc075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6e0aa7700 1 -- 172.21.15.55:0/2632366905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc1a8030 con 0x7fd6dc1094e0 2024-04-26T21:51:34.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.122+0000 7fd6db7fe700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc103e40 0x7fd6dc075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:51:34.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.123+0000 7fd6e0aa7700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc1094e0 0x7fd6dc0761a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400efc0 tx=0x7fd6d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:34.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.123+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40092e0 con 0x7fd6dc1094e0 2024-04-26T21:51:34.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.123+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc1a82c0 con 0x7fd6dc1094e0 2024-04-26T21:51:34.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.123+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d4004bb0 con 0x7fd6dc1094e0 2024-04-26T21:51:34.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.123+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4004020 con 0x7fd6dc1094e0 2024-04-26T21:51:34.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.123+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc1a8800 con 0x7fd6dc1094e0 2024-04-26T21:51:34.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.124+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6dc063b30 con 0x7fd6dc1094e0 2024-04-26T21:51:34.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.125+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd6d401b030 con 0x7fd6dc1094e0 2024-04-26T21:51:34.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.126+0000 7fd6d97fa700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d50 0x7fd6c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:34.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.126+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd6d40cfff0 con 0x7fd6dc1094e0 2024-04-26T21:51:34.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.126+0000 7fd6dbfff700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d50 0x7fd6c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:34.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.126+0000 7fd6d97fa700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] conn(0x7fd6c40ad530 0x7fd6c40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:34.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.126+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 --> [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6c40b0000 con 0x7fd6c40ad530 2024-04-26T21:51:34.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.126+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd6d40d0400 con 0x7fd6dc1094e0 2024-04-26T21:51:34.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.127+0000 7fd6db7fe700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] conn(0x7fd6c40ad530 0x7fd6c40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:34.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.127+0000 7fd6dbfff700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d50 0x7fd6c40a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b2e0 tx=0x7fd6cc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:34.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.127+0000 7fd6db7fe700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] conn(0x7fd6c40ad530 0x7fd6c40af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:34.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.128+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 <== osd.10 v2:172.21.15.64:6824/3001316208 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6c40b0000 con 0x7fd6c40ad530 2024-04-26T21:51:34.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 --> [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd6dc063f30 con 0x7fd6c40ad530 2024-04-26T21:51:34.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6d97fa700 1 -- 172.21.15.55:0/2632366905 <== osd.10 v2:172.21.15.64:6824/3001316208 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd6dc063f30 con 0x7fd6c40ad530 2024-04-26T21:51:34.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] conn(0x7fd6c40ad530 msgr2=0x7fd6c40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e250a700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] conn(0x7fd6c40ad530 0x7fd6c40af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d50 msgr2=0x7fd6c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e250a700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d50 0x7fd6c40a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b2e0 tx=0x7fd6cc005960 comp rx=0 tx=0).stop 2024-04-26T21:51:34.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc1094e0 msgr2=0x7fd6dc0761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:34.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e250a700 1 --2- 172.21.15.55:0/2632366905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc1094e0 0x7fd6dc0761a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400efc0 tx=0x7fd6d400c3a0 comp rx=0 tx=0).stop 2024-04-26T21:51:34.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e0aa7700 1 -- 172.21.15.55:0/2632366905 reap_dead start 2024-04-26T21:51:34.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.145+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 shutdown_connections 2024-04-26T21:51:34.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.146+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 >> 172.21.15.55:0/2632366905 conn(0x7fd6dc07a4f0 msgr2=0x7fd6dc1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:34.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.146+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 shutdown_connections 2024-04-26T21:51:34.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:34.146+0000 7fd6e250a700 1 -- 172.21.15.55:0/2632366905 wait complete. 2024-04-26T21:51:34.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:34 smithi055 ceph-mon[29221]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T21:51:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:34 smithi064 ceph-mon[32765]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T21:51:34.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:34 smithi165 ceph-mon[33296]: pgmap v307: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T21:51:35.364 INFO:teuthology.orchestra.run.smithi055.stdout:107374182462 2024-04-26T21:51:35.364 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T21:51:36.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 -- 172.21.15.55:0/1538723612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec107640 msgr2=0x7f58ec10f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:36.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 --2- 172.21.15.55:0/1538723612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec107640 0x7f58ec10f210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b0d0 tx=0x7f58dc00b3e0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 -- 172.21.15.55:0/1538723612 shutdown_connections 2024-04-26T21:51:36.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 --2- 172.21.15.55:0/1538723612 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec107640 0x7f58ec10f210 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 --2- 172.21.15.55:0/1538723612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58ec1025b0 0x7f58ec102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 --2- 172.21.15.55:0/1538723612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58ec102f60 0x7f58ec106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 -- 172.21.15.55:0/1538723612 >> 172.21.15.55:0/1538723612 conn(0x7f58ec0792d0 msgr2=0x7f58ec0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:36.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 -- 172.21.15.55:0/1538723612 shutdown_connections 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.360+0000 7f58f1214700 1 -- 172.21.15.55:0/1538723612 wait complete. 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.361+0000 7f58f1214700 1 Processor -- start 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.361+0000 7f58f1214700 1 -- start start 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.361+0000 7f58f1214700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec1025b0 0x7f58ec19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.361+0000 7f58f1214700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58ec102f60 0x7f58ec19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.361+0000 7f58f1214700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58ec107640 0x7f58ec199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58f1214700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec112500 con 0x7f58ec1025b0 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58f1214700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec112380 con 0x7f58ec107640 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58f1214700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec112680 con 0x7f58ec102f60 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec1025b0 0x7f58ec19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ea59c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58ec102f60 0x7f58ec19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec1025b0 0x7f58ec19f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51492/0 (socket says 172.21.15.55:51492) 2024-04-26T21:51:36.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 -- 172.21.15.55:0/2224558532 learned_addr learned my addr 172.21.15.55:0/2224558532 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58eb59e700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58ec107640 0x7f58ec199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 -- 172.21.15.55:0/2224558532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58ec102f60 msgr2=0x7f58ec19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58ec102f60 0x7f58ec19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 -- 172.21.15.55:0/2224558532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58ec107640 msgr2=0x7f58ec199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58ec107640 0x7f58ec199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 -- 172.21.15.55:0/2224558532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec19a0d0 con 0x7f58ec1025b0 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58eb59e700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58ec107640 0x7f58ec199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.362+0000 7f58ead9d700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec1025b0 0x7f58ec19f7b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58e400e9a0 tx=0x7f58e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.363+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e400ca10 con 0x7f58ec1025b0 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.363+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec19a3c0 con 0x7f58ec1025b0 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.363+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec103f30 con 0x7f58ec1025b0 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.363+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e400cbb0 con 0x7f58ec1025b0 2024-04-26T21:51:36.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.363+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4018680 con 0x7f58ec1025b0 2024-04-26T21:51:36.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.364+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58cc001030 con 0x7f58ec1025b0 2024-04-26T21:51:36.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.366+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f58e4010430 con 0x7f58ec1025b0 2024-04-26T21:51:36.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.366+0000 7f58e3fff700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58d80a3cd0 0x7f58d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:36.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.367+0000 7f58ea59c700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58d80a3cd0 0x7f58d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:36.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.367+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f58e40d10e0 con 0x7f58ec1025b0 2024-04-26T21:51:36.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.367+0000 7f58e3fff700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] conn(0x7f58d80ad510 0x7f58d80af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:36.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.367+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 --> [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58e4004500 con 0x7f58d80ad510 2024-04-26T21:51:36.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.367+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f58e40d1490 con 0x7f58ec1025b0 2024-04-26T21:51:36.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.367+0000 7f58eb59e700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] conn(0x7f58d80ad510 0x7f58d80af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:36.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.367+0000 7f58ea59c700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58d80a3cd0 0x7f58d80a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f58ec102220 tx=0x7f58d4012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:36.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.367+0000 7f58eb59e700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] conn(0x7f58d80ad510 0x7f58d80af930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:36.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.369+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 <== osd.5 v2:172.21.15.165:6808/1409137199 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58e4004500 con 0x7f58d80ad510 2024-04-26T21:51:36.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.388+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 --> [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58cc002e60 con 0x7f58d80ad510 2024-04-26T21:51:36.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58e3fff700 1 -- 172.21.15.55:0/2224558532 <== osd.5 v2:172.21.15.165:6808/1409137199 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f58cc002e60 con 0x7f58d80ad510 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 >> [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] conn(0x7f58d80ad510 msgr2=0x7f58d80af930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] conn(0x7f58d80ad510 0x7f58d80af930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58d80a3cd0 msgr2=0x7f58d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58d80a3cd0 0x7f58d80a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f58ec102220 tx=0x7f58d4012040 comp rx=0 tx=0).stop 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec1025b0 msgr2=0x7f58ec19f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 --2- 172.21.15.55:0/2224558532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f58ec1025b0 0x7f58ec19f7b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58e400e9a0 tx=0x7f58e400ecb0 comp rx=0 tx=0).stop 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58eb59e700 1 -- 172.21.15.55:0/2224558532 reap_dead start 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 shutdown_connections 2024-04-26T21:51:36.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 >> 172.21.15.55:0/2224558532 conn(0x7f58ec0792d0 msgr2=0x7f58ec06be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:36.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 shutdown_connections 2024-04-26T21:51:36.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:36.389+0000 7f58f1214700 1 -- 172.21.15.55:0/2224558532 wait complete. 2024-04-26T21:51:36.629 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:36 smithi055 ceph-mon[29221]: pgmap v308: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T21:51:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:36 smithi064 ceph-mon[32765]: pgmap v308: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T21:51:36.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:36 smithi165 ceph-mon[33296]: pgmap v308: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T21:51:37.298 INFO:teuthology.orchestra.run.smithi055.stdout:309237645338 2024-04-26T21:51:37.298 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T21:51:37.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.577+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/869640899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c981094e0 msgr2=0x7f2c981098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.577+0000 7f2c9cf6e700 1 --2- 172.21.15.55:0/869640899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c981094e0 0x7f2c981098c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009990 tx=0x7f2c88009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.577+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/869640899 shutdown_connections 2024-04-26T21:51:37.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.578+0000 7f2c9cf6e700 1 --2- 172.21.15.55:0/869640899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c98103e60 0x7f2c981102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.578+0000 7f2c9cf6e700 1 --2- 172.21.15.55:0/869640899 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c981094e0 0x7f2c981098c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.578+0000 7f2c9cf6e700 1 --2- 172.21.15.55:0/869640899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c981034a0 0x7f2c98103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.578+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/869640899 >> 172.21.15.55:0/869640899 conn(0x7f2c9807a590 msgr2=0x7f2c9807a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:37.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.578+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/869640899 shutdown_connections 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.578+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/869640899 wait complete. 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.578+0000 7f2c9cf6e700 1 Processor -- start 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.578+0000 7f2c9cf6e700 1 -- start start 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9cf6e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c981034a0 0x7f2c9819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9cf6e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c98103e60 0x7f2c9819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9cf6e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c981094e0 0x7f2c981a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9cf6e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98113610 con 0x7f2c98103e60 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9cf6e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98113490 con 0x7f2c981034a0 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9cf6e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98113790 con 0x7f2c981094e0 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c98103e60 0x7f2c9819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c98103e60 0x7f2c9819c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51504/0 (socket says 172.21.15.55:51504) 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 -- 172.21.15.55:0/749028378 learned_addr learned my addr 172.21.15.55:0/749028378 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c96d9d700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c981094e0 0x7f2c981a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9659c700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c981034a0 0x7f2c9819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 -- 172.21.15.55:0/749028378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c981094e0 msgr2=0x7f2c981a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c981094e0 0x7f2c981a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 -- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c981034a0 msgr2=0x7f2c9819bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c981034a0 0x7f2c9819bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 -- 172.21.15.55:0/749028378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c981a0db0 con 0x7f2c98103e60 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9659c700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c981034a0 0x7f2c9819bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c95d9b700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c98103e60 0x7f2c9819c2e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00c970 tx=0x7f2c8c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c007aa0 con 0x7f2c98103e60 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c8c004ce0 con 0x7f2c98103e60 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.579+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c981a10a0 con 0x7f2c98103e60 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.580+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c005510 con 0x7f2c98103e60 2024-04-26T21:51:37.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.580+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981a8920 con 0x7f2c98103e60 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.580+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c981102b0 con 0x7f2c98103e60 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.582+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2c8c007c40 con 0x7f2c98103e60 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.582+0000 7f2c877fe700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2c800a3cd0 0x7f2c800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.582+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f2c8c0d05d0 con 0x7f2c98103e60 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.583+0000 7f2c877fe700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] conn(0x7f2c800ad4b0 0x7f2c800af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.583+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 --> [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c800aff80 con 0x7f2c800ad4b0 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.583+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f2c8c0d0ab0 con 0x7f2c98103e60 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.583+0000 7f2c9659c700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2c800a3cd0 0x7f2c800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.583+0000 7f2c96d9d700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] conn(0x7f2c800ad4b0 0x7f2c800af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.583+0000 7f2c9659c700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2c800a3cd0 0x7f2c800a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c88006010 tx=0x7f2c8800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.583+0000 7f2c96d9d700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] conn(0x7f2c800ad4b0 0x7f2c800af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.584+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 <== osd.1 v2:172.21.15.64:6800/794827991 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c800aff80 con 0x7f2c800ad4b0 2024-04-26T21:51:37.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.602+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 --> [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2c98104b80 con 0x7f2c800ad4b0 2024-04-26T21:51:37.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c877fe700 1 -- 172.21.15.55:0/749028378 <== osd.1 v2:172.21.15.64:6800/794827991 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2c98104b80 con 0x7f2c800ad4b0 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] conn(0x7f2c800ad4b0 msgr2=0x7f2c800af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] conn(0x7f2c800ad4b0 0x7f2c800af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2c800a3cd0 msgr2=0x7f2c800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2c800a3cd0 0x7f2c800a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c88006010 tx=0x7f2c8800b540 comp rx=0 tx=0).stop 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c98103e60 msgr2=0x7f2c9819c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 --2- 172.21.15.55:0/749028378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c98103e60 0x7f2c9819c2e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00c970 tx=0x7f2c8c00cc80 comp rx=0 tx=0).stop 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c96d9d700 1 -- 172.21.15.55:0/749028378 reap_dead start 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 shutdown_connections 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 >> 172.21.15.55:0/749028378 conn(0x7f2c9807a590 msgr2=0x7f2c9810e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 shutdown_connections 2024-04-26T21:51:37.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.603+0000 7f2c9cf6e700 1 -- 172.21.15.55:0/749028378 wait complete. 2024-04-26T21:51:37.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 -- 172.21.15.55:0/943869077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc104380 msgr2=0x7f40bc06bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 --2- 172.21.15.55:0/943869077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc104380 0x7f40bc06bb50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009990 tx=0x7f40b0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 -- 172.21.15.55:0/943869077 shutdown_connections 2024-04-26T21:51:37.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 --2- 172.21.15.55:0/943869077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40bc10fd40 0x7f40bc112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 --2- 172.21.15.55:0/943869077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc104380 0x7f40bc06bb50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 --2- 172.21.15.55:0/943869077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40bc06c090 0x7f40bc06c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 -- 172.21.15.55:0/943869077 >> 172.21.15.55:0/943869077 conn(0x7f40bc0790f0 msgr2=0x7f40bc079500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 -- 172.21.15.55:0/943869077 shutdown_connections 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 -- 172.21.15.55:0/943869077 wait complete. 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 Processor -- start 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 -- start start 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.785+0000 7f40c2efe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc06c090 0x7f40bc1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c2efe700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40bc104380 0x7f40bc1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c2efe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40bc10fd40 0x7f40bc19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c2efe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc115840 con 0x7f40bc06c090 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c2efe700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1156c0 con 0x7f40bc10fd40 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c2efe700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc1159c0 con 0x7f40bc104380 2024-04-26T21:51:37.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc06c090 0x7f40bc1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc06c090 0x7f40bc1a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51526/0 (socket says 172.21.15.55:51526) 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 -- 172.21.15.55:0/3227413873 learned_addr learned my addr 172.21.15.55:0/3227413873 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40bbfff700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40bc104380 0x7f40bc1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c149b700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40bc10fd40 0x7f40bc19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 -- 172.21.15.55:0/3227413873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40bc104380 msgr2=0x7f40bc1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40bc104380 0x7f40bc1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 -- 172.21.15.55:0/3227413873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40bc10fd40 msgr2=0x7f40bc19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40bc10fd40 0x7f40bc19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 -- 172.21.15.55:0/3227413873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc19c220 con 0x7f40bc06c090 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40bbfff700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f40bc104380 0x7f40bc1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:51:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c0c9a700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc06c090 0x7f40bc1a1900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000c00 tx=0x7f40b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0027070 con 0x7f40bc06c090 2024-04-26T21:51:37.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b0003fa0 con 0x7f40bc06c090 2024-04-26T21:51:37.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc19c4b0 con 0x7f40bc06c090 2024-04-26T21:51:37.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.786+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0022db0 con 0x7f40bc06c090 2024-04-26T21:51:37.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.787+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc1a87d0 con 0x7f40bc06c090 2024-04-26T21:51:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.787+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f40bc102090 con 0x7f40bc06c090 2024-04-26T21:51:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.789+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f40b002a070 con 0x7f40bc06c090 2024-04-26T21:51:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.789+0000 7f40b9ffb700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ac0a3cd0 0x7f40ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.790+0000 7f40bbfff700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ac0a3cd0 0x7f40ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.790+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f40b00d1820 con 0x7f40bc06c090 2024-04-26T21:51:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.790+0000 7f40b9ffb700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] conn(0x7f40ac0ad4b0 0x7f40ac0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.790+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 --> [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40b00043d0 con 0x7f40ac0ad4b0 2024-04-26T21:51:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.790+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f40b00d1bd0 con 0x7f40bc06c090 2024-04-26T21:51:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.790+0000 7f40c149b700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] conn(0x7f40ac0ad4b0 0x7f40ac0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.790+0000 7f40bbfff700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ac0a3cd0 0x7f40ac0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f40bc079910 tx=0x7f40a8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.790+0000 7f40c149b700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] conn(0x7f40ac0ad4b0 0x7f40ac0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.791+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 <== osd.7 v2:172.21.15.64:6816/1454699976 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40b00043d0 con 0x7f40ac0ad4b0 2024-04-26T21:51:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.808+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 --> [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f40bc102970 con 0x7f40ac0ad4b0 2024-04-26T21:51:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40b9ffb700 1 -- 172.21.15.55:0/3227413873 <== osd.7 v2:172.21.15.64:6816/1454699976 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f40bc102970 con 0x7f40ac0ad4b0 2024-04-26T21:51:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 >> [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] conn(0x7f40ac0ad4b0 msgr2=0x7f40ac0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] conn(0x7f40ac0ad4b0 0x7f40ac0af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ac0a3cd0 msgr2=0x7f40ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ac0a3cd0 0x7f40ac0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f40bc079910 tx=0x7f40a8005f50 comp rx=0 tx=0).stop 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc06c090 msgr2=0x7f40bc1a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 --2- 172.21.15.55:0/3227413873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40bc06c090 0x7f40bc1a1900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000c00 tx=0x7f40b00047b0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c149b700 1 -- 172.21.15.55:0/3227413873 reap_dead start 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 shutdown_connections 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 >> 172.21.15.55:0/3227413873 conn(0x7f40bc0790f0 msgr2=0x7f40bc101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 shutdown_connections 2024-04-26T21:51:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.809+0000 7f40c2efe700 1 -- 172.21.15.55:0/3227413873 wait complete. 2024-04-26T21:51:37.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 -- 172.21.15.55:0/1188974752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b006baf0 msgr2=0x7fc5b006bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 --2- 172.21.15.55:0/1188974752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b006baf0 0x7fc5b006bf50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009990 tx=0x7fc5a4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 -- 172.21.15.55:0/1188974752 shutdown_connections 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 --2- 172.21.15.55:0/1188974752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b006c490 0x7fc5b0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 --2- 172.21.15.55:0/1188974752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b01094c0 0x7fc5b01098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 --2- 172.21.15.55:0/1188974752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b006baf0 0x7fc5b006bf50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 -- 172.21.15.55:0/1188974752 >> 172.21.15.55:0/1188974752 conn(0x7fc5b00792d0 msgr2=0x7fc5b00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 -- 172.21.15.55:0/1188974752 shutdown_connections 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.952+0000 7fc5b688a700 1 -- 172.21.15.55:0/1188974752 wait complete. 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.953+0000 7fc5b688a700 1 Processor -- start 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.953+0000 7fc5b688a700 1 -- start start 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.953+0000 7fc5b688a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b006baf0 0x7fc5b019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.953+0000 7fc5b688a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b006c490 0x7fc5b019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.953+0000 7fc5b688a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b01094c0 0x7fc5b0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.953+0000 7fc5b688a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0113630 con 0x7fc5b01094c0 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.953+0000 7fc5b688a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01134b0 con 0x7fc5b006c490 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.953+0000 7fc5b688a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01137b0 con 0x7fc5b006baf0 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b01094c0 0x7fc5b0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b01094c0 0x7fc5b0199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51546/0 (socket says 172.21.15.55:51546) 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 -- 172.21.15.55:0/2266688505 learned_addr learned my addr 172.21.15.55:0/2266688505 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5affff700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b006baf0 0x7fc5b019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5af7fe700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b006c490 0x7fc5b019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 -- 172.21.15.55:0/2266688505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b006baf0 msgr2=0x7fc5b019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5b006baf0 0x7fc5b019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 -- 172.21.15.55:0/2266688505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b006c490 msgr2=0x7fc5b019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b006c490 0x7fc5b019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 -- 172.21.15.55:0/2266688505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b019a170 con 0x7fc5b01094c0 2024-04-26T21:51:37.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b4e27700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b01094c0 0x7fc5b0199a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800ada0 tx=0x7fc5a800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8017020 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a800ce30 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b019a460 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8005350 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.954+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01a6720 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.955+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5b0063b70 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.957+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc5a800c980 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.957+0000 7fc5ad7fa700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5980a3cd0 0x7fc5980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.957+0000 7fc5affff700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5980a3cd0 0x7fc5980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.957+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc5a80d04f0 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.957+0000 7fc5ad7fa700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] conn(0x7fc5980ad4b0 0x7fc5980af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.957+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 --> [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5a800fe50 con 0x7fc5980ad4b0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.957+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fc5a80d09d0 con 0x7fc5b01094c0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.958+0000 7fc5af7fe700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] conn(0x7fc5980ad4b0 0x7fc5980af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.958+0000 7fc5affff700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5980a3cd0 0x7fc5980a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc5b019ab50 tx=0x7fc5a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.958+0000 7fc5af7fe700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] conn(0x7fc5980ad4b0 0x7fc5980af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:37.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.959+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 <== osd.4 v2:172.21.15.64:6808/526876100 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5a800fe50 con 0x7fc5980ad4b0 2024-04-26T21:51:37.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.976+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 --> [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5b0063ff0 con 0x7fc5980ad4b0 2024-04-26T21:51:37.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.976+0000 7fc5ad7fa700 1 -- 172.21.15.55:0/2266688505 <== osd.4 v2:172.21.15.64:6808/526876100 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc5b0063ff0 con 0x7fc5980ad4b0 2024-04-26T21:51:37.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 >> [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] conn(0x7fc5980ad4b0 msgr2=0x7fc5980af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] conn(0x7fc5980ad4b0 0x7fc5980af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5980a3cd0 msgr2=0x7fc5980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5980a3cd0 0x7fc5980a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc5b019ab50 tx=0x7fc5a0009190 comp rx=0 tx=0).stop 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b01094c0 msgr2=0x7fc5b0199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 --2- 172.21.15.55:0/2266688505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5b01094c0 0x7fc5b0199a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800ada0 tx=0x7fc5a800c330 comp rx=0 tx=0).stop 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b4e27700 1 -- 172.21.15.55:0/2266688505 reap_dead start 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 shutdown_connections 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 >> 172.21.15.55:0/2266688505 conn(0x7fc5b00792d0 msgr2=0x7fc5b01008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 shutdown_connections 2024-04-26T21:51:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:37.977+0000 7fc5b688a700 1 -- 172.21.15.55:0/2266688505 wait complete. 2024-04-26T21:51:38.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:38 smithi055 ceph-mon[29221]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:38.733 INFO:teuthology.orchestra.run.smithi055.stdout:141733920826 2024-04-26T21:51:38.733 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T21:51:38.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:38 smithi064 ceph-mon[32765]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:38.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:38 smithi165 ceph-mon[33296]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:39.577 INFO:teuthology.orchestra.run.smithi055.stdout:55834574919 2024-04-26T21:51:39.578 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T21:51:40.111 INFO:teuthology.orchestra.run.smithi055.stdout:219043332137 2024-04-26T21:51:40.111 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T21:51:40.803 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:40 smithi055 ceph-mon[29221]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:51:40.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:40 smithi165 ceph-mon[33296]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:51:41.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:40 smithi064 ceph-mon[32765]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 -- 172.21.15.55:0/3433102130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102ce0 msgr2=0x7f603010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 --2- 172.21.15.55:0/3433102130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102ce0 0x7f603010f1d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f602800b0a0 tx=0x7f602800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 -- 172.21.15.55:0/3433102130 shutdown_connections 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 --2- 172.21.15.55:0/3433102130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102ce0 0x7f603010f1d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 --2- 172.21.15.55:0/3433102130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6030108390 0x7f6030108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 --2- 172.21.15.55:0/3433102130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6030102320 0x7f60301027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 -- 172.21.15.55:0/3433102130 >> 172.21.15.55:0/3433102130 conn(0x7f60300792d0 msgr2=0x7f60300796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 -- 172.21.15.55:0/3433102130 shutdown_connections 2024-04-26T21:51:41.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.319+0000 7f6036428700 1 -- 172.21.15.55:0/3433102130 wait complete. 2024-04-26T21:51:41.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f6036428700 1 Processor -- start 2024-04-26T21:51:41.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f6036428700 1 -- start start 2024-04-26T21:51:41.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f6036428700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102320 0x7f603019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f6036428700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6030102ce0 0x7f603019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f6036428700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6030108390 0x7f60301a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f6036428700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f60301124b0 con 0x7f6030102320 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f6036428700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6030112330 con 0x7f6030108390 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f6036428700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6030112630 con 0x7f6030102ce0 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f602ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102320 0x7f603019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f602ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102320 0x7f603019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37764/0 (socket says 172.21.15.55:37764) 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f602ffff700 1 -- 172.21.15.55:0/1193773713 learned_addr learned my addr 172.21.15.55:0/1193773713 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f602f7fe700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6030102ce0 0x7f603019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.320+0000 7f60349c5700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6030108390 0x7f60301a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602ffff700 1 -- 172.21.15.55:0/1193773713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6030102ce0 msgr2=0x7f603019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602ffff700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6030102ce0 0x7f603019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602ffff700 1 -- 172.21.15.55:0/1193773713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6030108390 msgr2=0x7f60301a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602ffff700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6030108390 0x7f60301a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602ffff700 1 -- 172.21.15.55:0/1193773713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60301a5e90 con 0x7f6030102320 2024-04-26T21:51:41.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602ffff700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102320 0x7f603019a8d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f602400e9a0 tx=0x7f602400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:41.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602400ca10 con 0x7f6030102320 2024-04-26T21:51:41.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602400cbb0 con 0x7f6030102320 2024-04-26T21:51:41.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6030198720 con 0x7f6030102320 2024-04-26T21:51:41.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024018680 con 0x7f6030102320 2024-04-26T21:51:41.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.321+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6030198c60 con 0x7f6030102320 2024-04-26T21:51:41.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.322+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6030063b30 con 0x7f6030102320 2024-04-26T21:51:41.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.323+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6024010430 con 0x7f6030102320 2024-04-26T21:51:41.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.324+0000 7f602d7fa700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f60180a3cd0 0x7f60180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.324+0000 7f602f7fe700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f60180a3cd0 0x7f60180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.324+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f60240d0b40 con 0x7f6030102320 2024-04-26T21:51:41.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.324+0000 7f602d7fa700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] conn(0x7f60180ad4b0 0x7f60180af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.324+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 --> [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6024019ec0 con 0x7f60180ad4b0 2024-04-26T21:51:41.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.324+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f60240d0ef0 con 0x7f6030102320 2024-04-26T21:51:41.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.324+0000 7f602f7fe700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f60180a3cd0 0x7f60180a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f60300798b0 tx=0x7f602000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:41.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.324+0000 7f60349c5700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] conn(0x7f60180ad4b0 0x7f60180af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.325+0000 7f60349c5700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] conn(0x7f60180ad4b0 0x7f60180af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:41.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.326+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 <== osd.8 v2:172.21.15.165:6816/2092227616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6024019ec0 con 0x7f60180ad4b0 2024-04-26T21:51:41.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.342+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 --> [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6030063fb0 con 0x7f60180ad4b0 2024-04-26T21:51:41.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f602d7fa700 1 -- 172.21.15.55:0/1193773713 <== osd.8 v2:172.21.15.165:6816/2092227616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6030063fb0 con 0x7f60180ad4b0 2024-04-26T21:51:41.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 >> [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] conn(0x7f60180ad4b0 msgr2=0x7f60180af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f6036428700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] conn(0x7f60180ad4b0 0x7f60180af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f60180a3cd0 msgr2=0x7f60180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f6036428700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f60180a3cd0 0x7f60180a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f60300798b0 tx=0x7f602000b410 comp rx=0 tx=0).stop 2024-04-26T21:51:41.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102320 msgr2=0x7f603019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f6036428700 1 --2- 172.21.15.55:0/1193773713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6030102320 0x7f603019a8d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f602400e9a0 tx=0x7f602400ecb0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f60349c5700 1 -- 172.21.15.55:0/1193773713 reap_dead start 2024-04-26T21:51:41.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 shutdown_connections 2024-04-26T21:51:41.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.343+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 >> 172.21.15.55:0/1193773713 conn(0x7f60300792d0 msgr2=0x7f6030100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:41.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.344+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 shutdown_connections 2024-04-26T21:51:41.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.344+0000 7f6036428700 1 -- 172.21.15.55:0/1193773713 wait complete. 2024-04-26T21:51:41.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.216+0000 7f0ad783a700 1 -- 172.21.15.55:0/883932069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006c360 msgr2=0x7f0ad01102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.216+0000 7f0ad783a700 1 --2- 172.21.15.55:0/883932069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006c360 0x7f0ad01102d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009990 tx=0x7f0ac8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 -- 172.21.15.55:0/883932069 shutdown_connections 2024-04-26T21:51:41.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 --2- 172.21.15.55:0/883932069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006c360 0x7f0ad01102d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 --2- 172.21.15.55:0/883932069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad0109490 0x7f0ad0109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 --2- 172.21.15.55:0/883932069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad006b9c0 0x7f0ad006be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 -- 172.21.15.55:0/883932069 >> 172.21.15.55:0/883932069 conn(0x7f0ad0079290 msgr2=0x7f0ad00796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 -- 172.21.15.55:0/883932069 shutdown_connections 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 -- 172.21.15.55:0/883932069 wait complete. 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 Processor -- start 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.217+0000 7f0ad783a700 1 -- start start 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad783a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006b9c0 0x7f0ad01023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad783a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad006c360 0x7f0ad0102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad783a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad0109490 0x7f0ad0102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad783a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad01135c0 con 0x7f0ad006b9c0 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad783a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad0113440 con 0x7f0ad006c360 2024-04-26T21:51:41.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad783a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad0113740 con 0x7f0ad0109490 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006b9c0 0x7f0ad01023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006b9c0 0x7f0ad01023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37758/0 (socket says 172.21.15.55:37758) 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 -- 172.21.15.55:0/2709341425 learned_addr learned my addr 172.21.15.55:0/2709341425 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad4dd5700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad006c360 0x7f0ad0102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad5dd7700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad0109490 0x7f0ad0102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 -- 172.21.15.55:0/2709341425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad0109490 msgr2=0x7f0ad0102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad0109490 0x7f0ad0102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 -- 172.21.15.55:0/2709341425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad006c360 msgr2=0x7f0ad0102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad006c360 0x7f0ad0102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 -- 172.21.15.55:0/2709341425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad01a7fd0 con 0x7f0ad006b9c0 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad5dd7700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad0109490 0x7f0ad0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad4dd5700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ad006c360 0x7f0ad0102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad55d6700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006b9c0 0x7f0ad01023e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00ef10 tx=0x7f0acc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:41.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc00cad0 con 0x7f0ad006b9c0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0acc019070 con 0x7f0ad006b9c0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad01a8260 con 0x7f0ad006b9c0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.218+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc0146e0 con 0x7f0ad006b9c0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.219+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad01a87f0 con 0x7f0ad006b9c0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.219+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ad0063b30 con 0x7f0ad006b9c0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.221+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0acc012050 con 0x7f0ad006b9c0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.221+0000 7f0ac67fc700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0abc0a3d50 0x7f0abc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.222+0000 7f0ad4dd5700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0abc0a3d50 0x7f0abc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.222+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0acc0d0ed0 con 0x7f0ad006b9c0 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.222+0000 7f0ac67fc700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] conn(0x7f0abc0ad530 0x7f0abc0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.222+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 --> [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0abc0b0000 con 0x7f0abc0ad530 2024-04-26T21:51:41.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.222+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f0acc0d1280 con 0x7f0ad006b9c0 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.222+0000 7f0ad4dd5700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0abc0a3d50 0x7f0abc0a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0079970 tx=0x7f0ac000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.222+0000 7f0ad5dd7700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] conn(0x7f0abc0ad530 0x7f0abc0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.222+0000 7f0ad5dd7700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] conn(0x7f0abc0ad530 0x7f0abc0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.224+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 <== osd.0 v2:172.21.15.55:6802/2219207145 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0abc0b0000 con 0x7f0abc0ad530 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.240+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 --> [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ad0063f30 con 0x7f0abc0ad530 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.240+0000 7f0ac67fc700 1 -- 172.21.15.55:0/2709341425 <== osd.0 v2:172.21.15.55:6802/2219207145 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ad0063f30 con 0x7f0abc0ad530 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] conn(0x7f0abc0ad530 msgr2=0x7f0abc0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] conn(0x7f0abc0ad530 0x7f0abc0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0abc0a3d50 msgr2=0x7f0abc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0abc0a3d50 0x7f0abc0a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0079970 tx=0x7f0ac000b2c0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006b9c0 msgr2=0x7f0ad01023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:41.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 --2- 172.21.15.55:0/2709341425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ad006b9c0 0x7f0ad01023e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00ef10 tx=0x7f0acc00c3a0 comp rx=0 tx=0).stop 2024-04-26T21:51:41.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad5dd7700 1 -- 172.21.15.55:0/2709341425 reap_dead start 2024-04-26T21:51:41.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 shutdown_connections 2024-04-26T21:51:41.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 >> 172.21.15.55:0/2709341425 conn(0x7f0ad0079290 msgr2=0x7f0ad0100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:41.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 shutdown_connections 2024-04-26T21:51:41.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:41.241+0000 7f0ad783a700 1 -- 172.21.15.55:0/2709341425 wait complete. 2024-04-26T21:51:42.222 INFO:teuthology.orchestra.run.smithi055.stdout:34359738447 2024-04-26T21:51:42.222 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T21:51:42.784 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:42 smithi055 ceph-mon[29221]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:42.820 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:42.820 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:42.821 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:42.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:42 smithi165 ceph-mon[33296]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:42.865 INFO:teuthology.orchestra.run.smithi055.stdout:163208757300 2024-04-26T21:51:42.865 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T21:51:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:42 smithi064 ceph-mon[32765]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:43.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.016+0000 7f3f49589700 1 -- 172.21.15.55:0/1631865168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f44103470 msgr2=0x7f3f441038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:43.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.016+0000 7f3f49589700 1 --2- 172.21.15.55:0/1631865168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f44103470 0x7f3f441038f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009990 tx=0x7f3f38009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:43.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 -- 172.21.15.55:0/1631865168 shutdown_connections 2024-04-26T21:51:43.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 --2- 172.21.15.55:0/1631865168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f44103e30 0x7f3f44110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:43.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 --2- 172.21.15.55:0/1631865168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f441094c0 0x7f3f441098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:43.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 --2- 172.21.15.55:0/1631865168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f44103470 0x7f3f441038f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:43.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 -- 172.21.15.55:0/1631865168 >> 172.21.15.55:0/1631865168 conn(0x7f3f4407a580 msgr2=0x7f3f4407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:43.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 -- 172.21.15.55:0/1631865168 shutdown_connections 2024-04-26T21:51:43.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 -- 172.21.15.55:0/1631865168 wait complete. 2024-04-26T21:51:43.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 Processor -- start 2024-04-26T21:51:43.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.017+0000 7f3f49589700 1 -- start start 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f49589700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f44103470 0x7f3f4419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f49589700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f44103e30 0x7f3f4419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f49589700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f441094c0 0x7f3f44199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f49589700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3f441135c0 con 0x7f3f44103470 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f49589700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44113440 con 0x7f3f441094c0 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f49589700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44113740 con 0x7f3f44103e30 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f427fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f44103e30 0x7f3f4419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f427fc700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f44103e30 0x7f3f4419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:44622/0 (socket says 172.21.15.55:44622) 2024-04-26T21:51:43.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f427fc700 1 -- 172.21.15.55:0/3150396131 learned_addr learned my addr 172.21.15.55:0/3150396131 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f42ffd700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f44103470 0x7f3f4419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f437fe700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f441094c0 0x7f3f44199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f42ffd700 1 -- 172.21.15.55:0/3150396131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f44103e30 msgr2=0x7f3f4419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f42ffd700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f44103e30 0x7f3f4419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f42ffd700 1 -- 172.21.15.55:0/3150396131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f441094c0 msgr2=0x7f3f44199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f42ffd700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f441094c0 0x7f3f44199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f42ffd700 1 -- 172.21.15.55:0/3150396131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4419a060 con 0x7f3f44103470 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f427fc700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f44103e30 0x7f3f4419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.018+0000 7f3f42ffd700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f44103470 0x7f3f4419f740 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400d750 tx=0x7f3f3400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.019+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34014030 con 0x7f3f44103470 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.019+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f340044d0 con 0x7f3f44103470 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.019+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4419a350 con 0x7f3f44103470 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.019+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34005420 con 0x7f3f44103470 2024-04-26T21:51:43.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.019+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f44075650 con 0x7f3f44103470 2024-04-26T21:51:43.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.020+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f44063b30 con 0x7f3f44103470 2024-04-26T21:51:43.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.022+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3f3400f690 con 0x7f3f44103470 2024-04-26T21:51:43.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.022+0000 7f3f2bfff700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f2c0a3cd0 0x7f3f2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:43.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.022+0000 7f3f427fc700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f2c0a3cd0 0x7f3f2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:43.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.022+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3f340cff30 con 0x7f3f44103470 2024-04-26T21:51:43.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.023+0000 7f3f2bfff700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] conn(0x7f3f2c0ad4b0 0x7f3f2c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:43.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.023+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 --> [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f3401ae50 con 0x7f3f2c0ad4b0 2024-04-26T21:51:43.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.023+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f3f340d2620 con 0x7f3f44103470 2024-04-26T21:51:43.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.023+0000 7f3f427fc700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f2c0a3cd0 0x7f3f2c0a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f38003800 tx=0x7f3f38005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:43.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.023+0000 7f3f437fe700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] conn(0x7f3f2c0ad4b0 0x7f3f2c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:43.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.023+0000 7f3f437fe700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] conn(0x7f3f2c0ad4b0 0x7f3f2c0af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:43.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.024+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 <== osd.11 v2:172.21.15.165:6824/1470164936 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f3401ae50 con 0x7f3f2c0ad4b0 2024-04-26T21:51:43.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.041+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 --> [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f44063f30 con 0x7f3f2c0ad4b0 2024-04-26T21:51:43.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.041+0000 7f3f2bfff700 1 -- 172.21.15.55:0/3150396131 <== osd.11 v2:172.21.15.165:6824/1470164936 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3f44063f30 con 0x7f3f2c0ad4b0 2024-04-26T21:51:43.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.041+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 >> [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] conn(0x7f3f2c0ad4b0 msgr2=0x7f3f2c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.041+0000 7f3f49589700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] conn(0x7f3f2c0ad4b0 0x7f3f2c0af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f2c0a3cd0 msgr2=0x7f3f2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f49589700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f2c0a3cd0 0x7f3f2c0a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3f38003800 tx=0x7f3f38005c00 comp rx=0 tx=0).stop 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f44103470 msgr2=0x7f3f4419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f49589700 1 --2- 172.21.15.55:0/3150396131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f44103470 0x7f3f4419f740 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400d750 tx=0x7f3f3400da60 comp rx=0 tx=0).stop 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f437fe700 1 -- 172.21.15.55:0/3150396131 reap_dead start 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 shutdown_connections 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 >> 172.21.15.55:0/3150396131 conn(0x7f3f4407a580 msgr2=0x7f3f441022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 shutdown_connections 2024-04-26T21:51:43.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:43.042+0000 7f3f49589700 1 -- 172.21.15.55:0/3150396131 wait complete. 2024-04-26T21:51:43.528 INFO:teuthology.orchestra.run.smithi055.stdout:253403070502 2024-04-26T21:51:43.529 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T21:51:44.453 INFO:teuthology.orchestra.run.smithi055.stdout:343597383703 2024-04-26T21:51:44.453 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T21:51:44.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:44 smithi055 ceph-mon[29221]: pgmap v312: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T21:51:44.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:44 smithi064 ceph-mon[32765]: pgmap v312: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T21:51:44.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:44 smithi165 ceph-mon[33296]: pgmap v312: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T21:51:46.790 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:46.791 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:46.792 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:46.793 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:46.796 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:46.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:46 smithi165 ceph-mon[33296]: pgmap v313: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T21:51:46.898 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:46 smithi055 ceph-mon[29221]: pgmap v313: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T21:51:47.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:46 smithi064 ceph-mon[32765]: pgmap v313: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-26T21:51:47.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/4279513356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c41094c0 msgr2=0x7ff1c41098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:47.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/4279513356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c41094c0 0x7ff1c41098a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009990 tx=0x7ff1bc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/4279513356 shutdown_connections 2024-04-26T21:51:47.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/4279513356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1c4103e30 0x7ff1c4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/4279513356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c41094c0 0x7ff1c41098a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/4279513356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c4103470 0x7ff1c41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/4279513356 >> 172.21.15.55:0/4279513356 conn(0x7ff1c407a580 msgr2=0x7ff1c407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:47.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/4279513356 shutdown_connections 2024-04-26T21:51:47.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.690+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/4279513356 wait complete. 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c8b0e700 1 Processor -- start 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c8b0e700 1 -- start start 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c8b0e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c4103470 0x7ff1c419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c8b0e700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1c4103e30 0x7ff1c419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c8b0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41094c0 0x7ff1c4199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c8b0e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41135c0 con 0x7ff1c4103470 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c8b0e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4113440 con 0x7ff1c41094c0 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c8b0e700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4113740 con 0x7ff1c4103e30 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c4103470 0x7ff1c419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c4103470 0x7ff1c419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37794/0 (socket says 172.21.15.55:37794) 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.691+0000 7ff1c259c700 1 -- 172.21.15.55:0/3039184131 learned_addr learned my addr 172.21.15.55:0/3039184131 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:47.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c2d9d700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41094c0 0x7ff1c4199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c1d9b700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1c4103e30 0x7ff1c419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c259c700 1 -- 172.21.15.55:0/3039184131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1c4103e30 msgr2=0x7ff1c419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c259c700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1c4103e30 0x7ff1c419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c259c700 1 -- 172.21.15.55:0/3039184131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41094c0 msgr2=0x7ff1c4199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c259c700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41094c0 0x7ff1c4199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c259c700 1 -- 172.21.15.55:0/3039184131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c4199fe0 con 0x7ff1c4103470 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c259c700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c4103470 0x7ff1c419f6c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1bb7fe700 1 -- 172.21.15.55:0/3039184131 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc027070 con 0x7ff1c4103470 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1bb7fe700 1 -- 172.21.15.55:0/3039184131 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1bc003f20 con 0x7ff1c4103470 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1bb7fe700 1 -- 172.21.15.55:0/3039184131 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc022cf0 con 0x7ff1c4103470 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c419a270 con 0x7ff1c4103470 2024-04-26T21:51:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.692+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c41a6590 con 0x7ff1c4103470 2024-04-26T21:51:47.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.693+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c406a110 con 0x7ff1c4103470 2024-04-26T21:51:47.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.694+0000 7ff1bb7fe700 1 -- 172.21.15.55:0/3039184131 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff1bc02a070 con 0x7ff1c4103470 2024-04-26T21:51:47.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.695+0000 7ff1bb7fe700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff1b00a3d50 0x7ff1b00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:47.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.695+0000 7ff1bb7fe700 1 -- 172.21.15.55:0/3039184131 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff1bc0d1120 con 0x7ff1c4103470 2024-04-26T21:51:47.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.695+0000 7ff1c1d9b700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff1b00a3d50 0x7ff1b00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:47.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.695+0000 7ff1c1d9b700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff1b00a3d50 0x7ff1b00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac005950 tx=0x7ff1ac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:47.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.700+0000 7ff1bb7fe700 1 -- 172.21.15.55:0/3039184131 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff1bc013070 con 0x7ff1c4103470 2024-04-26T21:51:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.917+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff1c402e490 con 0x7ff1c4103470 2024-04-26T21:51:47.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.917+0000 7ff1bb7fe700 1 -- 172.21.15.55:0/3039184131 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff1bc09dbe0 con 0x7ff1c4103470 2024-04-26T21:51:47.918 INFO:teuthology.orchestra.run.smithi055.stdout:287762808866 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff1b00a3d50 msgr2=0x7ff1b00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff1b00a3d50 0x7ff1b00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac005950 tx=0x7ff1ac0058e0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c4103470 msgr2=0x7ff1c419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c4103470 0x7ff1c419f6c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc0047b0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 shutdown_connections 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff1b00a3d50 0x7ff1b00a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41094c0 0x7ff1c4199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1c4103470 0x7ff1c419f6c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 --2- 172.21.15.55:0/3039184131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1c4103e30 0x7ff1c419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 >> 172.21.15.55:0/3039184131 conn(0x7ff1c407a580 msgr2=0x7ff1c4102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 shutdown_connections 2024-04-26T21:51:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:47.919+0000 7ff1c8b0e700 1 -- 172.21.15.55:0/3039184131 wait complete. 2024-04-26T21:51:48.255 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:48.256 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:48.789 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:48 smithi055 ceph-mon[29221]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:48.789 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:48 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3039184131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:51:48.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:48 smithi165 ceph-mon[33296]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:48.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:48 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3039184131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:51:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:48 smithi064 ceph-mon[32765]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:51:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:48 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3039184131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T21:51:49.571 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:49.571 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:51:49.673 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-26T21:51:49.674 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:49 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-26T21:51:49.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:49 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T21:51:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-04-26T21:51:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-26T21:51:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T21:51:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-26T21:51:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-04-26T21:51:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-26T21:51:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-26T21:51:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]: dispatch 2024-04-26T21:51:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-26T21:51:50.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:49 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T21:51:50.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:50 smithi055 ceph-mon[29221]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:51:50.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T21:51:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:50 smithi165 ceph-mon[33296]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:51:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:51:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-04-26T21:51:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-26T21:51:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T21:51:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-26T21:51:51.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-04-26T21:51:51.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-26T21:51:51.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-26T21:51:51.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 7]}]': finished 2024-04-26T21:51:51.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-26T21:51:51.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-26T21:51:51.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:50 smithi064 ceph-mon[32765]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T21:51:51.606 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:51 smithi055 ceph-mon[29221]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:51:51.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:51 smithi165 ceph-mon[33296]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:51:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:51 smithi064 ceph-mon[32765]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T21:51:52.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808862 got 287762808866 for osd.9 2024-04-26T21:51:52.245 DEBUG:teuthology.parallel:result is None 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 -- 172.21.15.55:0/2274091343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8109600 msgr2=0x7f00a81099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/2274091343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8109600 0x7f00a81099e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f009c009990 tx=0x7f009c009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 -- 172.21.15.55:0/2274091343 shutdown_connections 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/2274091343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a8102810 0x7f00a8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/2274091343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8109600 0x7f00a81099e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/2274091343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8101e50 0x7f00a81022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 -- 172.21.15.55:0/2274091343 >> 172.21.15.55:0/2274091343 conn(0x7f00a8077590 msgr2=0x7f00a80779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 -- 172.21.15.55:0/2274091343 shutdown_connections 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 -- 172.21.15.55:0/2274091343 wait complete. 2024-04-26T21:51:52.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 Processor -- start 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 -- start start 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.667+0000 7f00aeb5f700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8101e50 0x7f00a819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00aeb5f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8102810 0x7f00a819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00aeb5f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a8109600 0x7f00a81998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00aeb5f700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8113650 con 0x7f00a8101e50 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00aeb5f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81134d0 con 0x7f00a8109600 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00aeb5f700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81137d0 con 0x7f00a8102810 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8101e50 0x7f00a819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:52.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8101e50 0x7f00a819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45880/0 (socket says 172.21.15.55:45880) 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 -- 172.21.15.55:0/896016459 learned_addr learned my addr 172.21.15.55:0/896016459 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00a7fff700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8102810 0x7f00a819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ad0fc700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a8109600 0x7f00a81998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 -- 172.21.15.55:0/896016459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8102810 msgr2=0x7f00a819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8102810 0x7f00a819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 -- 172.21.15.55:0/896016459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a8109600 msgr2=0x7f00a81998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a8109600 0x7f00a81998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 -- 172.21.15.55:0/896016459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a819a050 con 0x7f00a8101e50 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00ac8fb700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8101e50 0x7f00a819f730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f009c0057f0 tx=0x7f009c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00a5ffb700 1 -- 172.21.15.55:0/896016459 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c027070 con 0x7f00a8101e50 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00a5ffb700 1 -- 172.21.15.55:0/896016459 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f009c0041c0 con 0x7f00a8101e50 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a819a2e0 con 0x7f00a8101e50 2024-04-26T21:51:52.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.668+0000 7f00a5ffb700 1 -- 172.21.15.55:0/896016459 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009c022e60 con 0x7f00a8101e50 2024-04-26T21:51:52.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.669+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a81a6600 con 0x7f00a8101e50 2024-04-26T21:51:52.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.670+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a806a110 con 0x7f00a8101e50 2024-04-26T21:51:52.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.671+0000 7f00a5ffb700 1 -- 172.21.15.55:0/896016459 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f009c02a070 con 0x7f00a8101e50 2024-04-26T21:51:52.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.671+0000 7f00a5ffb700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00980a3cd0 0x7f00980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:52.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.671+0000 7f00a7fff700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00980a3cd0 0x7f00980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:52.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.671+0000 7f00a5ffb700 1 -- 172.21.15.55:0/896016459 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f009c0d1320 con 0x7f00a8101e50 2024-04-26T21:51:52.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.672+0000 7f00a7fff700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00980a3cd0 0x7f00980a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f00a8077b20 tx=0x7f0094005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:52.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.676+0000 7f00a5ffb700 1 -- 172.21.15.55:0/896016459 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f009c013070 con 0x7f00a8101e50 2024-04-26T21:51:52.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:52 smithi165 ceph-mon[33296]: pgmap v318: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:51:52.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:52 smithi055 ceph-mon[29221]: pgmap v318: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:51:52.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.895+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f00a802e490 con 0x7f00a8101e50 2024-04-26T21:51:52.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.895+0000 7f00a5ffb700 1 -- 172.21.15.55:0/896016459 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f009c09dbb0 con 0x7f00a8101e50 2024-04-26T21:51:52.896 INFO:teuthology.orchestra.run.smithi055.stdout:77309411398 2024-04-26T21:51:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00980a3cd0 msgr2=0x7f00980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00980a3cd0 0x7f00980a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f00a8077b20 tx=0x7f0094005eb0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8101e50 msgr2=0x7f00a819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8101e50 0x7f00a819f730 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f009c0057f0 tx=0x7f009c003730 comp rx=0 tx=0).stop 2024-04-26T21:51:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 shutdown_connections 2024-04-26T21:51:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00980a3cd0 0x7f00980a6190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a8109600 0x7f00a81998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8101e50 0x7f00a819f730 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 --2- 172.21.15.55:0/896016459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a8102810 0x7f00a819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 >> 172.21.15.55:0/896016459 conn(0x7f00a8077590 msgr2=0x7f00a8100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 shutdown_connections 2024-04-26T21:51:52.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:52.897+0000 7f00aeb5f700 1 -- 172.21.15.55:0/896016459 wait complete. 2024-04-26T21:51:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:52 smithi064 ceph-mon[32765]: pgmap v318: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:51:53.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:53 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/896016459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:51:53.892 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:53 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/896016459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:51:54.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:53 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/896016459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T21:51:54.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:54 smithi165 ceph-mon[33296]: pgmap v319: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:54.900 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:54 smithi055 ceph-mon[29221]: pgmap v319: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:55.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:54 smithi064 ceph-mon[32765]: pgmap v319: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:55.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:55 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:55.842 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:55 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:56.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:55 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:56.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:56 smithi064 ceph-mon[32765]: pgmap v320: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 op/s; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:56.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:56 smithi165 ceph-mon[33296]: pgmap v320: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 op/s; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:56.868 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:56 smithi055 ceph-mon[29221]: pgmap v320: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 op/s; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:56.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 -- 172.21.15.55:0/4118865902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a01094e0 msgr2=0x7f32a01098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:56.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/4118865902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a01094e0 0x7f32a01098c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3294009990 tx=0x7f3294009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 -- 172.21.15.55:0/4118865902 shutdown_connections 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/4118865902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a0103e60 0x7f32a01102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/4118865902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a01094e0 0x7f32a01098c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/4118865902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32a01034a0 0x7f32a0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 -- 172.21.15.55:0/4118865902 >> 172.21.15.55:0/4118865902 conn(0x7f32a007a590 msgr2=0x7f32a007a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 -- 172.21.15.55:0/4118865902 shutdown_connections 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.867+0000 7f32a7bb4700 1 -- 172.21.15.55:0/4118865902 wait complete. 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a7bb4700 1 Processor -- start 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a7bb4700 1 -- start start 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a7bb4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a01034a0 0x7f32a019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a7bb4700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32a0103e60 0x7f32a019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a7bb4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a01094e0 0x7f32a01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a7bb4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0113660 con 0x7f32a01094e0 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a7bb4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01134e0 con 0x7f32a01034a0 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a7bb4700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01137e0 con 0x7f32a0103e60 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a5950700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a01034a0 0x7f32a019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a6151700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a01094e0 0x7f32a01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a5950700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a01034a0 0x7f32a019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35042/0 (socket says 172.21.15.55:35042) 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a514f700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32a0103e60 0x7f32a019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.868+0000 7f32a5950700 1 -- 172.21.15.55:0/1486123523 learned_addr learned my addr 172.21.15.55:0/1486123523 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a5950700 1 -- 172.21.15.55:0/1486123523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32a0103e60 msgr2=0x7f32a019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a5950700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32a0103e60 0x7f32a019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a5950700 1 -- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a01094e0 msgr2=0x7f32a01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a5950700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a01094e0 0x7f32a01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a5950700 1 -- 172.21.15.55:0/1486123523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a019a100 con 0x7f32a01034a0 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a514f700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32a0103e60 0x7f32a019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a5950700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a01034a0 0x7f32a019f7e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3294005910 tx=0x7f329400b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f3292ffd700 1 -- 172.21.15.55:0/1486123523 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294027070 con 0x7f32a01034a0 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f3292ffd700 1 -- 172.21.15.55:0/1486123523 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3294022470 con 0x7f32a01034a0 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f3292ffd700 1 -- 172.21.15.55:0/1486123523 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294017650 con 0x7f32a01034a0 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a019a390 con 0x7f32a01034a0 2024-04-26T21:51:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.869+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a0075650 con 0x7f32a01034a0 2024-04-26T21:51:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.870+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a010b950 con 0x7f32a01034a0 2024-04-26T21:51:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.871+0000 7f3292ffd700 1 -- 172.21.15.55:0/1486123523 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f329402a070 con 0x7f32a01034a0 2024-04-26T21:51:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.872+0000 7f3292ffd700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f328c0a3d50 0x7f328c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.872+0000 7f32a514f700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f328c0a3d50 0x7f328c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.872+0000 7f3292ffd700 1 -- 172.21.15.55:0/1486123523 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f32940d11c0 con 0x7f32a01034a0 2024-04-26T21:51:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.872+0000 7f32a514f700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f328c0a3d50 0x7f328c0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f32a007acd0 tx=0x7f329c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:56.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:56.876+0000 7f3292ffd700 1 -- 172.21.15.55:0/1486123523 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3294013070 con 0x7f32a01034a0 2024-04-26T21:51:57.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.086+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f32a019aed0 con 0x7f32a01034a0 2024-04-26T21:51:57.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.086+0000 7f3292ffd700 1 -- 172.21.15.55:0/1486123523 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f329409da50 con 0x7f32a01034a0 2024-04-26T21:51:57.087 INFO:teuthology.orchestra.run.smithi055.stdout:141733920830 2024-04-26T21:51:57.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.088+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f328c0a3d50 msgr2=0x7f328c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:57.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.088+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f328c0a3d50 0x7f328c0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f32a007acd0 tx=0x7f329c009260 comp rx=0 tx=0).stop 2024-04-26T21:51:57.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a01034a0 msgr2=0x7f32a019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:57.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a01034a0 0x7f32a019f7e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3294005910 tx=0x7f329400b670 comp rx=0 tx=0).stop 2024-04-26T21:51:57.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 shutdown_connections 2024-04-26T21:51:57.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f32a01094e0 0x7f32a01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f328c0a3d50 0x7f328c0a6210 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32a01034a0 0x7f32a019f7e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 --2- 172.21.15.55:0/1486123523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32a0103e60 0x7f32a019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:57.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 >> 172.21.15.55:0/1486123523 conn(0x7f32a007a590 msgr2=0x7f32a0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:57.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 shutdown_connections 2024-04-26T21:51:57.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:57.089+0000 7f32a7bb4700 1 -- 172.21.15.55:0/1486123523 wait complete. 2024-04-26T21:51:57.668 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:57 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1486123523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:51:57.668 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:57.668 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:57.669 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:57.669 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:57 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1486123523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:51:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:57 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1486123523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T21:51:58.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:51:58.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 -- 172.21.15.55:0/1391277251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006b130 msgr2=0x7fdb9006b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 --2- 172.21.15.55:0/1391277251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006b130 0x7fdb9006b590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009990 tx=0x7fdb84009ca0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 -- 172.21.15.55:0/1391277251 shutdown_connections 2024-04-26T21:51:58.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 --2- 172.21.15.55:0/1391277251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb9006bad0 0x7fdb901102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 --2- 172.21.15.55:0/1391277251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb901094a0 0x7fdb90109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 --2- 172.21.15.55:0/1391277251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006b130 0x7fdb9006b590 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 -- 172.21.15.55:0/1391277251 >> 172.21.15.55:0/1391277251 conn(0x7fdb900792d0 msgr2=0x7fdb900796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 -- 172.21.15.55:0/1391277251 shutdown_connections 2024-04-26T21:51:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.447+0000 7fdb95952700 1 -- 172.21.15.55:0/1391277251 wait complete. 2024-04-26T21:51:58.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.448+0000 7fdb95952700 1 Processor -- start 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.448+0000 7fdb95952700 1 -- start start 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.448+0000 7fdb95952700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb9006b130 0x7fdb9019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.448+0000 7fdb95952700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006bad0 0x7fdb9019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.448+0000 7fdb95952700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb901094a0 0x7fdb90199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb95952700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901135b0 con 0x7fdb9006bad0 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb95952700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90113430 con 0x7fdb9006b130 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb95952700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90113730 con 0x7fdb901094a0 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006bad0 0x7fdb9019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006bad0 0x7fdb9019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45906/0 (socket says 172.21.15.55:45906) 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 -- 172.21.15.55:0/2036720510 learned_addr learned my addr 172.21.15.55:0/2036720510 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8f7fe700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb901094a0 0x7fdb90199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8effd700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb9006b130 0x7fdb9019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 -- 172.21.15.55:0/2036720510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb901094a0 msgr2=0x7fdb90199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb901094a0 0x7fdb90199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 -- 172.21.15.55:0/2036720510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb9006b130 msgr2=0x7fdb9019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb9006b130 0x7fdb9019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 -- 172.21.15.55:0/2036720510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb90199ff0 con 0x7fdb9006bad0 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb8e7fc700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006bad0 0x7fdb9019fc10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdb84000c00 tx=0x7fdb840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb94950700 1 -- 172.21.15.55:0/2036720510 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84027070 con 0x7fdb9006bad0 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb94950700 1 -- 172.21.15.55:0/2036720510 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb84003f20 con 0x7fdb9006bad0 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9019a280 con 0x7fdb9006bad0 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.449+0000 7fdb94950700 1 -- 172.21.15.55:0/2036720510 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84022ca0 con 0x7fdb9006bad0 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.450+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb901a65a0 con 0x7fdb9006bad0 2024-04-26T21:51:58.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.450+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9006a110 con 0x7fdb9006bad0 2024-04-26T21:51:58.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.452+0000 7fdb94950700 1 -- 172.21.15.55:0/2036720510 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb8402a070 con 0x7fdb9006bad0 2024-04-26T21:51:58.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.452+0000 7fdb94950700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdb780a3cd0 0x7fdb780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:51:58.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.452+0000 7fdb8effd700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdb780a3cd0 0x7fdb780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:51:58.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.452+0000 7fdb94950700 1 -- 172.21.15.55:0/2036720510 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fdb840d1ad0 con 0x7fdb9006bad0 2024-04-26T21:51:58.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.453+0000 7fdb8effd700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdb780a3cd0 0x7fdb780a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb9019a9d0 tx=0x7fdb8000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:51:58.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.456+0000 7fdb94950700 1 -- 172.21.15.55:0/2036720510 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdb84031070 con 0x7fdb9006bad0 2024-04-26T21:51:58.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.681+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdb9002e490 con 0x7fdb9006bad0 2024-04-26T21:51:58.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.681+0000 7fdb94950700 1 -- 172.21.15.55:0/2036720510 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdb84013070 con 0x7fdb9006bad0 2024-04-26T21:51:58.684 INFO:teuthology.orchestra.run.smithi055.stdout:197568495668 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdb780a3cd0 msgr2=0x7fdb780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdb780a3cd0 0x7fdb780a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb9019a9d0 tx=0x7fdb8000a380 comp rx=0 tx=0).stop 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006bad0 msgr2=0x7fdb9019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006bad0 0x7fdb9019fc10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdb84000c00 tx=0x7fdb840047b0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 shutdown_connections 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb901094a0 0x7fdb90199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdb780a3cd0 0x7fdb780a6190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdb9006b130 0x7fdb9019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 --2- 172.21.15.55:0/2036720510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb9006bad0 0x7fdb9019fc10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:51:58.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 >> 172.21.15.55:0/2036720510 conn(0x7fdb900792d0 msgr2=0x7fdb90100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:51:58.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 shutdown_connections 2024-04-26T21:51:58.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:51:58.684+0000 7fdb95952700 1 -- 172.21.15.55:0/2036720510 wait complete. 2024-04-26T21:51:58.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:58 smithi055 ceph-mon[29221]: pgmap v321: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:59.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:58 smithi064 ceph-mon[32765]: pgmap v321: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:58 smithi165 ceph-mon[33296]: pgmap v321: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 91 B/s, 1 objects/s recovering 2024-04-26T21:51:59.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:51:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2036720510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:52:00.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:51:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2036720510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:52:00.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:51:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2036720510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T21:52:00.631 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:00 smithi055 ceph-mon[29221]: pgmap v322: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 op/s; 74 B/s, 1 objects/s recovering 2024-04-26T21:52:00.632 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:00 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/159030031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:52:00.664 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920830 for osd.4 2024-04-26T21:52:00.664 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:00.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 -- 172.21.15.55:0/467338469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9888103e30 msgr2=0x7f9888110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:00.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 --2- 172.21.15.55:0/467338469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9888103e30 0x7f9888110290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f988000b0a0 tx=0x7f988000b3b0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 -- 172.21.15.55:0/467338469 shutdown_connections 2024-04-26T21:52:00.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 --2- 172.21.15.55:0/467338469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9888103e30 0x7f9888110290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 --2- 172.21.15.55:0/467338469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98881094c0 0x7f98881098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 --2- 172.21.15.55:0/467338469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9888103470 0x7f98881038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 -- 172.21.15.55:0/467338469 >> 172.21.15.55:0/467338469 conn(0x7f988807a580 msgr2=0x7f988807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 -- 172.21.15.55:0/467338469 shutdown_connections 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.144+0000 7f988e767700 1 -- 172.21.15.55:0/467338469 wait complete. 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988e767700 1 Processor -- start 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988e767700 1 -- start start 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988e767700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9888103470 0x7f9888075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988e767700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9888103e30 0x7f9888075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988e767700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98881094c0 0x7f98880761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988e767700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9888113640 con 0x7f98881094c0 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988e767700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f98881134c0 con 0x7f9888103e30 2024-04-26T21:52:00.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988e767700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f98881137c0 con 0x7f9888103470 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988cd04700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98881094c0 0x7f98880761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988cd04700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98881094c0 0x7f98880761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45918/0 (socket says 172.21.15.55:45918) 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.145+0000 7f988cd04700 1 -- 172.21.15.55:0/159030031 learned_addr learned my addr 172.21.15.55:0/159030031 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f9887fff700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9888103470 0x7f9888075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f98877fe700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9888103e30 0x7f9888075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f988cd04700 1 -- 172.21.15.55:0/159030031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9888103470 msgr2=0x7f9888075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f988cd04700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9888103470 0x7f9888075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f988cd04700 1 -- 172.21.15.55:0/159030031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9888103e30 msgr2=0x7f9888075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f988cd04700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9888103e30 0x7f9888075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f988cd04700 1 -- 172.21.15.55:0/159030031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98881a8050 con 0x7f98881094c0 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f98877fe700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9888103e30 0x7f9888075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f9887fff700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9888103470 0x7f9888075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f988cd04700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98881094c0 0x7f98880761a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f98800062a0 tx=0x7f9880004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f98857fa700 1 -- 172.21.15.55:0/159030031 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880021070 con 0x7f98881094c0 2024-04-26T21:52:00.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f98857fa700 1 -- 172.21.15.55:0/159030031 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9880007d00 con 0x7f98881094c0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98881a82e0 con 0x7f98881094c0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f98857fa700 1 -- 172.21.15.55:0/159030031 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988001d450 con 0x7f98881094c0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.146+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98881a87d0 con 0x7f98881094c0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.147+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988806a110 con 0x7f98881094c0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.148+0000 7f98857fa700 1 -- 172.21.15.55:0/159030031 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9880004020 con 0x7f98881094c0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.149+0000 7f98857fa700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f98700a3cd0 0x7f98700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.149+0000 7f9887fff700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f98700a3cd0 0x7f98700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.149+0000 7f98857fa700 1 -- 172.21.15.55:0/159030031 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f98800d17c0 con 0x7f98881094c0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.150+0000 7f9887fff700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f98700a3cd0 0x7f98700a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f98780099d0 tx=0x7f9878008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.154+0000 7f98857fa700 1 -- 172.21.15.55:0/159030031 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f988009e0b0 con 0x7f98881094c0 2024-04-26T21:52:00.980 INFO:teuthology.orchestra.run.smithi055.stdout:219043332141 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.363+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f988802e650 con 0x7f98881094c0 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.364+0000 7f98857fa700 1 -- 172.21.15.55:0/159030031 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f988009d9b0 con 0x7f98881094c0 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f98700a3cd0 msgr2=0x7f98700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f98700a3cd0 0x7f98700a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f98780099d0 tx=0x7f9878008040 comp rx=0 tx=0).stop 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98881094c0 msgr2=0x7f98880761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98881094c0 0x7f98880761a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f98800062a0 tx=0x7f9880004ed0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 shutdown_connections 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98881094c0 0x7f98880761a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9888103470 0x7f9888075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f98700a3cd0 0x7f98700a6190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 --2- 172.21.15.55:0/159030031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9888103e30 0x7f9888075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:00.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 >> 172.21.15.55:0/159030031 conn(0x7f988807a580 msgr2=0x7f9888102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:00.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 shutdown_connections 2024-04-26T21:52:00.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:00.366+0000 7f988e767700 1 -- 172.21.15.55:0/159030031 wait complete. 2024-04-26T21:52:00.982 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:00 smithi165 ceph-mon[33296]: pgmap v322: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 op/s; 74 B/s, 1 objects/s recovering 2024-04-26T21:52:00.982 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:00 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/159030031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:52:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:00 smithi064 ceph-mon[32765]: pgmap v322: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 op/s; 74 B/s, 1 objects/s recovering 2024-04-26T21:52:01.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:00 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/159030031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T21:52:01.647 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332141 for osd.7 2024-04-26T21:52:01.647 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2997783229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f010fe20 msgr2=0x7f66f0112210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2997783229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f010fe20 0x7f66f0112210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f66e800b0a0 tx=0x7f66e800b3b0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2997783229 shutdown_connections 2024-04-26T21:52:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2997783229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f010fe20 0x7f66f0112210 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2997783229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f0075650 0x7f66f0075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2997783229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66f0075f70 0x7f66f010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2997783229 >> 172.21.15.55:0/2997783229 conn(0x7f66f007a580 msgr2=0x7f66f007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2997783229 shutdown_connections 2024-04-26T21:52:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.792+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2997783229 wait complete. 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66f4fcf700 1 Processor -- start 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66f4fcf700 1 -- start start 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66f4fcf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f0075650 0x7f66f01a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66f4fcf700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f0075f70 0x7f66f01a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66f4fcf700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66f010fe20 0x7f66f019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66f4fcf700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01157c0 con 0x7f66f0075f70 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66f4fcf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0115640 con 0x7f66f0075650 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66f4fcf700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0115940 con 0x7f66f010fe20 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66edd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f0075f70 0x7f66f01a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66edd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f0075f70 0x7f66f01a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40478/0 (socket says 172.21.15.55:40478) 2024-04-26T21:52:01.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66edd9b700 1 -- 172.21.15.55:0/2856528689 learned_addr learned my addr 172.21.15.55:0/2856528689 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66ee59c700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f0075650 0x7f66f01a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66edd9b700 1 -- 172.21.15.55:0/2856528689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66f010fe20 msgr2=0x7f66f019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66edd9b700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66f010fe20 0x7f66f019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66edd9b700 1 -- 172.21.15.55:0/2856528689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f0075650 msgr2=0x7f66f01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66edd9b700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f0075650 0x7f66f01a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66edd9b700 1 -- 172.21.15.55:0/2856528689 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f019c260 con 0x7f66f0075f70 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.793+0000 7f66ee59c700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f0075650 0x7f66f01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.794+0000 7f66edd9b700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f0075f70 0x7f66f01a1e30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e9c0 tx=0x7f66e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.794+0000 7f66df7fe700 1 -- 172.21.15.55:0/2856528689 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e400cbd0 con 0x7f66f0075f70 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.794+0000 7f66df7fe700 1 -- 172.21.15.55:0/2856528689 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66e400cd70 con 0x7f66f0075f70 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.794+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f019c550 con 0x7f66f0075f70 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.794+0000 7f66df7fe700 1 -- 172.21.15.55:0/2856528689 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4005610 con 0x7f66f0075f70 2024-04-26T21:52:01.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.794+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f01a87c0 con 0x7f66f0075f70 2024-04-26T21:52:01.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.795+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f006a110 con 0x7f66f0075f70 2024-04-26T21:52:01.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.796+0000 7f66df7fe700 1 -- 172.21.15.55:0/2856528689 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f66e4004750 con 0x7f66f0075f70 2024-04-26T21:52:01.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.797+0000 7f66df7fe700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66d80a3d50 0x7f66d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:01.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.797+0000 7f66ee59c700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66d80a3d50 0x7f66d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:01.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.797+0000 7f66df7fe700 1 -- 172.21.15.55:0/2856528689 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f66e40d07d0 con 0x7f66f0075f70 2024-04-26T21:52:01.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.797+0000 7f66ee59c700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66d80a3d50 0x7f66d80a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66f01029e0 tx=0x7f66e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:01.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:01.801+0000 7f66df7fe700 1 -- 172.21.15.55:0/2856528689 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f66e409d030 con 0x7f66f0075f70 2024-04-26T21:52:02.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.012+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f66f002e490 con 0x7f66f0075f70 2024-04-26T21:52:02.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.012+0000 7f66df7fe700 1 -- 172.21.15.55:0/2856528689 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f66e409c930 con 0x7f66f0075f70 2024-04-26T21:52:02.013 INFO:teuthology.orchestra.run.smithi055.stdout:34359738451 2024-04-26T21:52:02.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.015+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66d80a3d50 msgr2=0x7f66d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.015+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66d80a3d50 0x7f66d80a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66f01029e0 tx=0x7f66e0009260 comp rx=0 tx=0).stop 2024-04-26T21:52:02.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.015+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f0075f70 msgr2=0x7f66f01a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.015+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f0075f70 0x7f66f01a1e30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f66e400e9c0 tx=0x7f66e400ecd0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.016+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 shutdown_connections 2024-04-26T21:52:02.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.016+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66f010fe20 0x7f66f019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.016+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f66d80a3d50 0x7f66d80a6210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.016+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f66f0075650 0x7f66f01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.016+0000 7f66f4fcf700 1 --2- 172.21.15.55:0/2856528689 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66f0075f70 0x7f66f01a1e30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.016+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 >> 172.21.15.55:0/2856528689 conn(0x7f66f007a580 msgr2=0x7f66f0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:02.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.016+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 shutdown_connections 2024-04-26T21:52:02.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.016+0000 7f66f4fcf700 1 -- 172.21.15.55:0/2856528689 wait complete. 2024-04-26T21:52:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 -- 172.21.15.55:0/2589847938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 msgr2=0x7f5bd010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2589847938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 0x7f5bd010f250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800b0d0 tx=0x7f5bc800b3e0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 -- 172.21.15.55:0/2589847938 shutdown_connections 2024-04-26T21:52:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2589847938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 0x7f5bd010f250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2589847938 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd0102960 0x7f5bd0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2589847938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bd0103310 0x7f5bd0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 -- 172.21.15.55:0/2589847938 >> 172.21.15.55:0/2589847938 conn(0x7f5bd00792d0 msgr2=0x7f5bd00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:02.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 -- 172.21.15.55:0/2589847938 shutdown_connections 2024-04-26T21:52:02.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.325+0000 7f5bd4863700 1 -- 172.21.15.55:0/2589847938 wait complete. 2024-04-26T21:52:02.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.326+0000 7f5bd4863700 1 Processor -- start 2024-04-26T21:52:02.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.326+0000 7f5bd4863700 1 -- start start 2024-04-26T21:52:02.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.326+0000 7f5bd4863700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd0102960 0x7f5bd019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.326+0000 7f5bd4863700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bd0103310 0x7f5bd019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.326+0000 7f5bd4863700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 0x7f5bd0199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.326+0000 7f5bd4863700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd0112540 con 0x7f5bd01076b0 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.326+0000 7f5bd4863700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd01123c0 con 0x7f5bd0102960 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.326+0000 7f5bd4863700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd01126c0 con 0x7f5bd0103310 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 0x7f5bd0199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 0x7f5bd0199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40504/0 (socket says 172.21.15.55:40504) 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 -- 172.21.15.55:0/2351495687 learned_addr learned my addr 172.21.15.55:0/2351495687 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bcdd9b700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bd0103310 0x7f5bd019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bce59c700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd0102960 0x7f5bd019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 -- 172.21.15.55:0/2351495687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bd0103310 msgr2=0x7f5bd019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bd0103310 0x7f5bd019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 -- 172.21.15.55:0/2351495687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd0102960 msgr2=0x7f5bd019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd0102960 0x7f5bd019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 -- 172.21.15.55:0/2351495687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd019a0d0 con 0x7f5bd01076b0 2024-04-26T21:52:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bce59c700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd0102960 0x7f5bd019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bced9d700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 0x7f5bd0199970 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009fd0 tx=0x7f5bc8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bc77fe700 1 -- 172.21.15.55:0/2351495687 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc801f070 con 0x7f5bd01076b0 2024-04-26T21:52:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bc77fe700 1 -- 172.21.15.55:0/2351495687 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bc8003c00 con 0x7f5bd01076b0 2024-04-26T21:52:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd019a360 con 0x7f5bd01076b0 2024-04-26T21:52:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bc77fe700 1 -- 172.21.15.55:0/2351495687 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc801b7b0 con 0x7f5bd01076b0 2024-04-26T21:52:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.327+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd01a6680 con 0x7f5bd01076b0 2024-04-26T21:52:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.328+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd006b3a0 con 0x7f5bd01076b0 2024-04-26T21:52:02.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.330+0000 7f5bc77fe700 1 -- 172.21.15.55:0/2351495687 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5bc8022070 con 0x7f5bd01076b0 2024-04-26T21:52:02.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.330+0000 7f5bc77fe700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5bbc0a3cd0 0x7f5bbc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:02.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.330+0000 7f5bce59c700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5bbc0a3cd0 0x7f5bbc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:02.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.330+0000 7f5bc77fe700 1 -- 172.21.15.55:0/2351495687 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f5bc80d1790 con 0x7f5bd01076b0 2024-04-26T21:52:02.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.331+0000 7f5bce59c700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5bbc0a3cd0 0x7f5bbc0a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0005950 tx=0x7f5bc0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:02.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.334+0000 7f5bc77fe700 1 -- 172.21.15.55:0/2351495687 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5bc809e080 con 0x7f5bd01076b0 2024-04-26T21:52:02.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.546+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5bd0103a60 con 0x7f5bd01076b0 2024-04-26T21:52:02.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.546+0000 7f5bc77fe700 1 -- 172.21.15.55:0/2351495687 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5bc809d980 con 0x7f5bd01076b0 2024-04-26T21:52:02.547 INFO:teuthology.orchestra.run.smithi055.stdout:309237645343 2024-04-26T21:52:02.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.548+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5bbc0a3cd0 msgr2=0x7f5bbc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.548+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5bbc0a3cd0 0x7f5bbc0a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0005950 tx=0x7f5bc0005f50 comp rx=0 tx=0).stop 2024-04-26T21:52:02.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.548+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 msgr2=0x7f5bd0199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:02.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 0x7f5bd0199970 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009fd0 tx=0x7f5bc8007a20 comp rx=0 tx=0).stop 2024-04-26T21:52:02.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 shutdown_connections 2024-04-26T21:52:02.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bd01076b0 0x7f5bd0199970 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5bbc0a3cd0 0x7f5bbc0a6190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5bd0102960 0x7f5bd019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 --2- 172.21.15.55:0/2351495687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bd0103310 0x7f5bd019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:02.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 >> 172.21.15.55:0/2351495687 conn(0x7f5bd00792d0 msgr2=0x7f5bd006bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:02.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 shutdown_connections 2024-04-26T21:52:02.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:02.549+0000 7f5bd4863700 1 -- 172.21.15.55:0/2351495687 wait complete. 2024-04-26T21:52:02.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:02 smithi055 ceph-mon[29221]: pgmap v323: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1 op/s; 67 B/s, 1 objects/s recovering 2024-04-26T21:52:02.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:02 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2856528689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:52:02.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:02 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2351495687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:52:02.981 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411398 for osd.2 2024-04-26T21:52:02.981 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:02 smithi064 ceph-mon[32765]: pgmap v323: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1 op/s; 67 B/s, 1 objects/s recovering 2024-04-26T21:52:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:02 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2856528689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:52:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:02 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2351495687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:52:03.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:02 smithi165 ceph-mon[33296]: pgmap v323: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1 op/s; 67 B/s, 1 objects/s recovering 2024-04-26T21:52:03.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:02 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2856528689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T21:52:03.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:02 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2351495687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T21:52:03.090 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495668 for osd.6 2024-04-26T21:52:03.090 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 -- 172.21.15.55:0/607233727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0102960 msgr2=0x7f17c0102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 --2- 172.21.15.55:0/607233727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0102960 0x7f17c0102d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f17b00099c0 tx=0x7f17b0009cd0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 -- 172.21.15.55:0/607233727 shutdown_connections 2024-04-26T21:52:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 --2- 172.21.15.55:0/607233727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c01076b0 0x7f17c010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 --2- 172.21.15.55:0/607233727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0102960 0x7f17c0102d40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 --2- 172.21.15.55:0/607233727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c0103310 0x7f17c0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 -- 172.21.15.55:0/607233727 >> 172.21.15.55:0/607233727 conn(0x7f17c00792d0 msgr2=0x7f17c00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 -- 172.21.15.55:0/607233727 shutdown_connections 2024-04-26T21:52:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.268+0000 7f17c6859700 1 -- 172.21.15.55:0/607233727 wait complete. 2024-04-26T21:52:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c6859700 1 Processor -- start 2024-04-26T21:52:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c6859700 1 -- start start 2024-04-26T21:52:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c6859700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c0102960 0x7f17c019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c6859700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0103310 0x7f17c019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c6859700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c01076b0 0x7f17c01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c6859700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f17c0112540 con 0x7f17c0103310 2024-04-26T21:52:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c6859700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17c01123c0 con 0x7f17c0102960 2024-04-26T21:52:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c6859700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17c01126c0 con 0x7f17c01076b0 2024-04-26T21:52:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0103310 0x7f17c019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0103310 0x7f17c019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40518/0 (socket says 172.21.15.55:40518) 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17bf7fe700 1 -- 172.21.15.55:0/915504933 learned_addr learned my addr 172.21.15.55:0/915504933 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17c4df6700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c01076b0 0x7f17c01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17bffff700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c0102960 0x7f17c019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.269+0000 7f17bf7fe700 1 -- 172.21.15.55:0/915504933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c01076b0 msgr2=0x7f17c01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bf7fe700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c01076b0 0x7f17c01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bf7fe700 1 -- 172.21.15.55:0/915504933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c0102960 msgr2=0x7f17c019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bf7fe700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c0102960 0x7f17c019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bf7fe700 1 -- 172.21.15.55:0/915504933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c019a130 con 0x7f17c0103310 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bffff700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c0102960 0x7f17c019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bf7fe700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0103310 0x7f17c019fd50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c8c0 tx=0x7f17a800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bd7fa700 1 -- 172.21.15.55:0/915504933 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8004220 con 0x7f17c0103310 2024-04-26T21:52:03.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bd7fa700 1 -- 172.21.15.55:0/915504933 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a80043c0 con 0x7f17c0103310 2024-04-26T21:52:03.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17bd7fa700 1 -- 172.21.15.55:0/915504933 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8007710 con 0x7f17c0103310 2024-04-26T21:52:03.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c019a420 con 0x7f17c0103310 2024-04-26T21:52:03.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.270+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c01a66e0 con 0x7f17c0103310 2024-04-26T21:52:03.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.271+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c0103a60 con 0x7f17c0103310 2024-04-26T21:52:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.272+0000 7f17bd7fa700 1 -- 172.21.15.55:0/915504933 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f17a8004750 con 0x7f17c0103310 2024-04-26T21:52:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.273+0000 7f17bd7fa700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f17ac0a3cd0 0x7f17ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.273+0000 7f17bffff700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f17ac0a3cd0 0x7f17ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.273+0000 7f17bd7fa700 1 -- 172.21.15.55:0/915504933 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f17a80d0120 con 0x7f17c0103310 2024-04-26T21:52:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.273+0000 7f17bffff700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f17ac0a3cd0 0x7f17ac0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b580 tx=0x7f17b0009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:03.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.277+0000 7f17bd7fa700 1 -- 172.21.15.55:0/915504933 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f17a8014030 con 0x7f17c0103310 2024-04-26T21:52:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.488+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f17c002e490 con 0x7f17c0103310 2024-04-26T21:52:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.488+0000 7f17bd7fa700 1 -- 172.21.15.55:0/915504933 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f17a809c980 con 0x7f17c0103310 2024-04-26T21:52:03.488 INFO:teuthology.orchestra.run.smithi055.stdout:343597383706 2024-04-26T21:52:03.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f17ac0a3cd0 msgr2=0x7f17ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:03.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f17ac0a3cd0 0x7f17ac0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b580 tx=0x7f17b0009530 comp rx=0 tx=0).stop 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0103310 msgr2=0x7f17c019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0103310 0x7f17c019fd50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c8c0 tx=0x7f17a800cbd0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 shutdown_connections 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17c01076b0 0x7f17c01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f17ac0a3cd0 0x7f17ac0a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17c0102960 0x7f17c019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 --2- 172.21.15.55:0/915504933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17c0103310 0x7f17c019fd50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 >> 172.21.15.55:0/915504933 conn(0x7f17c00792d0 msgr2=0x7f17c006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 shutdown_connections 2024-04-26T21:52:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:03.490+0000 7f17c6859700 1 -- 172.21.15.55:0/915504933 wait complete. 2024-04-26T21:52:03.865 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:03 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/915504933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:52:04.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:03 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/915504933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:52:04.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:03 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/915504933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T21:52:04.195 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383703 got 343597383706 for osd.11 2024-04-26T21:52:04.195 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:04.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 -- 172.21.15.55:0/2699354466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 msgr2=0x7f24e4102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:04.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 --2- 172.21.15.55:0/2699354466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 0x7f24e4102af0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f24d40099c0 tx=0x7f24d4009cd0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 -- 172.21.15.55:0/2699354466 shutdown_connections 2024-04-26T21:52:04.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 --2- 172.21.15.55:0/2699354466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e4103030 0x7f24e410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 --2- 172.21.15.55:0/2699354466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24e41073a0 0x7f24e4107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 --2- 172.21.15.55:0/2699354466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 0x7f24e4102af0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 -- 172.21.15.55:0/2699354466 >> 172.21.15.55:0/2699354466 conn(0x7f24e4078190 msgr2=0x7f24e40785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:04.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 -- 172.21.15.55:0/2699354466 shutdown_connections 2024-04-26T21:52:04.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.365+0000 7f24e9748700 1 -- 172.21.15.55:0/2699354466 wait complete. 2024-04-26T21:52:04.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e9748700 1 Processor -- start 2024-04-26T21:52:04.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e9748700 1 -- start start 2024-04-26T21:52:04.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e9748700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 0x7f24e4104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e9748700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24e4103030 0x7f24e4104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e9748700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e41073a0 0x7f24e4104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e9748700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f24e4112570 con 0x7f24e4102670 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e9748700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41123f0 con 0x7f24e4103030 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e9748700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24e41126f0 con 0x7f24e41073a0 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 0x7f24e4104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 0x7f24e4104050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40530/0 (socket says 172.21.15.55:40530) 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.366+0000 7f24e2ffd700 1 -- 172.21.15.55:0/1575642908 learned_addr learned my addr 172.21.15.55:0/1575642908 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e27fc700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24e4103030 0x7f24e4104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e37fe700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e41073a0 0x7f24e4104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e2ffd700 1 -- 172.21.15.55:0/1575642908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e41073a0 msgr2=0x7f24e4104ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e2ffd700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e41073a0 0x7f24e4104ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e2ffd700 1 -- 172.21.15.55:0/1575642908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24e4103030 msgr2=0x7f24e4104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e2ffd700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24e4103030 0x7f24e4104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e2ffd700 1 -- 172.21.15.55:0/1575642908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e41a7fe0 con 0x7f24e4102670 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e27fc700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24e4103030 0x7f24e4104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e2ffd700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 0x7f24e4104050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00d950 tx=0x7f24cc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24dbfff700 1 -- 172.21.15.55:0/1575642908 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc016070 con 0x7f24e4102670 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e41a82d0 con 0x7f24e4102670 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24dbfff700 1 -- 172.21.15.55:0/1575642908 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24cc010460 con 0x7f24e4102670 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24dbfff700 1 -- 172.21.15.55:0/1575642908 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00f620 con 0x7f24e4102670 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.367+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e41a8830 con 0x7f24e4102670 2024-04-26T21:52:04.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.368+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e4103a90 con 0x7f24e4102670 2024-04-26T21:52:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.369+0000 7f24dbfff700 1 -- 172.21.15.55:0/1575642908 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f24cc019070 con 0x7f24e4102670 2024-04-26T21:52:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.370+0000 7f24dbfff700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f24d00a3cd0 0x7f24d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.370+0000 7f24e27fc700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f24d00a3cd0 0x7f24d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.370+0000 7f24dbfff700 1 -- 172.21.15.55:0/1575642908 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f24cc0d01a0 con 0x7f24e4102670 2024-04-26T21:52:04.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.371+0000 7f24e27fc700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f24d00a3cd0 0x7f24d00a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f24e4105ae0 tx=0x7f24d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:04.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.375+0000 7f24dbfff700 1 -- 172.21.15.55:0/1575642908 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f24cc09ca00 con 0x7f24e4102670 2024-04-26T21:52:04.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.586+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f24e402e490 con 0x7f24e4102670 2024-04-26T21:52:04.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.586+0000 7f24dbfff700 1 -- 172.21.15.55:0/1575642908 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f24cc09c300 con 0x7f24e4102670 2024-04-26T21:52:04.587 INFO:teuthology.orchestra.run.smithi055.stdout:253403070507 2024-04-26T21:52:04.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f24d00a3cd0 msgr2=0x7f24d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:04.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f24d00a3cd0 0x7f24d00a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f24e4105ae0 tx=0x7f24d401a040 comp rx=0 tx=0).stop 2024-04-26T21:52:04.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 msgr2=0x7f24e4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:04.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 0x7f24e4104050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00d950 tx=0x7f24cc00dd10 comp rx=0 tx=0).stop 2024-04-26T21:52:04.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 shutdown_connections 2024-04-26T21:52:04.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e41073a0 0x7f24e4104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24e4102670 0x7f24e4104050 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f24d00a3cd0 0x7f24d00a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 --2- 172.21.15.55:0/1575642908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f24e4103030 0x7f24e4104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:04.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.588+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 >> 172.21.15.55:0/1575642908 conn(0x7f24e4078190 msgr2=0x7f24e406bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:04.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.589+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 shutdown_connections 2024-04-26T21:52:04.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:04.589+0000 7f24e9748700 1 -- 172.21.15.55:0/1575642908 wait complete. 2024-04-26T21:52:04.914 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:04 smithi055 ceph-mon[29221]: pgmap v324: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 op/s; 61 B/s, 0 objects/s recovering 2024-04-26T21:52:04.915 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:04 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1575642908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:52:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:04 smithi064 ceph-mon[32765]: pgmap v324: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 op/s; 61 B/s, 0 objects/s recovering 2024-04-26T21:52:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:04 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1575642908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:52:05.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:04 smithi165 ceph-mon[33296]: pgmap v324: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 op/s; 61 B/s, 0 objects/s recovering 2024-04-26T21:52:05.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:04 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1575642908' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T21:52:06.003 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645338 got 309237645343 for osd.10 2024-04-26T21:52:06.004 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:06.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:06 smithi055 ceph-mon[29221]: pgmap v325: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:52:06.861 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738451 for osd.0 2024-04-26T21:52:06.862 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:06 smithi064 ceph-mon[32765]: pgmap v325: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:52:07.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:06 smithi165 ceph-mon[33296]: pgmap v325: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T21:52:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/1419012541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 msgr2=0x7f0eb81098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/1419012541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 0x7f0eb81098a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009990 tx=0x7f0ea8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/1419012541 shutdown_connections 2024-04-26T21:52:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/1419012541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eb8103e30 0x7f0eb8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/1419012541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 0x7f0eb81098a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/1419012541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb8103470 0x7f0eb81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/1419012541 >> 172.21.15.55:0/1419012541 conn(0x7f0eb807a580 msgr2=0x7f0eb807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/1419012541 shutdown_connections 2024-04-26T21:52:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.302+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/1419012541 wait complete. 2024-04-26T21:52:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebe3f4700 1 Processor -- start 2024-04-26T21:52:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebe3f4700 1 -- start start 2024-04-26T21:52:07.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebe3f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eb8103470 0x7f0eb819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebe3f4700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb8103e30 0x7f0eb819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebe3f4700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 0x7f0eb8199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebe3f4700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81135c0 con 0x7f0eb81094c0 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebe3f4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8113440 con 0x7f0eb8103470 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebe3f4700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8113740 con 0x7f0eb8103e30 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebc991700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 0x7f0eb8199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebc991700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 0x7f0eb8199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40538/0 (socket says 172.21.15.55:40538) 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0ebc991700 1 -- 172.21.15.55:0/2092254313 learned_addr learned my addr 172.21.15.55:0/2092254313 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0eb7fff700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eb8103470 0x7f0eb819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.303+0000 7f0eb77fe700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb8103e30 0x7f0eb819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0ebc991700 1 -- 172.21.15.55:0/2092254313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb8103e30 msgr2=0x7f0eb819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:07.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0ebc991700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb8103e30 0x7f0eb819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0ebc991700 1 -- 172.21.15.55:0/2092254313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eb8103470 msgr2=0x7f0eb819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0ebc991700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eb8103470 0x7f0eb819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0ebc991700 1 -- 172.21.15.55:0/2092254313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb819a080 con 0x7f0eb81094c0 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0ebc991700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 0x7f0eb8199920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0007ee0 tx=0x7f0eb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0eb57fa700 1 -- 172.21.15.55:0/2092254313 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000dbd0 con 0x7f0eb81094c0 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb819a370 con 0x7f0eb81094c0 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0eb57fa700 1 -- 172.21.15.55:0/2092254313 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb000dd70 con 0x7f0eb81094c0 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0eb57fa700 1 -- 172.21.15.55:0/2092254313 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0005990 con 0x7f0eb81094c0 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.304+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a6630 con 0x7f0eb81094c0 2024-04-26T21:52:07.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.305+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb806a110 con 0x7f0eb81094c0 2024-04-26T21:52:07.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.306+0000 7f0eb57fa700 1 -- 172.21.15.55:0/2092254313 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0eb001a020 con 0x7f0eb81094c0 2024-04-26T21:52:07.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.307+0000 7f0eb57fa700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0ea00a3cd0 0x7f0ea00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:07.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.307+0000 7f0eb7fff700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0ea00a3cd0 0x7f0ea00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:07.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.307+0000 7f0eb57fa700 1 -- 172.21.15.55:0/2092254313 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0eb00d03a0 con 0x7f0eb81094c0 2024-04-26T21:52:07.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.307+0000 7f0eb7fff700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0ea00a3cd0 0x7f0ea00a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0eb819aa60 tx=0x7f0ea8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:07.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.310+0000 7f0eb57fa700 1 -- 172.21.15.55:0/2092254313 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0eb0010070 con 0x7f0eb81094c0 2024-04-26T21:52:07.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.520+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0eb802e490 con 0x7f0eb81094c0 2024-04-26T21:52:07.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.520+0000 7f0eb57fa700 1 -- 172.21.15.55:0/2092254313 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0eb009cc00 con 0x7f0eb81094c0 2024-04-26T21:52:07.521 INFO:teuthology.orchestra.run.smithi055.stdout:55834574925 2024-04-26T21:52:07.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0ea00a3cd0 msgr2=0x7f0ea00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:07.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0ea00a3cd0 0x7f0ea00a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0eb819aa60 tx=0x7f0ea8005960 comp rx=0 tx=0).stop 2024-04-26T21:52:07.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 msgr2=0x7f0eb8199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:07.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 0x7f0eb8199920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0007ee0 tx=0x7f0eb000d3b0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 shutdown_connections 2024-04-26T21:52:07.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0eb81094c0 0x7f0eb8199920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0ea00a3cd0 0x7f0ea00a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0eb8103470 0x7f0eb819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 --2- 172.21.15.55:0/2092254313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0eb8103e30 0x7f0eb819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:07.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 >> 172.21.15.55:0/2092254313 conn(0x7f0eb807a580 msgr2=0x7f0eb81022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:07.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 shutdown_connections 2024-04-26T21:52:07.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:07.522+0000 7f0ebe3f4700 1 -- 172.21.15.55:0/2092254313 wait complete. 2024-04-26T21:52:07.665 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:07 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2092254313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:52:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:07 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2092254313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:52:08.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:07 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2092254313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T21:52:08.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.115+0000 7facd11aa700 1 -- 172.21.15.55:0/42959364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc103e60 msgr2=0x7faccc1102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.115+0000 7facd11aa700 1 --2- 172.21.15.55:0/42959364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc103e60 0x7faccc1102b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7facc4009990 tx=0x7facc4009ca0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.116+0000 7facd11aa700 1 -- 172.21.15.55:0/42959364 shutdown_connections 2024-04-26T21:52:08.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.116+0000 7facd11aa700 1 --2- 172.21.15.55:0/42959364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc103e60 0x7faccc1102b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.116+0000 7facd11aa700 1 --2- 172.21.15.55:0/42959364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faccc1094e0 0x7faccc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.116+0000 7facd11aa700 1 --2- 172.21.15.55:0/42959364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faccc1034a0 0x7faccc103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.116+0000 7facd11aa700 1 -- 172.21.15.55:0/42959364 >> 172.21.15.55:0/42959364 conn(0x7faccc07a590 msgr2=0x7faccc07a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:08.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.116+0000 7facd11aa700 1 -- 172.21.15.55:0/42959364 shutdown_connections 2024-04-26T21:52:08.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.116+0000 7facd11aa700 1 -- 172.21.15.55:0/42959364 wait complete. 2024-04-26T21:52:08.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.116+0000 7facd11aa700 1 Processor -- start 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7facd11aa700 1 -- start start 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7facd11aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faccc1034a0 0x7faccc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7facd11aa700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faccc103e60 0x7faccc19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7facd11aa700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc1094e0 0x7faccc199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7facd11aa700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1135e0 con 0x7faccc1094e0 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7facd11aa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faccc113460 con 0x7faccc1034a0 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7facd11aa700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faccc113760 con 0x7faccc103e60 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc1094e0 0x7faccc199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc1094e0 0x7faccc199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40558/0 (socket says 172.21.15.55:40558) 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 -- 172.21.15.55:0/2692181122 learned_addr learned my addr 172.21.15.55:0/2692181122 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7facca59c700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faccc103e60 0x7faccc19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccad9d700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faccc1034a0 0x7faccc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 -- 172.21.15.55:0/2692181122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faccc103e60 msgr2=0x7faccc19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faccc103e60 0x7faccc19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 -- 172.21.15.55:0/2692181122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faccc1034a0 msgr2=0x7faccc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faccc1034a0 0x7faccc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 -- 172.21.15.55:0/2692181122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc199fe0 con 0x7faccc1094e0 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.117+0000 7faccb59e700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc1094e0 0x7faccc199800 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7facc4005910 tx=0x7facc40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.118+0000 7facc3fff700 1 -- 172.21.15.55:0/2692181122 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4027070 con 0x7faccc1094e0 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.118+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc19a270 con 0x7faccc1094e0 2024-04-26T21:52:08.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.118+0000 7facc3fff700 1 -- 172.21.15.55:0/2692181122 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc4003f20 con 0x7faccc1094e0 2024-04-26T21:52:08.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.118+0000 7facc3fff700 1 -- 172.21.15.55:0/2692181122 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4022d00 con 0x7faccc1094e0 2024-04-26T21:52:08.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.118+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc1a64e0 con 0x7faccc1094e0 2024-04-26T21:52:08.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.119+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faccc104b80 con 0x7faccc1094e0 2024-04-26T21:52:08.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.120+0000 7facc3fff700 1 -- 172.21.15.55:0/2692181122 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7facc402a070 con 0x7faccc1094e0 2024-04-26T21:52:08.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.120+0000 7facc3fff700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7facb80a3cd0 0x7facb80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.121+0000 7facc3fff700 1 -- 172.21.15.55:0/2692181122 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7facc40d1bc0 con 0x7faccc1094e0 2024-04-26T21:52:08.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.121+0000 7faccad9d700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7facb80a3cd0 0x7facb80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.121+0000 7faccad9d700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7facb80a3cd0 0x7facb80a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7facbc007fd0 tx=0x7facbc00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:08.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.124+0000 7facc3fff700 1 -- 172.21.15.55:0/2692181122 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7facc4031070 con 0x7faccc1094e0 2024-04-26T21:52:08.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 -- 172.21.15.55:0/1899813168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730109560 msgr2=0x7f8730109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 --2- 172.21.15.55:0/1899813168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730109560 0x7f8730109940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f87240099c0 tx=0x7f8724009cd0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 -- 172.21.15.55:0/1899813168 shutdown_connections 2024-04-26T21:52:08.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 --2- 172.21.15.55:0/1899813168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8730103e80 0x7f8730110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 --2- 172.21.15.55:0/1899813168 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730109560 0x7f8730109940 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 --2- 172.21.15.55:0/1899813168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87301034c0 0x7f8730103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 -- 172.21.15.55:0/1899813168 >> 172.21.15.55:0/1899813168 conn(0x7f873007a5d0 msgr2=0x7f873007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:08.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 -- 172.21.15.55:0/1899813168 shutdown_connections 2024-04-26T21:52:08.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.304+0000 7f8736e2a700 1 -- 172.21.15.55:0/1899813168 wait complete. 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f8736e2a700 1 Processor -- start 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f8736e2a700 1 -- start start 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f8736e2a700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87301034c0 0x7f873019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f8736e2a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730103e80 0x7f873019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f8736e2a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8730109560 0x7f87301999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f8736e2a700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8730113710 con 0x7f8730103e80 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f8736e2a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8730113590 con 0x7f8730109560 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f8736e2a700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8730113890 con 0x7f87301034c0 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f872ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730103e80 0x7f873019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f872ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730103e80 0x7f873019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40590/0 (socket says 172.21.15.55:40590) 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f872ffff700 1 -- 172.21.15.55:0/2970774897 learned_addr learned my addr 172.21.15.55:0/2970774897 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.305+0000 7f87353c7700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8730109560 0x7f87301999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872ffff700 1 -- 172.21.15.55:0/2970774897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87301034c0 msgr2=0x7f873019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872ffff700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87301034c0 0x7f873019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872ffff700 1 -- 172.21.15.55:0/2970774897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8730109560 msgr2=0x7f87301999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872ffff700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8730109560 0x7f87301999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872ffff700 1 -- 172.21.15.55:0/2970774897 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873019a150 con 0x7f8730103e80 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872ffff700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730103e80 0x7f873019fd20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f871c00b530 tx=0x7f871c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872dffb700 1 -- 172.21.15.55:0/2970774897 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c015020 con 0x7f8730103e80 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872dffb700 1 -- 172.21.15.55:0/2970774897 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f871c011ce0 con 0x7f8730103e80 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873019a440 con 0x7f8730103e80 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f872dffb700 1 -- 172.21.15.55:0/2970774897 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c010e40 con 0x7f8730103e80 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.306+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87301a66b0 con 0x7f8730103e80 2024-04-26T21:52:08.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.307+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f873006a110 con 0x7f8730103e80 2024-04-26T21:52:08.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.308+0000 7f872dffb700 1 -- 172.21.15.55:0/2970774897 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f871c018020 con 0x7f8730103e80 2024-04-26T21:52:08.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.309+0000 7f872dffb700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f87200a3d50 0x7f87200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:08.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.309+0000 7f872dffb700 1 -- 172.21.15.55:0/2970774897 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f871c0cfc80 con 0x7f8730103e80 2024-04-26T21:52:08.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.309+0000 7f8734bc6700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f87200a3d50 0x7f87200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:08.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.310+0000 7f8734bc6700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f87200a3d50 0x7f87200a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f87240098e0 tx=0x7f8724000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:08.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.314+0000 7f872dffb700 1 -- 172.21.15.55:0/2970774897 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f871c01f020 con 0x7f8730103e80 2024-04-26T21:52:08.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.339+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7faccc02e490 con 0x7faccc1094e0 2024-04-26T21:52:08.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.340+0000 7facc3fff700 1 -- 172.21.15.55:0/2692181122 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7facc4013070 con 0x7faccc1094e0 2024-04-26T21:52:08.340 INFO:teuthology.orchestra.run.smithi055.stdout:163208757306 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7facb80a3cd0 msgr2=0x7facb80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7facb80a3cd0 0x7facb80a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7facbc007fd0 tx=0x7facbc00c040 comp rx=0 tx=0).stop 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc1094e0 msgr2=0x7faccc199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc1094e0 0x7faccc199800 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7facc4005910 tx=0x7facc40047b0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 shutdown_connections 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faccc1094e0 0x7faccc199800 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7facb80a3cd0 0x7facb80a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faccc1034a0 0x7faccc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 --2- 172.21.15.55:0/2692181122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faccc103e60 0x7faccc19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.342+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 >> 172.21.15.55:0/2692181122 conn(0x7faccc07a590 msgr2=0x7faccc102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.343+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 shutdown_connections 2024-04-26T21:52:08.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.343+0000 7facd11aa700 1 -- 172.21.15.55:0/2692181122 wait complete. 2024-04-26T21:52:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.532+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f873002e490 con 0x7f8730103e80 2024-04-26T21:52:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.532+0000 7f872dffb700 1 -- 172.21.15.55:0/2970774897 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f871c09c5f0 con 0x7f8730103e80 2024-04-26T21:52:08.533 INFO:teuthology.orchestra.run.smithi055.stdout:107374182469 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f87200a3d50 msgr2=0x7f87200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f87200a3d50 0x7f87200a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f87240098e0 tx=0x7f8724000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730103e80 msgr2=0x7f873019fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730103e80 0x7f873019fd20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f871c00b530 tx=0x7f871c00b840 comp rx=0 tx=0).stop 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 shutdown_connections 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f87200a3d50 0x7f87200a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8730109560 0x7f87301999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87301034c0 0x7f873019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 --2- 172.21.15.55:0/2970774897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8730103e80 0x7f873019fd20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:08.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 >> 172.21.15.55:0/2970774897 conn(0x7f873007a5d0 msgr2=0x7f87301021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:08.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 shutdown_connections 2024-04-26T21:52:08.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:08.534+0000 7f8736e2a700 1 -- 172.21.15.55:0/2970774897 wait complete. 2024-04-26T21:52:08.902 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:08 smithi055 ceph-mon[29221]: pgmap v326: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:08.902 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:08 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2692181122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:52:08.902 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:08 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2970774897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:52:08.928 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070502 got 253403070507 for osd.8 2024-04-26T21:52:08.929 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:08 smithi064 ceph-mon[32765]: pgmap v326: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:08 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2692181122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:52:09.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:08 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2970774897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:52:09.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:08 smithi165 ceph-mon[33296]: pgmap v326: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:09.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:08 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2692181122' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T21:52:09.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:08 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2970774897' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T21:52:09.326 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574925 for osd.1 2024-04-26T21:52:09.327 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:09.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757306 for osd.5 2024-04-26T21:52:09.823 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:10.523 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182469 for osd.3 2024-04-26T21:52:10.524 DEBUG:teuthology.parallel:result is None 2024-04-26T21:52:10.524 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T21:52:10.524 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:52:10.854 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:10 smithi055 ceph-mon[29221]: pgmap v327: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:11.010 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:10 smithi064 ceph-mon[32765]: pgmap v327: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:11.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:10 smithi165 ceph-mon[33296]: pgmap v327: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:11.334 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:12.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:12 smithi055 ceph-mon[29221]: pgmap v328: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:13.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:12 smithi064 ceph-mon[32765]: pgmap v328: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 -- 172.21.15.55:0/154414531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 msgr2=0x7fdda410f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 --2- 172.21.15.55:0/154414531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 0x7fdda410f1d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00b0a0 tx=0x7fdd9c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 -- 172.21.15.55:0/154414531 shutdown_connections 2024-04-26T21:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 --2- 172.21.15.55:0/154414531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 0x7fdda410f1d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 --2- 172.21.15.55:0/154414531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdda4108390 0x7fdda4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 --2- 172.21.15.55:0/154414531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda4102320 0x7fdda41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 -- 172.21.15.55:0/154414531 >> 172.21.15.55:0/154414531 conn(0x7fdda40792d0 msgr2=0x7fdda40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:13.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 -- 172.21.15.55:0/154414531 shutdown_connections 2024-04-26T21:52:13.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.012+0000 7fdda8c87700 1 -- 172.21.15.55:0/154414531 wait complete. 2024-04-26T21:52:13.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdda8c87700 1 Processor -- start 2024-04-26T21:52:13.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdda8c87700 1 -- start start 2024-04-26T21:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdda8c87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda4102320 0x7fdda419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdda8c87700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 0x7fdda419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdda8c87700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdda4108390 0x7fdda41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdda8c87700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41124d0 con 0x7fdda4102ce0 2024-04-26T21:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdda8c87700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4112350 con 0x7fdda4102320 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdda8c87700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4112650 con 0x7fdda4108390 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdd9bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 0x7fdda419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdd9bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 0x7fdda419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47862/0 (socket says 172.21.15.55:47862) 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.013+0000 7fdd9bfff700 1 -- 172.21.15.55:0/1866969538 learned_addr learned my addr 172.21.15.55:0/1866969538 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdda2d9d700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdda4108390 0x7fdda41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdda259c700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda4102320 0x7fdda419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9bfff700 1 -- 172.21.15.55:0/1866969538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdda4108390 msgr2=0x7fdda41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9bfff700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdda4108390 0x7fdda41998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9bfff700 1 -- 172.21.15.55:0/1866969538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda4102320 msgr2=0x7fdda419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9bfff700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda4102320 0x7fdda419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9bfff700 1 -- 172.21.15.55:0/1866969538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda419a030 con 0x7fdda4102ce0 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdda2d9d700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdda4108390 0x7fdda41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:13.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9bfff700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 0x7fdda419fc00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c920 tx=0x7fdd8c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:13.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9b7fe700 1 -- 172.21.15.55:0/1866969538 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c007b00 con 0x7fdda4102ce0 2024-04-26T21:52:13.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda419a320 con 0x7fdda4102ce0 2024-04-26T21:52:13.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9b7fe700 1 -- 172.21.15.55:0/1866969538 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd8c004ce0 con 0x7fdda4102ce0 2024-04-26T21:52:13.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdd9b7fe700 1 -- 172.21.15.55:0/1866969538 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c005650 con 0x7fdda4102ce0 2024-04-26T21:52:13.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.014+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda41a6590 con 0x7fdda4102ce0 2024-04-26T21:52:13.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.016+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda406a110 con 0x7fdda4102ce0 2024-04-26T21:52:13.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.016+0000 7fdd9b7fe700 1 -- 172.21.15.55:0/1866969538 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdd8c007ca0 con 0x7fdda4102ce0 2024-04-26T21:52:13.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.017+0000 7fdd9b7fe700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd900a3d50 0x7fdd900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:13.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.017+0000 7fdda259c700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd900a3d50 0x7fdd900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:13.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.017+0000 7fdd9b7fe700 1 -- 172.21.15.55:0/1866969538 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fdd8c0d04c0 con 0x7fdda4102ce0 2024-04-26T21:52:13.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.018+0000 7fdda259c700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd900a3d50 0x7fdd900a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd94005e10 tx=0x7fdd94005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:13.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.022+0000 7fdd9b7fe700 1 -- 172.21.15.55:0/1866969538 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdd8c014030 con 0x7fdda4102ce0 2024-04-26T21:52:13.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:12 smithi165 ceph-mon[33296]: pgmap v328: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.237+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdda4106140 con 0x7fdd900a3d50 2024-04-26T21:52:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.244+0000 7fdd9b7fe700 1 -- 172.21.15.55:0/1866969538 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258986 (secure 0 0 0) 0x7fdda4106140 con 0x7fdd900a3d50 2024-04-26T21:52:13.250 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd900a3d50 msgr2=0x7fdd900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd900a3d50 0x7fdd900a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd94005e10 tx=0x7fdd94005d40 comp rx=0 tx=0).stop 2024-04-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 msgr2=0x7fdda419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 0x7fdda419fc00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c920 tx=0x7fdd8c00cce0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 shutdown_connections 2024-04-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdda4108390 0x7fdda41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd900a3d50 0x7fdd900a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdda4102320 0x7fdda419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 --2- 172.21.15.55:0/1866969538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda4102ce0 0x7fdda419fc00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 >> 172.21.15.55:0/1866969538 conn(0x7fdda40792d0 msgr2=0x7fdda4101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 shutdown_connections 2024-04-26T21:52:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:13.247+0000 7fdda8c87700 1 -- 172.21.15.55:0/1866969538 wait complete. 2024-04-26T21:52:13.253 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-04-26T21:52:13.941 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-04-26T21:52:11.360287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644080,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":509,"num_read_kb":383,"num_write":336,"num_write_kb":3035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352416,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589408,"statfs":{"total":1151940427776,"available":1151579553792,"internally_reserved":0,"allocated":11796480,"data_stored":7401804,"data_compressed":183945,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002982"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545722+0000","last_change":"2024-04-26T21:50:57.139486+0000","last_active":"2024-04-26T21:51:50.545722+0000","last_peered":"2024-04-26T21:51:50.545722+0000","last_clean":"2024-04-26T21:51:50.545722+0000","last_became_active":"2024-04-26T21:50:57.139133+0000","last_became_peered":"2024-04-26T21:50:57.139133+0000","last_unstale":"2024-04-26T21:51:50.545722+0000","last_undegraded":"2024-04-26T21:51:50.545722+0000","last_fullsized":"2024-04-26T21:51:50.545722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:33.211627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539345+0000","last_change":"2024-04-26T21:50:57.138519+0000","last_active":"2024-04-26T21:51:50.539345+0000","last_peered":"2024-04-26T21:51:50.539345+0000","last_clean":"2024-04-26T21:51:50.539345+0000","last_became_active":"2024-04-26T21:50:57.138131+0000","last_became_peered":"2024-04-26T21:50:57.138131+0000","last_unstale":"2024-04-26T21:51:50.539345+0000","last_undegraded":"2024-04-26T21:51:50.539345+0000","last_fullsized":"2024-04-26T21:51:50.539345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:34.349694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.538994+0000","last_change":"2024-04-26T21:50:57.146584+0000","last_active":"2024-04-26T21:51:50.538994+0000","last_peered":"2024-04-26T21:51:50.538994+0000","last_clean":"2024-04-26T21:51:50.538994+0000","last_became_active":"2024-04-26T21:50:57.146262+0000","last_became_peered":"2024-04-26T21:50:57.146262+0000","last_unstale":"2024-04-26T21:51:50.538994+0000","last_undegraded":"2024-04-26T21:51:50.538994+0000","last_fullsized":"2024-04-26T21:51:50.538994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:49.946882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.548238+0000","last_change":"2024-04-26T21:51:50.546008+0000","last_active":"2024-04-26T21:51:50.548238+0000","last_peered":"2024-04-26T21:51:50.548238+0000","last_clean":"2024-04-26T21:51:50.548238+0000","last_became_active":"2024-04-26T21:51:50.545739+0000","last_became_peered":"2024-04-26T21:51:50.545739+0000","last_unstale":"2024-04-26T21:51:50.548238+0000","last_undegraded":"2024-04-26T21:51:50.548238+0000","last_fullsized":"2024-04-26T21:51:50.548238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.494950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539285+0000","last_change":"2024-04-26T21:50:57.136292+0000","last_active":"2024-04-26T21:51:50.539285+0000","last_peered":"2024-04-26T21:51:50.539285+0000","last_clean":"2024-04-26T21:51:50.539285+0000","last_became_active":"2024-04-26T21:50:57.135936+0000","last_became_peered":"2024-04-26T21:50:57.135936+0000","last_unstale":"2024-04-26T21:51:50.539285+0000","last_undegraded":"2024-04-26T21:51:50.539285+0000","last_fullsized":"2024-04-26T21:51:50.539285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:35.415440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544083+0000","last_change":"2024-04-26T21:50:57.147764+0000","last_active":"2024-04-26T21:51:50.544083+0000","last_peered":"2024-04-26T21:51:50.544083+0000","last_clean":"2024-04-26T21:51:50.544083+0000","last_became_active":"2024-04-26T21:50:57.147034+0000","last_became_peered":"2024-04-26T21:50:57.147034+0000","last_unstale":"2024-04-26T21:51:50.544083+0000","last_undegraded":"2024-04-26T21:51:50.544083+0000","last_fullsized":"2024-04-26T21:51:50.544083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:44.833213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541303+0000","last_change":"2024-04-26T21:50:57.144813+0000","last_active":"2024-04-26T21:51:50.541303+0000","last_peered":"2024-04-26T21:51:50.541303+0000","last_clean":"2024-04-26T21:51:50.541303+0000","last_became_active":"2024-04-26T21:50:57.144442+0000","last_became_peered":"2024-04-26T21:50:57.144442+0000","last_unstale":"2024-04-26T21:51:50.541303+0000","last_undegraded":"2024-04-26T21:51:50.541303+0000","last_fullsized":"2024-04-26T21:51:50.541303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:59.032263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.548896+0000","last_change":"2024-04-26T21:51:50.547307+0000","last_active":"2024-04-26T21:51:50.548896+0000","last_peered":"2024-04-26T21:51:50.548896+0000","last_clean":"2024-04-26T21:51:50.548896+0000","last_became_active":"2024-04-26T21:51:50.546888+0000","last_became_peered":"2024-04-26T21:51:50.546888+0000","last_unstale":"2024-04-26T21:51:50.548896+0000","last_undegraded":"2024-04-26T21:51:50.548896+0000","last_fullsized":"2024-04-26T21:51:50.548896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:36.137772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.938023+0000","last_change":"2024-04-26T21:50:57.132957+0000","last_active":"2024-04-26T21:51:50.938023+0000","last_peered":"2024-04-26T21:51:50.938023+0000","last_clean":"2024-04-26T21:51:50.938023+0000","last_became_active":"2024-04-26T21:50:57.132606+0000","last_became_peered":"2024-04-26T21:50:57.132606+0000","last_unstale":"2024-04-26T21:51:50.938023+0000","last_undegraded":"2024-04-26T21:51:50.938023+0000","last_fullsized":"2024-04-26T21:51:50.938023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:00.994370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544743+0000","last_change":"2024-04-26T21:50:57.150598+0000","last_active":"2024-04-26T21:51:50.544743+0000","last_peered":"2024-04-26T21:51:50.544743+0000","last_clean":"2024-04-26T21:51:50.544743+0000","last_became_active":"2024-04-26T21:50:57.149697+0000","last_became_peered":"2024-04-26T21:50:57.149697+0000","last_unstale":"2024-04-26T21:51:50.544743+0000","last_undegraded":"2024-04-26T21:51:50.544743+0000","last_fullsized":"2024-04-26T21:51:50.544743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.621137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542183+0000","last_change":"2024-04-26T21:50:57.147264+0000","last_active":"2024-04-26T21:51:50.542183+0000","last_peered":"2024-04-26T21:51:50.542183+0000","last_clean":"2024-04-26T21:51:50.542183+0000","last_became_active":"2024-04-26T21:50:57.146888+0000","last_became_peered":"2024-04-26T21:50:57.146888+0000","last_unstale":"2024-04-26T21:51:50.542183+0000","last_undegraded":"2024-04-26T21:51:50.542183+0000","last_fullsized":"2024-04-26T21:51:50.542183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:49.584327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544007+0000","last_change":"2024-04-26T21:50:57.142005+0000","last_active":"2024-04-26T21:51:50.544007+0000","last_peered":"2024-04-26T21:51:50.544007+0000","last_clean":"2024-04-26T21:51:50.544007+0000","last_became_active":"2024-04-26T21:50:57.141605+0000","last_became_peered":"2024-04-26T21:50:57.141605+0000","last_unstale":"2024-04-26T21:51:50.544007+0000","last_undegraded":"2024-04-26T21:51:50.544007+0000","last_fullsized":"2024-04-26T21:51:50.544007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:43.485295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543887+0000","last_change":"2024-04-26T21:50:57.138049+0000","last_active":"2024-04-26T21:51:50.543887+0000","last_peered":"2024-04-26T21:51:50.543887+0000","last_clean":"2024-04-26T21:51:50.543887+0000","last_became_active":"2024-04-26T21:50:57.137717+0000","last_became_peered":"2024-04-26T21:50:57.137717+0000","last_unstale":"2024-04-26T21:51:50.543887+0000","last_undegraded":"2024-04-26T21:51:50.543887+0000","last_fullsized":"2024-04-26T21:51:50.543887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:28.918224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.538938+0000","last_change":"2024-04-26T21:50:57.139183+0000","last_active":"2024-04-26T21:51:50.538938+0000","last_peered":"2024-04-26T21:51:50.538938+0000","last_clean":"2024-04-26T21:51:50.538938+0000","last_became_active":"2024-04-26T21:50:57.138759+0000","last_became_peered":"2024-04-26T21:50:57.138759+0000","last_unstale":"2024-04-26T21:51:50.538938+0000","last_undegraded":"2024-04-26T21:51:50.538938+0000","last_fullsized":"2024-04-26T21:51:50.538938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:12.003122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.538904+0000","last_change":"2024-04-26T21:50:57.143110+0000","last_active":"2024-04-26T21:51:50.538904+0000","last_peered":"2024-04-26T21:51:50.538904+0000","last_clean":"2024-04-26T21:51:50.538904+0000","last_became_active":"2024-04-26T21:50:57.142754+0000","last_became_peered":"2024-04-26T21:50:57.142754+0000","last_unstale":"2024-04-26T21:51:50.538904+0000","last_undegraded":"2024-04-26T21:51:50.538904+0000","last_fullsized":"2024-04-26T21:51:50.538904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:08.321000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937309+0000","last_change":"2024-04-26T21:50:57.146514+0000","last_active":"2024-04-26T21:51:50.937309+0000","last_peered":"2024-04-26T21:51:50.937309+0000","last_clean":"2024-04-26T21:51:50.937309+0000","last_became_active":"2024-04-26T21:50:57.146046+0000","last_became_peered":"2024-04-26T21:50:57.146046+0000","last_unstale":"2024-04-26T21:51:50.937309+0000","last_undegraded":"2024-04-26T21:51:50.937309+0000","last_fullsized":"2024-04-26T21:51:50.937309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:21.130220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545839+0000","last_change":"2024-04-26T21:50:57.140492+0000","last_active":"2024-04-26T21:51:50.545839+0000","last_peered":"2024-04-26T21:51:50.545839+0000","last_clean":"2024-04-26T21:51:50.545839+0000","last_became_active":"2024-04-26T21:50:57.140147+0000","last_became_peered":"2024-04-26T21:50:57.140147+0000","last_unstale":"2024-04-26T21:51:50.545839+0000","last_undegraded":"2024-04-26T21:51:50.545839+0000","last_fullsized":"2024-04-26T21:51:50.545839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:21.679202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.540908+0000","last_change":"2024-04-26T21:50:57.140992+0000","last_active":"2024-04-26T21:51:50.540908+0000","last_peered":"2024-04-26T21:51:50.540908+0000","last_clean":"2024-04-26T21:51:50.540908+0000","last_became_active":"2024-04-26T21:50:57.140644+0000","last_became_peered":"2024-04-26T21:50:57.140644+0000","last_unstale":"2024-04-26T21:51:50.540908+0000","last_undegraded":"2024-04-26T21:51:50.540908+0000","last_fullsized":"2024-04-26T21:51:50.540908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541570+0000","last_change":"2024-04-26T21:50:57.141242+0000","last_active":"2024-04-26T21:51:50.541570+0000","last_peered":"2024-04-26T21:51:50.541570+0000","last_clean":"2024-04-26T21:51:50.541570+0000","last_became_active":"2024-04-26T21:50:57.140909+0000","last_became_peered":"2024-04-26T21:50:57.140909+0000","last_unstale":"2024-04-26T21:51:50.541570+0000","last_undegraded":"2024-04-26T21:51:50.541570+0000","last_fullsized":"2024-04-26T21:51:50.541570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:21.996984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541007+0000","last_change":"2024-04-26T21:50:56.371876+0000","last_active":"2024-04-26T21:51:50.541007+0000","last_peered":"2024-04-26T21:51:50.541007+0000","last_clean":"2024-04-26T21:51:50.541007+0000","last_became_active":"2024-04-26T21:50:56.371509+0000","last_became_peered":"2024-04-26T21:50:56.371509+0000","last_unstale":"2024-04-26T21:51:50.541007+0000","last_undegraded":"2024-04-26T21:51:50.541007+0000","last_fullsized":"2024-04-26T21:51:50.541007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:50.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594536+0000","last_change":"2024-04-26T21:50:57.145654+0000","last_active":"2024-04-26T21:51:50.594536+0000","last_peered":"2024-04-26T21:51:50.594536+0000","last_clean":"2024-04-26T21:51:50.594536+0000","last_became_active":"2024-04-26T21:50:57.145355+0000","last_became_peered":"2024-04-26T21:50:57.145355+0000","last_unstale":"2024-04-26T21:51:50.594536+0000","last_undegraded":"2024-04-26T21:51:50.594536+0000","last_fullsized":"2024-04-26T21:51:50.594536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:51.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594661+0000","last_change":"2024-04-26T21:50:56.362758+0000","last_active":"2024-04-26T21:51:50.594661+0000","last_peered":"2024-04-26T21:51:50.594661+0000","last_clean":"2024-04-26T21:51:50.594661+0000","last_became_active":"2024-04-26T21:50:56.361338+0000","last_became_peered":"2024-04-26T21:50:56.361338+0000","last_unstale":"2024-04-26T21:51:50.594661+0000","last_undegraded":"2024-04-26T21:51:50.594661+0000","last_fullsized":"2024-04-26T21:51:50.594661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:44.603426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539534+0000","last_change":"2024-04-26T21:50:57.150701+0000","last_active":"2024-04-26T21:51:50.539534+0000","last_peered":"2024-04-26T21:51:50.539534+0000","last_clean":"2024-04-26T21:51:50.539534+0000","last_became_active":"2024-04-26T21:50:57.150388+0000","last_became_peered":"2024-04-26T21:50:57.150388+0000","last_unstale":"2024-04-26T21:51:50.539534+0000","last_undegraded":"2024-04-26T21:51:50.539534+0000","last_fullsized":"2024-04-26T21:51:50.539534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:43:07.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539956+0000","last_change":"2024-04-26T21:50:56.371379+0000","last_active":"2024-04-26T21:51:50.539956+0000","last_peered":"2024-04-26T21:51:50.539956+0000","last_clean":"2024-04-26T21:51:50.539956+0000","last_became_active":"2024-04-26T21:50:56.371025+0000","last_became_peered":"2024-04-26T21:50:56.371025+0000","last_unstale":"2024-04-26T21:51:50.539956+0000","last_undegraded":"2024-04-26T21:51:50.539956+0000","last_fullsized":"2024-04-26T21:51:50.539956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:00.633393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937662+0000","last_change":"2024-04-26T21:50:57.144777+0000","last_active":"2024-04-26T21:51:50.937662+0000","last_peered":"2024-04-26T21:51:50.937662+0000","last_clean":"2024-04-26T21:51:50.937662+0000","last_became_active":"2024-04-26T21:50:57.144431+0000","last_became_peered":"2024-04-26T21:50:57.144431+0000","last_unstale":"2024-04-26T21:51:50.937662+0000","last_undegraded":"2024-04-26T21:51:50.937662+0000","last_fullsized":"2024-04-26T21:51:50.937662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:42.164240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539612+0000","last_change":"2024-04-26T21:50:56.360843+0000","last_active":"2024-04-26T21:51:50.539612+0000","last_peered":"2024-04-26T21:51:50.539612+0000","last_clean":"2024-04-26T21:51:50.539612+0000","last_became_active":"2024-04-26T21:50:56.360440+0000","last_became_peered":"2024-04-26T21:50:56.360440+0000","last_unstale":"2024-04-26T21:51:50.539612+0000","last_undegraded":"2024-04-26T21:51:50.539612+0000","last_fullsized":"2024-04-26T21:51:50.539612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:47.585306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544573+0000","last_change":"2024-04-26T21:50:57.146449+0000","last_active":"2024-04-26T21:51:50.544573+0000","last_peered":"2024-04-26T21:51:50.544573+0000","last_clean":"2024-04-26T21:51:50.544573+0000","last_became_active":"2024-04-26T21:50:57.146128+0000","last_became_peered":"2024-04-26T21:50:57.146128+0000","last_unstale":"2024-04-26T21:51:50.544573+0000","last_undegraded":"2024-04-26T21:51:50.544573+0000","last_fullsized":"2024-04-26T21:51:50.544573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:09.165868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:04.567911+0000","last_change":"2024-04-26T21:51:50.551697+0000","last_active":"2024-04-26T21:52:04.567911+0000","last_peered":"2024-04-26T21:52:04.567911+0000","last_clean":"2024-04-26T21:52:04.567911+0000","last_became_active":"2024-04-26T21:51:50.546274+0000","last_became_peered":"2024-04-26T21:51:50.546274+0000","last_unstale":"2024-04-26T21:52:04.567911+0000","last_undegraded":"2024-04-26T21:52:04.567911+0000","last_fullsized":"2024-04-26T21:52:04.567911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:44.549374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539254+0000","last_change":"2024-04-26T21:50:57.138817+0000","last_active":"2024-04-26T21:51:50.539254+0000","last_peered":"2024-04-26T21:51:50.539254+0000","last_clean":"2024-04-26T21:51:50.539254+0000","last_became_active":"2024-04-26T21:50:57.138489+0000","last_became_peered":"2024-04-26T21:50:57.138489+0000","last_unstale":"2024-04-26T21:51:50.539254+0000","last_undegraded":"2024-04-26T21:51:50.539254+0000","last_fullsized":"2024-04-26T21:51:50.539254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:56.072063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.941723+0000","last_change":"2024-04-26T21:51:50.941723+0000","last_active":"2024-04-26T21:51:50.941723+0000","last_peered":"2024-04-26T21:51:50.941723+0000","last_clean":"2024-04-26T21:51:50.941723+0000","last_became_active":"2024-04-26T21:51:50.938689+0000","last_became_peered":"2024-04-26T21:51:50.938689+0000","last_unstale":"2024-04-26T21:51:50.941723+0000","last_undegraded":"2024-04-26T21:51:50.941723+0000","last_fullsized":"2024-04-26T21:51:50.941723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:12.119245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.595771+0000","last_change":"2024-04-26T21:51:50.595771+0000","last_active":"2024-04-26T21:51:50.595771+0000","last_peered":"2024-04-26T21:51:50.595771+0000","last_clean":"2024-04-26T21:51:50.595771+0000","last_became_active":"2024-04-26T21:51:50.595441+0000","last_became_peered":"2024-04-26T21:51:50.595441+0000","last_unstale":"2024-04-26T21:51:50.595771+0000","last_undegraded":"2024-04-26T21:51:50.595771+0000","last_fullsized":"2024-04-26T21:51:50.595771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:53.753035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543667+0000","last_change":"2024-04-26T21:50:56.362272+0000","last_active":"2024-04-26T21:51:50.543667+0000","last_peered":"2024-04-26T21:51:50.543667+0000","last_clean":"2024-04-26T21:51:50.543667+0000","last_became_active":"2024-04-26T21:50:56.361941+0000","last_became_peered":"2024-04-26T21:50:56.361941+0000","last_unstale":"2024-04-26T21:51:50.543667+0000","last_undegraded":"2024-04-26T21:51:50.543667+0000","last_fullsized":"2024-04-26T21:51:50.543667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594673+0000","last_change":"2024-04-26T21:50:57.135566+0000","last_active":"2024-04-26T21:51:50.594673+0000","last_peered":"2024-04-26T21:51:50.594673+0000","last_clean":"2024-04-26T21:51:50.594673+0000","last_became_active":"2024-04-26T21:50:57.135231+0000","last_became_peered":"2024-04-26T21:50:57.135231+0000","last_unstale":"2024-04-26T21:51:50.594673+0000","last_undegraded":"2024-04-26T21:51:50.594673+0000","last_fullsized":"2024-04-26T21:51:50.594673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:09.381936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544881+0000","last_change":"2024-04-26T21:50:56.372420+0000","last_active":"2024-04-26T21:51:50.544881+0000","last_peered":"2024-04-26T21:51:50.544881+0000","last_clean":"2024-04-26T21:51:50.544881+0000","last_became_active":"2024-04-26T21:50:56.372096+0000","last_became_peered":"2024-04-26T21:50:56.372096+0000","last_unstale":"2024-04-26T21:51:50.544881+0000","last_undegraded":"2024-04-26T21:51:50.544881+0000","last_fullsized":"2024-04-26T21:51:50.544881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:03.515804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.540837+0000","last_change":"2024-04-26T21:50:57.140071+0000","last_active":"2024-04-26T21:51:50.540837+0000","last_peered":"2024-04-26T21:51:50.540837+0000","last_clean":"2024-04-26T21:51:50.540837+0000","last_became_active":"2024-04-26T21:50:57.139693+0000","last_became_peered":"2024-04-26T21:50:57.139693+0000","last_unstale":"2024-04-26T21:51:50.540837+0000","last_undegraded":"2024-04-26T21:51:50.540837+0000","last_fullsized":"2024-04-26T21:51:50.540837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:17.963966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:07.571400+0000","last_change":"2024-04-26T21:50:56.363748+0000","last_active":"2024-04-26T21:52:07.571400+0000","last_peered":"2024-04-26T21:52:07.571400+0000","last_clean":"2024-04-26T21:52:07.571400+0000","last_became_active":"2024-04-26T21:50:56.363328+0000","last_became_peered":"2024-04-26T21:50:56.363328+0000","last_unstale":"2024-04-26T21:52:07.571400+0000","last_undegraded":"2024-04-26T21:52:07.571400+0000","last_fullsized":"2024-04-26T21:52:07.571400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:24.450981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937846+0000","last_change":"2024-04-26T21:50:57.143064+0000","last_active":"2024-04-26T21:51:50.937846+0000","last_peered":"2024-04-26T21:51:50.937846+0000","last_clean":"2024-04-26T21:51:50.937846+0000","last_became_active":"2024-04-26T21:50:57.142715+0000","last_became_peered":"2024-04-26T21:50:57.142715+0000","last_unstale":"2024-04-26T21:51:50.937846+0000","last_undegraded":"2024-04-26T21:51:50.937846+0000","last_fullsized":"2024-04-26T21:51:50.937846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:35.901855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543472+0000","last_change":"2024-04-26T21:50:56.369898+0000","last_active":"2024-04-26T21:51:50.543472+0000","last_peered":"2024-04-26T21:51:50.543472+0000","last_clean":"2024-04-26T21:51:50.543472+0000","last_became_active":"2024-04-26T21:50:56.369349+0000","last_became_peered":"2024-04-26T21:50:56.369349+0000","last_unstale":"2024-04-26T21:51:50.543472+0000","last_undegraded":"2024-04-26T21:51:50.543472+0000","last_fullsized":"2024-04-26T21:51:50.543472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:41.378193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539538+0000","last_change":"2024-04-26T21:50:57.141417+0000","last_active":"2024-04-26T21:51:50.539538+0000","last_peered":"2024-04-26T21:51:50.539538+0000","last_clean":"2024-04-26T21:51:50.539538+0000","last_became_active":"2024-04-26T21:50:57.141091+0000","last_became_peered":"2024-04-26T21:50:57.141091+0000","last_unstale":"2024-04-26T21:51:50.539538+0000","last_undegraded":"2024-04-26T21:51:50.539538+0000","last_fullsized":"2024-04-26T21:51:50.539538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:45.357827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545426+0000","last_change":"2024-04-26T21:50:56.364439+0000","last_active":"2024-04-26T21:51:50.545426+0000","last_peered":"2024-04-26T21:51:50.545426+0000","last_clean":"2024-04-26T21:51:50.545426+0000","last_became_active":"2024-04-26T21:50:56.364090+0000","last_became_peered":"2024-04-26T21:50:56.364090+0000","last_unstale":"2024-04-26T21:51:50.545426+0000","last_undegraded":"2024-04-26T21:51:50.545426+0000","last_fullsized":"2024-04-26T21:51:50.545426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:43.562189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937656+0000","last_change":"2024-04-26T21:50:57.145976+0000","last_active":"2024-04-26T21:51:50.937656+0000","last_peered":"2024-04-26T21:51:50.937656+0000","last_clean":"2024-04-26T21:51:50.937656+0000","last_became_active":"2024-04-26T21:50:57.145620+0000","last_became_peered":"2024-04-26T21:50:57.145620+0000","last_unstale":"2024-04-26T21:51:50.937656+0000","last_undegraded":"2024-04-26T21:51:50.937656+0000","last_fullsized":"2024-04-26T21:51:50.937656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:26.351362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'25","reported_seq":187,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:08.572604+0000","last_change":"2024-04-26T21:50:56.365786+0000","last_active":"2024-04-26T21:52:08.572604+0000","last_peered":"2024-04-26T21:52:08.572604+0000","last_clean":"2024-04-26T21:52:08.572604+0000","last_became_active":"2024-04-26T21:50:56.365410+0000","last_became_peered":"2024-04-26T21:50:56.365410+0000","last_unstale":"2024-04-26T21:52:08.572604+0000","last_undegraded":"2024-04-26T21:52:08.572604+0000","last_fullsized":"2024-04-26T21:52:08.572604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:01.321661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49477,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":143,"num_read_kb":119,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539601+0000","last_change":"2024-04-26T21:50:57.137333+0000","last_active":"2024-04-26T21:51:50.539601+0000","last_peered":"2024-04-26T21:51:50.539601+0000","last_clean":"2024-04-26T21:51:50.539601+0000","last_became_active":"2024-04-26T21:50:57.136983+0000","last_became_peered":"2024-04-26T21:50:57.136983+0000","last_unstale":"2024-04-26T21:51:50.539601+0000","last_undegraded":"2024-04-26T21:51:50.539601+0000","last_fullsized":"2024-04-26T21:51:50.539601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:18.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542083+0000","last_change":"2024-04-26T21:50:56.358340+0000","last_active":"2024-04-26T21:51:50.542083+0000","last_peered":"2024-04-26T21:51:50.542083+0000","last_clean":"2024-04-26T21:51:50.542083+0000","last_became_active":"2024-04-26T21:50:56.357972+0000","last_became_peered":"2024-04-26T21:50:56.357972+0000","last_unstale":"2024-04-26T21:51:50.542083+0000","last_undegraded":"2024-04-26T21:51:50.542083+0000","last_fullsized":"2024-04-26T21:51:50.542083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:52.256528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543770+0000","last_change":"2024-04-26T21:50:57.137027+0000","last_active":"2024-04-26T21:51:50.543770+0000","last_peered":"2024-04-26T21:51:50.543770+0000","last_clean":"2024-04-26T21:51:50.543770+0000","last_became_active":"2024-04-26T21:50:57.136400+0000","last_became_peered":"2024-04-26T21:50:57.136400+0000","last_unstale":"2024-04-26T21:51:50.543770+0000","last_undegraded":"2024-04-26T21:51:50.543770+0000","last_fullsized":"2024-04-26T21:51:50.543770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:45.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543952+0000","last_change":"2024-04-26T21:50:56.369784+0000","last_active":"2024-04-26T21:51:50.543952+0000","last_peered":"2024-04-26T21:51:50.543952+0000","last_clean":"2024-04-26T21:51:50.543952+0000","last_became_active":"2024-04-26T21:50:56.369093+0000","last_became_peered":"2024-04-26T21:50:56.369093+0000","last_unstale":"2024-04-26T21:51:50.543952+0000","last_undegraded":"2024-04-26T21:51:50.543952+0000","last_fullsized":"2024-04-26T21:51:50.543952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:27.595146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543136+0000","last_change":"2024-04-26T21:50:57.141404+0000","last_active":"2024-04-26T21:51:50.543136+0000","last_peered":"2024-04-26T21:51:50.543136+0000","last_clean":"2024-04-26T21:51:50.543136+0000","last_became_active":"2024-04-26T21:50:57.140973+0000","last_became_peered":"2024-04-26T21:50:57.140973+0000","last_unstale":"2024-04-26T21:51:50.543136+0000","last_undegraded":"2024-04-26T21:51:50.543136+0000","last_fullsized":"2024-04-26T21:51:50.543136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:09.405982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:08.448580+0000","last_change":"2024-04-26T21:50:56.370727+0000","last_active":"2024-04-26T21:52:08.448580+0000","last_peered":"2024-04-26T21:52:08.448580+0000","last_clean":"2024-04-26T21:52:08.448580+0000","last_became_active":"2024-04-26T21:50:56.370373+0000","last_became_peered":"2024-04-26T21:50:56.370373+0000","last_unstale":"2024-04-26T21:52:08.448580+0000","last_undegraded":"2024-04-26T21:52:08.448580+0000","last_fullsized":"2024-04-26T21:52:08.448580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:12.689699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542026+0000","last_change":"2024-04-26T21:50:57.145771+0000","last_active":"2024-04-26T21:51:50.542026+0000","last_peered":"2024-04-26T21:51:50.542026+0000","last_clean":"2024-04-26T21:51:50.542026+0000","last_became_active":"2024-04-26T21:50:57.145424+0000","last_became_peered":"2024-04-26T21:50:57.145424+0000","last_unstale":"2024-04-26T21:51:50.542026+0000","last_undegraded":"2024-04-26T21:51:50.542026+0000","last_fullsized":"2024-04-26T21:51:50.542026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:10.485045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.560040+0000","last_change":"2024-04-26T21:51:50.560040+0000","last_active":"2024-04-26T21:51:50.560040+0000","last_peered":"2024-04-26T21:51:50.560040+0000","last_clean":"2024-04-26T21:51:50.560040+0000","last_became_active":"2024-04-26T21:51:50.546278+0000","last_became_peered":"2024-04-26T21:51:50.546278+0000","last_unstale":"2024-04-26T21:51:50.560040+0000","last_undegraded":"2024-04-26T21:51:50.560040+0000","last_fullsized":"2024-04-26T21:51:50.560040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:59.851719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.938948+0000","last_change":"2024-04-26T21:51:50.938948+0000","last_active":"2024-04-26T21:51:50.938948+0000","last_peered":"2024-04-26T21:51:50.938948+0000","last_clean":"2024-04-26T21:51:50.938948+0000","last_became_active":"2024-04-26T21:51:50.938501+0000","last_became_peered":"2024-04-26T21:51:50.938501+0000","last_unstale":"2024-04-26T21:51:50.938948+0000","last_undegraded":"2024-04-26T21:51:50.938948+0000","last_fullsized":"2024-04-26T21:51:50.938948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:44.936013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539918+0000","last_change":"2024-04-26T21:50:56.358270+0000","last_active":"2024-04-26T21:51:50.539918+0000","last_peered":"2024-04-26T21:51:50.539918+0000","last_clean":"2024-04-26T21:51:50.539918+0000","last_became_active":"2024-04-26T21:50:56.357909+0000","last_became_peered":"2024-04-26T21:50:56.357909+0000","last_unstale":"2024-04-26T21:51:50.539918+0000","last_undegraded":"2024-04-26T21:51:50.539918+0000","last_fullsized":"2024-04-26T21:51:50.539918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:45.627352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545482+0000","last_change":"2024-04-26T21:50:57.142181+0000","last_active":"2024-04-26T21:51:50.545482+0000","last_peered":"2024-04-26T21:51:50.545482+0000","last_clean":"2024-04-26T21:51:50.545482+0000","last_became_active":"2024-04-26T21:50:57.141837+0000","last_became_peered":"2024-04-26T21:50:57.141837+0000","last_unstale":"2024-04-26T21:51:50.545482+0000","last_undegraded":"2024-04-26T21:51:50.545482+0000","last_fullsized":"2024-04-26T21:51:50.545482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:42.150522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.547619+0000","last_change":"2024-04-26T21:51:50.547619+0000","last_active":"2024-04-26T21:51:50.547619+0000","last_peered":"2024-04-26T21:51:50.547619+0000","last_clean":"2024-04-26T21:51:50.547619+0000","last_became_active":"2024-04-26T21:51:50.547180+0000","last_became_peered":"2024-04-26T21:51:50.547180+0000","last_unstale":"2024-04-26T21:51:50.547619+0000","last_undegraded":"2024-04-26T21:51:50.547619+0000","last_fullsized":"2024-04-26T21:51:50.547619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:17.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544546+0000","last_change":"2024-04-26T21:50:57.146682+0000","last_active":"2024-04-26T21:51:50.544546+0000","last_peered":"2024-04-26T21:51:50.544546+0000","last_clean":"2024-04-26T21:51:50.544546+0000","last_became_active":"2024-04-26T21:50:57.146329+0000","last_became_peered":"2024-04-26T21:50:57.146329+0000","last_unstale":"2024-04-26T21:51:50.544546+0000","last_undegraded":"2024-04-26T21:51:50.544546+0000","last_fullsized":"2024-04-26T21:51:50.544546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:41.035526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541572+0000","last_change":"2024-04-26T21:50:56.370533+0000","last_active":"2024-04-26T21:51:50.541572+0000","last_peered":"2024-04-26T21:51:50.541572+0000","last_clean":"2024-04-26T21:51:50.541572+0000","last_became_active":"2024-04-26T21:50:56.370029+0000","last_became_peered":"2024-04-26T21:50:56.370029+0000","last_unstale":"2024-04-26T21:51:50.541572+0000","last_undegraded":"2024-04-26T21:51:50.541572+0000","last_fullsized":"2024-04-26T21:51:50.541572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.005405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544183+0000","last_change":"2024-04-26T21:50:57.142420+0000","last_active":"2024-04-26T21:51:50.544183+0000","last_peered":"2024-04-26T21:51:50.544183+0000","last_clean":"2024-04-26T21:51:50.544183+0000","last_became_active":"2024-04-26T21:50:57.142023+0000","last_became_peered":"2024-04-26T21:50:57.142023+0000","last_unstale":"2024-04-26T21:51:50.544183+0000","last_undegraded":"2024-04-26T21:51:50.544183+0000","last_fullsized":"2024-04-26T21:51:50.544183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:31.567136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545145+0000","last_change":"2024-04-26T21:50:57.139924+0000","last_active":"2024-04-26T21:51:50.545145+0000","last_peered":"2024-04-26T21:51:50.545145+0000","last_clean":"2024-04-26T21:51:50.545145+0000","last_became_active":"2024-04-26T21:50:57.139561+0000","last_became_peered":"2024-04-26T21:50:57.139561+0000","last_unstale":"2024-04-26T21:51:50.545145+0000","last_undegraded":"2024-04-26T21:51:50.545145+0000","last_fullsized":"2024-04-26T21:51:50.545145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:52.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:09.449652+0000","last_change":"2024-04-26T21:50:56.365256+0000","last_active":"2024-04-26T21:52:09.449652+0000","last_peered":"2024-04-26T21:52:09.449652+0000","last_clean":"2024-04-26T21:52:09.449652+0000","last_became_active":"2024-04-26T21:50:56.364516+0000","last_became_peered":"2024-04-26T21:50:56.364516+0000","last_unstale":"2024-04-26T21:52:09.449652+0000","last_undegraded":"2024-04-26T21:52:09.449652+0000","last_fullsized":"2024-04-26T21:52:09.449652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:13.550129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594301+0000","last_change":"2024-04-26T21:50:57.138177+0000","last_active":"2024-04-26T21:51:50.594301+0000","last_peered":"2024-04-26T21:51:50.594301+0000","last_clean":"2024-04-26T21:51:50.594301+0000","last_became_active":"2024-04-26T21:50:57.137725+0000","last_became_peered":"2024-04-26T21:50:57.137725+0000","last_unstale":"2024-04-26T21:51:50.594301+0000","last_undegraded":"2024-04-26T21:51:50.594301+0000","last_fullsized":"2024-04-26T21:51:50.594301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:14.113935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542929+0000","last_change":"2024-04-26T21:50:56.358280+0000","last_active":"2024-04-26T21:51:50.542929+0000","last_peered":"2024-04-26T21:51:50.542929+0000","last_clean":"2024-04-26T21:51:50.542929+0000","last_became_active":"2024-04-26T21:50:56.357854+0000","last_became_peered":"2024-04-26T21:50:56.357854+0000","last_unstale":"2024-04-26T21:51:50.542929+0000","last_undegraded":"2024-04-26T21:51:50.542929+0000","last_fullsized":"2024-04-26T21:51:50.542929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.499124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542214+0000","last_change":"2024-04-26T21:49:42.292617+0000","last_active":"2024-04-26T21:51:50.542214+0000","last_peered":"2024-04-26T21:51:50.542214+0000","last_clean":"2024-04-26T21:51:50.542214+0000","last_became_active":"2024-04-26T21:49:42.292160+0000","last_became_peered":"2024-04-26T21:49:42.292160+0000","last_unstale":"2024-04-26T21:51:50.542214+0000","last_undegraded":"2024-04-26T21:51:50.542214+0000","last_fullsized":"2024-04-26T21:51:50.542214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_clean_scrub_stamp":"2024-04-26T21:46:13.787665+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:24.663652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543539+0000","last_change":"2024-04-26T21:50:57.145641+0000","last_active":"2024-04-26T21:51:50.543539+0000","last_peered":"2024-04-26T21:51:50.543539+0000","last_clean":"2024-04-26T21:51:50.543539+0000","last_became_active":"2024-04-26T21:50:57.145304+0000","last_became_peered":"2024-04-26T21:50:57.145304+0000","last_unstale":"2024-04-26T21:51:50.543539+0000","last_undegraded":"2024-04-26T21:51:50.543539+0000","last_fullsized":"2024-04-26T21:51:50.543539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:11.389728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.549862+0000","last_change":"2024-04-26T21:51:50.549862+0000","last_active":"2024-04-26T21:51:50.549862+0000","last_peered":"2024-04-26T21:51:50.549862+0000","last_clean":"2024-04-26T21:51:50.549862+0000","last_became_active":"2024-04-26T21:51:50.546054+0000","last_became_peered":"2024-04-26T21:51:50.546054+0000","last_unstale":"2024-04-26T21:51:50.549862+0000","last_undegraded":"2024-04-26T21:51:50.549862+0000","last_fullsized":"2024-04-26T21:51:50.549862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:33.817688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539705+0000","last_change":"2024-04-26T21:50:56.364074+0000","last_active":"2024-04-26T21:51:50.539705+0000","last_peered":"2024-04-26T21:51:50.539705+0000","last_clean":"2024-04-26T21:51:50.539705+0000","last_became_active":"2024-04-26T21:50:56.363688+0000","last_became_peered":"2024-04-26T21:50:56.363688+0000","last_unstale":"2024-04-26T21:51:50.539705+0000","last_undegraded":"2024-04-26T21:51:50.539705+0000","last_fullsized":"2024-04-26T21:51:50.539705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:35.526125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544008+0000","last_change":"2024-04-26T21:50:57.136811+0000","last_active":"2024-04-26T21:51:50.544008+0000","last_peered":"2024-04-26T21:51:50.544008+0000","last_clean":"2024-04-26T21:51:50.544008+0000","last_became_active":"2024-04-26T21:50:57.136252+0000","last_became_peered":"2024-04-26T21:50:57.136252+0000","last_unstale":"2024-04-26T21:51:50.544008+0000","last_undegraded":"2024-04-26T21:51:50.544008+0000","last_fullsized":"2024-04-26T21:51:50.544008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:01.061283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.540898+0000","last_change":"2024-04-26T21:50:56.364317+0000","last_active":"2024-04-26T21:51:50.540898+0000","last_peered":"2024-04-26T21:51:50.540898+0000","last_clean":"2024-04-26T21:51:50.540898+0000","last_became_active":"2024-04-26T21:50:56.363882+0000","last_became_peered":"2024-04-26T21:50:56.363882+0000","last_unstale":"2024-04-26T21:51:50.540898+0000","last_undegraded":"2024-04-26T21:51:50.540898+0000","last_fullsized":"2024-04-26T21:51:50.540898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:12.994293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594575+0000","last_change":"2024-04-26T21:50:57.144856+0000","last_active":"2024-04-26T21:51:50.594575+0000","last_peered":"2024-04-26T21:51:50.594575+0000","last_clean":"2024-04-26T21:51:50.594575+0000","last_became_active":"2024-04-26T21:50:57.143717+0000","last_became_peered":"2024-04-26T21:50:57.143717+0000","last_unstale":"2024-04-26T21:51:50.594575+0000","last_undegraded":"2024-04-26T21:51:50.594575+0000","last_fullsized":"2024-04-26T21:51:50.594575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:53.049243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543543+0000","last_change":"2024-04-26T21:50:56.370284+0000","last_active":"2024-04-26T21:51:50.543543+0000","last_peered":"2024-04-26T21:51:50.543543+0000","last_clean":"2024-04-26T21:51:50.543543+0000","last_became_active":"2024-04-26T21:50:56.369738+0000","last_became_peered":"2024-04-26T21:50:56.369738+0000","last_unstale":"2024-04-26T21:51:50.543543+0000","last_undegraded":"2024-04-26T21:51:50.543543+0000","last_fullsized":"2024-04-26T21:51:50.543543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:23.390553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539402+0000","last_change":"2024-04-26T21:50:57.142687+0000","last_active":"2024-04-26T21:51:50.539402+0000","last_peered":"2024-04-26T21:51:50.539402+0000","last_clean":"2024-04-26T21:51:50.539402+0000","last_became_active":"2024-04-26T21:50:57.142365+0000","last_became_peered":"2024-04-26T21:50:57.142365+0000","last_unstale":"2024-04-26T21:51:50.539402+0000","last_undegraded":"2024-04-26T21:51:50.539402+0000","last_fullsized":"2024-04-26T21:51:50.539402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:14.261741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937372+0000","last_change":"2024-04-26T21:50:56.364163+0000","last_active":"2024-04-26T21:51:50.937372+0000","last_peered":"2024-04-26T21:51:50.937372+0000","last_clean":"2024-04-26T21:51:50.937372+0000","last_became_active":"2024-04-26T21:50:56.363729+0000","last_became_peered":"2024-04-26T21:50:56.363729+0000","last_unstale":"2024-04-26T21:51:50.937372+0000","last_undegraded":"2024-04-26T21:51:50.937372+0000","last_fullsized":"2024-04-26T21:51:50.937372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:54.620884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539129+0000","last_change":"2024-04-26T21:50:57.142692+0000","last_active":"2024-04-26T21:51:50.539129+0000","last_peered":"2024-04-26T21:51:50.539129+0000","last_clean":"2024-04-26T21:51:50.539129+0000","last_became_active":"2024-04-26T21:50:57.142137+0000","last_became_peered":"2024-04-26T21:50:57.142137+0000","last_unstale":"2024-04-26T21:51:50.539129+0000","last_undegraded":"2024-04-26T21:51:50.539129+0000","last_fullsized":"2024-04-26T21:51:50.539129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.996485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594135+0000","last_change":"2024-04-26T21:50:56.366374+0000","last_active":"2024-04-26T21:51:50.594135+0000","last_peered":"2024-04-26T21:51:50.594135+0000","last_clean":"2024-04-26T21:51:50.594135+0000","last_became_active":"2024-04-26T21:50:56.366014+0000","last_became_peered":"2024-04-26T21:50:56.366014+0000","last_unstale":"2024-04-26T21:51:50.594135+0000","last_undegraded":"2024-04-26T21:51:50.594135+0000","last_fullsized":"2024-04-26T21:51:50.594135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:41.867792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543744+0000","last_change":"2024-04-26T21:50:57.142833+0000","last_active":"2024-04-26T21:51:50.543744+0000","last_peered":"2024-04-26T21:51:50.543744+0000","last_clean":"2024-04-26T21:51:50.543744+0000","last_became_active":"2024-04-26T21:50:57.142061+0000","last_became_peered":"2024-04-26T21:50:57.142061+0000","last_unstale":"2024-04-26T21:51:50.543744+0000","last_undegraded":"2024-04-26T21:51:50.543744+0000","last_fullsized":"2024-04-26T21:51:50.543744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:33.023159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541867+0000","last_change":"2024-04-26T21:50:56.363172+0000","last_active":"2024-04-26T21:51:50.541867+0000","last_peered":"2024-04-26T21:51:50.541867+0000","last_clean":"2024-04-26T21:51:50.541867+0000","last_became_active":"2024-04-26T21:50:56.362782+0000","last_became_peered":"2024-04-26T21:50:56.362782+0000","last_unstale":"2024-04-26T21:51:50.541867+0000","last_undegraded":"2024-04-26T21:51:50.541867+0000","last_fullsized":"2024-04-26T21:51:50.541867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:37.183497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594433+0000","last_change":"2024-04-26T21:50:57.142088+0000","last_active":"2024-04-26T21:51:50.594433+0000","last_peered":"2024-04-26T21:51:50.594433+0000","last_clean":"2024-04-26T21:51:50.594433+0000","last_became_active":"2024-04-26T21:50:57.141685+0000","last_became_peered":"2024-04-26T21:50:57.141685+0000","last_unstale":"2024-04-26T21:51:50.594433+0000","last_undegraded":"2024-04-26T21:51:50.594433+0000","last_fullsized":"2024-04-26T21:51:50.594433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:24.033787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539417+0000","last_change":"2024-04-26T21:50:56.371717+0000","last_active":"2024-04-26T21:51:50.539417+0000","last_peered":"2024-04-26T21:51:50.539417+0000","last_clean":"2024-04-26T21:51:50.539417+0000","last_became_active":"2024-04-26T21:50:56.370930+0000","last_became_peered":"2024-04-26T21:50:56.370930+0000","last_unstale":"2024-04-26T21:51:50.539417+0000","last_undegraded":"2024-04-26T21:51:50.539417+0000","last_fullsized":"2024-04-26T21:51:50.539417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.977392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541695+0000","last_change":"2024-04-26T21:50:57.139671+0000","last_active":"2024-04-26T21:51:50.541695+0000","last_peered":"2024-04-26T21:51:50.541695+0000","last_clean":"2024-04-26T21:51:50.541695+0000","last_became_active":"2024-04-26T21:50:57.139245+0000","last_became_peered":"2024-04-26T21:50:57.139245+0000","last_unstale":"2024-04-26T21:51:50.541695+0000","last_undegraded":"2024-04-26T21:51:50.541695+0000","last_fullsized":"2024-04-26T21:51:50.541695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:26.903905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541758+0000","last_change":"2024-04-26T21:50:56.369836+0000","last_active":"2024-04-26T21:51:50.541758+0000","last_peered":"2024-04-26T21:51:50.541758+0000","last_clean":"2024-04-26T21:51:50.541758+0000","last_became_active":"2024-04-26T21:50:56.369522+0000","last_became_peered":"2024-04-26T21:50:56.369522+0000","last_unstale":"2024-04-26T21:51:50.541758+0000","last_undegraded":"2024-04-26T21:51:50.541758+0000","last_fullsized":"2024-04-26T21:51:50.541758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:40.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543740+0000","last_change":"2024-04-26T21:50:57.138551+0000","last_active":"2024-04-26T21:51:50.543740+0000","last_peered":"2024-04-26T21:51:50.543740+0000","last_clean":"2024-04-26T21:51:50.543740+0000","last_became_active":"2024-04-26T21:50:57.138187+0000","last_became_peered":"2024-04-26T21:50:57.138187+0000","last_unstale":"2024-04-26T21:51:50.543740+0000","last_undegraded":"2024-04-26T21:51:50.543740+0000","last_fullsized":"2024-04-26T21:51:50.543740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:46.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543990+0000","last_change":"2024-04-26T21:50:56.362009+0000","last_active":"2024-04-26T21:51:50.543990+0000","last_peered":"2024-04-26T21:51:50.543990+0000","last_clean":"2024-04-26T21:51:50.543990+0000","last_became_active":"2024-04-26T21:50:56.361646+0000","last_became_peered":"2024-04-26T21:50:56.361646+0000","last_unstale":"2024-04-26T21:51:50.543990+0000","last_undegraded":"2024-04-26T21:51:50.543990+0000","last_fullsized":"2024-04-26T21:51:50.543990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:25.316195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543113+0000","last_change":"2024-04-26T21:50:57.139983+0000","last_active":"2024-04-26T21:51:50.543113+0000","last_peered":"2024-04-26T21:51:50.543113+0000","last_clean":"2024-04-26T21:51:50.543113+0000","last_became_active":"2024-04-26T21:50:57.138426+0000","last_became_peered":"2024-04-26T21:50:57.138426+0000","last_unstale":"2024-04-26T21:51:50.543113+0000","last_undegraded":"2024-04-26T21:51:50.543113+0000","last_fullsized":"2024-04-26T21:51:50.543113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:53.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544138+0000","last_change":"2024-04-26T21:50:56.369527+0000","last_active":"2024-04-26T21:51:50.544138+0000","last_peered":"2024-04-26T21:51:50.544138+0000","last_clean":"2024-04-26T21:51:50.544138+0000","last_became_active":"2024-04-26T21:50:56.369168+0000","last_became_peered":"2024-04-26T21:50:56.369168+0000","last_unstale":"2024-04-26T21:51:50.544138+0000","last_undegraded":"2024-04-26T21:51:50.544138+0000","last_fullsized":"2024-04-26T21:51:50.544138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:38.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542958+0000","last_change":"2024-04-26T21:50:57.137957+0000","last_active":"2024-04-26T21:51:50.542958+0000","last_peered":"2024-04-26T21:51:50.542958+0000","last_clean":"2024-04-26T21:51:50.542958+0000","last_became_active":"2024-04-26T21:50:57.137595+0000","last_became_peered":"2024-04-26T21:50:57.137595+0000","last_unstale":"2024-04-26T21:51:50.542958+0000","last_undegraded":"2024-04-26T21:51:50.542958+0000","last_fullsized":"2024-04-26T21:51:50.542958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:09.355937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543859+0000","last_change":"2024-04-26T21:50:57.143098+0000","last_active":"2024-04-26T21:51:50.543859+0000","last_peered":"2024-04-26T21:51:50.543859+0000","last_clean":"2024-04-26T21:51:50.543859+0000","last_became_active":"2024-04-26T21:50:57.142524+0000","last_became_peered":"2024-04-26T21:50:57.142524+0000","last_unstale":"2024-04-26T21:51:50.543859+0000","last_undegraded":"2024-04-26T21:51:50.543859+0000","last_fullsized":"2024-04-26T21:51:50.543859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:33.333382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541844+0000","last_change":"2024-04-26T21:50:57.143123+0000","last_active":"2024-04-26T21:51:50.541844+0000","last_peered":"2024-04-26T21:51:50.541844+0000","last_clean":"2024-04-26T21:51:50.541844+0000","last_became_active":"2024-04-26T21:50:57.142805+0000","last_became_peered":"2024-04-26T21:50:57.142805+0000","last_unstale":"2024-04-26T21:51:50.541844+0000","last_undegraded":"2024-04-26T21:51:50.541844+0000","last_fullsized":"2024-04-26T21:51:50.541844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:19.330202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541368+0000","last_change":"2024-04-26T21:50:57.140268+0000","last_active":"2024-04-26T21:51:50.541368+0000","last_peered":"2024-04-26T21:51:50.541368+0000","last_clean":"2024-04-26T21:51:50.541368+0000","last_became_active":"2024-04-26T21:50:57.139927+0000","last_became_peered":"2024-04-26T21:50:57.139927+0000","last_unstale":"2024-04-26T21:51:50.541368+0000","last_undegraded":"2024-04-26T21:51:50.541368+0000","last_fullsized":"2024-04-26T21:51:50.541368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:01.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539146+0000","last_change":"2024-04-26T21:50:57.143564+0000","last_active":"2024-04-26T21:51:50.539146+0000","last_peered":"2024-04-26T21:51:50.539146+0000","last_clean":"2024-04-26T21:51:50.539146+0000","last_became_active":"2024-04-26T21:50:57.142740+0000","last_became_peered":"2024-04-26T21:50:57.142740+0000","last_unstale":"2024-04-26T21:51:50.539146+0000","last_undegraded":"2024-04-26T21:51:50.539146+0000","last_fullsized":"2024-04-26T21:51:50.539146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:58.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544034+0000","last_change":"2024-04-26T21:50:57.146883+0000","last_active":"2024-04-26T21:51:50.544034+0000","last_peered":"2024-04-26T21:51:50.544034+0000","last_clean":"2024-04-26T21:51:50.544034+0000","last_became_active":"2024-04-26T21:50:57.146540+0000","last_became_peered":"2024-04-26T21:50:57.146540+0000","last_unstale":"2024-04-26T21:51:50.544034+0000","last_undegraded":"2024-04-26T21:51:50.544034+0000","last_fullsized":"2024-04-26T21:51:50.544034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.787618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545677+0000","last_change":"2024-04-26T21:50:57.136938+0000","last_active":"2024-04-26T21:51:50.545677+0000","last_peered":"2024-04-26T21:51:50.545677+0000","last_clean":"2024-04-26T21:51:50.545677+0000","last_became_active":"2024-04-26T21:50:57.136474+0000","last_became_peered":"2024-04-26T21:50:57.136474+0000","last_unstale":"2024-04-26T21:51:50.545677+0000","last_undegraded":"2024-04-26T21:51:50.545677+0000","last_fullsized":"2024-04-26T21:51:50.545677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:42.534342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542048+0000","last_change":"2024-04-26T21:50:57.150958+0000","last_active":"2024-04-26T21:51:50.542048+0000","last_peered":"2024-04-26T21:51:50.542048+0000","last_clean":"2024-04-26T21:51:50.542048+0000","last_became_active":"2024-04-26T21:50:57.150593+0000","last_became_peered":"2024-04-26T21:50:57.150593+0000","last_unstale":"2024-04-26T21:51:50.542048+0000","last_undegraded":"2024-04-26T21:51:50.542048+0000","last_fullsized":"2024-04-26T21:51:50.542048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:26.583983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594463+0000","last_change":"2024-04-26T21:50:57.147766+0000","last_active":"2024-04-26T21:51:50.594463+0000","last_peered":"2024-04-26T21:51:50.594463+0000","last_clean":"2024-04-26T21:51:50.594463+0000","last_became_active":"2024-04-26T21:50:57.147442+0000","last_became_peered":"2024-04-26T21:50:57.147442+0000","last_unstale":"2024-04-26T21:51:50.594463+0000","last_undegraded":"2024-04-26T21:51:50.594463+0000","last_fullsized":"2024-04-26T21:51:50.594463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.489117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.595948+0000","last_change":"2024-04-26T21:51:50.595554+0000","last_active":"2024-04-26T21:51:50.595948+0000","last_peered":"2024-04-26T21:51:50.595948+0000","last_clean":"2024-04-26T21:51:50.595948+0000","last_became_active":"2024-04-26T21:51:50.595248+0000","last_became_peered":"2024-04-26T21:51:50.595248+0000","last_unstale":"2024-04-26T21:51:50.595948+0000","last_undegraded":"2024-04-26T21:51:50.595948+0000","last_fullsized":"2024-04-26T21:51:50.595948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:36.853146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594067+0000","last_change":"2024-04-26T21:50:57.142804+0000","last_active":"2024-04-26T21:51:50.594067+0000","last_peered":"2024-04-26T21:51:50.594067+0000","last_clean":"2024-04-26T21:51:50.594067+0000","last_became_active":"2024-04-26T21:50:57.141902+0000","last_became_peered":"2024-04-26T21:50:57.141902+0000","last_unstale":"2024-04-26T21:51:50.594067+0000","last_undegraded":"2024-04-26T21:51:50.594067+0000","last_fullsized":"2024-04-26T21:51:50.594067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:37.590730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937945+0000","last_change":"2024-04-26T21:50:57.138637+0000","last_active":"2024-04-26T21:51:50.937945+0000","last_peered":"2024-04-26T21:51:50.937945+0000","last_clean":"2024-04-26T21:51:50.937945+0000","last_became_active":"2024-04-26T21:50:57.138295+0000","last_became_peered":"2024-04-26T21:50:57.138295+0000","last_unstale":"2024-04-26T21:51:50.937945+0000","last_undegraded":"2024-04-26T21:51:50.937945+0000","last_fullsized":"2024-04-26T21:51:50.937945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:19.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543859+0000","last_change":"2024-04-26T21:50:57.137377+0000","last_active":"2024-04-26T21:51:50.543859+0000","last_peered":"2024-04-26T21:51:50.543859+0000","last_clean":"2024-04-26T21:51:50.543859+0000","last_became_active":"2024-04-26T21:50:57.136893+0000","last_became_peered":"2024-04-26T21:50:57.136893+0000","last_unstale":"2024-04-26T21:51:50.543859+0000","last_undegraded":"2024-04-26T21:51:50.543859+0000","last_fullsized":"2024-04-26T21:51:50.543859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:44.257438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545214+0000","last_change":"2024-04-26T21:50:57.141439+0000","last_active":"2024-04-26T21:51:50.545214+0000","last_peered":"2024-04-26T21:51:50.545214+0000","last_clean":"2024-04-26T21:51:50.545214+0000","last_became_active":"2024-04-26T21:50:57.141060+0000","last_became_peered":"2024-04-26T21:50:57.141060+0000","last_unstale":"2024-04-26T21:51:50.545214+0000","last_undegraded":"2024-04-26T21:51:50.545214+0000","last_fullsized":"2024-04-26T21:51:50.545214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:27.174348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250381,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53478,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":383,"num_read_kb":274,"num_write":153,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":160434,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":457001,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":1097728,"data_compressed":44071,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456615,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456535,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1048465,"data_compressed":42217,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":456004,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":505274,"data_compressed":7602,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":457275,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":506512,"data_compressed":7602,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":456315,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456944,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1047136,"data_compressed":42217,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":49477,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":50715,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51446,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:52:13.945 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T21:52:14.218 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:14.467 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:14.468 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:14.468 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:14.468 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:14.468 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:14.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:14.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:14.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:14.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:14.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:14.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:14.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:14.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:14.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:14.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:52:15.406 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:15 smithi055 ceph-mon[29221]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:15.406 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:15 smithi055 ceph-mon[29221]: pgmap v329: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:15 smithi064 ceph-mon[32765]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:15 smithi064 ceph-mon[32765]: pgmap v329: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:15.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:15 smithi165 ceph-mon[33296]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:15.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:15 smithi165 ceph-mon[33296]: pgmap v329: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/2847300777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 msgr2=0x7f0bc81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/2847300777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 0x7f0bc81027a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009990 tx=0x7f0bbc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:52:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/2847300777 shutdown_connections 2024-04-26T21:52:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/2847300777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bc8102ce0 0x7f0bc810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/2847300777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bc8108390 0x7f0bc8108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/2847300777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 0x7f0bc81027a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/2847300777 >> 172.21.15.55:0/2847300777 conn(0x7f0bc80792d0 msgr2=0x7f0bc80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/2847300777 shutdown_connections 2024-04-26T21:52:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.951+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/2847300777 wait complete. 2024-04-26T21:52:15.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.952+0000 7f0bcc8b1700 1 Processor -- start 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.952+0000 7f0bcc8b1700 1 -- start start 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bcc8b1700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 0x7f0bc8104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bcc8b1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bc8102ce0 0x7f0bc8104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bcc8b1700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bc8108390 0x7f0bc8104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bcc8b1700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81124b0 con 0x7f0bc8102320 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bcc8b1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8112330 con 0x7f0bc8102ce0 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bcc8b1700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8112630 con 0x7f0bc8108390 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 0x7f0bc8104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 0x7f0bc8104000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47878/0 (socket says 172.21.15.55:47878) 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 -- 172.21.15.55:0/3655051968 learned_addr learned my addr 172.21.15.55:0/3655051968 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc5d9b700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bc8102ce0 0x7f0bc8104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc6d9d700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bc8108390 0x7f0bc8104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 -- 172.21.15.55:0/3655051968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bc8108390 msgr2=0x7f0bc8104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:15.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bc8108390 0x7f0bc8104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 -- 172.21.15.55:0/3655051968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bc8102ce0 msgr2=0x7f0bc8104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bc8102ce0 0x7f0bc8104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 -- 172.21.15.55:0/3655051968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc81a7f80 con 0x7f0bc8102320 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc5d9b700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bc8102ce0 0x7f0bc8104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.953+0000 7f0bc659c700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 0x7f0bc8104000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800d950 tx=0x7f0bb800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.954+0000 7f0bb77fe700 1 -- 172.21.15.55:0/3655051968 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8016070 con 0x7f0bc8102320 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.954+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc81a8270 con 0x7f0bc8102320 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.954+0000 7f0bb77fe700 1 -- 172.21.15.55:0/3655051968 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb8010460 con 0x7f0bc8102320 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.954+0000 7f0bb77fe700 1 -- 172.21.15.55:0/3655051968 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800f620 con 0x7f0bc8102320 2024-04-26T21:52:15.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.954+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc81a8800 con 0x7f0bc8102320 2024-04-26T21:52:15.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.955+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc806a110 con 0x7f0bc8102320 2024-04-26T21:52:15.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.956+0000 7f0bb77fe700 1 -- 172.21.15.55:0/3655051968 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0bb8019070 con 0x7f0bc8102320 2024-04-26T21:52:15.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.956+0000 7f0bb77fe700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0bb00a3d50 0x7f0bb00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:15.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.956+0000 7f0bc5d9b700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0bb00a3d50 0x7f0bb00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:15.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.957+0000 7f0bb77fe700 1 -- 172.21.15.55:0/3655051968 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0bb80d04e0 con 0x7f0bc8102320 2024-04-26T21:52:15.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.957+0000 7f0bc5d9b700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0bb00a3d50 0x7f0bb00a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:15.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:15.961+0000 7f0bb77fe700 1 -- 172.21.15.55:0/3655051968 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0bb809cd40 con 0x7f0bc8102320 2024-04-26T21:52:16.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.162+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bc8106120 con 0x7f0bb00a3d50 2024-04-26T21:52:16.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.169+0000 7f0bb77fe700 1 -- 172.21.15.55:0/3655051968 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258987 (secure 0 0 0) 0x7f0bc8106120 con 0x7f0bb00a3d50 2024-04-26T21:52:16.173 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0bb00a3d50 msgr2=0x7f0bb00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0bb00a3d50 0x7f0bb00a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc01a040 comp rx=0 tx=0).stop 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 msgr2=0x7f0bc8104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 0x7f0bc8104000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800d950 tx=0x7f0bb800dd10 comp rx=0 tx=0).stop 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 shutdown_connections 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bc8108390 0x7f0bc8104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bc8102320 0x7f0bc8104000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0bb00a3d50 0x7f0bb00a6210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 --2- 172.21.15.55:0/3655051968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bc8102ce0 0x7f0bc8104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:16.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 >> 172.21.15.55:0/3655051968 conn(0x7f0bc80792d0 msgr2=0x7f0bc8101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:16.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 shutdown_connections 2024-04-26T21:52:16.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:16.173+0000 7f0bcc8b1700 1 -- 172.21.15.55:0/3655051968 wait complete. 2024-04-26T21:52:16.176 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-04-26T21:52:16.894 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-04-26T21:52:15.361401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644080,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":534,"num_read_kb":393,"num_write":336,"num_write_kb":3035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352416,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589408,"statfs":{"total":1151940427776,"available":1151579553792,"internally_reserved":0,"allocated":11796480,"data_stored":7401804,"data_compressed":183945,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003252"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545722+0000","last_change":"2024-04-26T21:50:57.139486+0000","last_active":"2024-04-26T21:51:50.545722+0000","last_peered":"2024-04-26T21:51:50.545722+0000","last_clean":"2024-04-26T21:51:50.545722+0000","last_became_active":"2024-04-26T21:50:57.139133+0000","last_became_peered":"2024-04-26T21:50:57.139133+0000","last_unstale":"2024-04-26T21:51:50.545722+0000","last_undegraded":"2024-04-26T21:51:50.545722+0000","last_fullsized":"2024-04-26T21:51:50.545722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:33.211627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539345+0000","last_change":"2024-04-26T21:50:57.138519+0000","last_active":"2024-04-26T21:51:50.539345+0000","last_peered":"2024-04-26T21:51:50.539345+0000","last_clean":"2024-04-26T21:51:50.539345+0000","last_became_active":"2024-04-26T21:50:57.138131+0000","last_became_peered":"2024-04-26T21:50:57.138131+0000","last_unstale":"2024-04-26T21:51:50.539345+0000","last_undegraded":"2024-04-26T21:51:50.539345+0000","last_fullsized":"2024-04-26T21:51:50.539345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:34.349694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.538994+0000","last_change":"2024-04-26T21:50:57.146584+0000","last_active":"2024-04-26T21:51:50.538994+0000","last_peered":"2024-04-26T21:51:50.538994+0000","last_clean":"2024-04-26T21:51:50.538994+0000","last_became_active":"2024-04-26T21:50:57.146262+0000","last_became_peered":"2024-04-26T21:50:57.146262+0000","last_unstale":"2024-04-26T21:51:50.538994+0000","last_undegraded":"2024-04-26T21:51:50.538994+0000","last_fullsized":"2024-04-26T21:51:50.538994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:49.946882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.548238+0000","last_change":"2024-04-26T21:51:50.546008+0000","last_active":"2024-04-26T21:51:50.548238+0000","last_peered":"2024-04-26T21:51:50.548238+0000","last_clean":"2024-04-26T21:51:50.548238+0000","last_became_active":"2024-04-26T21:51:50.545739+0000","last_became_peered":"2024-04-26T21:51:50.545739+0000","last_unstale":"2024-04-26T21:51:50.548238+0000","last_undegraded":"2024-04-26T21:51:50.548238+0000","last_fullsized":"2024-04-26T21:51:50.548238+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.494950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539285+0000","last_change":"2024-04-26T21:50:57.136292+0000","last_active":"2024-04-26T21:51:50.539285+0000","last_peered":"2024-04-26T21:51:50.539285+0000","last_clean":"2024-04-26T21:51:50.539285+0000","last_became_active":"2024-04-26T21:50:57.135936+0000","last_became_peered":"2024-04-26T21:50:57.135936+0000","last_unstale":"2024-04-26T21:51:50.539285+0000","last_undegraded":"2024-04-26T21:51:50.539285+0000","last_fullsized":"2024-04-26T21:51:50.539285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:35.415440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544083+0000","last_change":"2024-04-26T21:50:57.147764+0000","last_active":"2024-04-26T21:51:50.544083+0000","last_peered":"2024-04-26T21:51:50.544083+0000","last_clean":"2024-04-26T21:51:50.544083+0000","last_became_active":"2024-04-26T21:50:57.147034+0000","last_became_peered":"2024-04-26T21:50:57.147034+0000","last_unstale":"2024-04-26T21:51:50.544083+0000","last_undegraded":"2024-04-26T21:51:50.544083+0000","last_fullsized":"2024-04-26T21:51:50.544083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:44.833213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541303+0000","last_change":"2024-04-26T21:50:57.144813+0000","last_active":"2024-04-26T21:51:50.541303+0000","last_peered":"2024-04-26T21:51:50.541303+0000","last_clean":"2024-04-26T21:51:50.541303+0000","last_became_active":"2024-04-26T21:50:57.144442+0000","last_became_peered":"2024-04-26T21:50:57.144442+0000","last_unstale":"2024-04-26T21:51:50.541303+0000","last_undegraded":"2024-04-26T21:51:50.541303+0000","last_fullsized":"2024-04-26T21:51:50.541303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:59.032263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.548896+0000","last_change":"2024-04-26T21:51:50.547307+0000","last_active":"2024-04-26T21:51:50.548896+0000","last_peered":"2024-04-26T21:51:50.548896+0000","last_clean":"2024-04-26T21:51:50.548896+0000","last_became_active":"2024-04-26T21:51:50.546888+0000","last_became_peered":"2024-04-26T21:51:50.546888+0000","last_unstale":"2024-04-26T21:51:50.548896+0000","last_undegraded":"2024-04-26T21:51:50.548896+0000","last_fullsized":"2024-04-26T21:51:50.548896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:36.137772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.938023+0000","last_change":"2024-04-26T21:50:57.132957+0000","last_active":"2024-04-26T21:51:50.938023+0000","last_peered":"2024-04-26T21:51:50.938023+0000","last_clean":"2024-04-26T21:51:50.938023+0000","last_became_active":"2024-04-26T21:50:57.132606+0000","last_became_peered":"2024-04-26T21:50:57.132606+0000","last_unstale":"2024-04-26T21:51:50.938023+0000","last_undegraded":"2024-04-26T21:51:50.938023+0000","last_fullsized":"2024-04-26T21:51:50.938023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:00.994370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544743+0000","last_change":"2024-04-26T21:50:57.150598+0000","last_active":"2024-04-26T21:51:50.544743+0000","last_peered":"2024-04-26T21:51:50.544743+0000","last_clean":"2024-04-26T21:51:50.544743+0000","last_became_active":"2024-04-26T21:50:57.149697+0000","last_became_peered":"2024-04-26T21:50:57.149697+0000","last_unstale":"2024-04-26T21:51:50.544743+0000","last_undegraded":"2024-04-26T21:51:50.544743+0000","last_fullsized":"2024-04-26T21:51:50.544743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.621137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542183+0000","last_change":"2024-04-26T21:50:57.147264+0000","last_active":"2024-04-26T21:51:50.542183+0000","last_peered":"2024-04-26T21:51:50.542183+0000","last_clean":"2024-04-26T21:51:50.542183+0000","last_became_active":"2024-04-26T21:50:57.146888+0000","last_became_peered":"2024-04-26T21:50:57.146888+0000","last_unstale":"2024-04-26T21:51:50.542183+0000","last_undegraded":"2024-04-26T21:51:50.542183+0000","last_fullsized":"2024-04-26T21:51:50.542183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:49.584327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544007+0000","last_change":"2024-04-26T21:50:57.142005+0000","last_active":"2024-04-26T21:51:50.544007+0000","last_peered":"2024-04-26T21:51:50.544007+0000","last_clean":"2024-04-26T21:51:50.544007+0000","last_became_active":"2024-04-26T21:50:57.141605+0000","last_became_peered":"2024-04-26T21:50:57.141605+0000","last_unstale":"2024-04-26T21:51:50.544007+0000","last_undegraded":"2024-04-26T21:51:50.544007+0000","last_fullsized":"2024-04-26T21:51:50.544007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:43.485295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543887+0000","last_change":"2024-04-26T21:50:57.138049+0000","last_active":"2024-04-26T21:51:50.543887+0000","last_peered":"2024-04-26T21:51:50.543887+0000","last_clean":"2024-04-26T21:51:50.543887+0000","last_became_active":"2024-04-26T21:50:57.137717+0000","last_became_peered":"2024-04-26T21:50:57.137717+0000","last_unstale":"2024-04-26T21:51:50.543887+0000","last_undegraded":"2024-04-26T21:51:50.543887+0000","last_fullsized":"2024-04-26T21:51:50.543887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:28.918224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.538938+0000","last_change":"2024-04-26T21:50:57.139183+0000","last_active":"2024-04-26T21:51:50.538938+0000","last_peered":"2024-04-26T21:51:50.538938+0000","last_clean":"2024-04-26T21:51:50.538938+0000","last_became_active":"2024-04-26T21:50:57.138759+0000","last_became_peered":"2024-04-26T21:50:57.138759+0000","last_unstale":"2024-04-26T21:51:50.538938+0000","last_undegraded":"2024-04-26T21:51:50.538938+0000","last_fullsized":"2024-04-26T21:51:50.538938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:12.003122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.538904+0000","last_change":"2024-04-26T21:50:57.143110+0000","last_active":"2024-04-26T21:51:50.538904+0000","last_peered":"2024-04-26T21:51:50.538904+0000","last_clean":"2024-04-26T21:51:50.538904+0000","last_became_active":"2024-04-26T21:50:57.142754+0000","last_became_peered":"2024-04-26T21:50:57.142754+0000","last_unstale":"2024-04-26T21:51:50.538904+0000","last_undegraded":"2024-04-26T21:51:50.538904+0000","last_fullsized":"2024-04-26T21:51:50.538904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:08.321000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937309+0000","last_change":"2024-04-26T21:50:57.146514+0000","last_active":"2024-04-26T21:51:50.937309+0000","last_peered":"2024-04-26T21:51:50.937309+0000","last_clean":"2024-04-26T21:51:50.937309+0000","last_became_active":"2024-04-26T21:50:57.146046+0000","last_became_peered":"2024-04-26T21:50:57.146046+0000","last_unstale":"2024-04-26T21:51:50.937309+0000","last_undegraded":"2024-04-26T21:51:50.937309+0000","last_fullsized":"2024-04-26T21:51:50.937309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:21.130220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545839+0000","last_change":"2024-04-26T21:50:57.140492+0000","last_active":"2024-04-26T21:51:50.545839+0000","last_peered":"2024-04-26T21:51:50.545839+0000","last_clean":"2024-04-26T21:51:50.545839+0000","last_became_active":"2024-04-26T21:50:57.140147+0000","last_became_peered":"2024-04-26T21:50:57.140147+0000","last_unstale":"2024-04-26T21:51:50.545839+0000","last_undegraded":"2024-04-26T21:51:50.545839+0000","last_fullsized":"2024-04-26T21:51:50.545839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:21.679202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.540908+0000","last_change":"2024-04-26T21:50:57.140992+0000","last_active":"2024-04-26T21:51:50.540908+0000","last_peered":"2024-04-26T21:51:50.540908+0000","last_clean":"2024-04-26T21:51:50.540908+0000","last_became_active":"2024-04-26T21:50:57.140644+0000","last_became_peered":"2024-04-26T21:50:57.140644+0000","last_unstale":"2024-04-26T21:51:50.540908+0000","last_undegraded":"2024-04-26T21:51:50.540908+0000","last_fullsized":"2024-04-26T21:51:50.540908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541570+0000","last_change":"2024-04-26T21:50:57.141242+0000","last_active":"2024-04-26T21:51:50.541570+0000","last_peered":"2024-04-26T21:51:50.541570+0000","last_clean":"2024-04-26T21:51:50.541570+0000","last_became_active":"2024-04-26T21:50:57.140909+0000","last_became_peered":"2024-04-26T21:50:57.140909+0000","last_unstale":"2024-04-26T21:51:50.541570+0000","last_undegraded":"2024-04-26T21:51:50.541570+0000","last_fullsized":"2024-04-26T21:51:50.541570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:21.996984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541007+0000","last_change":"2024-04-26T21:50:56.371876+0000","last_active":"2024-04-26T21:51:50.541007+0000","last_peered":"2024-04-26T21:51:50.541007+0000","last_clean":"2024-04-26T21:51:50.541007+0000","last_became_active":"2024-04-26T21:50:56.371509+0000","last_became_peered":"2024-04-26T21:50:56.371509+0000","last_unstale":"2024-04-26T21:51:50.541007+0000","last_undegraded":"2024-04-26T21:51:50.541007+0000","last_fullsized":"2024-04-26T21:51:50.541007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:50.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594536+0000","last_change":"2024-04-26T21:50:57.145654+0000","last_active":"2024-04-26T21:51:50.594536+0000","last_peered":"2024-04-26T21:51:50.594536+0000","last_clean":"2024-04-26T21:51:50.594536+0000","last_became_active":"2024-04-26T21:50:57.145355+0000","last_became_peered":"2024-04-26T21:50:57.145355+0000","last_unstale":"2024-04-26T21:51:50.594536+0000","last_undegraded":"2024-04-26T21:51:50.594536+0000","last_fullsized":"2024-04-26T21:51:50.594536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:51.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594661+0000","last_change":"2024-04-26T21:50:56.362758+0000","last_active":"2024-04-26T21:51:50.594661+0000","last_peered":"2024-04-26T21:51:50.594661+0000","last_clean":"2024-04-26T21:51:50.594661+0000","last_became_active":"2024-04-26T21:50:56.361338+0000","last_became_peered":"2024-04-26T21:50:56.361338+0000","last_unstale":"2024-04-26T21:51:50.594661+0000","last_undegraded":"2024-04-26T21:51:50.594661+0000","last_fullsized":"2024-04-26T21:51:50.594661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:44.603426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539534+0000","last_change":"2024-04-26T21:50:57.150701+0000","last_active":"2024-04-26T21:51:50.539534+0000","last_peered":"2024-04-26T21:51:50.539534+0000","last_clean":"2024-04-26T21:51:50.539534+0000","last_became_active":"2024-04-26T21:50:57.150388+0000","last_became_peered":"2024-04-26T21:50:57.150388+0000","last_unstale":"2024-04-26T21:51:50.539534+0000","last_undegraded":"2024-04-26T21:51:50.539534+0000","last_fullsized":"2024-04-26T21:51:50.539534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:43:07.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539956+0000","last_change":"2024-04-26T21:50:56.371379+0000","last_active":"2024-04-26T21:51:50.539956+0000","last_peered":"2024-04-26T21:51:50.539956+0000","last_clean":"2024-04-26T21:51:50.539956+0000","last_became_active":"2024-04-26T21:50:56.371025+0000","last_became_peered":"2024-04-26T21:50:56.371025+0000","last_unstale":"2024-04-26T21:51:50.539956+0000","last_undegraded":"2024-04-26T21:51:50.539956+0000","last_fullsized":"2024-04-26T21:51:50.539956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:00.633393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937662+0000","last_change":"2024-04-26T21:50:57.144777+0000","last_active":"2024-04-26T21:51:50.937662+0000","last_peered":"2024-04-26T21:51:50.937662+0000","last_clean":"2024-04-26T21:51:50.937662+0000","last_became_active":"2024-04-26T21:50:57.144431+0000","last_became_peered":"2024-04-26T21:50:57.144431+0000","last_unstale":"2024-04-26T21:51:50.937662+0000","last_undegraded":"2024-04-26T21:51:50.937662+0000","last_fullsized":"2024-04-26T21:51:50.937662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:42.164240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539612+0000","last_change":"2024-04-26T21:50:56.360843+0000","last_active":"2024-04-26T21:51:50.539612+0000","last_peered":"2024-04-26T21:51:50.539612+0000","last_clean":"2024-04-26T21:51:50.539612+0000","last_became_active":"2024-04-26T21:50:56.360440+0000","last_became_peered":"2024-04-26T21:50:56.360440+0000","last_unstale":"2024-04-26T21:51:50.539612+0000","last_undegraded":"2024-04-26T21:51:50.539612+0000","last_fullsized":"2024-04-26T21:51:50.539612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:47.585306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544573+0000","last_change":"2024-04-26T21:50:57.146449+0000","last_active":"2024-04-26T21:51:50.544573+0000","last_peered":"2024-04-26T21:51:50.544573+0000","last_clean":"2024-04-26T21:51:50.544573+0000","last_became_active":"2024-04-26T21:50:57.146128+0000","last_became_peered":"2024-04-26T21:50:57.146128+0000","last_unstale":"2024-04-26T21:51:50.544573+0000","last_undegraded":"2024-04-26T21:51:50.544573+0000","last_fullsized":"2024-04-26T21:51:50.544573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:09.165868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:09.574682+0000","last_change":"2024-04-26T21:51:50.551697+0000","last_active":"2024-04-26T21:52:09.574682+0000","last_peered":"2024-04-26T21:52:09.574682+0000","last_clean":"2024-04-26T21:52:09.574682+0000","last_became_active":"2024-04-26T21:51:50.546274+0000","last_became_peered":"2024-04-26T21:51:50.546274+0000","last_unstale":"2024-04-26T21:52:09.574682+0000","last_undegraded":"2024-04-26T21:52:09.574682+0000","last_fullsized":"2024-04-26T21:52:09.574682+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:44.549374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539254+0000","last_change":"2024-04-26T21:50:57.138817+0000","last_active":"2024-04-26T21:51:50.539254+0000","last_peered":"2024-04-26T21:51:50.539254+0000","last_clean":"2024-04-26T21:51:50.539254+0000","last_became_active":"2024-04-26T21:50:57.138489+0000","last_became_peered":"2024-04-26T21:50:57.138489+0000","last_unstale":"2024-04-26T21:51:50.539254+0000","last_undegraded":"2024-04-26T21:51:50.539254+0000","last_fullsized":"2024-04-26T21:51:50.539254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:56.072063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.941723+0000","last_change":"2024-04-26T21:51:50.941723+0000","last_active":"2024-04-26T21:51:50.941723+0000","last_peered":"2024-04-26T21:51:50.941723+0000","last_clean":"2024-04-26T21:51:50.941723+0000","last_became_active":"2024-04-26T21:51:50.938689+0000","last_became_peered":"2024-04-26T21:51:50.938689+0000","last_unstale":"2024-04-26T21:51:50.941723+0000","last_undegraded":"2024-04-26T21:51:50.941723+0000","last_fullsized":"2024-04-26T21:51:50.941723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:12.119245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.595771+0000","last_change":"2024-04-26T21:51:50.595771+0000","last_active":"2024-04-26T21:51:50.595771+0000","last_peered":"2024-04-26T21:51:50.595771+0000","last_clean":"2024-04-26T21:51:50.595771+0000","last_became_active":"2024-04-26T21:51:50.595441+0000","last_became_peered":"2024-04-26T21:51:50.595441+0000","last_unstale":"2024-04-26T21:51:50.595771+0000","last_undegraded":"2024-04-26T21:51:50.595771+0000","last_fullsized":"2024-04-26T21:51:50.595771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:53.753035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543667+0000","last_change":"2024-04-26T21:50:56.362272+0000","last_active":"2024-04-26T21:51:50.543667+0000","last_peered":"2024-04-26T21:51:50.543667+0000","last_clean":"2024-04-26T21:51:50.543667+0000","last_became_active":"2024-04-26T21:50:56.361941+0000","last_became_peered":"2024-04-26T21:50:56.361941+0000","last_unstale":"2024-04-26T21:51:50.543667+0000","last_undegraded":"2024-04-26T21:51:50.543667+0000","last_fullsized":"2024-04-26T21:51:50.543667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594673+0000","last_change":"2024-04-26T21:50:57.135566+0000","last_active":"2024-04-26T21:51:50.594673+0000","last_peered":"2024-04-26T21:51:50.594673+0000","last_clean":"2024-04-26T21:51:50.594673+0000","last_became_active":"2024-04-26T21:50:57.135231+0000","last_became_peered":"2024-04-26T21:50:57.135231+0000","last_unstale":"2024-04-26T21:51:50.594673+0000","last_undegraded":"2024-04-26T21:51:50.594673+0000","last_fullsized":"2024-04-26T21:51:50.594673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:09.381936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544881+0000","last_change":"2024-04-26T21:50:56.372420+0000","last_active":"2024-04-26T21:51:50.544881+0000","last_peered":"2024-04-26T21:51:50.544881+0000","last_clean":"2024-04-26T21:51:50.544881+0000","last_became_active":"2024-04-26T21:50:56.372096+0000","last_became_peered":"2024-04-26T21:50:56.372096+0000","last_unstale":"2024-04-26T21:51:50.544881+0000","last_undegraded":"2024-04-26T21:51:50.544881+0000","last_fullsized":"2024-04-26T21:51:50.544881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:03.515804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.540837+0000","last_change":"2024-04-26T21:50:57.140071+0000","last_active":"2024-04-26T21:51:50.540837+0000","last_peered":"2024-04-26T21:51:50.540837+0000","last_clean":"2024-04-26T21:51:50.540837+0000","last_became_active":"2024-04-26T21:50:57.139693+0000","last_became_peered":"2024-04-26T21:50:57.139693+0000","last_unstale":"2024-04-26T21:51:50.540837+0000","last_undegraded":"2024-04-26T21:51:50.540837+0000","last_fullsized":"2024-04-26T21:51:50.540837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:17.963966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:12.578586+0000","last_change":"2024-04-26T21:50:56.363748+0000","last_active":"2024-04-26T21:52:12.578586+0000","last_peered":"2024-04-26T21:52:12.578586+0000","last_clean":"2024-04-26T21:52:12.578586+0000","last_became_active":"2024-04-26T21:50:56.363328+0000","last_became_peered":"2024-04-26T21:50:56.363328+0000","last_unstale":"2024-04-26T21:52:12.578586+0000","last_undegraded":"2024-04-26T21:52:12.578586+0000","last_fullsized":"2024-04-26T21:52:12.578586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:24.450981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937846+0000","last_change":"2024-04-26T21:50:57.143064+0000","last_active":"2024-04-26T21:51:50.937846+0000","last_peered":"2024-04-26T21:51:50.937846+0000","last_clean":"2024-04-26T21:51:50.937846+0000","last_became_active":"2024-04-26T21:50:57.142715+0000","last_became_peered":"2024-04-26T21:50:57.142715+0000","last_unstale":"2024-04-26T21:51:50.937846+0000","last_undegraded":"2024-04-26T21:51:50.937846+0000","last_fullsized":"2024-04-26T21:51:50.937846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:35.901855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543472+0000","last_change":"2024-04-26T21:50:56.369898+0000","last_active":"2024-04-26T21:51:50.543472+0000","last_peered":"2024-04-26T21:51:50.543472+0000","last_clean":"2024-04-26T21:51:50.543472+0000","last_became_active":"2024-04-26T21:50:56.369349+0000","last_became_peered":"2024-04-26T21:50:56.369349+0000","last_unstale":"2024-04-26T21:51:50.543472+0000","last_undegraded":"2024-04-26T21:51:50.543472+0000","last_fullsized":"2024-04-26T21:51:50.543472+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:41.378193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539538+0000","last_change":"2024-04-26T21:50:57.141417+0000","last_active":"2024-04-26T21:51:50.539538+0000","last_peered":"2024-04-26T21:51:50.539538+0000","last_clean":"2024-04-26T21:51:50.539538+0000","last_became_active":"2024-04-26T21:50:57.141091+0000","last_became_peered":"2024-04-26T21:50:57.141091+0000","last_unstale":"2024-04-26T21:51:50.539538+0000","last_undegraded":"2024-04-26T21:51:50.539538+0000","last_fullsized":"2024-04-26T21:51:50.539538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:45.357827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545426+0000","last_change":"2024-04-26T21:50:56.364439+0000","last_active":"2024-04-26T21:51:50.545426+0000","last_peered":"2024-04-26T21:51:50.545426+0000","last_clean":"2024-04-26T21:51:50.545426+0000","last_became_active":"2024-04-26T21:50:56.364090+0000","last_became_peered":"2024-04-26T21:50:56.364090+0000","last_unstale":"2024-04-26T21:51:50.545426+0000","last_undegraded":"2024-04-26T21:51:50.545426+0000","last_fullsized":"2024-04-26T21:51:50.545426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:43.562189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937656+0000","last_change":"2024-04-26T21:50:57.145976+0000","last_active":"2024-04-26T21:51:50.937656+0000","last_peered":"2024-04-26T21:51:50.937656+0000","last_clean":"2024-04-26T21:51:50.937656+0000","last_became_active":"2024-04-26T21:50:57.145620+0000","last_became_peered":"2024-04-26T21:50:57.145620+0000","last_unstale":"2024-04-26T21:51:50.937656+0000","last_undegraded":"2024-04-26T21:51:50.937656+0000","last_fullsized":"2024-04-26T21:51:50.937656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:26.351362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'25","reported_seq":197,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:13.580170+0000","last_change":"2024-04-26T21:50:56.365786+0000","last_active":"2024-04-26T21:52:13.580170+0000","last_peered":"2024-04-26T21:52:13.580170+0000","last_clean":"2024-04-26T21:52:13.580170+0000","last_became_active":"2024-04-26T21:50:56.365410+0000","last_became_peered":"2024-04-26T21:50:56.365410+0000","last_unstale":"2024-04-26T21:52:13.580170+0000","last_undegraded":"2024-04-26T21:52:13.580170+0000","last_fullsized":"2024-04-26T21:52:13.580170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:01.321661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49477,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":153,"num_read_kb":124,"num_write":26,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539601+0000","last_change":"2024-04-26T21:50:57.137333+0000","last_active":"2024-04-26T21:51:50.539601+0000","last_peered":"2024-04-26T21:51:50.539601+0000","last_clean":"2024-04-26T21:51:50.539601+0000","last_became_active":"2024-04-26T21:50:57.136983+0000","last_became_peered":"2024-04-26T21:50:57.136983+0000","last_unstale":"2024-04-26T21:51:50.539601+0000","last_undegraded":"2024-04-26T21:51:50.539601+0000","last_fullsized":"2024-04-26T21:51:50.539601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:18.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542083+0000","last_change":"2024-04-26T21:50:56.358340+0000","last_active":"2024-04-26T21:51:50.542083+0000","last_peered":"2024-04-26T21:51:50.542083+0000","last_clean":"2024-04-26T21:51:50.542083+0000","last_became_active":"2024-04-26T21:50:56.357972+0000","last_became_peered":"2024-04-26T21:50:56.357972+0000","last_unstale":"2024-04-26T21:51:50.542083+0000","last_undegraded":"2024-04-26T21:51:50.542083+0000","last_fullsized":"2024-04-26T21:51:50.542083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:52.256528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543770+0000","last_change":"2024-04-26T21:50:57.137027+0000","last_active":"2024-04-26T21:51:50.543770+0000","last_peered":"2024-04-26T21:51:50.543770+0000","last_clean":"2024-04-26T21:51:50.543770+0000","last_became_active":"2024-04-26T21:50:57.136400+0000","last_became_peered":"2024-04-26T21:50:57.136400+0000","last_unstale":"2024-04-26T21:51:50.543770+0000","last_undegraded":"2024-04-26T21:51:50.543770+0000","last_fullsized":"2024-04-26T21:51:50.543770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:45.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543952+0000","last_change":"2024-04-26T21:50:56.369784+0000","last_active":"2024-04-26T21:51:50.543952+0000","last_peered":"2024-04-26T21:51:50.543952+0000","last_clean":"2024-04-26T21:51:50.543952+0000","last_became_active":"2024-04-26T21:50:56.369093+0000","last_became_peered":"2024-04-26T21:50:56.369093+0000","last_unstale":"2024-04-26T21:51:50.543952+0000","last_undegraded":"2024-04-26T21:51:50.543952+0000","last_fullsized":"2024-04-26T21:51:50.543952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:27.595146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543136+0000","last_change":"2024-04-26T21:50:57.141404+0000","last_active":"2024-04-26T21:51:50.543136+0000","last_peered":"2024-04-26T21:51:50.543136+0000","last_clean":"2024-04-26T21:51:50.543136+0000","last_became_active":"2024-04-26T21:50:57.140973+0000","last_became_peered":"2024-04-26T21:50:57.140973+0000","last_unstale":"2024-04-26T21:51:50.543136+0000","last_undegraded":"2024-04-26T21:51:50.543136+0000","last_fullsized":"2024-04-26T21:51:50.543136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:09.405982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:13.458001+0000","last_change":"2024-04-26T21:50:56.370727+0000","last_active":"2024-04-26T21:52:13.458001+0000","last_peered":"2024-04-26T21:52:13.458001+0000","last_clean":"2024-04-26T21:52:13.458001+0000","last_became_active":"2024-04-26T21:50:56.370373+0000","last_became_peered":"2024-04-26T21:50:56.370373+0000","last_unstale":"2024-04-26T21:52:13.458001+0000","last_undegraded":"2024-04-26T21:52:13.458001+0000","last_fullsized":"2024-04-26T21:52:13.458001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:12.689699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542026+0000","last_change":"2024-04-26T21:50:57.145771+0000","last_active":"2024-04-26T21:51:50.542026+0000","last_peered":"2024-04-26T21:51:50.542026+0000","last_clean":"2024-04-26T21:51:50.542026+0000","last_became_active":"2024-04-26T21:50:57.145424+0000","last_became_peered":"2024-04-26T21:50:57.145424+0000","last_unstale":"2024-04-26T21:51:50.542026+0000","last_undegraded":"2024-04-26T21:51:50.542026+0000","last_fullsized":"2024-04-26T21:51:50.542026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:10.485045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.560040+0000","last_change":"2024-04-26T21:51:50.560040+0000","last_active":"2024-04-26T21:51:50.560040+0000","last_peered":"2024-04-26T21:51:50.560040+0000","last_clean":"2024-04-26T21:51:50.560040+0000","last_became_active":"2024-04-26T21:51:50.546278+0000","last_became_peered":"2024-04-26T21:51:50.546278+0000","last_unstale":"2024-04-26T21:51:50.560040+0000","last_undegraded":"2024-04-26T21:51:50.560040+0000","last_fullsized":"2024-04-26T21:51:50.560040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:59.851719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.938948+0000","last_change":"2024-04-26T21:51:50.938948+0000","last_active":"2024-04-26T21:51:50.938948+0000","last_peered":"2024-04-26T21:51:50.938948+0000","last_clean":"2024-04-26T21:51:50.938948+0000","last_became_active":"2024-04-26T21:51:50.938501+0000","last_became_peered":"2024-04-26T21:51:50.938501+0000","last_unstale":"2024-04-26T21:51:50.938948+0000","last_undegraded":"2024-04-26T21:51:50.938948+0000","last_fullsized":"2024-04-26T21:51:50.938948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:44.936013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539918+0000","last_change":"2024-04-26T21:50:56.358270+0000","last_active":"2024-04-26T21:51:50.539918+0000","last_peered":"2024-04-26T21:51:50.539918+0000","last_clean":"2024-04-26T21:51:50.539918+0000","last_became_active":"2024-04-26T21:50:56.357909+0000","last_became_peered":"2024-04-26T21:50:56.357909+0000","last_unstale":"2024-04-26T21:51:50.539918+0000","last_undegraded":"2024-04-26T21:51:50.539918+0000","last_fullsized":"2024-04-26T21:51:50.539918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:45.627352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545482+0000","last_change":"2024-04-26T21:50:57.142181+0000","last_active":"2024-04-26T21:51:50.545482+0000","last_peered":"2024-04-26T21:51:50.545482+0000","last_clean":"2024-04-26T21:51:50.545482+0000","last_became_active":"2024-04-26T21:50:57.141837+0000","last_became_peered":"2024-04-26T21:50:57.141837+0000","last_unstale":"2024-04-26T21:51:50.545482+0000","last_undegraded":"2024-04-26T21:51:50.545482+0000","last_fullsized":"2024-04-26T21:51:50.545482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:42.150522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.547619+0000","last_change":"2024-04-26T21:51:50.547619+0000","last_active":"2024-04-26T21:51:50.547619+0000","last_peered":"2024-04-26T21:51:50.547619+0000","last_clean":"2024-04-26T21:51:50.547619+0000","last_became_active":"2024-04-26T21:51:50.547180+0000","last_became_peered":"2024-04-26T21:51:50.547180+0000","last_unstale":"2024-04-26T21:51:50.547619+0000","last_undegraded":"2024-04-26T21:51:50.547619+0000","last_fullsized":"2024-04-26T21:51:50.547619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:17.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544546+0000","last_change":"2024-04-26T21:50:57.146682+0000","last_active":"2024-04-26T21:51:50.544546+0000","last_peered":"2024-04-26T21:51:50.544546+0000","last_clean":"2024-04-26T21:51:50.544546+0000","last_became_active":"2024-04-26T21:50:57.146329+0000","last_became_peered":"2024-04-26T21:50:57.146329+0000","last_unstale":"2024-04-26T21:51:50.544546+0000","last_undegraded":"2024-04-26T21:51:50.544546+0000","last_fullsized":"2024-04-26T21:51:50.544546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:41.035526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541572+0000","last_change":"2024-04-26T21:50:56.370533+0000","last_active":"2024-04-26T21:51:50.541572+0000","last_peered":"2024-04-26T21:51:50.541572+0000","last_clean":"2024-04-26T21:51:50.541572+0000","last_became_active":"2024-04-26T21:50:56.370029+0000","last_became_peered":"2024-04-26T21:50:56.370029+0000","last_unstale":"2024-04-26T21:51:50.541572+0000","last_undegraded":"2024-04-26T21:51:50.541572+0000","last_fullsized":"2024-04-26T21:51:50.541572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.005405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544183+0000","last_change":"2024-04-26T21:50:57.142420+0000","last_active":"2024-04-26T21:51:50.544183+0000","last_peered":"2024-04-26T21:51:50.544183+0000","last_clean":"2024-04-26T21:51:50.544183+0000","last_became_active":"2024-04-26T21:50:57.142023+0000","last_became_peered":"2024-04-26T21:50:57.142023+0000","last_unstale":"2024-04-26T21:51:50.544183+0000","last_undegraded":"2024-04-26T21:51:50.544183+0000","last_fullsized":"2024-04-26T21:51:50.544183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:31.567136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545145+0000","last_change":"2024-04-26T21:50:57.139924+0000","last_active":"2024-04-26T21:51:50.545145+0000","last_peered":"2024-04-26T21:51:50.545145+0000","last_clean":"2024-04-26T21:51:50.545145+0000","last_became_active":"2024-04-26T21:50:57.139561+0000","last_became_peered":"2024-04-26T21:50:57.139561+0000","last_unstale":"2024-04-26T21:51:50.545145+0000","last_undegraded":"2024-04-26T21:51:50.545145+0000","last_fullsized":"2024-04-26T21:51:50.545145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:52.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:52:09.449652+0000","last_change":"2024-04-26T21:50:56.365256+0000","last_active":"2024-04-26T21:52:09.449652+0000","last_peered":"2024-04-26T21:52:09.449652+0000","last_clean":"2024-04-26T21:52:09.449652+0000","last_became_active":"2024-04-26T21:50:56.364516+0000","last_became_peered":"2024-04-26T21:50:56.364516+0000","last_unstale":"2024-04-26T21:52:09.449652+0000","last_undegraded":"2024-04-26T21:52:09.449652+0000","last_fullsized":"2024-04-26T21:52:09.449652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:13.550129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594301+0000","last_change":"2024-04-26T21:50:57.138177+0000","last_active":"2024-04-26T21:51:50.594301+0000","last_peered":"2024-04-26T21:51:50.594301+0000","last_clean":"2024-04-26T21:51:50.594301+0000","last_became_active":"2024-04-26T21:50:57.137725+0000","last_became_peered":"2024-04-26T21:50:57.137725+0000","last_unstale":"2024-04-26T21:51:50.594301+0000","last_undegraded":"2024-04-26T21:51:50.594301+0000","last_fullsized":"2024-04-26T21:51:50.594301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:14.113935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542929+0000","last_change":"2024-04-26T21:50:56.358280+0000","last_active":"2024-04-26T21:51:50.542929+0000","last_peered":"2024-04-26T21:51:50.542929+0000","last_clean":"2024-04-26T21:51:50.542929+0000","last_became_active":"2024-04-26T21:50:56.357854+0000","last_became_peered":"2024-04-26T21:50:56.357854+0000","last_unstale":"2024-04-26T21:51:50.542929+0000","last_undegraded":"2024-04-26T21:51:50.542929+0000","last_fullsized":"2024-04-26T21:51:50.542929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.499124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542214+0000","last_change":"2024-04-26T21:49:42.292617+0000","last_active":"2024-04-26T21:51:50.542214+0000","last_peered":"2024-04-26T21:51:50.542214+0000","last_clean":"2024-04-26T21:51:50.542214+0000","last_became_active":"2024-04-26T21:49:42.292160+0000","last_became_peered":"2024-04-26T21:49:42.292160+0000","last_unstale":"2024-04-26T21:51:50.542214+0000","last_undegraded":"2024-04-26T21:51:50.542214+0000","last_fullsized":"2024-04-26T21:51:50.542214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_clean_scrub_stamp":"2024-04-26T21:46:13.787665+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:24.663652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543539+0000","last_change":"2024-04-26T21:50:57.145641+0000","last_active":"2024-04-26T21:51:50.543539+0000","last_peered":"2024-04-26T21:51:50.543539+0000","last_clean":"2024-04-26T21:51:50.543539+0000","last_became_active":"2024-04-26T21:50:57.145304+0000","last_became_peered":"2024-04-26T21:50:57.145304+0000","last_unstale":"2024-04-26T21:51:50.543539+0000","last_undegraded":"2024-04-26T21:51:50.543539+0000","last_fullsized":"2024-04-26T21:51:50.543539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:11.389728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.549862+0000","last_change":"2024-04-26T21:51:50.549862+0000","last_active":"2024-04-26T21:51:50.549862+0000","last_peered":"2024-04-26T21:51:50.549862+0000","last_clean":"2024-04-26T21:51:50.549862+0000","last_became_active":"2024-04-26T21:51:50.546054+0000","last_became_peered":"2024-04-26T21:51:50.546054+0000","last_unstale":"2024-04-26T21:51:50.549862+0000","last_undegraded":"2024-04-26T21:51:50.549862+0000","last_fullsized":"2024-04-26T21:51:50.549862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:33.817688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539705+0000","last_change":"2024-04-26T21:50:56.364074+0000","last_active":"2024-04-26T21:51:50.539705+0000","last_peered":"2024-04-26T21:51:50.539705+0000","last_clean":"2024-04-26T21:51:50.539705+0000","last_became_active":"2024-04-26T21:50:56.363688+0000","last_became_peered":"2024-04-26T21:50:56.363688+0000","last_unstale":"2024-04-26T21:51:50.539705+0000","last_undegraded":"2024-04-26T21:51:50.539705+0000","last_fullsized":"2024-04-26T21:51:50.539705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:35.526125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544008+0000","last_change":"2024-04-26T21:50:57.136811+0000","last_active":"2024-04-26T21:51:50.544008+0000","last_peered":"2024-04-26T21:51:50.544008+0000","last_clean":"2024-04-26T21:51:50.544008+0000","last_became_active":"2024-04-26T21:50:57.136252+0000","last_became_peered":"2024-04-26T21:50:57.136252+0000","last_unstale":"2024-04-26T21:51:50.544008+0000","last_undegraded":"2024-04-26T21:51:50.544008+0000","last_fullsized":"2024-04-26T21:51:50.544008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:01.061283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.540898+0000","last_change":"2024-04-26T21:50:56.364317+0000","last_active":"2024-04-26T21:51:50.540898+0000","last_peered":"2024-04-26T21:51:50.540898+0000","last_clean":"2024-04-26T21:51:50.540898+0000","last_became_active":"2024-04-26T21:50:56.363882+0000","last_became_peered":"2024-04-26T21:50:56.363882+0000","last_unstale":"2024-04-26T21:51:50.540898+0000","last_undegraded":"2024-04-26T21:51:50.540898+0000","last_fullsized":"2024-04-26T21:51:50.540898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:12.994293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594575+0000","last_change":"2024-04-26T21:50:57.144856+0000","last_active":"2024-04-26T21:51:50.594575+0000","last_peered":"2024-04-26T21:51:50.594575+0000","last_clean":"2024-04-26T21:51:50.594575+0000","last_became_active":"2024-04-26T21:50:57.143717+0000","last_became_peered":"2024-04-26T21:50:57.143717+0000","last_unstale":"2024-04-26T21:51:50.594575+0000","last_undegraded":"2024-04-26T21:51:50.594575+0000","last_fullsized":"2024-04-26T21:51:50.594575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:53.049243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543543+0000","last_change":"2024-04-26T21:50:56.370284+0000","last_active":"2024-04-26T21:51:50.543543+0000","last_peered":"2024-04-26T21:51:50.543543+0000","last_clean":"2024-04-26T21:51:50.543543+0000","last_became_active":"2024-04-26T21:50:56.369738+0000","last_became_peered":"2024-04-26T21:50:56.369738+0000","last_unstale":"2024-04-26T21:51:50.543543+0000","last_undegraded":"2024-04-26T21:51:50.543543+0000","last_fullsized":"2024-04-26T21:51:50.543543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:23.390553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539402+0000","last_change":"2024-04-26T21:50:57.142687+0000","last_active":"2024-04-26T21:51:50.539402+0000","last_peered":"2024-04-26T21:51:50.539402+0000","last_clean":"2024-04-26T21:51:50.539402+0000","last_became_active":"2024-04-26T21:50:57.142365+0000","last_became_peered":"2024-04-26T21:50:57.142365+0000","last_unstale":"2024-04-26T21:51:50.539402+0000","last_undegraded":"2024-04-26T21:51:50.539402+0000","last_fullsized":"2024-04-26T21:51:50.539402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:14.261741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937372+0000","last_change":"2024-04-26T21:50:56.364163+0000","last_active":"2024-04-26T21:51:50.937372+0000","last_peered":"2024-04-26T21:51:50.937372+0000","last_clean":"2024-04-26T21:51:50.937372+0000","last_became_active":"2024-04-26T21:50:56.363729+0000","last_became_peered":"2024-04-26T21:50:56.363729+0000","last_unstale":"2024-04-26T21:51:50.937372+0000","last_undegraded":"2024-04-26T21:51:50.937372+0000","last_fullsized":"2024-04-26T21:51:50.937372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:54.620884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539129+0000","last_change":"2024-04-26T21:50:57.142692+0000","last_active":"2024-04-26T21:51:50.539129+0000","last_peered":"2024-04-26T21:51:50.539129+0000","last_clean":"2024-04-26T21:51:50.539129+0000","last_became_active":"2024-04-26T21:50:57.142137+0000","last_became_peered":"2024-04-26T21:50:57.142137+0000","last_unstale":"2024-04-26T21:51:50.539129+0000","last_undegraded":"2024-04-26T21:51:50.539129+0000","last_fullsized":"2024-04-26T21:51:50.539129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.996485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594135+0000","last_change":"2024-04-26T21:50:56.366374+0000","last_active":"2024-04-26T21:51:50.594135+0000","last_peered":"2024-04-26T21:51:50.594135+0000","last_clean":"2024-04-26T21:51:50.594135+0000","last_became_active":"2024-04-26T21:50:56.366014+0000","last_became_peered":"2024-04-26T21:50:56.366014+0000","last_unstale":"2024-04-26T21:51:50.594135+0000","last_undegraded":"2024-04-26T21:51:50.594135+0000","last_fullsized":"2024-04-26T21:51:50.594135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:41.867792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543744+0000","last_change":"2024-04-26T21:50:57.142833+0000","last_active":"2024-04-26T21:51:50.543744+0000","last_peered":"2024-04-26T21:51:50.543744+0000","last_clean":"2024-04-26T21:51:50.543744+0000","last_became_active":"2024-04-26T21:50:57.142061+0000","last_became_peered":"2024-04-26T21:50:57.142061+0000","last_unstale":"2024-04-26T21:51:50.543744+0000","last_undegraded":"2024-04-26T21:51:50.543744+0000","last_fullsized":"2024-04-26T21:51:50.543744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:33.023159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541867+0000","last_change":"2024-04-26T21:50:56.363172+0000","last_active":"2024-04-26T21:51:50.541867+0000","last_peered":"2024-04-26T21:51:50.541867+0000","last_clean":"2024-04-26T21:51:50.541867+0000","last_became_active":"2024-04-26T21:50:56.362782+0000","last_became_peered":"2024-04-26T21:50:56.362782+0000","last_unstale":"2024-04-26T21:51:50.541867+0000","last_undegraded":"2024-04-26T21:51:50.541867+0000","last_fullsized":"2024-04-26T21:51:50.541867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:37.183497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594433+0000","last_change":"2024-04-26T21:50:57.142088+0000","last_active":"2024-04-26T21:51:50.594433+0000","last_peered":"2024-04-26T21:51:50.594433+0000","last_clean":"2024-04-26T21:51:50.594433+0000","last_became_active":"2024-04-26T21:50:57.141685+0000","last_became_peered":"2024-04-26T21:50:57.141685+0000","last_unstale":"2024-04-26T21:51:50.594433+0000","last_undegraded":"2024-04-26T21:51:50.594433+0000","last_fullsized":"2024-04-26T21:51:50.594433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:24.033787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539417+0000","last_change":"2024-04-26T21:50:56.371717+0000","last_active":"2024-04-26T21:51:50.539417+0000","last_peered":"2024-04-26T21:51:50.539417+0000","last_clean":"2024-04-26T21:51:50.539417+0000","last_became_active":"2024-04-26T21:50:56.370930+0000","last_became_peered":"2024-04-26T21:50:56.370930+0000","last_unstale":"2024-04-26T21:51:50.539417+0000","last_undegraded":"2024-04-26T21:51:50.539417+0000","last_fullsized":"2024-04-26T21:51:50.539417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.977392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541695+0000","last_change":"2024-04-26T21:50:57.139671+0000","last_active":"2024-04-26T21:51:50.541695+0000","last_peered":"2024-04-26T21:51:50.541695+0000","last_clean":"2024-04-26T21:51:50.541695+0000","last_became_active":"2024-04-26T21:50:57.139245+0000","last_became_peered":"2024-04-26T21:50:57.139245+0000","last_unstale":"2024-04-26T21:51:50.541695+0000","last_undegraded":"2024-04-26T21:51:50.541695+0000","last_fullsized":"2024-04-26T21:51:50.541695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:26.903905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541758+0000","last_change":"2024-04-26T21:50:56.369836+0000","last_active":"2024-04-26T21:51:50.541758+0000","last_peered":"2024-04-26T21:51:50.541758+0000","last_clean":"2024-04-26T21:51:50.541758+0000","last_became_active":"2024-04-26T21:50:56.369522+0000","last_became_peered":"2024-04-26T21:50:56.369522+0000","last_unstale":"2024-04-26T21:51:50.541758+0000","last_undegraded":"2024-04-26T21:51:50.541758+0000","last_fullsized":"2024-04-26T21:51:50.541758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:40.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543740+0000","last_change":"2024-04-26T21:50:57.138551+0000","last_active":"2024-04-26T21:51:50.543740+0000","last_peered":"2024-04-26T21:51:50.543740+0000","last_clean":"2024-04-26T21:51:50.543740+0000","last_became_active":"2024-04-26T21:50:57.138187+0000","last_became_peered":"2024-04-26T21:50:57.138187+0000","last_unstale":"2024-04-26T21:51:50.543740+0000","last_undegraded":"2024-04-26T21:51:50.543740+0000","last_fullsized":"2024-04-26T21:51:50.543740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:46.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543990+0000","last_change":"2024-04-26T21:50:56.362009+0000","last_active":"2024-04-26T21:51:50.543990+0000","last_peered":"2024-04-26T21:51:50.543990+0000","last_clean":"2024-04-26T21:51:50.543990+0000","last_became_active":"2024-04-26T21:50:56.361646+0000","last_became_peered":"2024-04-26T21:50:56.361646+0000","last_unstale":"2024-04-26T21:51:50.543990+0000","last_undegraded":"2024-04-26T21:51:50.543990+0000","last_fullsized":"2024-04-26T21:51:50.543990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:25.316195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543113+0000","last_change":"2024-04-26T21:50:57.139983+0000","last_active":"2024-04-26T21:51:50.543113+0000","last_peered":"2024-04-26T21:51:50.543113+0000","last_clean":"2024-04-26T21:51:50.543113+0000","last_became_active":"2024-04-26T21:50:57.138426+0000","last_became_peered":"2024-04-26T21:50:57.138426+0000","last_unstale":"2024-04-26T21:51:50.543113+0000","last_undegraded":"2024-04-26T21:51:50.543113+0000","last_fullsized":"2024-04-26T21:51:50.543113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:53.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544138+0000","last_change":"2024-04-26T21:50:56.369527+0000","last_active":"2024-04-26T21:51:50.544138+0000","last_peered":"2024-04-26T21:51:50.544138+0000","last_clean":"2024-04-26T21:51:50.544138+0000","last_became_active":"2024-04-26T21:50:56.369168+0000","last_became_peered":"2024-04-26T21:50:56.369168+0000","last_unstale":"2024-04-26T21:51:50.544138+0000","last_undegraded":"2024-04-26T21:51:50.544138+0000","last_fullsized":"2024-04-26T21:51:50.544138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:38.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542958+0000","last_change":"2024-04-26T21:50:57.137957+0000","last_active":"2024-04-26T21:51:50.542958+0000","last_peered":"2024-04-26T21:51:50.542958+0000","last_clean":"2024-04-26T21:51:50.542958+0000","last_became_active":"2024-04-26T21:50:57.137595+0000","last_became_peered":"2024-04-26T21:50:57.137595+0000","last_unstale":"2024-04-26T21:51:50.542958+0000","last_undegraded":"2024-04-26T21:51:50.542958+0000","last_fullsized":"2024-04-26T21:51:50.542958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:09.355937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543859+0000","last_change":"2024-04-26T21:50:57.143098+0000","last_active":"2024-04-26T21:51:50.543859+0000","last_peered":"2024-04-26T21:51:50.543859+0000","last_clean":"2024-04-26T21:51:50.543859+0000","last_became_active":"2024-04-26T21:50:57.142524+0000","last_became_peered":"2024-04-26T21:50:57.142524+0000","last_unstale":"2024-04-26T21:51:50.543859+0000","last_undegraded":"2024-04-26T21:51:50.543859+0000","last_fullsized":"2024-04-26T21:51:50.543859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:33.333382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541844+0000","last_change":"2024-04-26T21:50:57.143123+0000","last_active":"2024-04-26T21:51:50.541844+0000","last_peered":"2024-04-26T21:51:50.541844+0000","last_clean":"2024-04-26T21:51:50.541844+0000","last_became_active":"2024-04-26T21:50:57.142805+0000","last_became_peered":"2024-04-26T21:50:57.142805+0000","last_unstale":"2024-04-26T21:51:50.541844+0000","last_undegraded":"2024-04-26T21:51:50.541844+0000","last_fullsized":"2024-04-26T21:51:50.541844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:19.330202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.541368+0000","last_change":"2024-04-26T21:50:57.140268+0000","last_active":"2024-04-26T21:51:50.541368+0000","last_peered":"2024-04-26T21:51:50.541368+0000","last_clean":"2024-04-26T21:51:50.541368+0000","last_became_active":"2024-04-26T21:50:57.139927+0000","last_became_peered":"2024-04-26T21:50:57.139927+0000","last_unstale":"2024-04-26T21:51:50.541368+0000","last_undegraded":"2024-04-26T21:51:50.541368+0000","last_fullsized":"2024-04-26T21:51:50.541368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:01.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.539146+0000","last_change":"2024-04-26T21:50:57.143564+0000","last_active":"2024-04-26T21:51:50.539146+0000","last_peered":"2024-04-26T21:51:50.539146+0000","last_clean":"2024-04-26T21:51:50.539146+0000","last_became_active":"2024-04-26T21:50:57.142740+0000","last_became_peered":"2024-04-26T21:50:57.142740+0000","last_unstale":"2024-04-26T21:51:50.539146+0000","last_undegraded":"2024-04-26T21:51:50.539146+0000","last_fullsized":"2024-04-26T21:51:50.539146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:58.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.544034+0000","last_change":"2024-04-26T21:50:57.146883+0000","last_active":"2024-04-26T21:51:50.544034+0000","last_peered":"2024-04-26T21:51:50.544034+0000","last_clean":"2024-04-26T21:51:50.544034+0000","last_became_active":"2024-04-26T21:50:57.146540+0000","last_became_peered":"2024-04-26T21:50:57.146540+0000","last_unstale":"2024-04-26T21:51:50.544034+0000","last_undegraded":"2024-04-26T21:51:50.544034+0000","last_fullsized":"2024-04-26T21:51:50.544034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.787618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545677+0000","last_change":"2024-04-26T21:50:57.136938+0000","last_active":"2024-04-26T21:51:50.545677+0000","last_peered":"2024-04-26T21:51:50.545677+0000","last_clean":"2024-04-26T21:51:50.545677+0000","last_became_active":"2024-04-26T21:50:57.136474+0000","last_became_peered":"2024-04-26T21:50:57.136474+0000","last_unstale":"2024-04-26T21:51:50.545677+0000","last_undegraded":"2024-04-26T21:51:50.545677+0000","last_fullsized":"2024-04-26T21:51:50.545677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:42.534342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.542048+0000","last_change":"2024-04-26T21:50:57.150958+0000","last_active":"2024-04-26T21:51:50.542048+0000","last_peered":"2024-04-26T21:51:50.542048+0000","last_clean":"2024-04-26T21:51:50.542048+0000","last_became_active":"2024-04-26T21:50:57.150593+0000","last_became_peered":"2024-04-26T21:50:57.150593+0000","last_unstale":"2024-04-26T21:51:50.542048+0000","last_undegraded":"2024-04-26T21:51:50.542048+0000","last_fullsized":"2024-04-26T21:51:50.542048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:26.583983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594463+0000","last_change":"2024-04-26T21:50:57.147766+0000","last_active":"2024-04-26T21:51:50.594463+0000","last_peered":"2024-04-26T21:51:50.594463+0000","last_clean":"2024-04-26T21:51:50.594463+0000","last_became_active":"2024-04-26T21:50:57.147442+0000","last_became_peered":"2024-04-26T21:50:57.147442+0000","last_unstale":"2024-04-26T21:51:50.594463+0000","last_undegraded":"2024-04-26T21:51:50.594463+0000","last_fullsized":"2024-04-26T21:51:50.594463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.489117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.595948+0000","last_change":"2024-04-26T21:51:50.595554+0000","last_active":"2024-04-26T21:51:50.595948+0000","last_peered":"2024-04-26T21:51:50.595948+0000","last_clean":"2024-04-26T21:51:50.595948+0000","last_became_active":"2024-04-26T21:51:50.595248+0000","last_became_peered":"2024-04-26T21:51:50.595248+0000","last_unstale":"2024-04-26T21:51:50.595948+0000","last_undegraded":"2024-04-26T21:51:50.595948+0000","last_fullsized":"2024-04-26T21:51:50.595948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:36.853146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.594067+0000","last_change":"2024-04-26T21:50:57.142804+0000","last_active":"2024-04-26T21:51:50.594067+0000","last_peered":"2024-04-26T21:51:50.594067+0000","last_clean":"2024-04-26T21:51:50.594067+0000","last_became_active":"2024-04-26T21:50:57.141902+0000","last_became_peered":"2024-04-26T21:50:57.141902+0000","last_unstale":"2024-04-26T21:51:50.594067+0000","last_undegraded":"2024-04-26T21:51:50.594067+0000","last_fullsized":"2024-04-26T21:51:50.594067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:37.590730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.937945+0000","last_change":"2024-04-26T21:50:57.138637+0000","last_active":"2024-04-26T21:51:50.937945+0000","last_peered":"2024-04-26T21:51:50.937945+0000","last_clean":"2024-04-26T21:51:50.937945+0000","last_became_active":"2024-04-26T21:50:57.138295+0000","last_became_peered":"2024-04-26T21:50:57.138295+0000","last_unstale":"2024-04-26T21:51:50.937945+0000","last_undegraded":"2024-04-26T21:51:50.937945+0000","last_fullsized":"2024-04-26T21:51:50.937945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:19.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.543859+0000","last_change":"2024-04-26T21:50:57.137377+0000","last_active":"2024-04-26T21:51:50.543859+0000","last_peered":"2024-04-26T21:51:50.543859+0000","last_clean":"2024-04-26T21:51:50.543859+0000","last_became_active":"2024-04-26T21:50:57.136893+0000","last_became_peered":"2024-04-26T21:50:57.136893+0000","last_unstale":"2024-04-26T21:51:50.543859+0000","last_undegraded":"2024-04-26T21:51:50.543859+0000","last_fullsized":"2024-04-26T21:51:50.543859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:44.257438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-26T21:51:50.545214+0000","last_change":"2024-04-26T21:50:57.141439+0000","last_active":"2024-04-26T21:51:50.545214+0000","last_peered":"2024-04-26T21:51:50.545214+0000","last_clean":"2024-04-26T21:51:50.545214+0000","last_became_active":"2024-04-26T21:50:57.141060+0000","last_became_peered":"2024-04-26T21:50:57.141060+0000","last_unstale":"2024-04-26T21:51:50.545214+0000","last_undegraded":"2024-04-26T21:51:50.545214+0000","last_fullsized":"2024-04-26T21:51:50.545214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:27.174348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250381,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53478,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":408,"num_read_kb":284,"num_write":153,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":160434,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":457001,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":1097728,"data_compressed":44071,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456615,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456535,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1048465,"data_compressed":42217,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":456004,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":505274,"data_compressed":7602,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":457275,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":506512,"data_compressed":7602,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":456315,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456944,"data_compressed":5748,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1047136,"data_compressed":42217,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":49477,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":50715,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51446,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:52:16.898 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T21:52:16.898 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T21:52:16.899 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T21:52:16.899 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T21:52:17.188 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:17.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:17 smithi064 ceph-mon[32765]: pgmap v330: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:17.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:17 smithi064 ceph-mon[32765]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:17.544 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:17 smithi055 ceph-mon[29221]: pgmap v330: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:17.544 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:17 smithi055 ceph-mon[29221]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:17.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:17 smithi165 ceph-mon[33296]: pgmap v330: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:17.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:17 smithi165 ceph-mon[33296]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:18.274 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:18 smithi055 ceph-mon[29221]: pgmap v331: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:18.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:18 smithi064 ceph-mon[32765]: pgmap v331: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:18.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:18 smithi165 ceph-mon[33296]: pgmap v331: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:18.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 -- 172.21.15.55:0/540289450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f332806bb70 msgr2=0x7f332806bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:18.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 --2- 172.21.15.55:0/540289450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f332806bb70 0x7f332806bfd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3318009990 tx=0x7f3318009ca0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 -- 172.21.15.55:0/540289450 shutdown_connections 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 --2- 172.21.15.55:0/540289450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332806c510 0x7f33281102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 --2- 172.21.15.55:0/540289450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33281094a0 0x7f3328109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 --2- 172.21.15.55:0/540289450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f332806bb70 0x7f332806bfd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 -- 172.21.15.55:0/540289450 >> 172.21.15.55:0/540289450 conn(0x7f33280792f0 msgr2=0x7f3328079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 -- 172.21.15.55:0/540289450 shutdown_connections 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.833+0000 7f333049b700 1 -- 172.21.15.55:0/540289450 wait complete. 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f333049b700 1 Processor -- start 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f333049b700 1 -- start start 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f333049b700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f332806bb70 0x7f332819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f333049b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332806c510 0x7f332819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f333049b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33281094a0 0x7f33281a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f333049b700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3328113650 con 0x7f33281094a0 2024-04-26T21:52:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f333049b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f33281134d0 con 0x7f332806c510 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f333049b700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f33281137d0 con 0x7f332806bb70 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33281094a0 0x7f33281a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33281094a0 0x7f33281a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47892/0 (socket says 172.21.15.55:47892) 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 -- 172.21.15.55:0/2936367747 learned_addr learned my addr 172.21.15.55:0/2936367747 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332da36700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332806c510 0x7f332819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332e237700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f332806bb70 0x7f332819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 -- 172.21.15.55:0/2936367747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f332806bb70 msgr2=0x7f332819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f332806bb70 0x7f332819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 -- 172.21.15.55:0/2936367747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332806c510 msgr2=0x7f332819c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332806c510 0x7f332819c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 -- 172.21.15.55:0/2936367747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33281a0d50 con 0x7f33281094a0 2024-04-26T21:52:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.834+0000 7f332ea38700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33281094a0 0x7f33281a0610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f332000b970 tx=0x7f332000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.835+0000 7f331f7fe700 1 -- 172.21.15.55:0/2936367747 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332000d860 con 0x7f33281094a0 2024-04-26T21:52:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.835+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33281a1040 con 0x7f33281094a0 2024-04-26T21:52:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.835+0000 7f331f7fe700 1 -- 172.21.15.55:0/2936367747 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f332000e430 con 0x7f33281094a0 2024-04-26T21:52:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.835+0000 7f331f7fe700 1 -- 172.21.15.55:0/2936367747 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320017660 con 0x7f33281094a0 2024-04-26T21:52:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.835+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33281a88c0 con 0x7f33281094a0 2024-04-26T21:52:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.836+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3328104b50 con 0x7f33281094a0 2024-04-26T21:52:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.837+0000 7f331f7fe700 1 -- 172.21.15.55:0/2936367747 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f332000da00 con 0x7f33281094a0 2024-04-26T21:52:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.837+0000 7f331f7fe700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33140a3cd0 0x7f33140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.838+0000 7f332e237700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33140a3cd0 0x7f33140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.838+0000 7f331f7fe700 1 -- 172.21.15.55:0/2936367747 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f33200d05a0 con 0x7f33281094a0 2024-04-26T21:52:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.838+0000 7f332e237700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33140a3cd0 0x7f33140a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3324005950 tx=0x7f332400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:18.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:18.841+0000 7f331f7fe700 1 -- 172.21.15.55:0/2936367747 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3320013030 con 0x7f33281094a0 2024-04-26T21:52:19.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.096+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f332802e490 con 0x7f33281094a0 2024-04-26T21:52:19.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.096+0000 7f331f7fe700 1 -- 172.21.15.55:0/2936367747 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f332009ce00 con 0x7f33281094a0 2024-04-26T21:52:19.097 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:19.097 INFO:teuthology.orchestra.run.smithi055.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33140a3cd0 msgr2=0x7f33140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33140a3cd0 0x7f33140a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3324005950 tx=0x7f332400a300 comp rx=0 tx=0).stop 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33281094a0 msgr2=0x7f33281a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33281094a0 0x7f33281a0610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f332000b970 tx=0x7f332000bc80 comp rx=0 tx=0).stop 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 shutdown_connections 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33281094a0 0x7f33281a0610 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f332806bb70 0x7f332819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33140a3cd0 0x7f33140a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 --2- 172.21.15.55:0/2936367747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f332806c510 0x7f332819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:19.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 >> 172.21.15.55:0/2936367747 conn(0x7f33280792f0 msgr2=0x7f332810e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:19.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 shutdown_connections 2024-04-26T21:52:19.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:19.099+0000 7f333049b700 1 -- 172.21.15.55:0/2936367747 wait complete. 2024-04-26T21:52:19.392 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:19 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2936367747' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:52:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:19 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2936367747' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:52:19.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:19 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2936367747' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T21:52:19.753 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T21:52:20.221 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.328+0000 7f01ec4f1640 1 -- 172.21.15.55:0/2721532482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e41033f0 msgr2=0x7f01e41057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.328+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/2721532482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e41033f0 0x7f01e41057e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009900 tx=0x7f01e002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 -- 172.21.15.55:0/2721532482 shutdown_connections 2024-04-26T21:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/2721532482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01e4108650 0x7f01e410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/2721532482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e4105d20 0x7f01e4108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/2721532482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e41033f0 0x7f01e41057e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 -- 172.21.15.55:0/2721532482 >> 172.21.15.55:0/2721532482 conn(0x7f01e40fcde0 msgr2=0x7f01e40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 -- 172.21.15.55:0/2721532482 shutdown_connections 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 -- 172.21.15.55:0/2721532482 wait complete. 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 Processor -- start 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 -- start start 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01e41033f0 0x7f01e4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e4105d20 0x7f01e4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e4108650 0x7f01e41a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f01e410e4c0 con 0x7f01e4105d20 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f01e410e340 con 0x7f01e4108650 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.329+0000 7f01ec4f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01e410e640 con 0x7f01e41033f0 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e4105d20 0x7f01e4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e4105d20 0x7f01e4199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47908/0 (socket says 172.21.15.55:47908) 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 -- 172.21.15.55:0/3466467948 learned_addr learned my addr 172.21.15.55:0/3466467948 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01ea28d640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01e41033f0 0x7f01e4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01eaa8e640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e4108650 0x7f01e41a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 -- 172.21.15.55:0/3466467948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01e41033f0 msgr2=0x7f01e4198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01e41033f0 0x7f01e4198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 -- 172.21.15.55:0/3466467948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e4108650 msgr2=0x7f01e41a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e4108650 0x7f01e41a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 -- 172.21.15.55:0/3466467948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e41a0810 con 0x7f01e4105d20 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01e9a8c640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e4105d20 0x7f01e4199130 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01d400d770 tx=0x7f01d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01db7fe640 1 -- 172.21.15.55:0/3466467948 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4017070 con 0x7f01e4105d20 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01db7fe640 1 -- 172.21.15.55:0/3466467948 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d4009c30 con 0x7f01e4105d20 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01db7fe640 1 -- 172.21.15.55:0/3466467948 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d400f620 con 0x7f01e4105d20 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e41a0b00 con 0x7f01e4105d20 2024-04-26T21:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.330+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e41a1050 con 0x7f01e4105d20 2024-04-26T21:52:20.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.331+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e40fe8d0 con 0x7f01e4105d20 2024-04-26T21:52:20.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.333+0000 7f01db7fe640 1 -- 172.21.15.55:0/3466467948 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f01d401a070 con 0x7f01e4105d20 2024-04-26T21:52:20.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.334+0000 7f01db7fe640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01d00a3c40 0x7f01d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.334+0000 7f01db7fe640 1 -- 172.21.15.55:0/3466467948 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f01d40cffe0 con 0x7f01e4105d20 2024-04-26T21:52:20.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.334+0000 7f01ea28d640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01d00a3c40 0x7f01d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.334+0000 7f01ea28d640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01d00a3c40 0x7f01d00a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01e0006010 tx=0x7f01e00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.337+0000 7f01db7fe640 1 -- 172.21.15.55:0/3466467948 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f01d4009de0 con 0x7f01e4105d20 2024-04-26T21:52:20.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:20 smithi064 ceph-mon[32765]: pgmap v332: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:20.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.525+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01e402dcd0 con 0x7f01e4105d20 2024-04-26T21:52:20.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.526+0000 7f01db7fe640 1 -- 172.21.15.55:0/3466467948 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22713 (secure 0 0 0) 0x7f01d409c1f0 con 0x7f01e4105d20 2024-04-26T21:52:20.527 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:20.527 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:51:50.534100+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:20.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01d00a3c40 msgr2=0x7f01d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01d00a3c40 0x7f01d00a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01e0006010 tx=0x7f01e00314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e4105d20 msgr2=0x7f01e4199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e4105d20 0x7f01e4199130 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01d400d770 tx=0x7f01d400dc30 comp rx=0 tx=0).stop 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 shutdown_connections 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01d00a3c40 0x7f01d00a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01e4108650 0x7f01e41a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01e4105d20 0x7f01e4199130 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 --2- 172.21.15.55:0/3466467948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01e41033f0 0x7f01e4198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 >> 172.21.15.55:0/3466467948 conn(0x7f01e40fcde0 msgr2=0x7f01e4104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 shutdown_connections 2024-04-26T21:52:20.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.529+0000 7f01ec4f1640 1 -- 172.21.15.55:0/3466467948 wait complete. 2024-04-26T21:52:20.539 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:20 smithi055 ceph-mon[29221]: pgmap v332: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:20.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:52:20.541 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:52:20.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:20 smithi165 ceph-mon[33296]: pgmap v332: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:20.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 -- 172.21.15.55:0/3488485033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 msgr2=0x7ff820101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 --2- 172.21.15.55:0/3488485033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 0x7ff820101cb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff810009900 tx=0x7ff81002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:20.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 -- 172.21.15.55:0/3488485033 shutdown_connections 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 --2- 172.21.15.55:0/3488485033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff820103440 0x7ff820109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 --2- 172.21.15.55:0/3488485033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff820102a80 0x7ff820102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 --2- 172.21.15.55:0/3488485033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 0x7ff820101cb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 -- 172.21.15.55:0/3488485033 >> 172.21.15.55:0/3488485033 conn(0x7ff8200fce20 msgr2=0x7ff8200ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 -- 172.21.15.55:0/3488485033 shutdown_connections 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 -- 172.21.15.55:0/3488485033 wait complete. 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.647+0000 7ff825572640 1 Processor -- start 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff825572640 1 -- start start 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff825572640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 0x7ff82019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff825572640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff820102a80 0x7ff82019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff825572640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff820103440 0x7ff8201a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff825572640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010e540 con 0x7ff820101890 2024-04-26T21:52:20.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff825572640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010e3c0 con 0x7ff820103440 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff825572640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff82010e6c0 con 0x7ff820102a80 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 0x7ff82019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 0x7ff82019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47918/0 (socket says 172.21.15.55:47918) 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 -- 172.21.15.55:0/2144125606 learned_addr learned my addr 172.21.15.55:0/2144125606 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81f7fe640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff820103440 0x7ff8201a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81e7fc640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff820102a80 0x7ff82019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 -- 172.21.15.55:0/2144125606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff820102a80 msgr2=0x7ff82019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff820102a80 0x7ff82019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 -- 172.21.15.55:0/2144125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff820103440 msgr2=0x7ff8201a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff820103440 0x7ff8201a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 -- 172.21.15.55:0/2144125606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8201a2a00 con 0x7ff820101890 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81e7fc640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff820102a80 0x7ff82019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:20.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81f7fe640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff820103440 0x7ff8201a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:20.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff81effd640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 0x7ff82019ade0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff810031500 tx=0x7ff810003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:20.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.648+0000 7ff807fff640 1 -- 172.21.15.55:0/2144125606 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81003f070 con 0x7ff820101890 2024-04-26T21:52:20.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.649+0000 7ff807fff640 1 -- 172.21.15.55:0/2144125606 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff810003e70 con 0x7ff820101890 2024-04-26T21:52:20.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.649+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8201a2c90 con 0x7ff820101890 2024-04-26T21:52:20.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.649+0000 7ff807fff640 1 -- 172.21.15.55:0/2144125606 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810035030 con 0x7ff820101890 2024-04-26T21:52:20.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.649+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8201a3180 con 0x7ff820101890 2024-04-26T21:52:20.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.649+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8201a3350 con 0x7ff820101890 2024-04-26T21:52:20.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.651+0000 7ff807fff640 1 -- 172.21.15.55:0/2144125606 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff810042070 con 0x7ff820101890 2024-04-26T21:52:20.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.651+0000 7ff807fff640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff8080a3c40 0x7ff8080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.651+0000 7ff81e7fc640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff8080a3c40 0x7ff8080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.651+0000 7ff807fff640 1 -- 172.21.15.55:0/2144125606 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff8100f12b0 con 0x7ff820101890 2024-04-26T21:52:20.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.652+0000 7ff81e7fc640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff8080a3c40 0x7ff8080a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff82019c2f0 tx=0x7ff814005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:20.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.656+0000 7ff807fff640 1 -- 172.21.15.55:0/2144125606 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff8100bdb10 con 0x7ff820101890 2024-04-26T21:52:20.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.842+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8201a3520 con 0x7ff820101890 2024-04-26T21:52:20.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.842+0000 7ff807fff640 1 -- 172.21.15.55:0/2144125606 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7ff8100bd410 con 0x7ff820101890 2024-04-26T21:52:20.843 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:52:20.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff8080a3c40 msgr2=0x7ff8080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff8080a3c40 0x7ff8080a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff82019c2f0 tx=0x7ff814005d30 comp rx=0 tx=0).stop 2024-04-26T21:52:20.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 msgr2=0x7ff82019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 0x7ff82019ade0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff810031500 tx=0x7ff810003c80 comp rx=0 tx=0).stop 2024-04-26T21:52:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 shutdown_connections 2024-04-26T21:52:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff8080a3c40 0x7ff8080a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff820103440 0x7ff8201a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff820102a80 0x7ff82019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 --2- 172.21.15.55:0/2144125606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff820101890 0x7ff82019ade0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 >> 172.21.15.55:0/2144125606 conn(0x7ff8200fce20 msgr2=0x7ff82010a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 shutdown_connections 2024-04-26T21:52:20.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.845+0000 7ff825572640 1 -- 172.21.15.55:0/2144125606 wait complete. 2024-04-26T21:52:20.856 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:52:20.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.960+0000 7fada539d640 1 -- 172.21.15.55:0/2230195939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0103470 msgr2=0x7fada0105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 --2- 172.21.15.55:0/2230195939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0103470 0x7fada0105860 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fad90009900 tx=0x7fad9002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:20.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 -- 172.21.15.55:0/2230195939 shutdown_connections 2024-04-26T21:52:20.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 --2- 172.21.15.55:0/2230195939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada01086d0 0x7fada010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 --2- 172.21.15.55:0/2230195939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0105da0 0x7fada0108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 --2- 172.21.15.55:0/2230195939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0103470 0x7fada0105860 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 -- 172.21.15.55:0/2230195939 >> 172.21.15.55:0/2230195939 conn(0x7fada00fce20 msgr2=0x7fada00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 -- 172.21.15.55:0/2230195939 shutdown_connections 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 -- 172.21.15.55:0/2230195939 wait complete. 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 Processor -- start 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 -- start start 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103470 0x7fada0102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0105da0 0x7fada0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.961+0000 7fada539d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada01086d0 0x7fada0101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fada539d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fada010e530 con 0x7fada0105da0 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fada539d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fada010e3b0 con 0x7fada01086d0 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fada539d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fada010e6b0 con 0x7fada0103470 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0105da0 0x7fada0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0105da0 0x7fada0101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49608/0 (socket says 172.21.15.55:49608) 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 -- 172.21.15.55:0/1529158330 learned_addr learned my addr 172.21.15.55:0/1529158330 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9f7fe640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada01086d0 0x7fada0101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 -- 172.21.15.55:0/1529158330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103470 msgr2=0x7fada0102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9effd640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103470 0x7fada0102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103470 0x7fada0102de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 -- 172.21.15.55:0/1529158330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada01086d0 msgr2=0x7fada0101960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada01086d0 0x7fada0101960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 -- 172.21.15.55:0/1529158330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada01021f0 con 0x7fada0105da0 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9f7fe640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada01086d0 0x7fada0101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad9e7fc640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0105da0 0x7fada0101420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad9400b450 tx=0x7fad9400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad87fff640 1 -- 172.21.15.55:0/1529158330 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94015020 con 0x7fada0105da0 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad87fff640 1 -- 172.21.15.55:0/1529158330 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad94011d40 con 0x7fada0105da0 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada01a2dd0 con 0x7fada0105da0 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fad87fff640 1 -- 172.21.15.55:0/1529158330 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9400f340 con 0x7fada0105da0 2024-04-26T21:52:20.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.962+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada01a32b0 con 0x7fada0105da0 2024-04-26T21:52:20.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.963+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada00fe910 con 0x7fada0105da0 2024-04-26T21:52:20.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.965+0000 7fad87fff640 1 -- 172.21.15.55:0/1529158330 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fad94018020 con 0x7fada0105da0 2024-04-26T21:52:20.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.965+0000 7fad87fff640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad880a3cc0 0x7fad880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:20.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.965+0000 7fad9effd640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad880a3cc0 0x7fad880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:20.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.965+0000 7fad87fff640 1 -- 172.21.15.55:0/1529158330 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fad940cffa0 con 0x7fada0105da0 2024-04-26T21:52:20.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.966+0000 7fad9effd640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad880a3cc0 0x7fad880a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fad90005c40 tx=0x7fad90000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:20.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:20.970+0000 7fad87fff640 1 -- 172.21.15.55:0/1529158330 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fad9401f020 con 0x7fada0105da0 2024-04-26T21:52:21.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.157+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fada002dd60 con 0x7fada0105da0 2024-04-26T21:52:21.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.157+0000 7fad87fff640 1 -- 172.21.15.55:0/1529158330 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fad9400d040 con 0x7fada0105da0 2024-04-26T21:52:21.158 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.159+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad880a3cc0 msgr2=0x7fad880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.159+0000 7fada539d640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad880a3cc0 0x7fad880a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fad90005c40 tx=0x7fad90000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.159+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0105da0 msgr2=0x7fada0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.159+0000 7fada539d640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0105da0 0x7fada0101420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad9400b450 tx=0x7fad9400b910 comp rx=0 tx=0).stop 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.160+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 shutdown_connections 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.160+0000 7fada539d640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad880a3cc0 0x7fad880a6180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.160+0000 7fada539d640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fada01086d0 0x7fada0101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.160+0000 7fada539d640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fada0105da0 0x7fada0101420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.160+0000 7fada539d640 1 --2- 172.21.15.55:0/1529158330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada0103470 0x7fada0102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.160+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 >> 172.21.15.55:0/1529158330 conn(0x7fada00fce20 msgr2=0x7fada0103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.160+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 shutdown_connections 2024-04-26T21:52:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.160+0000 7fada539d640 1 -- 172.21.15.55:0/1529158330 wait complete. 2024-04-26T21:52:21.171 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:52:21.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.275+0000 7f7ced259640 1 -- 172.21.15.55:0/2399303497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 msgr2=0x7f7ce8101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.275+0000 7f7ced259640 1 --2- 172.21.15.55:0/2399303497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 0x7f7ce8101c90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009900 tx=0x7f7ce002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:21.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.276+0000 7f7ced259640 1 -- 172.21.15.55:0/2399303497 shutdown_connections 2024-04-26T21:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.276+0000 7f7ced259640 1 --2- 172.21.15.55:0/2399303497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ce8103420 0x7f7ce8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.276+0000 7f7ced259640 1 --2- 172.21.15.55:0/2399303497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce8102a60 0x7f7ce8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.276+0000 7f7ced259640 1 --2- 172.21.15.55:0/2399303497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 0x7f7ce8101c90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.276+0000 7f7ced259640 1 -- 172.21.15.55:0/2399303497 >> 172.21.15.55:0/2399303497 conn(0x7f7ce80fcde0 msgr2=0x7f7ce80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.276+0000 7f7ced259640 1 -- 172.21.15.55:0/2399303497 shutdown_connections 2024-04-26T21:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.276+0000 7f7ced259640 1 -- 172.21.15.55:0/2399303497 wait complete. 2024-04-26T21:52:21.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ced259640 1 Processor -- start 2024-04-26T21:52:21.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ced259640 1 -- start start 2024-04-26T21:52:21.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ced259640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 0x7f7ce8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ced259640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce8102a60 0x7f7ce8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ced259640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ce8103420 0x7f7ce81a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ced259640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810d3a0 con 0x7f7ce8101870 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ced259640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810d220 con 0x7f7ce8102a60 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ced259640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810d520 con 0x7f7ce8103420 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 0x7f7ce8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 0x7f7ce8198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49618/0 (socket says 172.21.15.55:49618) 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce6d9d640 1 -- 172.21.15.55:0/1251330987 learned_addr learned my addr 172.21.15.55:0/1251330987 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce759e640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ce8103420 0x7f7ce81a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce659c640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce8102a60 0x7f7ce8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce6d9d640 1 -- 172.21.15.55:0/1251330987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ce8103420 msgr2=0x7f7ce81a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce6d9d640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ce8103420 0x7f7ce81a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce6d9d640 1 -- 172.21.15.55:0/1251330987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce8102a60 msgr2=0x7f7ce8199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce6d9d640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce8102a60 0x7f7ce8199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce6d9d640 1 -- 172.21.15.55:0/1251330987 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce81a0810 con 0x7f7ce8101870 2024-04-26T21:52:21.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.277+0000 7f7ce659c640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce8102a60 0x7f7ce8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:21.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.278+0000 7f7ce6d9d640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 0x7f7ce8198bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0006010 tx=0x7f7ce00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:21.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.278+0000 7f7ccffff640 1 -- 172.21.15.55:0/1251330987 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce003f030 con 0x7f7ce8101870 2024-04-26T21:52:21.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.278+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce81a0aa0 con 0x7f7ce8101870 2024-04-26T21:52:21.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.278+0000 7f7ccffff640 1 -- 172.21.15.55:0/1251330987 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce0003cb0 con 0x7f7ce8101870 2024-04-26T21:52:21.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.278+0000 7f7ccffff640 1 -- 172.21.15.55:0/1251330987 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0035070 con 0x7f7ce8101870 2024-04-26T21:52:21.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.278+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce81a0fe0 con 0x7f7ce8101870 2024-04-26T21:52:21.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.279+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce81a11b0 con 0x7f7ce8101870 2024-04-26T21:52:21.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.280+0000 7f7ccffff640 1 -- 172.21.15.55:0/1251330987 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7ce0042030 con 0x7f7ce8101870 2024-04-26T21:52:21.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.281+0000 7f7ccffff640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cd00a3c40 0x7f7cd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.281+0000 7f7ce659c640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cd00a3c40 0x7f7cd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.281+0000 7f7ccffff640 1 -- 172.21.15.55:0/1251330987 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f7ce00f0ff0 con 0x7f7ce8101870 2024-04-26T21:52:21.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.281+0000 7f7ce659c640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cd00a3c40 0x7f7cd00a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc005d00 tx=0x7f7cdc005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:21.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.284+0000 7f7ccffff640 1 -- 172.21.15.55:0/1251330987 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7ce00bd8f0 con 0x7f7ce8101870 2024-04-26T21:52:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:21 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3466467948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:21 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2144125606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:21 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1529158330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.471+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ce81a1380 con 0x7f7ce8101870 2024-04-26T21:52:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.471+0000 7f7ccffff640 1 -- 172.21.15.55:0/1251330987 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f7ce00bd1f0 con 0x7f7ce8101870 2024-04-26T21:52:21.472 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T21:52:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cd00a3c40 msgr2=0x7f7cd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cd00a3c40 0x7f7cd00a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc005d00 tx=0x7f7cdc005ed0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 msgr2=0x7f7ce8198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 0x7f7ce8198bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0006010 tx=0x7f7ce00042f0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 shutdown_connections 2024-04-26T21:52:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cd00a3c40 0x7f7cd00a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ce8103420 0x7f7ce81a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ce8102a60 0x7f7ce8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 --2- 172.21.15.55:0/1251330987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ce8101870 0x7f7ce8198bf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.473+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 >> 172.21.15.55:0/1251330987 conn(0x7f7ce80fcde0 msgr2=0x7f7ce8069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.474+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 shutdown_connections 2024-04-26T21:52:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.474+0000 7f7ced259640 1 -- 172.21.15.55:0/1251330987 wait complete. 2024-04-26T21:52:21.485 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:21 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3466467948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:21 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2144125606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:21 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1529158330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:21.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:21 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3466467948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:21.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:21 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2144125606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:21.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:21 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1529158330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:21.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 -- 172.21.15.55:0/4137082299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950102980 msgr2=0x7fb950102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 --2- 172.21.15.55:0/4137082299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950102980 0x7fb950102da0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb940009900 tx=0x7fb94002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 -- 172.21.15.55:0/4137082299 shutdown_connections 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 --2- 172.21.15.55:0/4137082299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb950104530 0x7fb95010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 --2- 172.21.15.55:0/4137082299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950103b70 0x7fb950103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 --2- 172.21.15.55:0/4137082299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950102980 0x7fb950102da0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 -- 172.21.15.55:0/4137082299 >> 172.21.15.55:0/4137082299 conn(0x7fb9500fdf10 msgr2=0x7fb950100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 -- 172.21.15.55:0/4137082299 shutdown_connections 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.586+0000 7fb956259640 1 -- 172.21.15.55:0/4137082299 wait complete. 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb956259640 1 Processor -- start 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb956259640 1 -- start start 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb956259640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb950102980 0x7fb950198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb956259640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950103b70 0x7fb9501991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb956259640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950104530 0x7fb9501a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb956259640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb95010e500 con 0x7fb950104530 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb956259640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb95010e380 con 0x7fb950102980 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb956259640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb95010e680 con 0x7fb950103b70 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950104530 0x7fb9501a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950104530 0x7fb9501a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49622/0 (socket says 172.21.15.55:49622) 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94ffff640 1 -- 172.21.15.55:0/2788571301 learned_addr learned my addr 172.21.15.55:0/2788571301 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94f7fe640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb950102980 0x7fb950198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94effd640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950103b70 0x7fb9501991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94ffff640 1 -- 172.21.15.55:0/2788571301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950103b70 msgr2=0x7fb9501991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94ffff640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950103b70 0x7fb9501991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94ffff640 1 -- 172.21.15.55:0/2788571301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb950102980 msgr2=0x7fb950198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94ffff640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb950102980 0x7fb950198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.587+0000 7fb94ffff640 1 -- 172.21.15.55:0/2788571301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9501a0880 con 0x7fb950104530 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.588+0000 7fb94ffff640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950104530 0x7fb9501a01e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb94800eaa0 tx=0x7fb94800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.588+0000 7fb94cff9640 1 -- 172.21.15.55:0/2788571301 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94800cdb0 con 0x7fb950104530 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.588+0000 7fb94cff9640 1 -- 172.21.15.55:0/2788571301 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb948005020 con 0x7fb950104530 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.588+0000 7fb94cff9640 1 -- 172.21.15.55:0/2788571301 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480116d0 con 0x7fb950104530 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.588+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9501a0b70 con 0x7fb950104530 2024-04-26T21:52:21.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.588+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9501a1070 con 0x7fb950104530 2024-04-26T21:52:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.589+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb950109d70 con 0x7fb950104530 2024-04-26T21:52:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.590+0000 7fb94cff9640 1 -- 172.21.15.55:0/2788571301 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb9480051d0 con 0x7fb950104530 2024-04-26T21:52:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.591+0000 7fb94cff9640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9380a3c40 0x7fb9380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.591+0000 7fb94f7fe640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9380a3c40 0x7fb9380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.591+0000 7fb94cff9640 1 -- 172.21.15.55:0/2788571301 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fb9480d0c30 con 0x7fb950104530 2024-04-26T21:52:21.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.591+0000 7fb94f7fe640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9380a3c40 0x7fb9380a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb95019a140 tx=0x7fb9400314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:21.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.594+0000 7fb94cff9640 1 -- 172.21.15.55:0/2788571301 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb948016030 con 0x7fb950104530 2024-04-26T21:52:21.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.819+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb95002dad0 con 0x7fb950104530 2024-04-26T21:52:21.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.820+0000 7fb94cff9640 1 -- 172.21.15.55:0/2788571301 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11863 (secure 0 0 0) 0x7fb94809d490 con 0x7fb950104530 2024-04-26T21:52:21.820 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:21.820 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:51:01.298886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:52:21.820 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 20 2024-04-26T21:52:21.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9380a3c40 msgr2=0x7fb9380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9380a3c40 0x7fb9380a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb95019a140 tx=0x7fb9400314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950104530 msgr2=0x7fb9501a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950104530 0x7fb9501a01e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb94800eaa0 tx=0x7fb94800ef60 comp rx=0 tx=0).stop 2024-04-26T21:52:21.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 shutdown_connections 2024-04-26T21:52:21.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9380a3c40 0x7fb9380a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb950104530 0x7fb9501a01e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950103b70 0x7fb9501991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 --2- 172.21.15.55:0/2788571301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb950102980 0x7fb950198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 >> 172.21.15.55:0/2788571301 conn(0x7fb9500fdf10 msgr2=0x7fb950106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:21.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 shutdown_connections 2024-04-26T21:52:21.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.822+0000 7fb956259640 1 -- 172.21.15.55:0/2788571301 wait complete. 2024-04-26T21:52:21.833 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:21.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 -- 172.21.15.55:0/2964773884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500105da0 msgr2=0x7f9500108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 --2- 172.21.15.55:0/2964773884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500105da0 0x7f9500108190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009900 tx=0x7f94f002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:21.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 -- 172.21.15.55:0/2964773884 shutdown_connections 2024-04-26T21:52:21.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 --2- 172.21.15.55:0/2964773884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95001086d0 0x7f950010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 --2- 172.21.15.55:0/2964773884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500105da0 0x7f9500108190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 --2- 172.21.15.55:0/2964773884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500103470 0x7f9500105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 -- 172.21.15.55:0/2964773884 >> 172.21.15.55:0/2964773884 conn(0x7f95000fce20 msgr2=0x7f95000ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 -- 172.21.15.55:0/2964773884 shutdown_connections 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 -- 172.21.15.55:0/2964773884 wait complete. 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 Processor -- start 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 -- start start 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500103470 0x7f9500198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.935+0000 7f950847e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9500105da0 0x7f95001990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950847e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95001086d0 0x7f95001a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950847e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f950010e4e0 con 0x7f9500103470 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950847e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f950010e360 con 0x7f95001086d0 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950847e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f950010e660 con 0x7f9500105da0 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500103470 0x7f9500198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500103470 0x7f9500198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49642/0 (socket says 172.21.15.55:49642) 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 -- 172.21.15.55:0/1167332445 learned_addr learned my addr 172.21.15.55:0/1167332445 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f9505a19640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9500105da0 0x7f95001990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f9506a1b640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95001086d0 0x7f95001a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 -- 172.21.15.55:0/1167332445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9500105da0 msgr2=0x7f95001990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9500105da0 0x7f95001990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 -- 172.21.15.55:0/1167332445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95001086d0 msgr2=0x7f95001a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95001086d0 0x7f95001a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 -- 172.21.15.55:0/1167332445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95001a07b0 con 0x7f9500103470 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950621a640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500103470 0x7f9500198b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00c920 tx=0x7f94fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f94f77fe640 1 -- 172.21.15.55:0/1167332445 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc007c70 con 0x7f9500103470 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f94f77fe640 1 -- 172.21.15.55:0/1167332445 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94fc007e10 con 0x7f9500103470 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95001a0aa0 con 0x7f9500103470 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f94f77fe640 1 -- 172.21.15.55:0/1167332445 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc018820 con 0x7f9500103470 2024-04-26T21:52:21.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.936+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95001a0fc0 con 0x7f9500103470 2024-04-26T21:52:21.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.937+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95000fe910 con 0x7f9500103470 2024-04-26T21:52:21.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.938+0000 7f94f77fe640 1 -- 172.21.15.55:0/1167332445 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f94fc00f450 con 0x7f9500103470 2024-04-26T21:52:21.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.939+0000 7f94f77fe640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f94ec0a3c40 0x7f94ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:21.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.939+0000 7f94f77fe640 1 -- 172.21.15.55:0/1167332445 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f94fc0d09d0 con 0x7f9500103470 2024-04-26T21:52:21.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.939+0000 7f9505a19640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f94ec0a3c40 0x7f94ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:21.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.940+0000 7f9505a19640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f94ec0a3c40 0x7f94ec0a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006010 tx=0x7f94f00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:21.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:21.943+0000 7f94f77fe640 1 -- 172.21.15.55:0/1167332445 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f94fc014030 con 0x7f9500103470 2024-04-26T21:52:22.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.130+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f950002dd60 con 0x7f9500103470 2024-04-26T21:52:22.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.132+0000 7f94f77fe640 1 -- 172.21.15.55:0/1167332445 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22713 (secure 0 0 0) 0x7f94fc09d2c0 con 0x7f9500103470 2024-04-26T21:52:22.132 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:22.132 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:51:50.534100+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:22.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.133+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f94ec0a3c40 msgr2=0x7f94ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.133+0000 7f950847e640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f94ec0a3c40 0x7f94ec0a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94f0006010 tx=0x7f94f00314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.133+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500103470 msgr2=0x7f9500198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.133+0000 7f950847e640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500103470 0x7f9500198b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00c920 tx=0x7f94fc00cde0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.134+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 shutdown_connections 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.134+0000 7f950847e640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f94ec0a3c40 0x7f94ec0a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.134+0000 7f950847e640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95001086d0 0x7f95001a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.134+0000 7f950847e640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9500105da0 0x7f95001990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.134+0000 7f950847e640 1 --2- 172.21.15.55:0/1167332445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9500103470 0x7f9500198b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.134+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 >> 172.21.15.55:0/1167332445 conn(0x7f95000fce20 msgr2=0x7f9500103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.134+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 shutdown_connections 2024-04-26T21:52:22.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.134+0000 7f950847e640 1 -- 172.21.15.55:0/1167332445 wait complete. 2024-04-26T21:52:22.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:52:22.146 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:52:22.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.250+0000 7fcf01be6640 1 -- 172.21.15.55:0/1027466591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 msgr2=0x7fcefc10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.250+0000 7fcf01be6640 1 --2- 172.21.15.55:0/1027466591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 0x7fcefc10aac0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009ca0 tx=0x7fcef402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.250+0000 7fcf01be6640 1 -- 172.21.15.55:0/1027466591 shutdown_connections 2024-04-26T21:52:22.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.250+0000 7fcf01be6640 1 --2- 172.21.15.55:0/1027466591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 0x7fcefc10aac0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.250+0000 7fcf01be6640 1 --2- 172.21.15.55:0/1027466591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc105d40 0x7fcefc108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.250+0000 7fcf01be6640 1 --2- 172.21.15.55:0/1027466591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcefc103410 0x7fcefc105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.250+0000 7fcf01be6640 1 -- 172.21.15.55:0/1027466591 >> 172.21.15.55:0/1027466591 conn(0x7fcefc0fce20 msgr2=0x7fcefc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:22.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 -- 172.21.15.55:0/1027466591 shutdown_connections 2024-04-26T21:52:22.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 -- 172.21.15.55:0/1027466591 wait complete. 2024-04-26T21:52:22.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 Processor -- start 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 -- start start 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc103410 0x7fcefc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcefc105d40 0x7fcefc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 0x7fcefc1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc10e600 con 0x7fcefc108670 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc10e480 con 0x7fcefc105d40 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.251+0000 7fcf01be6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc10e780 con 0x7fcefc103410 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 0x7fcefc1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 0x7fcefc1a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49666/0 (socket says 172.21.15.55:49666) 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 -- 172.21.15.55:0/2728101952 learned_addr learned my addr 172.21.15.55:0/2728101952 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefaffd640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcefc105d40 0x7fcefc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefb7fe640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc103410 0x7fcefc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 -- 172.21.15.55:0/2728101952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc103410 msgr2=0x7fcefc198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc103410 0x7fcefc198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 -- 172.21.15.55:0/2728101952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcefc105d40 msgr2=0x7fcefc199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcefc105d40 0x7fcefc199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 -- 172.21.15.55:0/2728101952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc1a0940 con 0x7fcefc108670 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefb7fe640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc103410 0x7fcefc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcefbfff640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 0x7fcefc1a02a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcef4033040 tx=0x7fcef4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcef8ff9640 1 -- 172.21.15.55:0/2728101952 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4004040 con 0x7fcefc108670 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcef8ff9640 1 -- 172.21.15.55:0/2728101952 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef4004690 con 0x7fcefc108670 2024-04-26T21:52:22.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcef8ff9640 1 -- 172.21.15.55:0/2728101952 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef402fe70 con 0x7fcefc108670 2024-04-26T21:52:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc1a0bd0 con 0x7fcefc108670 2024-04-26T21:52:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.252+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc1a1120 con 0x7fcefc108670 2024-04-26T21:52:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.253+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcefc0fe910 con 0x7fcefc108670 2024-04-26T21:52:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.254+0000 7fcef8ff9640 1 -- 172.21.15.55:0/2728101952 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcef402f610 con 0x7fcefc108670 2024-04-26T21:52:22.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.255+0000 7fcef8ff9640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcee40a3c40 0x7fcee40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.255+0000 7fcefb7fe640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcee40a3c40 0x7fcee40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.255+0000 7fcef8ff9640 1 -- 172.21.15.55:0/2728101952 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fcef40f1e30 con 0x7fcefc108670 2024-04-26T21:52:22.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.256+0000 7fcefb7fe640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcee40a3c40 0x7fcee40a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fceec006fd0 tx=0x7fceec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:22.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.260+0000 7fcef8ff9640 1 -- 172.21.15.55:0/2728101952 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcef40be740 con 0x7fcefc108670 2024-04-26T21:52:22.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:22 smithi055 ceph-mon[29221]: pgmap v333: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:22.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:22 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1251330987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:22.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:22 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2788571301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:22.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:22 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1167332445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:22.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.443+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcefc02dd20 con 0x7fcefc108670 2024-04-26T21:52:22.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.444+0000 7fcef8ff9640 1 -- 172.21.15.55:0/2728101952 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fcef40be040 con 0x7fcefc108670 2024-04-26T21:52:22.444 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcee40a3c40 msgr2=0x7fcee40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcee40a3c40 0x7fcee40a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fceec006fd0 tx=0x7fceec008040 comp rx=0 tx=0).stop 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 msgr2=0x7fcefc1a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 0x7fcefc1a02a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcef4033040 tx=0x7fcef4003ba0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 shutdown_connections 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcee40a3c40 0x7fcee40a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcefc108670 0x7fcefc1a02a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcefc105d40 0x7fcefc199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 --2- 172.21.15.55:0/2728101952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcefc103410 0x7fcefc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 >> 172.21.15.55:0/2728101952 conn(0x7fcefc0fce20 msgr2=0x7fcefc103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 shutdown_connections 2024-04-26T21:52:22.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.446+0000 7fcf01be6640 1 -- 172.21.15.55:0/2728101952 wait complete. 2024-04-26T21:52:22.458 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:52:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:22 smithi064 ceph-mon[32765]: pgmap v333: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:22 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1251330987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:22 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2788571301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:22.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:22 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1167332445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 -- 172.21.15.55:0/3707949127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc102990 msgr2=0x7fc2fc102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 --2- 172.21.15.55:0/3707949127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc102990 0x7fc2fc102db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009900 tx=0x7fc2f002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 -- 172.21.15.55:0/3707949127 shutdown_connections 2024-04-26T21:52:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 --2- 172.21.15.55:0/3707949127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2fc104540 0x7fc2fc10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 --2- 172.21.15.55:0/3707949127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc103b80 0x7fc2fc104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 --2- 172.21.15.55:0/3707949127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc102990 0x7fc2fc102db0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 -- 172.21.15.55:0/3707949127 >> 172.21.15.55:0/3707949127 conn(0x7fc2fc0fdf00 msgr2=0x7fc2fc100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 -- 172.21.15.55:0/3707949127 shutdown_connections 2024-04-26T21:52:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.562+0000 7fc301a99640 1 -- 172.21.15.55:0/3707949127 wait complete. 2024-04-26T21:52:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc301a99640 1 Processor -- start 2024-04-26T21:52:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc301a99640 1 -- start start 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc301a99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc102990 0x7fc2fc0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc301a99640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2fc103b80 0x7fc2fc074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc301a99640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc104540 0x7fc2fc075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc301a99640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc10e490 con 0x7fc2fc104540 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc301a99640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc10e310 con 0x7fc2fc102990 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc301a99640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2fc10e610 con 0x7fc2fc103b80 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc2fbfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc104540 0x7fc2fc075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc2fbfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc104540 0x7fc2fc075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49680/0 (socket says 172.21.15.55:49680) 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc2fbfff640 1 -- 172.21.15.55:0/3173913480 learned_addr learned my addr 172.21.15.55:0/3173913480 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.563+0000 7fc2fb7fe640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc102990 0x7fc2fc0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2faffd640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2fc103b80 0x7fc2fc074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2fbfff640 1 -- 172.21.15.55:0/3173913480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2fc103b80 msgr2=0x7fc2fc074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2fbfff640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2fc103b80 0x7fc2fc074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2fbfff640 1 -- 172.21.15.55:0/3173913480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc102990 msgr2=0x7fc2fc0766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2fbfff640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc102990 0x7fc2fc0766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2fbfff640 1 -- 172.21.15.55:0/3173913480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2fc075af0 con 0x7fc2fc104540 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2fbfff640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc104540 0x7fc2fc075260 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400eaa0 tx=0x7fc2f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2f8ff9640 1 -- 172.21.15.55:0/3173913480 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f400cdb0 con 0x7fc2fc104540 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2fc1a2cf0 con 0x7fc2fc104540 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2f8ff9640 1 -- 172.21.15.55:0/3173913480 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f4005020 con 0x7fc2fc104540 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc2f8ff9640 1 -- 172.21.15.55:0/3173913480 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40116d0 con 0x7fc2fc104540 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.564+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2fc1a3190 con 0x7fc2fc104540 2024-04-26T21:52:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.565+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2fc109dd0 con 0x7fc2fc104540 2024-04-26T21:52:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.566+0000 7fc2f8ff9640 1 -- 172.21.15.55:0/3173913480 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc2f40051d0 con 0x7fc2fc104540 2024-04-26T21:52:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.567+0000 7fc2f8ff9640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2e40a3c40 0x7fc2e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.567+0000 7fc2fb7fe640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2e40a3c40 0x7fc2e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.567+0000 7fc2f8ff9640 1 -- 172.21.15.55:0/3173913480 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc2f40d07f0 con 0x7fc2fc104540 2024-04-26T21:52:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.567+0000 7fc2fb7fe640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2e40a3c40 0x7fc2e40a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009230 tx=0x7fc2f00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:22.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.571+0000 7fc2f8ff9640 1 -- 172.21.15.55:0/3173913480 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc2f4016030 con 0x7fc2fc104540 2024-04-26T21:52:22.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:22 smithi165 ceph-mon[33296]: pgmap v333: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:22.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:22 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1251330987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:22.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:22 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2788571301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:22.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:22 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1167332445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:22.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.756+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc2fc02dc80 con 0x7fc2fc104540 2024-04-26T21:52:22.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.756+0000 7fc2f8ff9640 1 -- 172.21.15.55:0/3173913480 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fc2f409d050 con 0x7fc2fc104540 2024-04-26T21:52:22.757 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2e40a3c40 msgr2=0x7fc2e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2e40a3c40 0x7fc2e40a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009230 tx=0x7fc2f00314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc104540 msgr2=0x7fc2fc075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc104540 0x7fc2fc075260 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400eaa0 tx=0x7fc2f400ef60 comp rx=0 tx=0).stop 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 shutdown_connections 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2e40a3c40 0x7fc2e40a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2fc104540 0x7fc2fc075260 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2fc103b80 0x7fc2fc074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 --2- 172.21.15.55:0/3173913480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2fc102990 0x7fc2fc0766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 >> 172.21.15.55:0/3173913480 conn(0x7fc2fc0fdf00 msgr2=0x7fc2fc106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:22.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 shutdown_connections 2024-04-26T21:52:22.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.758+0000 7fc301a99640 1 -- 172.21.15.55:0/3173913480 wait complete. 2024-04-26T21:52:22.770 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:52:22.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.874+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/1980731366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44102a80 msgr2=0x7f4f44102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.874+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/1980731366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44102a80 0x7f4f44102f00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009900 tx=0x7f4f3802a410 comp rx=0 tx=0).stop 2024-04-26T21:52:22.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.874+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/1980731366 shutdown_connections 2024-04-26T21:52:22.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.874+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/1980731366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f44103440 0x7f4f44109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.874+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/1980731366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44102a80 0x7f4f44102f00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.875+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/1980731366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f44101890 0x7f4f44101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.875+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/1980731366 >> 172.21.15.55:0/1980731366 conn(0x7f4f440fce20 msgr2=0x7f4f440ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:22.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.875+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/1980731366 shutdown_connections 2024-04-26T21:52:22.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.875+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/1980731366 wait complete. 2024-04-26T21:52:22.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.875+0000 7f4f4a8b1640 1 Processor -- start 2024-04-26T21:52:22.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.875+0000 7f4f4a8b1640 1 -- start start 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.875+0000 7f4f4a8b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44101890 0x7f4f44198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.875+0000 7f4f4a8b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f44102a80 0x7f4f441990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f4a8b1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f44103440 0x7f4f441a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f4a8b1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4410e4e0 con 0x7f4f44101890 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f4a8b1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4410e360 con 0x7f4f44102a80 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f4a8b1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4410e660 con 0x7f4f44103440 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44101890 0x7f4f44198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44101890 0x7f4f44198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49694/0 (socket says 172.21.15.55:49694) 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 -- 172.21.15.55:0/3648576936 learned_addr learned my addr 172.21.15.55:0/3648576936 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:22.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f437fe640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f44102a80 0x7f4f441990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 -- 172.21.15.55:0/3648576936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f44103440 msgr2=0x7f4f441a0110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f44103440 0x7f4f441a0110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 -- 172.21.15.55:0/3648576936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f44102a80 msgr2=0x7f4f441990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f44102a80 0x7f4f441990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 -- 172.21.15.55:0/3648576936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f441a0780 con 0x7f4f44101890 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f437fe640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f44102a80 0x7f4f441990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f43fff640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44101890 0x7f4f44198b60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c970 tx=0x7f4f3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f417fa640 1 -- 172.21.15.55:0/3648576936 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34007c10 con 0x7f4f44101890 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f441a0a70 con 0x7f4f44101890 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f417fa640 1 -- 172.21.15.55:0/3648576936 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34007db0 con 0x7f4f44101890 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f417fa640 1 -- 172.21.15.55:0/3648576936 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3400f870 con 0x7f4f44101890 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.876+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f441a1010 con 0x7f4f44101890 2024-04-26T21:52:22.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.877+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f44108c80 con 0x7f4f44101890 2024-04-26T21:52:22.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.879+0000 7f4f417fa640 1 -- 172.21.15.55:0/3648576936 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4f3400fa70 con 0x7f4f44101890 2024-04-26T21:52:22.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.879+0000 7f4f417fa640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4f2c0a3c40 0x7f4f2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:22.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.879+0000 7f4f437fe640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4f2c0a3c40 0x7f4f2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:22.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.879+0000 7f4f417fa640 1 -- 172.21.15.55:0/3648576936 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4f340d0ca0 con 0x7f4f44101890 2024-04-26T21:52:22.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.880+0000 7f4f437fe640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4f2c0a3c40 0x7f4f2c0a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f4419a070 tx=0x7f4f380314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:22.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:22.883+0000 7f4f417fa640 1 -- 172.21.15.55:0/3648576936 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4f3409d620 con 0x7f4f44101890 2024-04-26T21:52:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.068+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f4402dd30 con 0x7f4f44101890 2024-04-26T21:52:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.069+0000 7f4f417fa640 1 -- 172.21.15.55:0/3648576936 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f4f3409cf20 con 0x7f4f44101890 2024-04-26T21:52:23.069 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T21:52:23.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4f2c0a3c40 msgr2=0x7f4f2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4f2c0a3c40 0x7f4f2c0a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4f4419a070 tx=0x7f4f380314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44101890 msgr2=0x7f4f44198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44101890 0x7f4f44198b60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400c970 tx=0x7f4f3400ce30 comp rx=0 tx=0).stop 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 shutdown_connections 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4f2c0a3c40 0x7f4f2c0a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f44103440 0x7f4f441a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f44102a80 0x7f4f441990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 --2- 172.21.15.55:0/3648576936 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f44101890 0x7f4f44198b60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 >> 172.21.15.55:0/3648576936 conn(0x7f4f440fce20 msgr2=0x7f4f44105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 shutdown_connections 2024-04-26T21:52:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.071+0000 7f4f4a8b1640 1 -- 172.21.15.55:0/3648576936 wait complete. 2024-04-26T21:52:23.083 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:23.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.183+0000 7f62a6ef1640 1 -- 172.21.15.55:0/1505971630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 msgr2=0x7f62a010add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.183+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/1505971630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 0x7f62a010add0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6290009900 tx=0x7f629002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:23.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 -- 172.21.15.55:0/1505971630 shutdown_connections 2024-04-26T21:52:23.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/1505971630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 0x7f62a010add0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/1505971630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0103b80 0x7f62a0104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/1505971630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62a0102990 0x7f62a0102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 -- 172.21.15.55:0/1505971630 >> 172.21.15.55:0/1505971630 conn(0x7f62a00fdee0 msgr2=0x7f62a0100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 -- 172.21.15.55:0/1505971630 shutdown_connections 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 -- 172.21.15.55:0/1505971630 wait complete. 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 Processor -- start 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 -- start start 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62a0102990 0x7f62a0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0103b80 0x7f62a0199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 0x7f62a01a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f62a010e4a0 con 0x7f62a0104540 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62a010e320 con 0x7f62a0102990 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a6ef1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f62a010e620 con 0x7f62a0103b80 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a548e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 0x7f62a01a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a548e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 0x7f62a01a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49702/0 (socket says 172.21.15.55:49702) 2024-04-26T21:52:23.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.184+0000 7f62a548e640 1 -- 172.21.15.55:0/936116908 learned_addr learned my addr 172.21.15.55:0/936116908 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a4c8d640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62a0102990 0x7f62a0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f629ffff640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0103b80 0x7f62a0199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a548e640 1 -- 172.21.15.55:0/936116908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0103b80 msgr2=0x7f62a0199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a548e640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0103b80 0x7f62a0199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a548e640 1 -- 172.21.15.55:0/936116908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62a0102990 msgr2=0x7f62a0198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a548e640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62a0102990 0x7f62a0198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a548e640 1 -- 172.21.15.55:0/936116908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a01a0760 con 0x7f62a0104540 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a548e640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 0x7f62a01a00f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6290031500 tx=0x7f6290031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f629dffb640 1 -- 172.21.15.55:0/936116908 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629003f030 con 0x7f62a0104540 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f629dffb640 1 -- 172.21.15.55:0/936116908 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6290003aa0 con 0x7f62a0104540 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f629dffb640 1 -- 172.21.15.55:0/936116908 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290035070 con 0x7f62a0104540 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a01a09f0 con 0x7f62a0104540 2024-04-26T21:52:23.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.185+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a01a0f30 con 0x7f62a0104540 2024-04-26T21:52:23.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.186+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a0109cb0 con 0x7f62a0104540 2024-04-26T21:52:23.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.187+0000 7f629dffb640 1 -- 172.21.15.55:0/936116908 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6290042030 con 0x7f62a0104540 2024-04-26T21:52:23.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.188+0000 7f629dffb640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f62880a3cc0 0x7f62880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.188+0000 7f62a4c8d640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f62880a3cc0 0x7f62880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.188+0000 7f629dffb640 1 -- 172.21.15.55:0/936116908 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f62900f0af0 con 0x7f62a0104540 2024-04-26T21:52:23.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.189+0000 7f62a4c8d640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f62880a3cc0 0x7f62880a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f62a01039e0 tx=0x7f6298009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:23.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.193+0000 7f629dffb640 1 -- 172.21.15.55:0/936116908 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f62900bd350 con 0x7f62a0104540 2024-04-26T21:52:23.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.420+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a002dcc0 con 0x7f62a0104540 2024-04-26T21:52:23.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.421+0000 7f629dffb640 1 -- 172.21.15.55:0/936116908 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11863 (secure 0 0 0) 0x7f62900bcc50 con 0x7f62a0104540 2024-04-26T21:52:23.421 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:23.421 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:51:01.298886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:52:23.422 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 20 2024-04-26T21:52:23.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f62880a3cc0 msgr2=0x7f62880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f62880a3cc0 0x7f62880a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f62a01039e0 tx=0x7f6298009670 comp rx=0 tx=0).stop 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 msgr2=0x7f62a01a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 0x7f62a01a00f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6290031500 tx=0x7f6290031a50 comp rx=0 tx=0).stop 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 shutdown_connections 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f62880a3cc0 0x7f62880a6180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62a0104540 0x7f62a01a00f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0103b80 0x7f62a0199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 --2- 172.21.15.55:0/936116908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62a0102990 0x7f62a0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 >> 172.21.15.55:0/936116908 conn(0x7f62a00fdee0 msgr2=0x7f62a0106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 shutdown_connections 2024-04-26T21:52:23.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.423+0000 7f62a6ef1640 1 -- 172.21.15.55:0/936116908 wait complete. 2024-04-26T21:52:23.433 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:23 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2728101952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.433 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:23 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3173913480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.433 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:23 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3648576936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.434 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:23 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2728101952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:23 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3173913480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:23 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3648576936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1335139912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 msgr2=0x7f23e8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1335139912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 0x7f23e8104010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009900 tx=0x7f23dc02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1335139912 shutdown_connections 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1335139912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e8104550 0x7f23e810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1335139912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 0x7f23e8104010 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1335139912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e81029a0 0x7f23e8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1335139912 >> 172.21.15.55:0/1335139912 conn(0x7f23e80fdf30 msgr2=0x7f23e8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1335139912 shutdown_connections 2024-04-26T21:52:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.535+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1335139912 wait complete. 2024-04-26T21:52:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23ee1f7640 1 Processor -- start 2024-04-26T21:52:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23ee1f7640 1 -- start start 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23ee1f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e81029a0 0x7f23e80766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23ee1f7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 0x7f23e8074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23ee1f7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e8104550 0x7f23e8075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23ee1f7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f23e810e4d0 con 0x7f23e8103b90 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23ee1f7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23e810e350 con 0x7f23e81029a0 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23ee1f7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23e810e650 con 0x7f23e8104550 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23e6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 0x7f23e8074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23e6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 0x7f23e8074d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49706/0 (socket says 172.21.15.55:49706) 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23e6ffd640 1 -- 172.21.15.55:0/1774154544 learned_addr learned my addr 172.21.15.55:0/1774154544 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:23.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23e77fe640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e81029a0 0x7f23e80766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.536+0000 7f23e7fff640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e8104550 0x7f23e8075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e6ffd640 1 -- 172.21.15.55:0/1774154544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e8104550 msgr2=0x7f23e8075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e6ffd640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e8104550 0x7f23e8075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e6ffd640 1 -- 172.21.15.55:0/1774154544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e81029a0 msgr2=0x7f23e80766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e6ffd640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e81029a0 0x7f23e80766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e6ffd640 1 -- 172.21.15.55:0/1774154544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e8075af0 con 0x7f23e8103b90 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e6ffd640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 0x7f23e8074d20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f23dc005940 tx=0x7f23dc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e4ff9640 1 -- 172.21.15.55:0/1774154544 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc03f030 con 0x7f23e8103b90 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e4ff9640 1 -- 172.21.15.55:0/1774154544 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23dc003cb0 con 0x7f23e8103b90 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e81a2be0 con 0x7f23e8103b90 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23e4ff9640 1 -- 172.21.15.55:0/1774154544 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc035070 con 0x7f23e8103b90 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.537+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e81a3130 con 0x7f23e8103b90 2024-04-26T21:52:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.538+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e8109cc0 con 0x7f23e8103b90 2024-04-26T21:52:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.539+0000 7f23e4ff9640 1 -- 172.21.15.55:0/1774154544 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f23dc042030 con 0x7f23e8103b90 2024-04-26T21:52:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.540+0000 7f23e4ff9640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23d00a3c40 0x7f23d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.540+0000 7f23e77fe640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23d00a3c40 0x7f23d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.540+0000 7f23e4ff9640 1 -- 172.21.15.55:0/1774154544 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f23dc0f0f80 con 0x7f23e8103b90 2024-04-26T21:52:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.540+0000 7f23e77fe640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23d00a3c40 0x7f23d00a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f23e81039f0 tx=0x7f23d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:23.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.545+0000 7f23e4ff9640 1 -- 172.21.15.55:0/1774154544 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f23dc003e60 con 0x7f23e8103b90 2024-04-26T21:52:23.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:23 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2728101952' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:23 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3173913480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:23 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3648576936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:23.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.730+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23e802dd10 con 0x7f23e8103b90 2024-04-26T21:52:23.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.731+0000 7f23e4ff9640 1 -- 172.21.15.55:0/1774154544 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22713 (secure 0 0 0) 0x7f23dc0bd780 con 0x7f23e8103b90 2024-04-26T21:52:23.732 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:23.732 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:51:50.534100+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:23.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23d00a3c40 msgr2=0x7f23d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23d00a3c40 0x7f23d00a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f23e81039f0 tx=0x7f23d8005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:23.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 msgr2=0x7f23e8074d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 0x7f23e8074d20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f23dc005940 tx=0x7f23dc004430 comp rx=0 tx=0).stop 2024-04-26T21:52:23.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 shutdown_connections 2024-04-26T21:52:23.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23d00a3c40 0x7f23d00a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23e8104550 0x7f23e8075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23e8103b90 0x7f23e8074d20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 --2- 172.21.15.55:0/1774154544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23e81029a0 0x7f23e80766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.733+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 >> 172.21.15.55:0/1774154544 conn(0x7f23e80fdf30 msgr2=0x7f23e8106e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:23.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.734+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 shutdown_connections 2024-04-26T21:52:23.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.734+0000 7f23ee1f7640 1 -- 172.21.15.55:0/1774154544 wait complete. 2024-04-26T21:52:23.745 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 -- 172.21.15.55:0/2811829556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f81029a0 msgr2=0x7ff4f8102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 --2- 172.21.15.55:0/2811829556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f81029a0 0x7ff4f8102dc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009900 tx=0x7ff4f402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 -- 172.21.15.55:0/2811829556 shutdown_connections 2024-04-26T21:52:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 --2- 172.21.15.55:0/2811829556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4f8104550 0x7ff4f810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 --2- 172.21.15.55:0/2811829556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4f8103b90 0x7ff4f8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 --2- 172.21.15.55:0/2811829556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f81029a0 0x7ff4f8102dc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 -- 172.21.15.55:0/2811829556 >> 172.21.15.55:0/2811829556 conn(0x7ff4f80fdf30 msgr2=0x7ff4f8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 -- 172.21.15.55:0/2811829556 shutdown_connections 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.846+0000 7ff5004c8640 1 -- 172.21.15.55:0/2811829556 wait complete. 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff5004c8640 1 Processor -- start 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff5004c8640 1 -- start start 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff5004c8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4f81029a0 0x7ff4f8198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff5004c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f8103b90 0x7ff4f81990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff5004c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4f8104550 0x7ff4f81a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff5004c8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810e4a0 con 0x7ff4f8103b90 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff5004c8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810e320 con 0x7ff4f8104550 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff5004c8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810e620 con 0x7ff4f81029a0 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fda63640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f8103b90 0x7ff4f81990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fda63640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f8103b90 0x7ff4f81990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49726/0 (socket says 172.21.15.55:49726) 2024-04-26T21:52:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fda63640 1 -- 172.21.15.55:0/1627864623 learned_addr learned my addr 172.21.15.55:0/1627864623 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fe264640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4f81029a0 0x7ff4f8198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fea65640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4f8104550 0x7ff4f81a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fda63640 1 -- 172.21.15.55:0/1627864623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4f81029a0 msgr2=0x7ff4f8198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fda63640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4f81029a0 0x7ff4f8198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fda63640 1 -- 172.21.15.55:0/1627864623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4f8104550 msgr2=0x7ff4f81a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fda63640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4f8104550 0x7ff4f81a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fda63640 1 -- 172.21.15.55:0/1627864623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f81a0790 con 0x7ff4f8103b90 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.847+0000 7ff4fea65640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4f8104550 0x7ff4f81a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.848+0000 7ff4fda63640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f8103b90 0x7ff4f81990b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800c890 tx=0x7ff4e800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.848+0000 7ff4ef7fe640 1 -- 172.21.15.55:0/1627864623 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004220 con 0x7ff4f8103b90 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.848+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f81a0a80 con 0x7ff4f8103b90 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.848+0000 7ff4ef7fe640 1 -- 172.21.15.55:0/1627864623 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e80043c0 con 0x7ff4f8103b90 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.848+0000 7ff4ef7fe640 1 -- 172.21.15.55:0/1627864623 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004a00 con 0x7ff4f8103b90 2024-04-26T21:52:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.848+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a1020 con 0x7ff4f8103b90 2024-04-26T21:52:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.849+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f8109d60 con 0x7ff4f8103b90 2024-04-26T21:52:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.850+0000 7ff4ef7fe640 1 -- 172.21.15.55:0/1627864623 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff4e80079b0 con 0x7ff4f8103b90 2024-04-26T21:52:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.851+0000 7ff4ef7fe640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4e40a3c40 0x7ff4e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.851+0000 7ff4fe264640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4e40a3c40 0x7ff4e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:23.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.851+0000 7ff4ef7fe640 1 -- 172.21.15.55:0/1627864623 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff4e80d06f0 con 0x7ff4f8103b90 2024-04-26T21:52:23.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.851+0000 7ff4fe264640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4e40a3c40 0x7ff4e40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009230 tx=0x7ff4f40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:23.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:23.854+0000 7ff4ef7fe640 1 -- 172.21.15.55:0/1627864623 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4e8014030 con 0x7ff4f8103b90 2024-04-26T21:52:24.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.078+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4f802dc80 con 0x7ff4f8103b90 2024-04-26T21:52:24.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.078+0000 7ff4ef7fe640 1 -- 172.21.15.55:0/1627864623 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11863 (secure 0 0 0) 0x7ff4e809cf50 con 0x7ff4f8103b90 2024-04-26T21:52:24.079 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:24.079 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:51:01.298886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:52:24.079 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 20 2024-04-26T21:52:24.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.080+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4e40a3c40 msgr2=0x7ff4e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.080+0000 7ff5004c8640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4e40a3c40 0x7ff4e40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009230 tx=0x7ff4f40314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.080+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f8103b90 msgr2=0x7ff4f81990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.080+0000 7ff5004c8640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f8103b90 0x7ff4f81990b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800c890 tx=0x7ff4e800cd50 comp rx=0 tx=0).stop 2024-04-26T21:52:24.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.081+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 shutdown_connections 2024-04-26T21:52:24.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.081+0000 7ff5004c8640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4e40a3c40 0x7ff4e40a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.081+0000 7ff5004c8640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4f8104550 0x7ff4f81a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.081+0000 7ff5004c8640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4f8103b90 0x7ff4f81990b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.081+0000 7ff5004c8640 1 --2- 172.21.15.55:0/1627864623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4f81029a0 0x7ff4f8198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.081+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 >> 172.21.15.55:0/1627864623 conn(0x7ff4f80fdf30 msgr2=0x7ff4f8106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.081+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 shutdown_connections 2024-04-26T21:52:24.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.081+0000 7ff5004c8640 1 -- 172.21.15.55:0/1627864623 wait complete. 2024-04-26T21:52:24.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:50:56.719940+0000', 'modified': '2024-04-26T21:51:01.298886+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24551, 'mds_1': 24509, 'mds_2': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/125466680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 125466680}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 125466680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6835/2328208855', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 2328208855}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 2328208855}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/2954631237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2954631237}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2954631237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.165:6839/1115440142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 1115440142}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 1115440142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.64:6837/4014142220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 4014142220}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 4014142220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6839/463662731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 463662731}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 463662731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24502, 24509, 24551]} 2024-04-26T21:52:24.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T21:52:24.093 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.194+0000 7f4e74bf2640 1 -- 172.21.15.55:0/117168842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70101870 msgr2=0x7f4e70101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.194+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/117168842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70101870 0x7f4e70101c90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009900 tx=0x7f4e6002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 -- 172.21.15.55:0/117168842 shutdown_connections 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/117168842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e70103420 0x7f4e70109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/117168842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70102a60 0x7f4e70102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/117168842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70101870 0x7f4e70101c90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 -- 172.21.15.55:0/117168842 >> 172.21.15.55:0/117168842 conn(0x7f4e700fcde0 msgr2=0x7f4e700ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 -- 172.21.15.55:0/117168842 shutdown_connections 2024-04-26T21:52:24.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 -- 172.21.15.55:0/117168842 wait complete. 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 Processor -- start 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.195+0000 7f4e74bf2640 1 -- start start 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e74bf2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e70101870 0x7f4e70198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e74bf2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70102a60 0x7f4e70199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e74bf2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70103420 0x7f4e701a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e74bf2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010d3f0 con 0x7f4e70103420 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e74bf2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010d270 con 0x7f4e70101870 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e74bf2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7010d570 con 0x7f4e70102a60 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70103420 0x7f4e701a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70103420 0x7f4e701a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49744/0 (socket says 172.21.15.55:49744) 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 -- 172.21.15.55:0/1787999900 learned_addr learned my addr 172.21.15.55:0/1787999900 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e70101870 0x7f4e70198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70102a60 0x7f4e70199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 -- 172.21.15.55:0/1787999900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70102a60 msgr2=0x7f4e70199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70102a60 0x7f4e70199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 -- 172.21.15.55:0/1787999900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e70101870 msgr2=0x7f4e70198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e70101870 0x7f4e70198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 -- 172.21.15.55:0/1787999900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e701a0900 con 0x7f4e70103420 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e6ed9d640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70103420 0x7f4e701a0260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800eaa0 tx=0x7f4e6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e5f7fe640 1 -- 172.21.15.55:0/1787999900 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800cdb0 con 0x7f4e70103420 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e5f7fe640 1 -- 172.21.15.55:0/1787999900 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e68005020 con 0x7f4e70103420 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e74bf2640 1 -- 172.21.15.55:0/1787999900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e701a0bf0 con 0x7f4e70103420 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e5f7fe640 1 -- 172.21.15.55:0/1787999900 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e680116d0 con 0x7f4e70103420 2024-04-26T21:52:24.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.196+0000 7f4e74bf2640 1 -- 172.21.15.55:0/1787999900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a10f0 con 0x7f4e70103420 2024-04-26T21:52:24.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.199+0000 7f4e5f7fe640 1 -- 172.21.15.55:0/1787999900 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4e680051d0 con 0x7f4e70103420 2024-04-26T21:52:24.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.199+0000 7f4e779e0740 1 Processor -- start 2024-04-26T21:52:24.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.199+0000 7f4e779e0740 1 -- start start 2024-04-26T21:52:24.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.199+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba51ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.199+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba56b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba571f0 0x55fa1ba59680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e779e0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fa1b9b6740 con 0x55fa1ba51690 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e779e0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa1b9523a0 con 0x55fa1ba52080 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e779e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa1b9b2700 con 0x55fa1ba571f0 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba51ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba51ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49758/0 (socket says 172.21.15.55:49758) 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e5f7fe640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e580a3c40 0x7f4e580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/3808720966 learned_addr learned my addr 172.21.15.55:0/3808720966 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba56b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6ed9d640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e580a3c40 0x7f4e580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba571f0 0x55fa1ba59680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/3808720966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba571f0 msgr2=0x55fa1ba59680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba571f0 0x55fa1ba59680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/3808720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 msgr2=0x55fa1ba56b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba56b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e5f7fe640 1 -- 172.21.15.55:0/1787999900 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4e680d08b0 con 0x7f4e70103420 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/3808720966 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa1b96e750 con 0x55fa1ba51690 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba51ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4e64007ba0 tx=0x7f4e640073e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f7eb29410405f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e6ed9d640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e580a3c40 0x7f4e580a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e680a50f0 tx=0x7f4e68087040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e5cff9640 1 -- 172.21.15.55:0/3808720966 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6401a070 con 0x55fa1ba51690 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e5cff9640 1 -- 172.21.15.55:0/3808720966 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e64010040 con 0x55fa1ba51690 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.200+0000 7f4e5cff9640 1 -- 172.21.15.55:0/3808720966 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e640145b0 con 0x55fa1ba51690 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 -- 172.21.15.55:0/3808720966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 msgr2=0x55fa1ba51ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba51ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4e64007ba0 tx=0x7f4e640073e0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 -- 172.21.15.55:0/3808720966 shutdown_connections 2024-04-26T21:52:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba571f0 0x55fa1ba59680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 --2- 172.21.15.55:0/3808720966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba51ab0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 -- 172.21.15.55:0/3808720966 >> 172.21.15.55:0/3808720966 conn(0x55fa1ba4cd00 msgr2=0x55fa1ba4d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 -- 172.21.15.55:0/3808720966 shutdown_connections 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.201+0000 7f4e779e0740 1 -- 172.21.15.55:0/3808720966 wait complete. 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 1 -- start start 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 1 Processor -- start 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 0x55fa1ba5bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba52080 0x55fa1ba5c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 0x55fa1ba63090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ba5a1a0 con 0x55fa1ba52080 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa1b93dfe0 con 0x55fa1ba571f0 2024-04-26T21:52:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ba5a320 con 0x55fa1ba51690 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e779e0740 10 client.0 _ll_register_callbacks cb 0x55fa1b954150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba52080 0x55fa1ba5c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba52080 0x55fa1ba5c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49760/0 (socket says 172.21.15.55:49760) 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 -- 172.21.15.55:0/282987369 learned_addr learned my addr 172.21.15.55:0/282987369 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 0x55fa1ba63090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 0x55fa1ba5bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 -- 172.21.15.55:0/282987369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 msgr2=0x55fa1ba5bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 0x55fa1ba5bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 -- 172.21.15.55:0/282987369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 msgr2=0x55fa1ba63090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 0x55fa1ba63090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 -- 172.21.15.55:0/282987369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa1b6c82d0 con 0x55fa1ba52080 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 0x55fa1ba63090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba52080 0x55fa1ba5c050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e701028c0 tx=0x7f4e60031e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6404c0ed59eba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.202+0000 7f4e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e55ffb640 1 -- 172.21.15.55:0/282987369 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60035070 con 0x55fa1ba52080 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e55ffb640 1 -- 172.21.15.55:0/282987369 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4e60004240 con 0x55fa1ba52080 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 1 -- 172.21.15.55:0/282987369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e64010040 con 0x55fa1ba52080 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e55ffb640 1 -- 172.21.15.55:0/282987369 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4e6004b070 con 0x55fa1ba52080 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 10 client.15030 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 1 -- 172.21.15.55:0/282987369 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55fa1b7926b0 con 0x55fa1ba52080 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e55ffb640 1 -- 172.21.15.55:0/282987369 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e64010040 con 0x55fa1ba52080 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e55ffb640 1 -- 172.21.15.55:0/282987369 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f4e6003b430 con 0x55fa1ba52080 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 10 client.15030 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 10 client.15030 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 20 client.15030 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fa1ba685b0 0x55fa1ba69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 4 client.15030 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:52:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e779e0740 1 -- 172.21.15.55:0/282987369 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa1b9c4350 con 0x55fa1ba685b0 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.203+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fa1ba685b0 0x55fa1ba69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.204+0000 7f4e55ffb640 10 client.15030 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.204+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fa1ba685b0 0x55fa1ba69a20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=635f432a78e9d7f6 server_cookie=ffa24e0d72df50c1 in_seq=0 out_seq=0 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e55ffb640 1 -- 172.21.15.55:0/282987369 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa1b9c4350 con 0x55fa1ba685b0 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e55ffb640 10 client.15030 handle_command_reply: tid=0 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 client.15030 shutdown 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 -- 172.21.15.55:0/282987369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba52080 msgr2=0x55fa1ba5c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba52080 0x55fa1ba5c050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4e701028c0 tx=0x7f4e60031e30 comp rx=0 tx=0).stop 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 -- 172.21.15.55:0/282987369 shutdown_connections 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fa1ba685b0 0x55fa1ba69a20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 0x55fa1ba63090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba52080 0x55fa1ba5c050 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e55ffb640 0 client.15030 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:52:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 --2- 172.21.15.55:0/282987369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 0x55fa1ba5bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 -- 172.21.15.55:0/282987369 >> 172.21.15.55:0/282987369 conn(0x55fa1ba4cd00 msgr2=0x55fa1ba4bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 -- 172.21.15.55:0/282987369 shutdown_connections 2024-04-26T21:52:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 1 -- 172.21.15.55:0/282987369 wait complete. 2024-04-26T21:52:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.205+0000 7f4e779e0740 20 client.15030 trim_cache size 0 max 16384 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e779e0740 1 Processor -- start 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e779e0740 1 -- start start 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba52080 0x55fa1ba53a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 0x55fa1ba63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e779e0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ba67d60 con 0x55fa1ba51690 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e779e0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ba68430 con 0x55fa1ba571f0 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e779e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ba68b00 con 0x55fa1ba52080 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e6dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e6dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49762/0 (socket says 172.21.15.55:49762) 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.223+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/756651152 learned_addr learned my addr 172.21.15.55:0/756651152 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/756651152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba52080 msgr2=0x55fa1ba53a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/756651152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba52080 0x55fa1ba53a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/756651152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 msgr2=0x55fa1ba63050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/756651152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 0x55fa1ba63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/756651152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa1ba52460 con 0x55fa1ba51690 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/756651152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba534e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400f9e0 tx=0x7f4e64007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5670f627ca9efed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e57fff640 1 -- 172.21.15.55:0/756651152 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6402e070 con 0x55fa1ba51690 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e57fff640 1 -- 172.21.15.55:0/756651152 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e6402a050 con 0x55fa1ba51690 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.224+0000 7f4e57fff640 1 -- 172.21.15.55:0/756651152 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6401a040 con 0x55fa1ba51690 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- 172.21.15.55:0/756651152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 msgr2=0x55fa1ba534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 --2- 172.21.15.55:0/756651152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba534e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400f9e0 tx=0x7f4e64007f60 comp rx=0 tx=0).stop 2024-04-26T21:52:24.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- 172.21.15.55:0/756651152 shutdown_connections 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 --2- 172.21.15.55:0/756651152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba571f0 0x55fa1ba63050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 --2- 172.21.15.55:0/756651152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba52080 0x55fa1ba53a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 --2- 172.21.15.55:0/756651152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba51690 0x55fa1ba534e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- 172.21.15.55:0/756651152 >> 172.21.15.55:0/756651152 conn(0x55fa1ba4cd00 msgr2=0x55fa1ba5a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- 172.21.15.55:0/756651152 shutdown_connections 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- 172.21.15.55:0/756651152 wait complete. 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- start start 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 Processor -- start 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 0x55fa1ba5ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba5f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba571f0 0x55fa1ba623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ba698a0 con 0x55fa1ba571f0 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ba691d0 con 0x55fa1ba52080 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa1ba69f70 con 0x55fa1ba51690 2024-04-26T21:52:24.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e779e0740 10 client.0 _ll_register_callbacks cb 0x7f4e60005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e6dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba571f0 0x55fa1ba623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e6dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba571f0 0x55fa1ba623e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49766/0 (socket says 172.21.15.55:49766) 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/2067105769 learned_addr learned my addr 172.21.15.55:0/2067105769 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.225+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 0x55fa1ba5ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/2067105769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 msgr2=0x55fa1ba5ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 0x55fa1ba5ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/2067105769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 msgr2=0x55fa1ba5f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba5f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba5f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e6dd9b640 1 -- 172.21.15.55:0/2067105769 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa1ba629d0 con 0x55fa1ba571f0 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e6dd9b640 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba571f0 0x55fa1ba623e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55fa1ba66220 tx=0x7f4e640047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c96bb1265668eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:24.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e55ffb640 1 -- 172.21.15.55:0/2067105769 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6401a070 con 0x55fa1ba571f0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e55ffb640 1 -- 172.21.15.55:0/2067105769 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f4e6402aba0 con 0x55fa1ba571f0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e779e0740 1 -- 172.21.15.55:0/2067105769 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e6402a050 con 0x55fa1ba571f0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e55ffb640 1 -- 172.21.15.55:0/2067105769 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4e6402e040 con 0x55fa1ba571f0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e779e0740 10 client.15036 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e779e0740 1 -- 172.21.15.55:0/2067105769 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55fa1baceca0 con 0x55fa1ba571f0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e55ffb640 1 -- 172.21.15.55:0/2067105769 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e6402a050 con 0x55fa1ba571f0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e55ffb640 1 -- 172.21.15.55:0/2067105769 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f4e64020040 con 0x55fa1ba571f0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.226+0000 7f4e779e0740 10 client.15036 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.227+0000 7f4e779e0740 10 client.15036 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.227+0000 7f4e779e0740 20 client.15036 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.227+0000 7f4e779e0740 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fa1bad31e0 0x55fa1bad1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.227+0000 7f4e779e0740 4 client.15036 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.227+0000 7f4e779e0740 1 -- 172.21.15.55:0/2067105769 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "status"}) -- 0x55fa1ba52ff0 con 0x55fa1bad31e0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.227+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fa1bad31e0 0x55fa1bad1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.227+0000 7f4e6e59c640 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fa1bad31e0 0x55fa1bad1ef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=866b82595ec98e26 server_cookie=2dc67fe84ed5a4f6 in_seq=0 out_seq=0 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.227+0000 7f4e55ffb640 10 client.15036 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e55ffb640 1 -- 172.21.15.55:0/2067105769 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fa1ba52ff0 con 0x55fa1bad31e0 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e55ffb640 10 client.15036 handle_command_reply: tid=0 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 client.15036 shutdown 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 -- 172.21.15.55:0/2067105769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba571f0 msgr2=0x55fa1ba623e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba571f0 0x55fa1ba623e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55fa1ba66220 tx=0x7f4e640047d0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 -- 172.21.15.55:0/2067105769 shutdown_connections 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fa1bad31e0 0x55fa1bad1ef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa1ba571f0 0x55fa1ba623e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa1ba52080 0x55fa1ba5f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e55ffb640 0 client.15036 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 --2- 172.21.15.55:0/2067105769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa1ba51690 0x55fa1ba5ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 -- 172.21.15.55:0/2067105769 >> 172.21.15.55:0/2067105769 conn(0x55fa1ba4cd00 msgr2=0x55fa1ba4fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 -- 172.21.15.55:0/2067105769 shutdown_connections 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 1 -- 172.21.15.55:0/2067105769 wait complete. 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e779e0740 20 client.15036 trim_cache size 0 max 16384 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 2, 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 24502, 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-04-26T21:52:24.235 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 83.933642634999998, 2024-04-26T21:52:24.236 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 20, 2024-04-26T21:52:24.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 87, 2024-04-26T21:52:24.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 85, 2024-04-26T21:52:24.236 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 101.641456068 2024-04-26T21:52:24.236 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 -- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e580a3c40 msgr2=0x7f4e580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e580a3c40 0x7f4e580a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e680a50f0 tx=0x7f4e68087040 comp rx=0 tx=0).stop 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 -- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70103420 msgr2=0x7f4e701a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70103420 0x7f4e701a0260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800eaa0 tx=0x7f4e6800ef60 comp rx=0 tx=0).stop 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 -- 172.21.15.55:0/1787999900 shutdown_connections 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e580a3c40 0x7f4e580a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e70103420 0x7f4e701a0260 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e70102a60 0x7f4e70199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 --2- 172.21.15.55:0/1787999900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e70101870 0x7f4e70198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 -- 172.21.15.55:0/1787999900 >> 172.21.15.55:0/1787999900 conn(0x7f4e700fcde0 msgr2=0x7f4e70069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.228+0000 7f4e74bf2640 1 -- 172.21.15.55:0/1787999900 shutdown_connections 2024-04-26T21:52:24.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.229+0000 7f4e74bf2640 1 -- 172.21.15.55:0/1787999900 wait complete. 2024-04-26T21:52:24.240 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-26T21:52:24.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 -- 172.21.15.55:0/899175610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 msgr2=0x7f2b0010aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 --2- 172.21.15.55:0/899175610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 0x7f2b0010aac0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009ca0 tx=0x7f2af802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 -- 172.21.15.55:0/899175610 shutdown_connections 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 --2- 172.21.15.55:0/899175610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 0x7f2b0010aac0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 --2- 172.21.15.55:0/899175610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b00105d40 0x7f2b00108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 --2- 172.21.15.55:0/899175610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b00103410 0x7f2b00105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 -- 172.21.15.55:0/899175610 >> 172.21.15.55:0/899175610 conn(0x7f2b000fce20 msgr2=0x7f2b000ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 -- 172.21.15.55:0/899175610 shutdown_connections 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 -- 172.21.15.55:0/899175610 wait complete. 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 Processor -- start 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.342+0000 7f2b07c20640 1 -- start start 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b07c20640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b00103410 0x7f2b00198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b07c20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b00105d40 0x7f2b00199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b07c20640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 0x7f2b001a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b07c20640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0010e4e0 con 0x7f2b00108670 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b07c20640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0010e360 con 0x7f2b00105d40 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b07c20640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0010e660 con 0x7f2b00103410 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 0x7f2b001a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 0x7f2b001a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49788/0 (socket says 172.21.15.55:49788) 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 -- 172.21.15.55:0/3706425631 learned_addr learned my addr 172.21.15.55:0/3706425631 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b059bc640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b00103410 0x7f2b00198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b051bb640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b00105d40 0x7f2b00199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 -- 172.21.15.55:0/3706425631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b00103410 msgr2=0x7f2b00198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b00103410 0x7f2b00198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 -- 172.21.15.55:0/3706425631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b00105d40 msgr2=0x7f2b00199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b00105d40 0x7f2b00199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 -- 172.21.15.55:0/3706425631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b001a07e0 con 0x7f2b00108670 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b051bb640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b00105d40 0x7f2b00199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:24.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b061bd640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 0x7f2b001a0170 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2af80097b0 tx=0x7f2af8031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2af6ffd640 1 -- 172.21.15.55:0/3706425631 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8003b70 con 0x7f2b00108670 2024-04-26T21:52:24.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2af6ffd640 1 -- 172.21.15.55:0/3706425631 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af8003d10 con 0x7f2b00108670 2024-04-26T21:52:24.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b07c20640 1 -- 172.21.15.55:0/3706425631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b001a0a70 con 0x7f2b00108670 2024-04-26T21:52:24.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2af6ffd640 1 -- 172.21.15.55:0/3706425631 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af802fc10 con 0x7f2b00108670 2024-04-26T21:52:24.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.343+0000 7f2b07c20640 1 -- 172.21.15.55:0/3706425631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b001a0fc0 con 0x7f2b00108670 2024-04-26T21:52:24.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.346+0000 7f2af6ffd640 1 -- 172.21.15.55:0/3706425631 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2af8004040 con 0x7f2b00108670 2024-04-26T21:52:24.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.346+0000 7f2b0aa15740 1 Processor -- start 2024-04-26T21:52:24.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.346+0000 7f2b0aa15740 1 -- start start 2024-04-26T21:52:24.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.346+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046d690 0x55afb046dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.346+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046e080 0x55afb0472b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb0475680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55afb03d2740 con 0x55afb04731f0 2024-04-26T21:52:24.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55afb036e3a0 con 0x55afb046e080 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2af6ffd640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2aec0a3c40 0x7f2aec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55afb03ce700 con 0x55afb046d690 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb0475680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b061bd640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2aec0a3c40 0x7f2aec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb0475680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49790/0 (socket says 172.21.15.55:49790) 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 -- 172.21.15.55:0/4138923344 learned_addr learned my addr 172.21.15.55:0/4138923344 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b059bc640 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046e080 0x55afb0472b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046d690 0x55afb046dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2af6ffd640 1 -- 172.21.15.55:0/3706425631 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f2af80f1bf0 con 0x7f2b00108670 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 -- 172.21.15.55:0/4138923344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046d690 msgr2=0x55afb046dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046d690 0x55afb046dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 -- 172.21.15.55:0/4138923344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046e080 msgr2=0x55afb0472b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046e080 0x55afb0472b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 -- 172.21.15.55:0/4138923344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb038a750 con 0x55afb04731f0 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b061bd640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2aec0a3c40 0x7f2aec0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2af80ba830 tx=0x7f2af80bdcb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2b051bb640 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb0475680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2af0012550 tx=0x7f2af00097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3578c4fc41e9657 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2aebfff640 1 -- 172.21.15.55:0/4138923344 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0010710 con 0x55afb04731f0 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2aebfff640 1 -- 172.21.15.55:0/4138923344 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af0010d60 con 0x55afb04731f0 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.347+0000 7f2aebfff640 1 -- 172.21.15.55:0/4138923344 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af001ddf0 con 0x55afb04731f0 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 -- 172.21.15.55:0/4138923344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 msgr2=0x55afb0475680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb0475680 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2af0012550 tx=0x7f2af00097b0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 -- 172.21.15.55:0/4138923344 shutdown_connections 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb0475680 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046e080 0x55afb0472b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/4138923344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046d690 0x55afb046dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 -- 172.21.15.55:0/4138923344 >> 172.21.15.55:0/4138923344 conn(0x55afb0468d00 msgr2=0x55afb0469100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 -- 172.21.15.55:0/4138923344 shutdown_connections 2024-04-26T21:52:24.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.348+0000 7f2b0aa15740 1 -- 172.21.15.55:0/4138923344 wait complete. 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 1 -- start start 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 1 Processor -- start 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046d690 0x55afb0477b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 0x55afb0478090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb04731f0 0x55afb047f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55afb04761a0 con 0x55afb046d690 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55afb0359fe0 con 0x55afb04731f0 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55afb0476320 con 0x55afb046e080 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b0aa15740 10 client.0 _ll_register_callbacks cb 0x55afb0370150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b051bb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046d690 0x55afb0477b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b051bb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046d690 0x55afb0477b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49806/0 (socket says 172.21.15.55:49806) 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b051bb640 1 -- 172.21.15.55:0/685780736 learned_addr learned my addr 172.21.15.55:0/685780736 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b051bb640 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb04731f0 0x55afb047f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b051bb640 1 -- 172.21.15.55:0/685780736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 msgr2=0x55afb0478090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.349+0000 7f2b059bc640 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 0x55afb0478090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b051bb640 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 0x55afb0478090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b051bb640 1 -- 172.21.15.55:0/685780736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb04731f0 msgr2=0x55afb047f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b051bb640 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb04731f0 0x55afb047f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b051bb640 1 -- 172.21.15.55:0/685780736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb00e42d0 con 0x55afb046d690 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b051bb640 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046d690 0x55afb0477b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2af0006010 tx=0x7f2af00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ff107ccd2d1e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2ae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2ae97fa640 1 -- 172.21.15.55:0/685780736 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0004450 con 0x55afb046d690 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2ae97fa640 1 -- 172.21.15.55:0/685780736 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f2af0010510 con 0x55afb046d690 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b0aa15740 1 -- 172.21.15.55:0/685780736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af0010d60 con 0x55afb046d690 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2ae97fa640 1 -- 172.21.15.55:0/685780736 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2af0010710 con 0x55afb046d690 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b0aa15740 10 client.15048 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:24.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b0aa15740 1 -- 172.21.15.55:0/685780736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55afb01ae6b0 con 0x55afb046d690 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2ae97fa640 1 -- 172.21.15.55:0/685780736 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af0010d60 con 0x55afb046d690 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2ae97fa640 1 -- 172.21.15.55:0/685780736 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f2af001c070 con 0x55afb046d690 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b0aa15740 10 client.15048 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b0aa15740 10 client.15048 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.350+0000 7f2b0aa15740 20 client.15048 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.351+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55afb0484320 0x55afb0485790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.351+0000 7f2b0aa15740 4 client.15048 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.351+0000 7f2b0aa15740 1 -- 172.21.15.55:0/685780736 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afb03e0350 con 0x55afb0484320 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.351+0000 7f2b059bc640 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55afb0484320 0x55afb0485790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.351+0000 7f2b059bc640 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55afb0484320 0x55afb0485790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0e3fec94d4c8bfa server_cookie=f7c2d64751fc5772 in_seq=0 out_seq=0 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.351+0000 7f2ae97fa640 10 client.15048 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2ae97fa640 1 -- 172.21.15.55:0/685780736 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afb03e0350 con 0x55afb0484320 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2ae97fa640 10 client.15048 handle_command_reply: tid=0 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 client.15048 shutdown 2024-04-26T21:52:24.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 -- 172.21.15.55:0/685780736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046d690 msgr2=0x55afb0477b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046d690 0x55afb0477b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2af0006010 tx=0x7f2af00056d0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 -- 172.21.15.55:0/685780736 shutdown_connections 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55afb0484320 0x55afb0485790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb04731f0 0x55afb047f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 0x55afb0478090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2ae97fa640 0 client.15048 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/685780736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046d690 0x55afb0477b50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.352+0000 7f2b0aa15740 1 -- 172.21.15.55:0/685780736 >> 172.21.15.55:0/685780736 conn(0x55afb0468d00 msgr2=0x55afb0467f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.353+0000 7f2b0aa15740 1 -- 172.21.15.55:0/685780736 shutdown_connections 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.353+0000 7f2b0aa15740 1 -- 172.21.15.55:0/685780736 wait complete. 2024-04-26T21:52:24.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.353+0000 7f2b0aa15740 20 client.15048 trim_cache size 0 max 16384 2024-04-26T21:52:24.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b0aa15740 1 Processor -- start 2024-04-26T21:52:24.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b0aa15740 1 -- start start 2024-04-26T21:52:24.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb0477760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046e080 0x55afb047f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb04731f0 0x55afb0477ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55afb0483ad0 con 0x55afb046e080 2024-04-26T21:52:24.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55afb04841a0 con 0x55afb046d690 2024-04-26T21:52:24.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55afb0484870 con 0x55afb04731f0 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b059bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046e080 0x55afb047f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b059bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046e080 0x55afb047f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49816/0 (socket says 172.21.15.55:49816) 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.372+0000 7f2b059bc640 1 -- 172.21.15.55:0/434651583 learned_addr learned my addr 172.21.15.55:0/434651583 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b051bb640 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb0477760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b051bb640 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb04731f0 0x55afb0477ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b059bc640 1 -- 172.21.15.55:0/434651583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb04731f0 msgr2=0x55afb0477ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b059bc640 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb04731f0 0x55afb0477ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b059bc640 1 -- 172.21.15.55:0/434651583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 msgr2=0x55afb0477760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b059bc640 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb0477760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b059bc640 1 -- 172.21.15.55:0/434651583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afb0469a70 con 0x55afb046e080 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b051bb640 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb0477760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2b059bc640 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046e080 0x55afb047f060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55afb046e480 tx=0x7f2afc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62df52475ba9cb2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2aeb7fe640 1 -- 172.21.15.55:0/434651583 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc004050 con 0x55afb046e080 2024-04-26T21:52:24.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2aeb7fe640 1 -- 172.21.15.55:0/434651583 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc0041f0 con 0x55afb046e080 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.373+0000 7f2aeb7fe640 1 -- 172.21.15.55:0/434651583 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc003970 con 0x55afb046e080 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 -- 172.21.15.55:0/434651583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046e080 msgr2=0x55afb047f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046e080 0x55afb047f060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55afb046e480 tx=0x7f2afc012820 comp rx=0 tx=0).stop 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 -- 172.21.15.55:0/434651583 shutdown_connections 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb04731f0 0x55afb0477ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb046e080 0x55afb047f060 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/434651583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb0477760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 -- 172.21.15.55:0/434651583 >> 172.21.15.55:0/434651583 conn(0x55afb0468d00 msgr2=0x55afb04690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 -- 172.21.15.55:0/434651583 shutdown_connections 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 -- 172.21.15.55:0/434651583 wait complete. 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 -- start start 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 Processor -- start 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.374+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb047abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 0x55afb047b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b0aa15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb047e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55afb0485610 con 0x55afb04731f0 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55afb0484f40 con 0x55afb046d690 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b0aa15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55afb0485ce0 con 0x55afb046e080 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b0aa15740 10 client.0 _ll_register_callbacks cb 0x7f2afc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb047e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb047e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49832/0 (socket says 172.21.15.55:49832) 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 -- 172.21.15.55:0/1845210305 learned_addr learned my addr 172.21.15.55:0/1845210305 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb047abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b059bc640 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 0x55afb047b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 -- 172.21.15.55:0/1845210305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 msgr2=0x55afb047b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 0x55afb047b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 -- 172.21.15.55:0/1845210305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 msgr2=0x55afb047abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb047abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 -- 172.21.15.55:0/1845210305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afb047e6c0 con 0x55afb04731f0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b051bb640 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb047e0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2af001e000 tx=0x7f2af0020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894af5128b15a738 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2ae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2ae97fa640 1 -- 172.21.15.55:0/1845210305 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af003b040 con 0x55afb04731f0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2ae97fa640 1 -- 172.21.15.55:0/1845210305 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f2af00108b0 con 0x55afb04731f0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.375+0000 7f2b0aa15740 1 -- 172.21.15.55:0/1845210305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2afc0041f0 con 0x55afb04731f0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2ae97fa640 1 -- 172.21.15.55:0/1845210305 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2af0049070 con 0x55afb04731f0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b0aa15740 10 client.15054 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b0aa15740 1 -- 172.21.15.55:0/1845210305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55afb04ea600 con 0x55afb04731f0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2ae97fa640 1 -- 172.21.15.55:0/1845210305 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2afc0041f0 con 0x55afb04731f0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2ae97fa640 1 -- 172.21.15.55:0/1845210305 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f2af001c070 con 0x55afb04731f0 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b0aa15740 10 client.15054 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:24.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b0aa15740 10 client.15054 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b0aa15740 20 client.15054 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55afb0484320 0x55afb04ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b0aa15740 4 client.15054 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b0aa15740 1 -- 172.21.15.55:0/1845210305 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "status"}) -- 0x55afb03da1c0 con 0x55afb0484320 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.376+0000 7f2b059bc640 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55afb0484320 0x55afb04ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2ae97fa640 10 client.15054 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b059bc640 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55afb0484320 0x55afb04ede20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77ea05df6aa8d45a server_cookie=d4de9bb7720b5462 in_seq=0 out_seq=0 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2ae97fa640 1 -- 172.21.15.55:0/1845210305 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55afb03da1c0 con 0x55afb0484320 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2ae97fa640 10 client.15054 handle_command_reply: tid=0 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 client.15054 shutdown 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 -- 172.21.15.55:0/1845210305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 msgr2=0x55afb047e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb047e0d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2af001e000 tx=0x7f2af0020040 comp rx=0 tx=0).stop 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 -- 172.21.15.55:0/1845210305 shutdown_connections 2024-04-26T21:52:24.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55afb0484320 0x55afb04ede20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afb04731f0 0x55afb047e0d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afb046e080 0x55afb047b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2ae97fa640 0 client.15054 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 --2- 172.21.15.55:0/1845210305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afb046d690 0x55afb047abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 -- 172.21.15.55:0/1845210305 >> 172.21.15.55:0/1845210305 conn(0x55afb0468d00 msgr2=0x55afb0477760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 -- 172.21.15.55:0/1845210305 shutdown_connections 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 1 -- 172.21.15.55:0/1845210305 wait complete. 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.377+0000 7f2b0aa15740 20 client.15054 trim_cache size 0 max 16384 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:52:24.384 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 1, 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 24509, 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 85.103819736999995, 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 20, 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 87, 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 85, 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 104.589690967 2024-04-26T21:52:24.385 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:52:24.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 -- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2aec0a3c40 msgr2=0x7f2aec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2aec0a3c40 0x7f2aec0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2af80ba830 tx=0x7f2af80bdcb0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 -- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 msgr2=0x7f2b001a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 0x7f2b001a0170 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2af80097b0 tx=0x7f2af8031770 comp rx=0 tx=0).stop 2024-04-26T21:52:24.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 -- 172.21.15.55:0/3706425631 shutdown_connections 2024-04-26T21:52:24.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2aec0a3c40 0x7f2aec0a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b00108670 0x7f2b001a0170 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2b00105d40 0x7f2b00199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 --2- 172.21.15.55:0/3706425631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b00103410 0x7f2b00198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 -- 172.21.15.55:0/3706425631 >> 172.21.15.55:0/3706425631 conn(0x7f2b000fce20 msgr2=0x7f2b00103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 -- 172.21.15.55:0/3706425631 shutdown_connections 2024-04-26T21:52:24.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.378+0000 7f2b07c20640 1 -- 172.21.15.55:0/3706425631 wait complete. 2024-04-26T21:52:24.388 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:24 smithi055 ceph-mon[29221]: pgmap v334: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:24.388 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:24 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/936116908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.388 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:24 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1774154544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.388 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:24 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1627864623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.389 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 -- 172.21.15.55:0/2666053699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c102a80 msgr2=0x7ff72c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 --2- 172.21.15.55:0/2666053699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c102a80 0x7ff72c102f00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff714009900 tx=0x7ff71402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 -- 172.21.15.55:0/2666053699 shutdown_connections 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 --2- 172.21.15.55:0/2666053699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff72c103440 0x7ff72c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 --2- 172.21.15.55:0/2666053699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c102a80 0x7ff72c102f00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 --2- 172.21.15.55:0/2666053699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff72c101890 0x7ff72c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 -- 172.21.15.55:0/2666053699 >> 172.21.15.55:0/2666053699 conn(0x7ff72c0fce20 msgr2=0x7ff72c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 -- 172.21.15.55:0/2666053699 shutdown_connections 2024-04-26T21:52:24.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.491+0000 7ff7322f2640 1 -- 172.21.15.55:0/2666053699 wait complete. 2024-04-26T21:52:24.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff7322f2640 1 Processor -- start 2024-04-26T21:52:24.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff7322f2640 1 -- start start 2024-04-26T21:52:24.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff7322f2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff72c101890 0x7ff72c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff7322f2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff72c102a80 0x7ff72c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff7322f2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c103440 0x7ff72c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff7322f2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c10d3c0 con 0x7ff72c103440 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff7322f2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c10d240 con 0x7ff72c102a80 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff7322f2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c10d540 con 0x7ff72c101890 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff73088f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c103440 0x7ff72c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff73088f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c103440 0x7ff72c1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49846/0 (socket says 172.21.15.55:49846) 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff73088f640 1 -- 172.21.15.55:0/3529777276 learned_addr learned my addr 172.21.15.55:0/3529777276 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff72bfff640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff72c101890 0x7ff72c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff72c102a80 0x7ff72c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff73088f640 1 -- 172.21.15.55:0/3529777276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff72c101890 msgr2=0x7ff72c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff73088f640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff72c101890 0x7ff72c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff73088f640 1 -- 172.21.15.55:0/3529777276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff72c102a80 msgr2=0x7ff72c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff73088f640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff72c102a80 0x7ff72c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.492+0000 7ff73088f640 1 -- 172.21.15.55:0/3529777276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c1a0830 con 0x7ff72c103440 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.493+0000 7ff73088f640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c103440 0x7ff72c1a0190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff72400cad0 tx=0x7ff72400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.493+0000 7ff7297fa640 1 -- 172.21.15.55:0/3529777276 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72400ae20 con 0x7ff72c103440 2024-04-26T21:52:24.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.493+0000 7ff7297fa640 1 -- 172.21.15.55:0/3529777276 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff724005020 con 0x7ff72c103440 2024-04-26T21:52:24.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.493+0000 7ff7322f2640 1 -- 172.21.15.55:0/3529777276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c1a0b20 con 0x7ff72c103440 2024-04-26T21:52:24.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.493+0000 7ff7297fa640 1 -- 172.21.15.55:0/3529777276 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72400f6b0 con 0x7ff72c103440 2024-04-26T21:52:24.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.493+0000 7ff7322f2640 1 -- 172.21.15.55:0/3529777276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c1a1070 con 0x7ff72c103440 2024-04-26T21:52:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff7297fa640 1 -- 172.21.15.55:0/3529777276 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff7240051d0 con 0x7ff72c103440 2024-04-26T21:52:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff735005740 1 Processor -- start 2024-04-26T21:52:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff735005740 1 -- start start 2024-04-26T21:52:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff735005740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff735005740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff7297fa640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff7180a3c40 0x7ff7180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff735005740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 0x562c2e5b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff735005740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562c2e50d740 con 0x562c2e5ad490 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff735005740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562c2e4a93a0 con 0x562c2e5acaa0 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.495+0000 7ff735005740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c2e509700 con 0x562c2e5b2600 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff7180a3c40 0x7ff7180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5b1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49856/0 (socket says 172.21.15.55:49856) 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 -- 172.21.15.55:0/902787226 learned_addr learned my addr 172.21.15.55:0/902787226 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff7297fa640 1 -- 172.21.15.55:0/3529777276 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff7240d0560 con 0x7ff72c103440 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff73088f640 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 0x562c2e5b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 -- 172.21.15.55:0/902787226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 msgr2=0x562c2e5b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 0x562c2e5b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 -- 172.21.15.55:0/902787226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 msgr2=0x562c2e5acec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5acec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 -- 172.21.15.55:0/902787226 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2e4c5750 con 0x562c2e5ad490 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72bfff640 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5b1f90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00b750 tx=0x7ff71c00bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e493c517b095bbe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff7180a3c40 0x7ff7180a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff714000c00 tx=0x7ff7140046e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff7227fc640 1 -- 172.21.15.55:0/902787226 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00be60 con 0x562c2e5ad490 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff7227fc640 1 -- 172.21.15.55:0/902787226 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71c015070 con 0x562c2e5ad490 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.496+0000 7ff7227fc640 1 -- 172.21.15.55:0/902787226 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0106a0 con 0x562c2e5ad490 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 -- 172.21.15.55:0/902787226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 msgr2=0x562c2e5b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5b1f90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00b750 tx=0x7ff71c00bc10 comp rx=0 tx=0).stop 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 -- 172.21.15.55:0/902787226 shutdown_connections 2024-04-26T21:52:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 0x562c2e5b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5b1f90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 --2- 172.21.15.55:0/902787226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5acec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 -- 172.21.15.55:0/902787226 >> 172.21.15.55:0/902787226 conn(0x562c2e5a3d00 msgr2=0x562c2e5a4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 -- 172.21.15.55:0/902787226 shutdown_connections 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 -- 172.21.15.55:0/902787226 wait complete. 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.497+0000 7ff735005740 1 -- start start 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff735005740 1 Processor -- start 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff735005740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff735005740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5ad490 0x562c2e5b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff735005740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5b2600 0x562c2e5be440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff735005740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562c2e5b55b0 con 0x562c2e5b2600 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff735005740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562c2e494fe0 con 0x562c2e5acaa0 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff735005740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c2e5b5730 con 0x562c2e5ad490 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff735005740 10 client.0 _ll_register_callbacks cb 0x562c2e4ab150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5b2600 0x562c2e5be440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5b2600 0x562c2e5be440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49868/0 (socket says 172.21.15.55:49868) 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 -- 172.21.15.55:0/2045308730 learned_addr learned my addr 172.21.15.55:0/2045308730 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff72bfff640 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5ad490 0x562c2e5b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 -- 172.21.15.55:0/2045308730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5ad490 msgr2=0x562c2e5b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5ad490 0x562c2e5b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 -- 172.21.15.55:0/2045308730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 msgr2=0x562c2e5b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 -- 172.21.15.55:0/2045308730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2e21f2d0 con 0x562c2e5b2600 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff73088f640 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5b2600 0x562c2e5be440 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7240ce470 tx=0x7ff7240ce370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbb9bad44e79669 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff70b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.498+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2045308730 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72401d040 con 0x562c2e5b2600 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2045308730 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff7240054f0 con 0x562c2e5b2600 2024-04-26T21:52:24.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 1 -- 172.21.15.55:0/2045308730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71c015070 con 0x562c2e5b2600 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2045308730 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff72409c770 con 0x562c2e5b2600 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 10 client.15066 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 1 -- 172.21.15.55:0/2045308730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562c2e2e96b0 con 0x562c2e5b2600 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2045308730 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71c015070 con 0x562c2e5b2600 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2045308730 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7ff7240d0110 con 0x562c2e5b2600 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 10 client.15066 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 10 client.15066 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 20 client.15066 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2e5c3970 0x562c2e5c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 4 client.15066 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff735005740 1 -- 172.21.15.55:0/2045308730 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c2e51b350 con 0x562c2e5c3970 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.499+0000 7ff72bfff640 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2e5c3970 0x562c2e5c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.500+0000 7ff72bfff640 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2e5c3970 0x562c2e5c4de0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a607415fb48f7b99 server_cookie=416eadea0cfe4d5b in_seq=0 out_seq=0 2024-04-26T21:52:24.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.500+0000 7ff70b7fe640 10 client.15066 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:52:24.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2045308730 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c2e51b350 con 0x562c2e5c3970 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff70b7fe640 10 client.15066 handle_command_reply: tid=0 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 client.15066 shutdown 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 -- 172.21.15.55:0/2045308730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5b2600 msgr2=0x562c2e5be440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5b2600 0x562c2e5be440 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7240ce470 tx=0x7ff7240ce370 comp rx=0 tx=0).stop 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 -- 172.21.15.55:0/2045308730 shutdown_connections 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2e5c3970 0x562c2e5c4de0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5b2600 0x562c2e5be440 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5ad490 0x562c2e5b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff70b7fe640 0 client.15066 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 --2- 172.21.15.55:0/2045308730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 -- 172.21.15.55:0/2045308730 >> 172.21.15.55:0/2045308730 conn(0x562c2e5a3d00 msgr2=0x562c2e5a2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 -- 172.21.15.55:0/2045308730 shutdown_connections 2024-04-26T21:52:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 1 -- 172.21.15.55:0/2045308730 wait complete. 2024-04-26T21:52:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.501+0000 7ff735005740 20 client.15066 trim_cache size 0 max 16384 2024-04-26T21:52:24.512 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:24 smithi064 ceph-mon[32765]: pgmap v334: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:24.512 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:24 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/936116908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.512 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:24 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1774154544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.512 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:24 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1627864623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff735005740 1 Processor -- start 2024-04-26T21:52:24.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff735005740 1 -- start start 2024-04-26T21:52:24.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff735005740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff735005740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5ad490 0x562c2e5be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff735005740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 0x562c2e5b7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff735005740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562c2e5c3120 con 0x562c2e5acaa0 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff735005740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562c2e5c37f0 con 0x562c2e5ad490 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff735005740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c2e5c3ec0 con 0x562c2e5b2600 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.519+0000 7ff72bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49882/0 (socket says 172.21.15.55:49882) 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72bfff640 1 -- 172.21.15.55:0/1258177155 learned_addr learned my addr 172.21.15.55:0/1258177155 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5ad490 0x562c2e5be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72bfff640 1 -- 172.21.15.55:0/1258177155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 msgr2=0x562c2e5b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72bfff640 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 0x562c2e5b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72bfff640 1 -- 172.21.15.55:0/1258177155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5ad490 msgr2=0x562c2e5be400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72bfff640 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5ad490 0x562c2e5be400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72bfff640 1 -- 172.21.15.55:0/1258177155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2e5a4a70 con 0x562c2e5acaa0 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5ad490 0x562c2e5be400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:24.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff72bfff640 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ad0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00b750 tx=0x7ff71c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32094b273825a68b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff721ffb640 1 -- 172.21.15.55:0/1258177155 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c003dd0 con 0x562c2e5acaa0 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff721ffb640 1 -- 172.21.15.55:0/1258177155 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71c015040 con 0x562c2e5acaa0 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.520+0000 7ff721ffb640 1 -- 172.21.15.55:0/1258177155 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c01a8b0 con 0x562c2e5acaa0 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 -- 172.21.15.55:0/1258177155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5acaa0 msgr2=0x562c2e5b6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ad0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00b750 tx=0x7ff71c013040 comp rx=0 tx=0).stop 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 -- 172.21.15.55:0/1258177155 shutdown_connections 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5b2600 0x562c2e5b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5ad490 0x562c2e5be400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 --2- 172.21.15.55:0/1258177155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5acaa0 0x562c2e5b6ad0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 -- 172.21.15.55:0/1258177155 >> 172.21.15.55:0/1258177155 conn(0x562c2e5a3d00 msgr2=0x562c2e5ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 -- 172.21.15.55:0/1258177155 shutdown_connections 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 -- 172.21.15.55:0/1258177155 wait complete. 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 -- start start 2024-04-26T21:52:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 Processor -- start 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5acaa0 0x562c2e5ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.521+0000 7ff735005740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5b2600 0x562c2e5bda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff735005740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562c2e5c4c60 con 0x562c2e5ad490 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff735005740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562c2e5c4590 con 0x562c2e5b2600 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff735005740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c2e5c5330 con 0x562c2e5acaa0 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff735005740 10 client.0 _ll_register_callbacks cb 0x7ff72409c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49892/0 (socket says 172.21.15.55:49892) 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 -- 172.21.15.55:0/2203644726 learned_addr learned my addr 172.21.15.55:0/2203644726 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72bfff640 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5acaa0 0x562c2e5ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 -- 172.21.15.55:0/2203644726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5acaa0 msgr2=0x562c2e5ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5acaa0 0x562c2e5ba630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 -- 172.21.15.55:0/2203644726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5b2600 msgr2=0x562c2e5bda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5b2600 0x562c2e5bda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 -- 172.21.15.55:0/2203644726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2e5be0c0 con 0x562c2e5ad490 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff72b7fe640 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5bab70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff714009900 tx=0x7ff71402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62c2528bfca8780 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff70b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2203644726 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714035040 con 0x562c2e5ad490 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff735005740 1 -- 172.21.15.55:0/2203644726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff71c015040 con 0x562c2e5ad490 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2203644726 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7ff7140035f0 con 0x562c2e5ad490 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2203644726 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff7140037f0 con 0x562c2e5ad490 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff735005740 10 client.15072 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.522+0000 7ff735005740 1 -- 172.21.15.55:0/2203644726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562c2e62a400 con 0x562c2e5ad490 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2203644726 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71c015040 con 0x562c2e5ad490 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2203644726 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7ff71403da20 con 0x562c2e5ad490 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff735005740 10 client.15072 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:24.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff735005740 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff735005740 20 client.15072 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff735005740 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2e62f110 0x562c2e62de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff735005740 4 client.15072 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff735005740 1 -- 172.21.15.55:0/2203644726 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "status"}) -- 0x562c2e5151c0 con 0x562c2e62f110 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.523+0000 7ff72bfff640 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2e62f110 0x562c2e62de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff72bfff640 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2e62f110 0x562c2e62de20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7822308a38baab02 server_cookie=58d0e41311190831 in_seq=0 out_seq=0 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff70b7fe640 10 client.15072 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff70b7fe640 1 -- 172.21.15.55:0/2203644726 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562c2e5151c0 con 0x562c2e62f110 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff70b7fe640 10 client.15072 handle_command_reply: tid=0 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 client.15072 shutdown 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 -- 172.21.15.55:0/2203644726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 msgr2=0x562c2e5bab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5bab70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff714009900 tx=0x7ff71402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 -- 172.21.15.55:0/2203644726 shutdown_connections 2024-04-26T21:52:24.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2e5b2600 0x562c2e5bda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2e62f110 0x562c2e62de20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2e5ad490 0x562c2e5bab70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 --2- 172.21.15.55:0/2203644726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2e5acaa0 0x562c2e5ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff70b7fe640 0 client.15072 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 -- 172.21.15.55:0/2203644726 >> 172.21.15.55:0/2203644726 conn(0x562c2e5a3d00 msgr2=0x562c2e5b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 -- 172.21.15.55:0/2203644726 shutdown_connections 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 1 -- 172.21.15.55:0/2203644726 wait complete. 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.524+0000 7ff735005740 20 client.15072 trim_cache size 0 max 16384 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:52:24.531 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 24551, 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 87.174929055000007, 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 20, 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 87, 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 85, 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 93.424412646999997 2024-04-26T21:52:24.532 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:52:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 -- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff7180a3c40 msgr2=0x7ff7180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff7180a3c40 0x7ff7180a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff714000c00 tx=0x7ff7140046e0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 -- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c103440 msgr2=0x7ff72c1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c103440 0x7ff72c1a0190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff72400cad0 tx=0x7ff72400cf90 comp rx=0 tx=0).stop 2024-04-26T21:52:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 -- 172.21.15.55:0/3529777276 shutdown_connections 2024-04-26T21:52:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff7180a3c40 0x7ff7180a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff72c103440 0x7ff72c1a0190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff72c102a80 0x7ff72c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 --2- 172.21.15.55:0/3529777276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff72c101890 0x7ff72c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 -- 172.21.15.55:0/3529777276 >> 172.21.15.55:0/3529777276 conn(0x7ff72c0fce20 msgr2=0x7ff72c06af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 -- 172.21.15.55:0/3529777276 shutdown_connections 2024-04-26T21:52:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:24.525+0000 7ff7322f2640 1 -- 172.21.15.55:0/3529777276 wait complete. 2024-04-26T21:52:24.536 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T21:52:24.536 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:52:24.546 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi055.front.sepia.ceph.com 2024-04-26T21:52:24.547 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T21:52:24.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:24 smithi165 ceph-mon[33296]: pgmap v334: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:24.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:24 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/936116908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:24 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1774154544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:24 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1627864623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:24.794 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.497+0000 7fc4607de700 1 -- 172.21.15.55:0/2887849294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458103480 msgr2=0x7fc4581038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.497+0000 7fc4607de700 1 --2- 172.21.15.55:0/2887849294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458103480 0x7fc4581038a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc4540099c0 tx=0x7fc454009cd0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.497+0000 7fc4607de700 1 -- 172.21.15.55:0/2887849294 shutdown_connections 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.497+0000 7fc4607de700 1 --2- 172.21.15.55:0/2887849294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc458105020 0x7fc45810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 --2- 172.21.15.55:0/2887849294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458103480 0x7fc4581038a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 --2- 172.21.15.55:0/2887849294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc458104680 0x7fc458104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 -- 172.21.15.55:0/2887849294 >> 172.21.15.55:0/2887849294 conn(0x7fc4580fea20 msgr2=0x7fc458100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 -- 172.21.15.55:0/2887849294 shutdown_connections 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 -- 172.21.15.55:0/2887849294 wait complete. 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 Processor -- start 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 -- start start 2024-04-26T21:52:26.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc458103480 0x7fc4580771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458104680 0x7fc458075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc458105020 0x7fc458075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc45810ef20 con 0x7fc458104680 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc45810eda0 con 0x7fc458105020 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.498+0000 7fc4607de700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc45810f0a0 con 0x7fc458103480 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458104680 0x7fc458075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458104680 0x7fc458075810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49920/0 (socket says 172.21.15.55:49920) 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 -- 172.21.15.55:0/4086204995 learned_addr learned my addr 172.21.15.55:0/4086204995 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45e57a700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc458103480 0x7fc4580771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45ed7b700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc458105020 0x7fc458075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 -- 172.21.15.55:0/4086204995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc458103480 msgr2=0x7fc4580771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc458103480 0x7fc4580771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 -- 172.21.15.55:0/4086204995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc458105020 msgr2=0x7fc458075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc458105020 0x7fc458075d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 -- 172.21.15.55:0/4086204995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc458076600 con 0x7fc458104680 2024-04-26T21:52:26.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc45dd79700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458104680 0x7fc458075810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc44800c970 tx=0x7fc44800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:26.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc44f7fe700 1 -- 172.21.15.55:0/4086204995 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448007aa0 con 0x7fc458104680 2024-04-26T21:52:26.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc44f7fe700 1 -- 172.21.15.55:0/4086204995 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc448004ce0 con 0x7fc458104680 2024-04-26T21:52:26.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4581a3780 con 0x7fc458104680 2024-04-26T21:52:26.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc44f7fe700 1 -- 172.21.15.55:0/4086204995 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4480055c0 con 0x7fc458104680 2024-04-26T21:52:26.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.499+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4581a3c40 con 0x7fc458104680 2024-04-26T21:52:26.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.500+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45806a110 con 0x7fc458104680 2024-04-26T21:52:26.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.501+0000 7fc44f7fe700 1 -- 172.21.15.55:0/4086204995 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc448007c40 con 0x7fc458104680 2024-04-26T21:52:26.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.502+0000 7fc44f7fe700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4440a3cd0 0x7fc4440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:26.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.502+0000 7fc45e57a700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4440a3cd0 0x7fc4440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:26.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.502+0000 7fc44f7fe700 1 -- 172.21.15.55:0/4086204995 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fc4480d0470 con 0x7fc458104680 2024-04-26T21:52:26.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.502+0000 7fc45e57a700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4440a3cd0 0x7fc4440a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc45400b580 tx=0x7fc454005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:26.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.506+0000 7fc44f7fe700 1 -- 172.21.15.55:0/4086204995 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc44809cd00 con 0x7fc458104680 2024-04-26T21:52:26.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:26 smithi055 ceph-mon[29221]: pgmap v335: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:26.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.734+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fc458063860 con 0x7fc4440a3cd0 2024-04-26T21:52:26.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.735+0000 7fc44f7fe700 1 -- 172.21.15.55:0/4086204995 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc458063860 con 0x7fc4440a3cd0 2024-04-26T21:52:26.735 INFO:teuthology.orchestra.run.smithi055.stdout:Backend: cephadm 2024-04-26T21:52:26.735 INFO:teuthology.orchestra.run.smithi055.stdout:Available: Yes 2024-04-26T21:52:26.735 INFO:teuthology.orchestra.run.smithi055.stdout:Paused: No 2024-04-26T21:52:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4440a3cd0 msgr2=0x7fc4440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4440a3cd0 0x7fc4440a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc45400b580 tx=0x7fc454005fb0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458104680 msgr2=0x7fc458075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458104680 0x7fc458075810 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc44800c970 tx=0x7fc44800cd30 comp rx=0 tx=0).stop 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 shutdown_connections 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4440a3cd0 0x7fc4440a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc458105020 0x7fc458075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc458103480 0x7fc4580771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 --2- 172.21.15.55:0/4086204995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc458104680 0x7fc458075810 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 >> 172.21.15.55:0/4086204995 conn(0x7fc4580fea20 msgr2=0x7fc458107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 shutdown_connections 2024-04-26T21:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:26.737+0000 7fc4607de700 1 -- 172.21.15.55:0/4086204995 wait complete. 2024-04-26T21:52:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:26 smithi064 ceph-mon[32765]: pgmap v335: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:26 smithi165 ceph-mon[33296]: pgmap v335: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:27.454 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T21:52:27.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:27 smithi055 ceph-mon[29221]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:27.705 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:27.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:27 smithi064 ceph-mon[32765]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:27.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:27 smithi165 ceph-mon[33296]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:28.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:28 smithi055 ceph-mon[29221]: pgmap v336: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:28 smithi064 ceph-mon[32765]: pgmap v336: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:28.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:28 smithi165 ceph-mon[33296]: pgmap v336: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:29.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/560340333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 msgr2=0x7f1ac010b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/560340333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 0x7f1ac010b8f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800b0d0 tx=0x7f1ab800b3e0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/560340333 shutdown_connections 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/560340333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 0x7f1ac010b8f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/560340333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac01034a0 0x7f1ac01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/560340333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac01046a0 0x7f1ac0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/560340333 >> 172.21.15.55:0/560340333 conn(0x7f1ac00fea40 msgr2=0x7f1ac0100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/560340333 shutdown_connections 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/560340333 wait complete. 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.465+0000 7f1ac4fb0700 1 Processor -- start 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1ac4fb0700 1 -- start start 2024-04-26T21:52:29.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1ac4fb0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac01034a0 0x7f1ac00771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1ac4fb0700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac01046a0 0x7f1ac0075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1ac4fb0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 0x7f1ac0075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1ac4fb0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac010efe0 con 0x7f1ac0105040 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1ac4fb0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac010ee60 con 0x7f1ac01034a0 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1ac4fb0700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac010f160 con 0x7f1ac01046a0 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 0x7f1ac0075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 0x7f1ac0075d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49938/0 (socket says 172.21.15.55:49938) 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 -- 172.21.15.55:0/2886489822 learned_addr learned my addr 172.21.15.55:0/2886489822 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abe59c700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac01034a0 0x7f1ac00771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abdd9b700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac01046a0 0x7f1ac0075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 -- 172.21.15.55:0/2886489822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac01046a0 msgr2=0x7f1ac0075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:29.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac01046a0 0x7f1ac0075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 -- 172.21.15.55:0/2886489822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac01034a0 msgr2=0x7f1ac00771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac01034a0 0x7f1ac00771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 -- 172.21.15.55:0/2886489822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac0076600 con 0x7f1ac0105040 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abe59c700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac01034a0 0x7f1ac00771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1abed9d700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 0x7f1ac0075d50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009040 tx=0x7f1ab8004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.466+0000 7f1aaf7fe700 1 -- 172.21.15.55:0/2886489822 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab801f030 con 0x7f1ac0105040 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.467+0000 7f1aaf7fe700 1 -- 172.21.15.55:0/2886489822 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab8007a80 con 0x7f1ac0105040 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.467+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac01a36f0 con 0x7f1ac0105040 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.467+0000 7f1aaf7fe700 1 -- 172.21.15.55:0/2886489822 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80047e0 con 0x7f1ac0105040 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.467+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac01a3bc0 con 0x7f1ac0105040 2024-04-26T21:52:29.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.467+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac006a110 con 0x7f1ac0105040 2024-04-26T21:52:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.469+0000 7f1aaf7fe700 1 -- 172.21.15.55:0/2886489822 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1ab8003a00 con 0x7f1ac0105040 2024-04-26T21:52:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.469+0000 7f1aaf7fe700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1aa80a3cd0 0x7f1aa80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.469+0000 7f1abe59c700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1aa80a3cd0 0x7f1aa80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.470+0000 7f1aaf7fe700 1 -- 172.21.15.55:0/2886489822 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f1ab800f020 con 0x7f1ac0105040 2024-04-26T21:52:29.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.470+0000 7f1abe59c700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1aa80a3cd0 0x7f1aa80a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0104500 tx=0x7f1ab0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:29.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.473+0000 7f1aaf7fe700 1 -- 172.21.15.55:0/2886489822 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1ab8029020 con 0x7f1ac0105040 2024-04-26T21:52:29.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.689+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1ac0063860 con 0x7f1aa80a3cd0 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.702+0000 7f1aaf7fe700 1 -- 172.21.15.55:0/2886489822 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f1ac0063860 con 0x7f1aa80a3cd0 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stdout:mds.a smithi055 running (101s) 16s ago 101s 16.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 b8a6de870765 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stdout:mds.b smithi064 running (98s) 32s ago 98s 25.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 5568b9c16c3d 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stdout:mds.c smithi165 running (113s) 32s ago 112s 20.0M - 19.0.0-3306-gbc612ffb 55dc54019ad8 deb82a91f5eb 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stdout:mds.d smithi055 running (2m) 16s ago 2m 14.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 757b72f2bc21 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stdout:mds.e smithi064 running (118s) 32s ago 118s 18.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 318e305602ca 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stdout:mds.f smithi165 running (116s) 32s ago 115s 16.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 5fe755a3e503 2024-04-26T21:52:29.703 INFO:teuthology.orchestra.run.smithi055.stdout:mds.g smithi055 running (104s) 16s ago 104s 14.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 ad1578d168e4 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mds.h smithi064 running (110s) 32s ago 109s 16.8M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d2f9a90bf3b0 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mds.i smithi165 running (107s) 32s ago 107s 19.1M - 19.0.0-3306-gbc612ffb 55dc54019ad8 c86513383e0e 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mds.j smithi055 running (2m) 16s ago 2m 16.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 6442bc02a4d4 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mds.k smithi064 running (2m) 32s ago 2m 14.0M - 19.0.0-3306-gbc612ffb 55dc54019ad8 a6f81ec61086 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mds.l smithi165 running (2m) 32s ago 2m 15.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 1ab915fa5ffd 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mgr.x smithi055 *:9283,8765 running (9m) 16s ago 9m 517M - 19.0.0-3306-gbc612ffb 55dc54019ad8 60ea280c0780 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mgr.y smithi064 *:8443,8765 running (7m) 32s ago 7m 455M - 19.0.0-3306-gbc612ffb 55dc54019ad8 1e53c4197193 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mgr.z smithi165 *:8443,8765 running (7m) 32s ago 7m 455M - 19.0.0-3306-gbc612ffb 55dc54019ad8 fa46b4a04d46 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mon.a smithi055 running (9m) 16s ago 9m 52.1M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 8c4a4ee318a8 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mon.b smithi064 running (7m) 32s ago 7m 38.8M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 b8ed5b06c3bf 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:mon.c smithi165 running (7m) 32s ago 7m 39.2M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 2db75f7cb609 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:osd.0 smithi055 running (7m) 16s ago 7m 45.3M 4273M 19.0.0-3306-gbc612ffb 55dc54019ad8 40b044c2c7d6 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:osd.1 smithi064 running (6m) 32s ago 6m 48.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 893270bd8756 2024-04-26T21:52:29.704 INFO:teuthology.orchestra.run.smithi055.stdout:osd.2 smithi165 running (6m) 32s ago 6m 46.8M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 bb50a1830a01 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.3 smithi055 running (5m) 16s ago 5m 42.2M 4273M 19.0.0-3306-gbc612ffb 55dc54019ad8 5d6246f64f80 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.4 smithi064 running (5m) 32s ago 5m 42.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 b95b8e659f2e 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.5 smithi165 running (5m) 32s ago 5m 44.0M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 3cc43595ade5 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.6 smithi055 running (4m) 16s ago 4m 41.3M 4273M 19.0.0-3306-gbc612ffb 55dc54019ad8 fdf9272fffea 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.7 smithi064 running (4m) 32s ago 4m 42.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 025ddc51c965 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.8 smithi165 running (3m) 32s ago 3m 54.9M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 612a6165f1d8 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.9 smithi055 running (3m) 16s ago 3m 52.5M 4273M 19.0.0-3306-gbc612ffb 55dc54019ad8 15314487a054 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.10 smithi064 running (2m) 32s ago 2m 54.3M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 0b0de950e92f 2024-04-26T21:52:29.705 INFO:teuthology.orchestra.run.smithi055.stdout:osd.11 smithi165 running (2m) 32s ago 2m 52.3M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 12bd386a26db 2024-04-26T21:52:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1aa80a3cd0 msgr2=0x7f1aa80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1aa80a3cd0 0x7f1aa80a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0104500 tx=0x7f1ab0008040 comp rx=0 tx=0).stop 2024-04-26T21:52:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 msgr2=0x7f1ac0075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 0x7f1ac0075d50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009040 tx=0x7f1ab8004e00 comp rx=0 tx=0).stop 2024-04-26T21:52:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 shutdown_connections 2024-04-26T21:52:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ac0105040 0x7f1ac0075d50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1aa80a3cd0 0x7f1aa80a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac01034a0 0x7f1ac00771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 --2- 172.21.15.55:0/2886489822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac01046a0 0x7f1ac0075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 >> 172.21.15.55:0/2886489822 conn(0x7f1ac00fea40 msgr2=0x7f1ac01079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 shutdown_connections 2024-04-26T21:52:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:29.705+0000 7f1ac4fb0700 1 -- 172.21.15.55:0/2886489822 wait complete. 2024-04-26T21:52:30.544 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T21:52:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:30 smithi064 ceph-mon[32765]: pgmap v337: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:30 smithi064 ceph-mon[32765]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:30.797 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:30.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:30 smithi165 ceph-mon[33296]: pgmap v337: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:30.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:30 smithi165 ceph-mon[33296]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:30.919 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:30 smithi055 ceph-mon[29221]: pgmap v337: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:30.920 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:30 smithi055 ceph-mon[29221]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:32.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 -- 172.21.15.55:0/1355588736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec105030 msgr2=0x7f27ec10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:32.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 --2- 172.21.15.55:0/1355588736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec105030 0x7f27ec10b8e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f27e400b0d0 tx=0x7f27e400b3e0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 -- 172.21.15.55:0/1355588736 shutdown_connections 2024-04-26T21:52:32.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 --2- 172.21.15.55:0/1355588736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec105030 0x7f27ec10b8e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 --2- 172.21.15.55:0/1355588736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27ec103490 0x7f27ec1038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 --2- 172.21.15.55:0/1355588736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27ec104690 0x7f27ec104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 -- 172.21.15.55:0/1355588736 >> 172.21.15.55:0/1355588736 conn(0x7f27ec0fea30 msgr2=0x7f27ec100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 -- 172.21.15.55:0/1355588736 shutdown_connections 2024-04-26T21:52:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.531+0000 7f27f24d0700 1 -- 172.21.15.55:0/1355588736 wait complete. 2024-04-26T21:52:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f24d0700 1 Processor -- start 2024-04-26T21:52:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f24d0700 1 -- start start 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f24d0700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27ec103490 0x7f27ec1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f24d0700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec104690 0x7f27ec1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f24d0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27ec105030 0x7f27ec1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f24d0700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec10ef80 con 0x7f27ec104690 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f24d0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec10ee00 con 0x7f27ec105030 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f24d0700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec10f100 con 0x7f27ec103490 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec104690 0x7f27ec1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec104690 0x7f27ec1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45352/0 (socket says 172.21.15.55:45352) 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27eb7fe700 1 -- 172.21.15.55:0/2829479993 learned_addr learned my addr 172.21.15.55:0/2829479993 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27ebfff700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27ec103490 0x7f27ec1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.532+0000 7f27f0a6d700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27ec105030 0x7f27ec1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27eb7fe700 1 -- 172.21.15.55:0/2829479993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27ec103490 msgr2=0x7f27ec1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:32.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27eb7fe700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27ec103490 0x7f27ec1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27eb7fe700 1 -- 172.21.15.55:0/2829479993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27ec105030 msgr2=0x7f27ec1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27eb7fe700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27ec105030 0x7f27ec1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27eb7fe700 1 -- 172.21.15.55:0/2829479993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec1a1100 con 0x7f27ec104690 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27f0a6d700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27ec105030 0x7f27ec1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27ebfff700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27ec103490 0x7f27ec1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27eb7fe700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec104690 0x7f27ec1999e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b560 tx=0x7f27e000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27e97fa700 1 -- 172.21.15.55:0/2829479993 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0015020 con 0x7f27ec104690 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec1a13f0 con 0x7f27ec104690 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27e97fa700 1 -- 172.21.15.55:0/2829479993 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e0011ce0 con 0x7f27ec104690 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27e97fa700 1 -- 172.21.15.55:0/2829479993 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0010da0 con 0x7f27ec104690 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.533+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec1a19b0 con 0x7f27ec104690 2024-04-26T21:52:32.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.534+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec06a110 con 0x7f27ec104690 2024-04-26T21:52:32.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.535+0000 7f27e97fa700 1 -- 172.21.15.55:0/2829479993 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f27e0018020 con 0x7f27ec104690 2024-04-26T21:52:32.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.536+0000 7f27e97fa700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f27d40a3cd0 0x7f27d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:32.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.536+0000 7f27ebfff700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f27d40a3cd0 0x7f27d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:32.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.536+0000 7f27e97fa700 1 -- 172.21.15.55:0/2829479993 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f27e00cfd10 con 0x7f27ec104690 2024-04-26T21:52:32.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.536+0000 7f27ebfff700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f27d40a3cd0 0x7f27d40a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f27ec1044f0 tx=0x7f27dc00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:32.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.540+0000 7f27e97fa700 1 -- 172.21.15.55:0/2829479993 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f27e00d5020 con 0x7f27ec104690 2024-04-26T21:52:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:32 smithi064 ceph-mon[32765]: pgmap v338: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:32.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.756+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f27ec063860 con 0x7f27d40a3cd0 2024-04-26T21:52:32.757 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:32 smithi055 ceph-mon[29221]: pgmap v338: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:32.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.763+0000 7f27e97fa700 1 -- 172.21.15.55:0/2829479993 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f27ec063860 con 0x7f27d40a3cd0 2024-04-26T21:52:32.763 INFO:teuthology.orchestra.run.smithi055.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T21:52:32.763 INFO:teuthology.orchestra.run.smithi055.stdout:mds.all 12/12 35s ago 2m smithi055=a;smithi055=d;smithi055=g;smithi055=j;smithi064=b;smithi064=e;smithi064=h;smithi064=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-04-26T21:52:32.763 INFO:teuthology.orchestra.run.smithi055.stdout:mgr 3/3 35s ago 7m smithi055=x;smithi064=y;smithi165=z;count:3 2024-04-26T21:52:32.764 INFO:teuthology.orchestra.run.smithi055.stdout:mon 3/3 35s ago 8m smithi055:172.21.15.55=a;smithi064:172.21.15.64=b;smithi165:172.21.15.165=c;count:3 2024-04-26T21:52:32.764 INFO:teuthology.orchestra.run.smithi055.stdout:osd 12 35s ago - 2024-04-26T21:52:32.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f27d40a3cd0 msgr2=0x7f27d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f27d40a3cd0 0x7f27d40a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f27ec1044f0 tx=0x7f27dc00a680 comp rx=0 tx=0).stop 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec104690 msgr2=0x7f27ec1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec104690 0x7f27ec1999e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f27e000b560 tx=0x7f27e000b920 comp rx=0 tx=0).stop 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 shutdown_connections 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f27d40a3cd0 0x7f27d40a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27ec105030 0x7f27ec1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27ec103490 0x7f27ec1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 --2- 172.21.15.55:0/2829479993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f27ec104690 0x7f27ec1999e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 >> 172.21.15.55:0/2829479993 conn(0x7f27ec0fea30 msgr2=0x7f27ec1078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 shutdown_connections 2024-04-26T21:52:32.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:32.765+0000 7f27f24d0700 1 -- 172.21.15.55:0/2829479993 wait complete. 2024-04-26T21:52:32.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:32 smithi165 ceph-mon[33296]: pgmap v338: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:33.504 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T21:52:33.752 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:33.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:33 smithi064 ceph-mon[32765]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:33.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:33 smithi165 ceph-mon[33296]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:33.875 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:33 smithi055 ceph-mon[29221]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:34.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:34 smithi064 ceph-mon[32765]: pgmap v339: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T21:52:34.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:34 smithi165 ceph-mon[33296]: pgmap v339: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T21:52:34.902 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:34 smithi055 ceph-mon[29221]: pgmap v339: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 -- 172.21.15.55:0/4104772242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 msgr2=0x7f6fa4108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/4104772242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 0x7f6fa4108c30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009990 tx=0x7f6f98009ca0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 -- 172.21.15.55:0/4104772242 shutdown_connections 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/4104772242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fa4109170 0x7f6fa410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/4104772242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa4103f10 0x7f6fa4106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/4104772242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 0x7f6fa4108c30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 -- 172.21.15.55:0/4104772242 >> 172.21.15.55:0/4104772242 conn(0x7f6fa40fd920 msgr2=0x7f6fa40ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 -- 172.21.15.55:0/4104772242 shutdown_connections 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 -- 172.21.15.55:0/4104772242 wait complete. 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 Processor -- start 2024-04-26T21:52:35.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.432+0000 7f6fa9c67700 1 -- start start 2024-04-26T21:52:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa9c67700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa4103f10 0x7f6fa4103870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa9c67700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 0x7f6fa4101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa9c67700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fa4109170 0x7f6fa4102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa9c67700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410ef90 con 0x7f6fa4106840 2024-04-26T21:52:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa9c67700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410ee10 con 0x7f6fa4103f10 2024-04-26T21:52:35.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa9c67700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa410f110 con 0x7f6fa4109170 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 0x7f6fa4101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 0x7f6fa4101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45376/0 (socket says 172.21.15.55:45376) 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 -- 172.21.15.55:0/28814084 learned_addr learned my addr 172.21.15.55:0/28814084 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa37fe700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa4103f10 0x7f6fa4103870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa3fff700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fa4109170 0x7f6fa4102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 -- 172.21.15.55:0/28814084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fa4109170 msgr2=0x7f6fa4102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fa4109170 0x7f6fa4102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 -- 172.21.15.55:0/28814084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa4103f10 msgr2=0x7f6fa4103870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:35.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa4103f10 0x7f6fa4103870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 -- 172.21.15.55:0/28814084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa4102940 con 0x7f6fa4106840 2024-04-26T21:52:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa2ffd700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 0x7f6fa4101ec0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6f98005950 tx=0x7f6f980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa0ff9700 1 -- 172.21.15.55:0/28814084 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98027070 con 0x7f6fa4106840 2024-04-26T21:52:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa0ff9700 1 -- 172.21.15.55:0/28814084 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f98022470 con 0x7f6fa4106840 2024-04-26T21:52:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa4102bd0 con 0x7f6fa4106840 2024-04-26T21:52:35.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.433+0000 7f6fa0ff9700 1 -- 172.21.15.55:0/28814084 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9800f6a0 con 0x7f6fa4106840 2024-04-26T21:52:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.434+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa41a8070 con 0x7f6fa4106840 2024-04-26T21:52:35.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.434+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa406a110 con 0x7f6fa4106840 2024-04-26T21:52:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.436+0000 7f6fa0ff9700 1 -- 172.21.15.55:0/28814084 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6f9802a070 con 0x7f6fa4106840 2024-04-26T21:52:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.436+0000 7f6fa0ff9700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f8c0a3d50 0x7f6f8c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.436+0000 7f6fa37fe700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f8c0a3d50 0x7f6f8c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:35.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.436+0000 7f6fa0ff9700 1 -- 172.21.15.55:0/28814084 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6f980d1670 con 0x7f6fa4106840 2024-04-26T21:52:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.437+0000 7f6fa37fe700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f8c0a3d50 0x7f6f8c0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6f94005fd0 tx=0x7f6f9400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:35.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.440+0000 7f6fa0ff9700 1 -- 172.21.15.55:0/28814084 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6f98013070 con 0x7f6fa4106840 2024-04-26T21:52:35.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.654+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f6fa4063860 con 0x7f6f8c0a3d50 2024-04-26T21:52:35.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.656+0000 7f6fa0ff9700 1 -- 172.21.15.55:0/28814084 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f6fa4063860 con 0x7f6f8c0a3d50 2024-04-26T21:52:35.656 INFO:teuthology.orchestra.run.smithi055.stdout:HOST ADDR LABELS STATUS 2024-04-26T21:52:35.656 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055 172.21.15.55 2024-04-26T21:52:35.657 INFO:teuthology.orchestra.run.smithi055.stdout:smithi064 172.21.15.64 2024-04-26T21:52:35.657 INFO:teuthology.orchestra.run.smithi055.stdout:smithi165 172.21.15.165 2024-04-26T21:52:35.657 INFO:teuthology.orchestra.run.smithi055.stdout:3 hosts in cluster 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f8c0a3d50 msgr2=0x7f6f8c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f8c0a3d50 0x7f6f8c0a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6f94005fd0 tx=0x7f6f9400a560 comp rx=0 tx=0).stop 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 msgr2=0x7f6fa4101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 0x7f6fa4101ec0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6f98005950 tx=0x7f6f980047b0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 shutdown_connections 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fa4109170 0x7f6fa4102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f8c0a3d50 0x7f6f8c0a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fa4103f10 0x7f6fa4103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 --2- 172.21.15.55:0/28814084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6fa4106840 0x7f6fa4101ec0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.658+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 >> 172.21.15.55:0/28814084 conn(0x7f6fa40fd920 msgr2=0x7f6fa4104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.659+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 shutdown_connections 2024-04-26T21:52:35.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:35.659+0000 7f6fa9c67700 1 -- 172.21.15.55:0/28814084 wait complete. 2024-04-26T21:52:36.282 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T21:52:36.539 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:36.572 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:36 smithi055 ceph-mon[29221]: pgmap v340: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:36.572 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:36 smithi055 ceph-mon[29221]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:36 smithi064 ceph-mon[32765]: pgmap v340: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:36.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:36 smithi064 ceph-mon[32765]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:36.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:36 smithi165 ceph-mon[33296]: pgmap v340: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-26T21:52:36.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:36 smithi165 ceph-mon[33296]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 -- 172.21.15.55:0/3160169100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8968103f40 msgr2=0x7f896810a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3160169100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8968103f40 0x7f896810a7f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f896000b0d0 tx=0x7f896000b3e0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 -- 172.21.15.55:0/3160169100 shutdown_connections 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3160169100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8968103f40 0x7f896810a7f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3160169100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89681023a0 0x7f89681027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3160169100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89681035a0 0x7f8968103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 -- 172.21.15.55:0/3160169100 >> 172.21.15.55:0/3160169100 conn(0x7f89680fd940 msgr2=0x7f89680ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 -- 172.21.15.55:0/3160169100 shutdown_connections 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.279+0000 7f896e6b2700 1 -- 172.21.15.55:0/3160169100 wait complete. 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f896e6b2700 1 Processor -- start 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f896e6b2700 1 -- start start 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f896e6b2700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89681023a0 0x7f89681983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f896e6b2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89681035a0 0x7f8968198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f896e6b2700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8968103f40 0x7f896819f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f896e6b2700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f896810de60 con 0x7f89681023a0 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f896e6b2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f896810dce0 con 0x7f89681035a0 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f896e6b2700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f896810dfe0 con 0x7f8968103f40 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f8967fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89681023a0 0x7f89681983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f8967fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89681023a0 0x7f89681983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45404/0 (socket says 172.21.15.55:45404) 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.280+0000 7f8967fff700 1 -- 172.21.15.55:0/3189179721 learned_addr learned my addr 172.21.15.55:0/3189179721 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f89677fe700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89681035a0 0x7f8968198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f896cc4f700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8968103f40 0x7f896819f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f8967fff700 1 -- 172.21.15.55:0/3189179721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8968103f40 msgr2=0x7f896819f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f8967fff700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8968103f40 0x7f896819f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f8967fff700 1 -- 172.21.15.55:0/3189179721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89681035a0 msgr2=0x7f8968198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f8967fff700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89681035a0 0x7f8968198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f8967fff700 1 -- 172.21.15.55:0/3189179721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89681a0020 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f89677fe700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89681035a0 0x7f8968198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f8967fff700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89681023a0 0x7f89681983c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f895c00c9c0 tx=0x7f895c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f89657fa700 1 -- 172.21.15.55:0/3189179721 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c007ac0 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f89657fa700 1 -- 172.21.15.55:0/3189179721 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f895c007c60 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89681a0310 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f89657fa700 1 -- 172.21.15.55:0/3189179721 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c018660 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.281+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89681a08a0 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.282+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896806a110 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.283+0000 7f89657fa700 1 -- 172.21.15.55:0/3189179721 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f895c004020 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.284+0000 7f89657fa700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89500a3d50 0x7f89500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.284+0000 7f89677fe700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89500a3d50 0x7f89500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.284+0000 7f89657fa700 1 -- 172.21.15.55:0/3189179721 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f895c0d0780 con 0x7f89681023a0 2024-04-26T21:52:38.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.284+0000 7f89677fe700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89500a3d50 0x7f89500a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8958005950 tx=0x7f8958005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:38.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.287+0000 7f89657fa700 1 -- 172.21.15.55:0/3189179721 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f895c014030 con 0x7f89681023a0 2024-04-26T21:52:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.508+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8968063860 con 0x7f89500a3d50 2024-04-26T21:52:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.510+0000 7f89657fa700 1 -- 172.21.15.55:0/3189179721 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8968063860 con 0x7f89500a3d50 2024-04-26T21:52:38.510 INFO:teuthology.orchestra.run.smithi055.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T21:52:38.510 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102VS4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-04-26T21:52:38.510 INFO:teuthology.orchestra.run.smithi055.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-26T21:52:38.510 INFO:teuthology.orchestra.run.smithi055.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-26T21:52:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89500a3d50 msgr2=0x7f89500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89500a3d50 0x7f89500a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8958005950 tx=0x7f8958005f50 comp rx=0 tx=0).stop 2024-04-26T21:52:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89681023a0 msgr2=0x7f89681983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89681023a0 0x7f89681983c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f895c00c9c0 tx=0x7f895c00cd80 comp rx=0 tx=0).stop 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 shutdown_connections 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8968103f40 0x7f896819f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89681023a0 0x7f89681983c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89500a3d50 0x7f89500a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 --2- 172.21.15.55:0/3189179721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89681035a0 0x7f8968198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 >> 172.21.15.55:0/3189179721 conn(0x7f89680fd940 msgr2=0x7f89681067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 shutdown_connections 2024-04-26T21:52:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:38.512+0000 7f896e6b2700 1 -- 172.21.15.55:0/3189179721 wait complete. 2024-04-26T21:52:38.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:38 smithi064 ceph-mon[32765]: pgmap v341: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:38.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:38 smithi165 ceph-mon[33296]: pgmap v341: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:38.827 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:38 smithi055 ceph-mon[29221]: pgmap v341: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:39.151 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T21:52:39.160 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi055.front.sepia.ceph.com 2024-04-26T21:52:39.160 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T21:52:39.412 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:39.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:39 smithi064 ceph-mon[32765]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:39.774 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:39 smithi055 ceph-mon[29221]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:39.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:39 smithi165 ceph-mon[33296]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:52:40.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:40 smithi064 ceph-mon[32765]: pgmap v342: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:40.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:40 smithi055 ceph-mon[29221]: pgmap v342: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:40.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:40 smithi165 ceph-mon[33296]: pgmap v342: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 -- 172.21.15.55:0/502569365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc104b40 msgr2=0x7f3ecc104f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 --2- 172.21.15.55:0/502569365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc104b40 0x7f3ecc104f60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009990 tx=0x7f3ebc009ca0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 -- 172.21.15.55:0/502569365 shutdown_connections 2024-04-26T21:52:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 --2- 172.21.15.55:0/502569365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ecc101f60 0x7f3ecc10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 --2- 172.21.15.55:0/502569365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc104b40 0x7f3ecc104f60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 --2- 172.21.15.55:0/502569365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ecc1015c0 0x7f3ecc101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 -- 172.21.15.55:0/502569365 >> 172.21.15.55:0/502569365 conn(0x7f3ecc0fd180 msgr2=0x7f3ecc0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 -- 172.21.15.55:0/502569365 shutdown_connections 2024-04-26T21:52:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 -- 172.21.15.55:0/502569365 wait complete. 2024-04-26T21:52:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.159+0000 7f3ed1677700 1 Processor -- start 2024-04-26T21:52:41.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ed1677700 1 -- start start 2024-04-26T21:52:41.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ed1677700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc1015c0 0x7f3ecc104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:41.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ed1677700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ecc101f60 0x7f3ecc102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:41.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ed1677700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ecc104b40 0x7f3ecc102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:41.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ed1677700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc10f000 con 0x7f3ecc1015c0 2024-04-26T21:52:41.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ed1677700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc10ee80 con 0x7f3ecc101f60 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ed1677700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc10f180 con 0x7f3ecc104b40 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc1015c0 0x7f3ecc104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc1015c0 0x7f3ecc104240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54000/0 (socket says 172.21.15.55:54000) 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecaffd700 1 -- 172.21.15.55:0/2455237531 learned_addr learned my addr 172.21.15.55:0/2455237531 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3eca7fc700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ecc101f60 0x7f3ecc102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecb7fe700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ecc104b40 0x7f3ecc102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecaffd700 1 -- 172.21.15.55:0/2455237531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ecc104b40 msgr2=0x7f3ecc102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecaffd700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ecc104b40 0x7f3ecc102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecaffd700 1 -- 172.21.15.55:0/2455237531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ecc101f60 msgr2=0x7f3ecc102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecaffd700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ecc101f60 0x7f3ecc102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.160+0000 7f3ecaffd700 1 -- 172.21.15.55:0/2455237531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc103680 con 0x7f3ecc1015c0 2024-04-26T21:52:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.161+0000 7f3ecaffd700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc1015c0 0x7f3ecc104240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009fb0 tx=0x7f3ebc003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.161+0000 7f3eb3fff700 1 -- 172.21.15.55:0/2455237531 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc026070 con 0x7f3ecc1015c0 2024-04-26T21:52:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.161+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc1a37d0 con 0x7f3ecc1015c0 2024-04-26T21:52:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.161+0000 7f3eb3fff700 1 -- 172.21.15.55:0/2455237531 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ebc0047e0 con 0x7f3ecc1015c0 2024-04-26T21:52:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.161+0000 7f3eb3fff700 1 -- 172.21.15.55:0/2455237531 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0053b0 con 0x7f3ecc1015c0 2024-04-26T21:52:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.161+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc1a3c20 con 0x7f3ecc1015c0 2024-04-26T21:52:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.162+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ecc06a110 con 0x7f3ecc1015c0 2024-04-26T21:52:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.163+0000 7f3eb3fff700 1 -- 172.21.15.55:0/2455237531 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3ebc029070 con 0x7f3ecc1015c0 2024-04-26T21:52:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.164+0000 7f3eb3fff700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3eb40a3d50 0x7f3eb40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.164+0000 7f3eca7fc700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3eb40a3d50 0x7f3eb40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.164+0000 7f3eb3fff700 1 -- 172.21.15.55:0/2455237531 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f3ebc0d1540 con 0x7f3ecc1015c0 2024-04-26T21:52:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.164+0000 7f3eca7fc700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3eb40a3d50 0x7f3eb40a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0005950 tx=0x7f3ec000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:41.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.168+0000 7f3eb3fff700 1 -- 172.21.15.55:0/2455237531 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3ebc030020 con 0x7f3ecc1015c0 2024-04-26T21:52:41.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.423+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3ecc064030 con 0x7f3ecc1015c0 2024-04-26T21:52:41.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.423+0000 7f3eb3fff700 1 -- 172.21.15.55:0/2455237531 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3044 (secure 0 0 0) 0x7f3ebc013070 con 0x7f3ecc1015c0 2024-04-26T21:52:41.424 INFO:teuthology.orchestra.run.smithi055.stdout:e20 2024-04-26T21:52:41.424 INFO:teuthology.orchestra.run.smithi055.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T21:52:41.424 INFO:teuthology.orchestra.run.smithi055.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:52:41.424 INFO:teuthology.orchestra.run.smithi055.stdout:legacy client fscid: 1 2024-04-26T21:52:41.424 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:41.424 INFO:teuthology.orchestra.run.smithi055.stdout:Filesystem 'cephfs' (1) 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:fs_name cephfs 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:epoch 20 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:created 2024-04-26T21:50:56.719940+0000 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:modified 2024-04-26T21:51:01.298886+0000 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:tableserver 0 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:root 0 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:session_timeout 300 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:session_autoclose 300 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:max_file_size 1099511627776 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:max_xattr_size 65536 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:required_client_features {} 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:last_failure 0 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:last_failure_osd_epoch 0 2024-04-26T21:52:41.425 INFO:teuthology.orchestra.run.smithi055.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:max_mds 3 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:in 0,1,2 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:up {0=24551,1=24509,2=24502} 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:failed 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:damaged 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:stopped 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:data_pools [3] 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:metadata_pool 2 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:inline_data disabled 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:balancer 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:bal_rank_mask -1 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:standby_count_wanted 0 2024-04-26T21:52:41.426 INFO:teuthology.orchestra.run.smithi055.stdout:qdb_cluster leader: 24551 members: 24502,24509,24551 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b{0:24551} state up:active seq 3 addr [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c{0:24487} state up:standby-replay seq 1 addr [v2:172.21.15.165:6836/2954631237,v1:172.21.15.165:6837/2954631237] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.h{1:24509} state up:active seq 6 addr [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.e{1:24479} state up:standby-replay seq 1 addr [v2:172.21.15.64:6834/2328208855,v1:172.21.15.64:6835/2328208855] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.i{2:24502} state up:active seq 6 addr [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.f{2:24469} state up:standby-replay seq 1 addr [v2:172.21.15.165:6834/125466680,v1:172.21.15.165:6835/125466680] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:Standby daemons: 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.55:6834/1727764470,v1:172.21.15.55:6835/1727764470] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.55:6836/3725383900,v1:172.21.15.55:6837/3725383900] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.g{-1:14592} state up:standby seq 1 addr [v2:172.21.15.55:6838/2681477087,v1:172.21.15.55:6839/2681477087] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a{-1:14604} state up:standby seq 1 addr [v2:172.21.15.55:6840/1689426633,v1:172.21.15.55:6841/1689426633] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.427 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.l{-1:24427} state up:standby seq 1 addr [v2:172.21.15.165:6832/2896142285,v1:172.21.15.165:6833/2896142285] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.428 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.k{-1:24440} state up:standby seq 1 addr [v2:172.21.15.64:6832/689132199,v1:172.21.15.64:6833/689132199] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T21:52:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3eb40a3d50 msgr2=0x7f3eb40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3eb40a3d50 0x7f3eb40a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0005950 tx=0x7f3ec000a300 comp rx=0 tx=0).stop 2024-04-26T21:52:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc1015c0 msgr2=0x7f3ecc104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc1015c0 0x7f3ecc104240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009fb0 tx=0x7f3ebc003890 comp rx=0 tx=0).stop 2024-04-26T21:52:41.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 shutdown_connections 2024-04-26T21:52:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ecc104b40 0x7f3ecc102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ecc1015c0 0x7f3ecc104240 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3eb40a3d50 0x7f3eb40a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 --2- 172.21.15.55:0/2455237531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ecc101f60 0x7f3ecc102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 >> 172.21.15.55:0/2455237531 conn(0x7f3ecc0fd180 msgr2=0x7f3ecc107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 shutdown_connections 2024-04-26T21:52:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:41.427+0000 7f3ed1677700 1 -- 172.21.15.55:0/2455237531 wait complete. 2024-04-26T21:52:41.429 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 20 2024-04-26T21:52:41.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:41 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2455237531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:52:41.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:41 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2455237531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:52:41.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:41 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2455237531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T21:52:42.080 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T21:52:42.331 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:42.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:42 smithi064 ceph-mon[32765]: pgmap v343: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:42.760 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:42 smithi055 ceph-mon[29221]: pgmap v343: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:42.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:42 smithi165 ceph-mon[33296]: pgmap v343: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 -- 172.21.15.55:0/1045037292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 msgr2=0x7f5ad4104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 --2- 172.21.15.55:0/1045037292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 0x7f5ad4104ae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00099c0 tx=0x7f5ad0009cd0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 -- 172.21.15.55:0/1045037292 shutdown_connections 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 --2- 172.21.15.55:0/1045037292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ad4105020 0x7f5ad410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 --2- 172.21.15.55:0/1045037292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ad4103480 0x7f5ad41038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 --2- 172.21.15.55:0/1045037292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 0x7f5ad4104ae0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 -- 172.21.15.55:0/1045037292 >> 172.21.15.55:0/1045037292 conn(0x7f5ad40fea20 msgr2=0x7f5ad4100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 -- 172.21.15.55:0/1045037292 shutdown_connections 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 -- 172.21.15.55:0/1045037292 wait complete. 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 Processor -- start 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.162+0000 7f5adc095700 1 -- start start 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5adc095700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ad4103480 0x7f5ad40771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5adc095700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 0x7f5ad4075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5adc095700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ad4105020 0x7f5ad4075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5adc095700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad410f060 con 0x7f5ad4104680 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5adc095700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad410eee0 con 0x7f5ad4105020 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5adc095700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad410f1e0 con 0x7f5ad4103480 2024-04-26T21:52:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 0x7f5ad4075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 0x7f5ad4075810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54024/0 (socket says 172.21.15.55:54024) 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 -- 172.21.15.55:0/319997637 learned_addr learned my addr 172.21.15.55:0/319997637 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9e31700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ad4103480 0x7f5ad40771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ada632700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ad4105020 0x7f5ad4075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 -- 172.21.15.55:0/319997637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ad4103480 msgr2=0x7f5ad40771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ad4103480 0x7f5ad40771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 -- 172.21.15.55:0/319997637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ad4105020 msgr2=0x7f5ad4075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ad4105020 0x7f5ad4075d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 -- 172.21.15.55:0/319997637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad4076600 con 0x7f5ad4104680 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ada632700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ad4105020 0x7f5ad4075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5ad9630700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 0x7f5ad4075810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009f50 tx=0x7f5ad00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5acaffd700 1 -- 172.21.15.55:0/319997637 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0027070 con 0x7f5ad4104680 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5acaffd700 1 -- 172.21.15.55:0/319997637 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad0022470 con 0x7f5ad4104680 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad41a3830 con 0x7f5ad4104680 2024-04-26T21:52:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.163+0000 7f5acaffd700 1 -- 172.21.15.55:0/319997637 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000f6c0 con 0x7f5ad4104680 2024-04-26T21:52:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.164+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad41a3d00 con 0x7f5ad4104680 2024-04-26T21:52:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.164+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad406a110 con 0x7f5ad4104680 2024-04-26T21:52:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.166+0000 7f5acaffd700 1 -- 172.21.15.55:0/319997637 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ad002a070 con 0x7f5ad4104680 2024-04-26T21:52:44.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.166+0000 7f5acaffd700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5ac00a3cd0 0x7f5ac00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:44.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.166+0000 7f5ad9e31700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5ac00a3cd0 0x7f5ac00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:44.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.166+0000 7f5acaffd700 1 -- 172.21.15.55:0/319997637 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f5ad00d1d20 con 0x7f5ad4104680 2024-04-26T21:52:44.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.167+0000 7f5ad9e31700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5ac00a3cd0 0x7f5ac00a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5ad41044e0 tx=0x7f5ac4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:44.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.170+0000 7f5acaffd700 1 -- 172.21.15.55:0/319997637 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5ad0031070 con 0x7f5ad4104680 2024-04-26T21:52:44.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.381+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f5ad402e490 con 0x7f5ad4104680 2024-04-26T21:52:44.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.382+0000 7f5acaffd700 1 -- 172.21.15.55:0/319997637 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5326 (secure 0 0 0) 0x7f5ad0013070 con 0x7f5ad4104680 2024-04-26T21:52:44.382 INFO:teuthology.orchestra.run.smithi055.stdout:epoch 87 2024-04-26T21:52:44.382 INFO:teuthology.orchestra.run.smithi055.stdout:fsid c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T21:52:44.382 INFO:teuthology.orchestra.run.smithi055.stdout:created 2024-04-26T21:42:52.792981+0000 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:modified 2024-04-26T21:51:50.534100+0000 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:crush_version 26 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:full_ratio 0.95 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:backfillfull_ratio 0.9 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:nearfull_ratio 0.85 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:require_min_compat_client luminous 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:min_compat_client luminous 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:require_osd_release squid 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:stretch_mode_enabled false 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-26T21:52:44.383 INFO:teuthology.orchestra.run.smithi055.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.69 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:max_osd 12 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.55:6802/2219207145,v1:172.21.15.55:6803/2219207145] [v2:172.21.15.55:6804/2219207145,v1:172.21.15.55:6805/2219207145] exists,up e004f5b1-6940-4e27-9d6a-7b4c81d08f27 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6800/794827991,v1:172.21.15.64:6801/794827991] [v2:172.21.15.64:6802/794827991,v1:172.21.15.64:6803/794827991] exists,up a6715741-0a37-48df-8dd4-d07f539706f6 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/2376036424,v1:172.21.15.165:6801/2376036424] [v2:172.21.15.165:6802/2376036424,v1:172.21.15.165:6803/2376036424] exists,up 15bb5be1-55c9-4a43-8253-fcefc1d5041f 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.55:6810/3700452481,v1:172.21.15.55:6811/3700452481] [v2:172.21.15.55:6812/3700452481,v1:172.21.15.55:6813/3700452481] exists,up 2fcc948f-ba34-488a-be7b-ec4579b3a4c6 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6808/526876100,v1:172.21.15.64:6809/526876100] [v2:172.21.15.64:6810/526876100,v1:172.21.15.64:6811/526876100] exists,up daf304fa-67ee-45a4-a3ca-8140292c27ef 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/1409137199,v1:172.21.15.165:6809/1409137199] [v2:172.21.15.165:6810/1409137199,v1:172.21.15.165:6811/1409137199] exists,up 1f42d9d6-e34c-4085-809b-fe73d7da4281 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.55:6818/4025342485,v1:172.21.15.55:6819/4025342485] [v2:172.21.15.55:6820/4025342485,v1:172.21.15.55:6821/4025342485] exists,up c68fd703-709c-4adf-8437-6cdafc737f7d 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6816/1454699976,v1:172.21.15.64:6817/1454699976] [v2:172.21.15.64:6818/1454699976,v1:172.21.15.64:6819/1454699976] exists,up d96f9d6b-186d-48ba-9ade-8ce41775d9d3 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/2092227616,v1:172.21.15.165:6817/2092227616] [v2:172.21.15.165:6818/2092227616,v1:172.21.15.165:6819/2092227616] exists,up f954a5a8-dd31-40f3-88eb-34444e538d2a 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.55:6826/3581040676,v1:172.21.15.55:6827/3581040676] [v2:172.21.15.55:6828/3581040676,v1:172.21.15.55:6829/3581040676] exists,up b9769f00-30fe-4ecd-9e58-b81cc6e94d63 2024-04-26T21:52:44.384 INFO:teuthology.orchestra.run.smithi055.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6824/3001316208,v1:172.21.15.64:6825/3001316208] [v2:172.21.15.64:6826/3001316208,v1:172.21.15.64:6827/3001316208] exists,up f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/1470164936,v1:172.21.15.165:6825/1470164936] [v2:172.21.15.165:6826/1470164936,v1:172.21.15.165:6827/1470164936] exists,up 09405356-ca7d-48a4-8cc2-caf08446e063 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 2.0 [8,4] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 2.4 [2,11] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 2.10 [2,4] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 2.1a [8,1] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 2.1b [5,11] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 3.11 [4,8] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 3.1b [4,6] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 3.28 [5,7] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 3.38 [3,9] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:pg_upmap_items 3.3c [4,9] 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:0/1387634878 expires 2024-04-27T21:43:49.252173+0000 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:0/2133918566 expires 2024-04-27T21:43:49.252173+0000 2024-04-26T21:52:44.385 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:6801/2562126894 expires 2024-04-27T21:43:49.252173+0000 2024-04-26T21:52:44.386 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:6800/3795671875 expires 2024-04-27T21:43:21.996132+0000 2024-04-26T21:52:44.386 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:0/3938566585 expires 2024-04-27T21:43:21.996132+0000 2024-04-26T21:52:44.386 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:0/4026035232 expires 2024-04-27T21:43:21.996132+0000 2024-04-26T21:52:44.386 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:0/4038340190 expires 2024-04-27T21:43:21.996132+0000 2024-04-26T21:52:44.386 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:6801/3795671875 expires 2024-04-27T21:43:21.996132+0000 2024-04-26T21:52:44.386 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:0/1492865350 expires 2024-04-27T21:43:49.252173+0000 2024-04-26T21:52:44.386 INFO:teuthology.orchestra.run.smithi055.stdout:blocklist 172.21.15.55:6800/2562126894 expires 2024-04-27T21:43:49.252173+0000 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5ac00a3cd0 msgr2=0x7f5ac00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5ac00a3cd0 0x7f5ac00a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5ad41044e0 tx=0x7f5ac4009190 comp rx=0 tx=0).stop 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 msgr2=0x7f5ad4075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 0x7f5ad4075810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009f50 tx=0x7f5ad00047b0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 shutdown_connections 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5ac00a3cd0 0x7f5ac00a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ad4105020 0x7f5ad4075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ad4103480 0x7f5ad40771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 --2- 172.21.15.55:0/319997637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5ad4104680 0x7f5ad4075810 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:44.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 >> 172.21.15.55:0/319997637 conn(0x7f5ad40fea20 msgr2=0x7f5ad4107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:44.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 shutdown_connections 2024-04-26T21:52:44.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:44.385+0000 7f5adc095700 1 -- 172.21.15.55:0/319997637 wait complete. 2024-04-26T21:52:44.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:44 smithi165 ceph-mon[33296]: pgmap v344: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:44.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:44 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/319997637' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:52:44.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:44 smithi055 ceph-mon[29221]: pgmap v344: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:44.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:44 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/319997637' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:52:45.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:44 smithi064 ceph-mon[32765]: pgmap v344: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:52:45.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:44 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/319997637' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T21:52:45.090 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T21:52:45.101 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 -- 172.21.15.55:0/411935893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0103440 msgr2=0x7f2fb0109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 --2- 172.21.15.55:0/411935893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0103440 0x7f2fb0109cd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009ca0 tx=0x7f2fa802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 -- 172.21.15.55:0/411935893 shutdown_connections 2024-04-26T21:52:45.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 --2- 172.21.15.55:0/411935893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0103440 0x7f2fb0109cd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 --2- 172.21.15.55:0/411935893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb0102a80 0x7f2fb0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 --2- 172.21.15.55:0/411935893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb0101890 0x7f2fb0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 -- 172.21.15.55:0/411935893 >> 172.21.15.55:0/411935893 conn(0x7f2fb00fce20 msgr2=0x7f2fb00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 -- 172.21.15.55:0/411935893 shutdown_connections 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.212+0000 7f2fb7304640 1 -- 172.21.15.55:0/411935893 wait complete. 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb7304640 1 Processor -- start 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb7304640 1 -- start start 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb7304640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0101890 0x7f2fb0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb7304640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb0102a80 0x7f2fb01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb7304640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb0103440 0x7f2fb01a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb7304640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010d420 con 0x7f2fb0101890 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb7304640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010d2a0 con 0x7f2fb0103440 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb7304640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010d5a0 con 0x7f2fb0102a80 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0101890 0x7f2fb0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0101890 0x7f2fb0198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54044/0 (socket says 172.21.15.55:54044) 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 -- 172.21.15.55:0/1332152397 learned_addr learned my addr 172.21.15.55:0/1332152397 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb58a1640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb0103440 0x7f2fb01a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 -- 172.21.15.55:0/1332152397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb0102a80 msgr2=0x7f2fb01991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb0102a80 0x7f2fb01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 -- 172.21.15.55:0/1332152397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb0103440 msgr2=0x7f2fb01a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb0103440 0x7f2fb01a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 -- 172.21.15.55:0/1332152397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb01a07f0 con 0x7f2fb0101890 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb58a1640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb0103440 0x7f2fb01a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.213+0000 7f2fb50a0640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0101890 0x7f2fb0198c70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00d770 tx=0x7f2fac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.214+0000 7f2fa67fc640 1 -- 172.21.15.55:0/1332152397 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac017070 con 0x7f2fb0101890 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.214+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb01a0ae0 con 0x7f2fb0101890 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.214+0000 7f2fa67fc640 1 -- 172.21.15.55:0/1332152397 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fac009c30 con 0x7f2fb0101890 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.214+0000 7f2fa67fc640 1 -- 172.21.15.55:0/1332152397 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac003cc0 con 0x7f2fb0101890 2024-04-26T21:52:45.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.214+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb01a1030 con 0x7f2fb0101890 2024-04-26T21:52:45.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.215+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb01a1260 con 0x7f2fb0101890 2024-04-26T21:52:45.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.216+0000 7f2fa67fc640 1 -- 172.21.15.55:0/1332152397 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2fac003e60 con 0x7f2fb0101890 2024-04-26T21:52:45.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.217+0000 7f2fa67fc640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2f9c0a3c40 0x7f2f9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.217+0000 7f2fb489f640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2f9c0a3c40 0x7f2f9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.217+0000 7f2fa67fc640 1 -- 172.21.15.55:0/1332152397 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f2fac0d0010 con 0x7f2fb0101890 2024-04-26T21:52:45.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.217+0000 7f2fb489f640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2f9c0a3c40 0x7f2f9c0a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fb019a0e0 tx=0x7f2fa0006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:45.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.221+0000 7f2fa67fc640 1 -- 172.21.15.55:0/1332152397 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2fac09c870 con 0x7f2fb0101890 2024-04-26T21:52:45.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.411+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fb01a1660 con 0x7f2fb0101890 2024-04-26T21:52:45.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.412+0000 7f2fa67fc640 1 -- 172.21.15.55:0/1332152397 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22713 (secure 0 0 0) 0x7f2fac009de0 con 0x7f2fb0101890 2024-04-26T21:52:45.413 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:45.413 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:51:50.534100+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:45.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2f9c0a3c40 msgr2=0x7f2f9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2f9c0a3c40 0x7f2f9c0a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fb019a0e0 tx=0x7f2fa0006c90 comp rx=0 tx=0).stop 2024-04-26T21:52:45.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0101890 msgr2=0x7f2fb0198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0101890 0x7f2fb0198c70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00d770 tx=0x7f2fac00dc30 comp rx=0 tx=0).stop 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 shutdown_connections 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2f9c0a3c40 0x7f2f9c0a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb0103440 0x7f2fb01a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb0102a80 0x7f2fb01991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 --2- 172.21.15.55:0/1332152397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fb0101890 0x7f2fb0198c70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 >> 172.21.15.55:0/1332152397 conn(0x7f2fb00fce20 msgr2=0x7f2fb006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 shutdown_connections 2024-04-26T21:52:45.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.415+0000 7f2fb7304640 1 -- 172.21.15.55:0/1332152397 wait complete. 2024-04-26T21:52:45.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:52:45.427 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:52:45.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 -- 172.21.15.55:0/1694722755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4108650 msgr2=0x7fa0b410aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 --2- 172.21.15.55:0/1694722755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4108650 0x7fa0b410aaa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009ca0 tx=0x7fa0a402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 -- 172.21.15.55:0/1694722755 shutdown_connections 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 --2- 172.21.15.55:0/1694722755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4108650 0x7fa0b410aaa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 --2- 172.21.15.55:0/1694722755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b4105d20 0x7fa0b4108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 --2- 172.21.15.55:0/1694722755 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0b41033f0 0x7fa0b41057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 -- 172.21.15.55:0/1694722755 >> 172.21.15.55:0/1694722755 conn(0x7fa0b40fcde0 msgr2=0x7fa0b40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 -- 172.21.15.55:0/1694722755 shutdown_connections 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.531+0000 7fa0b9991640 1 -- 172.21.15.55:0/1694722755 wait complete. 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b9991640 1 Processor -- start 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b9991640 1 -- start start 2024-04-26T21:52:45.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b9991640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b41033f0 0x7fa0b419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b9991640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4105d20 0x7fa0b419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b9991640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0b4108650 0x7fa0b41a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b9991640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410e590 con 0x7fa0b4105d20 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b9991640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410e410 con 0x7fa0b41033f0 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b9991640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410e710 con 0x7fa0b4108650 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4105d20 0x7fa0b419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4105d20 0x7fa0b419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54056/0 (socket says 172.21.15.55:54056) 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.532+0000 7fa0b27fc640 1 -- 172.21.15.55:0/3890851075 learned_addr learned my addr 172.21.15.55:0/3890851075 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b2ffd640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b41033f0 0x7fa0b419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b37fe640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0b4108650 0x7fa0b41a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b27fc640 1 -- 172.21.15.55:0/3890851075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0b4108650 msgr2=0x7fa0b41a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0b4108650 0x7fa0b41a24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b27fc640 1 -- 172.21.15.55:0/3890851075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b41033f0 msgr2=0x7fa0b419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b41033f0 0x7fa0b419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b27fc640 1 -- 172.21.15.55:0/3890851075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b41a2b60 con 0x7fa0b4105d20 2024-04-26T21:52:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b2ffd640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b41033f0 0x7fa0b419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4105d20 0x7fa0b419b480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00b450 tx=0x7fa09c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b898f640 1 -- 172.21.15.55:0/3890851075 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c015020 con 0x7fa0b4105d20 2024-04-26T21:52:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b898f640 1 -- 172.21.15.55:0/3890851075 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09c011d40 con 0x7fa0b4105d20 2024-04-26T21:52:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b41a2e50 con 0x7fa0b4105d20 2024-04-26T21:52:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b898f640 1 -- 172.21.15.55:0/3890851075 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c00f340 con 0x7fa0b4105d20 2024-04-26T21:52:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.533+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41a3390 con 0x7fa0b4105d20 2024-04-26T21:52:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.534+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b410a140 con 0x7fa0b4105d20 2024-04-26T21:52:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.535+0000 7fa0b898f640 1 -- 172.21.15.55:0/3890851075 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa09c018020 con 0x7fa0b4105d20 2024-04-26T21:52:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.536+0000 7fa0b898f640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0a00a3c40 0x7fa0a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.536+0000 7fa0b2ffd640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0a00a3c40 0x7fa0a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.536+0000 7fa0b898f640 1 -- 172.21.15.55:0/3890851075 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fa09c0d0050 con 0x7fa0b4105d20 2024-04-26T21:52:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.536+0000 7fa0b2ffd640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0a00a3c40 0x7fa0a00a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:45.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.540+0000 7fa0b898f640 1 -- 172.21.15.55:0/3890851075 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa09c01f020 con 0x7fa0b4105d20 2024-04-26T21:52:45.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:45 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1332152397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:45.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.725+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0b402dcc0 con 0x7fa0b4105d20 2024-04-26T21:52:45.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.726+0000 7fa0b898f640 1 -- 172.21.15.55:0/3890851075 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fa09c00d040 con 0x7fa0b4105d20 2024-04-26T21:52:45.726 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:52:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0a00a3c40 msgr2=0x7fa0a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0a00a3c40 0x7fa0a00a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac008040 comp rx=0 tx=0).stop 2024-04-26T21:52:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4105d20 msgr2=0x7fa0b419b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4105d20 0x7fa0b419b480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa09c00b450 tx=0x7fa09c00b910 comp rx=0 tx=0).stop 2024-04-26T21:52:45.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 shutdown_connections 2024-04-26T21:52:45.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0a00a3c40 0x7fa0a00a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0b4108650 0x7fa0b41a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b4105d20 0x7fa0b419b480 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 --2- 172.21.15.55:0/3890851075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0b41033f0 0x7fa0b419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 >> 172.21.15.55:0/3890851075 conn(0x7fa0b40fcde0 msgr2=0x7fa0b4102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:45.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 shutdown_connections 2024-04-26T21:52:45.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.728+0000 7fa0b9991640 1 -- 172.21.15.55:0/3890851075 wait complete. 2024-04-26T21:52:45.740 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:52:45.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:45 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1332152397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 -- 172.21.15.55:0/3637483803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 msgr2=0x7fd28c108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 --2- 172.21.15.55:0/3637483803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 0x7fd28c108110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009900 tx=0x7fd27c02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 -- 172.21.15.55:0/3637483803 shutdown_connections 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 --2- 172.21.15.55:0/3637483803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28c108650 0x7fd28c10aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 --2- 172.21.15.55:0/3637483803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 0x7fd28c108110 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 --2- 172.21.15.55:0/3637483803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd28c1033f0 0x7fd28c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 -- 172.21.15.55:0/3637483803 >> 172.21.15.55:0/3637483803 conn(0x7fd28c0fcde0 msgr2=0x7fd28c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 -- 172.21.15.55:0/3637483803 shutdown_connections 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 -- 172.21.15.55:0/3637483803 wait complete. 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 Processor -- start 2024-04-26T21:52:45.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 -- start start 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd28c1033f0 0x7fd28c102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.844+0000 7fd293351640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 0x7fd28c1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd293351640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28c108650 0x7fd28c1018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd293351640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10e490 con 0x7fd28c105d20 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd293351640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10e310 con 0x7fd28c108650 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd293351640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10e610 con 0x7fd28c1033f0 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 0x7fd28c1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 0x7fd28c1013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54076/0 (socket says 172.21.15.55:54076) 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 -- 172.21.15.55:0/835320781 learned_addr learned my addr 172.21.15.55:0/835320781 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2910ed640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd28c1033f0 0x7fd28c102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 -- 172.21.15.55:0/835320781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd28c1033f0 msgr2=0x7fd28c102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd28c1033f0 0x7fd28c102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 -- 172.21.15.55:0/835320781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28c108650 msgr2=0x7fd28c1018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28c108650 0x7fd28c1018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 -- 172.21.15.55:0/835320781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c102170 con 0x7fd28c105d20 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2910ed640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd28c1033f0 0x7fd28c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2908ec640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 0x7fd28c1013a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd27c006010 tx=0x7fd27c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2827fc640 1 -- 172.21.15.55:0/835320781 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c03f030 con 0x7fd28c105d20 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c1a2c80 con 0x7fd28c105d20 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2827fc640 1 -- 172.21.15.55:0/835320781 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd27c003cb0 con 0x7fd28c105d20 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd2827fc640 1 -- 172.21.15.55:0/835320781 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c035070 con 0x7fd28c105d20 2024-04-26T21:52:45.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.845+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a31d0 con 0x7fd28c105d20 2024-04-26T21:52:45.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.846+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c0fe8d0 con 0x7fd28c105d20 2024-04-26T21:52:45.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.847+0000 7fd2827fc640 1 -- 172.21.15.55:0/835320781 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd27c042030 con 0x7fd28c105d20 2024-04-26T21:52:45.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.848+0000 7fd2827fc640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2780a3c40 0x7fd2780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:45.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.848+0000 7fd2910ed640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2780a3c40 0x7fd2780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:45.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.848+0000 7fd2827fc640 1 -- 172.21.15.55:0/835320781 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd27c0f1030 con 0x7fd28c105d20 2024-04-26T21:52:45.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.848+0000 7fd2910ed640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2780a3c40 0x7fd2780a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd28400abe0 tx=0x7fd28400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:45.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:45.852+0000 7fd2827fc640 1 -- 172.21.15.55:0/835320781 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd27c0bd890 con 0x7fd28c105d20 2024-04-26T21:52:46.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:45 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1332152397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:46.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.039+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd28c02dad0 con 0x7fd28c105d20 2024-04-26T21:52:46.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.039+0000 7fd2827fc640 1 -- 172.21.15.55:0/835320781 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fd27c0bd190 con 0x7fd28c105d20 2024-04-26T21:52:46.040 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T21:52:46.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2780a3c40 msgr2=0x7fd2780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2780a3c40 0x7fd2780a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd28400abe0 tx=0x7fd28400a5c0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 msgr2=0x7fd28c1013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 0x7fd28c1013a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd27c006010 tx=0x7fd27c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 shutdown_connections 2024-04-26T21:52:46.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd2780a3c40 0x7fd2780a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd28c108650 0x7fd28c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd28c105d20 0x7fd28c1013a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 --2- 172.21.15.55:0/835320781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd28c1033f0 0x7fd28c102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 >> 172.21.15.55:0/835320781 conn(0x7fd28c0fcde0 msgr2=0x7fd28c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 shutdown_connections 2024-04-26T21:52:46.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.041+0000 7fd293351640 1 -- 172.21.15.55:0/835320781 wait complete. 2024-04-26T21:52:46.053 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:52:46.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 -- 172.21.15.55:0/4154085839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 msgr2=0x7f5108102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 --2- 172.21.15.55:0/4154085839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 0x7f5108102ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009900 tx=0x7f50fc02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 -- 172.21.15.55:0/4154085839 shutdown_connections 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 --2- 172.21.15.55:0/4154085839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5108103420 0x7f5108109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 --2- 172.21.15.55:0/4154085839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 0x7f5108102ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 --2- 172.21.15.55:0/4154085839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5108101870 0x7f5108101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 -- 172.21.15.55:0/4154085839 >> 172.21.15.55:0/4154085839 conn(0x7f51080fcde0 msgr2=0x7f51080ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 -- 172.21.15.55:0/4154085839 shutdown_connections 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 -- 172.21.15.55:0/4154085839 wait complete. 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 Processor -- start 2024-04-26T21:52:46.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.157+0000 7f510e859640 1 -- start start 2024-04-26T21:52:46.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f510e859640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5108101870 0x7f5108197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f510e859640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 0x7f5108198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f510e859640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5108103420 0x7f510819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f510e859640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f510810d3f0 con 0x7f5108102a60 2024-04-26T21:52:46.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f510e859640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f510810d270 con 0x7f5108101870 2024-04-26T21:52:46.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f510e859640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f510810d570 con 0x7f5108103420 2024-04-26T21:52:46.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 0x7f5108198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 0x7f5108198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54100/0 (socket says 172.21.15.55:54100) 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 -- 172.21.15.55:0/3155383827 learned_addr learned my addr 172.21.15.55:0/3155383827 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f510cdf6640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5108103420 0x7f510819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f5107fff640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5108101870 0x7f5108197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 -- 172.21.15.55:0/3155383827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5108103420 msgr2=0x7f510819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5108103420 0x7f510819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 -- 172.21.15.55:0/3155383827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5108101870 msgr2=0x7f5108197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5108101870 0x7f5108197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 -- 172.21.15.55:0/3155383827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510819f6f0 con 0x7f5108102a60 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51077fe640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 0x7f5108198010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f50fc005ab0 tx=0x7f50fc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.158+0000 7f51057fa640 1 -- 172.21.15.55:0/3155383827 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc03f070 con 0x7f5108102a60 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.159+0000 7f51057fa640 1 -- 172.21.15.55:0/3155383827 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50fc031c50 con 0x7f5108102a60 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.159+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510819f980 con 0x7f5108102a60 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.159+0000 7f51057fa640 1 -- 172.21.15.55:0/3155383827 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc035030 con 0x7f5108102a60 2024-04-26T21:52:46.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.159+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510819fec0 con 0x7f5108102a60 2024-04-26T21:52:46.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.159+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5108108c10 con 0x7f5108102a60 2024-04-26T21:52:46.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.161+0000 7f51057fa640 1 -- 172.21.15.55:0/3155383827 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f50fc042070 con 0x7f5108102a60 2024-04-26T21:52:46.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.161+0000 7f51057fa640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50f00a3cc0 0x7f50f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.161+0000 7f5107fff640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50f00a3cc0 0x7f50f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.161+0000 7f51057fa640 1 -- 172.21.15.55:0/3155383827 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f50fc0f0c50 con 0x7f5108102a60 2024-04-26T21:52:46.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.162+0000 7f5107fff640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50f00a3cc0 0x7f50f00a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51081028c0 tx=0x7f50f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.166+0000 7f51057fa640 1 -- 172.21.15.55:0/3155383827 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f50fc0bce60 con 0x7f5108102a60 2024-04-26T21:52:46.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.351+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f510802dcf0 con 0x7f5108102a60 2024-04-26T21:52:46.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.351+0000 7f51057fa640 1 -- 172.21.15.55:0/3155383827 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f50fc0c1160 con 0x7f5108102a60 2024-04-26T21:52:46.352 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T21:52:46.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50f00a3cc0 msgr2=0x7f50f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50f00a3cc0 0x7f50f00a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51081028c0 tx=0x7f50f8005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 msgr2=0x7f5108198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 0x7f5108198010 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f50fc005ab0 tx=0x7f50fc003b90 comp rx=0 tx=0).stop 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 shutdown_connections 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50f00a3cc0 0x7f50f00a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5108103420 0x7f510819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5108102a60 0x7f5108198010 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 --2- 172.21.15.55:0/3155383827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5108101870 0x7f5108197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 >> 172.21.15.55:0/3155383827 conn(0x7f51080fcde0 msgr2=0x7f5108105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 shutdown_connections 2024-04-26T21:52:46.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.353+0000 7f510e859640 1 -- 172.21.15.55:0/3155383827 wait complete. 2024-04-26T21:52:46.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:46.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3757140336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf41029a0 msgr2=0x7f0bf4102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3757140336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf41029a0 0x7f0bf4102dc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009900 tx=0x7f0be802a410 comp rx=0 tx=0).stop 2024-04-26T21:52:46.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3757140336 shutdown_connections 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3757140336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf4104550 0x7f0bf410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3757140336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bf4103b90 0x7f0bf4104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3757140336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf41029a0 0x7f0bf4102dc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3757140336 >> 172.21.15.55:0/3757140336 conn(0x7f0bf40fdf30 msgr2=0x7f0bf4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3757140336 shutdown_connections 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3757140336 wait complete. 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 Processor -- start 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.466+0000 7f0bfb7af640 1 -- start start 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bfb7af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf41029a0 0x7f0bf40766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bfb7af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bf4103b90 0x7f0bf4074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bfb7af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf4104550 0x7f0bf4075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bfb7af640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410e4c0 con 0x7f0bf4104550 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bfb7af640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410e340 con 0x7f0bf41029a0 2024-04-26T21:52:46.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bfb7af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf410e640 con 0x7f0bf4103b90 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf4104550 0x7f0bf4075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf4104550 0x7f0bf4075260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54122/0 (socket says 172.21.15.55:54122) 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 -- 172.21.15.55:0/3969450511 learned_addr learned my addr 172.21.15.55:0/3969450511 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf8d4a640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bf4103b90 0x7f0bf4074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf954b640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf41029a0 0x7f0bf40766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 -- 172.21.15.55:0/3969450511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bf4103b90 msgr2=0x7f0bf4074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bf4103b90 0x7f0bf4074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 -- 172.21.15.55:0/3969450511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf41029a0 msgr2=0x7f0bf40766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf41029a0 0x7f0bf40766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 -- 172.21.15.55:0/3969450511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf4075af0 con 0x7f0bf4104550 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf954b640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf41029a0 0x7f0bf40766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bf9d4c640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf4104550 0x7f0bf4075260 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009fd0 tx=0x7f0bec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0be67fc640 1 -- 172.21.15.55:0/3969450511 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec017070 con 0x7f0bf4104550 2024-04-26T21:52:46.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0be67fc640 1 -- 172.21.15.55:0/3969450511 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bec003e20 con 0x7f0bf4104550 2024-04-26T21:52:46.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf41a2d60 con 0x7f0bf4104550 2024-04-26T21:52:46.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.467+0000 7f0be67fc640 1 -- 172.21.15.55:0/3969450511 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0106c0 con 0x7f0bf4104550 2024-04-26T21:52:46.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.468+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41a31a0 con 0x7f0bf4104550 2024-04-26T21:52:46.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.468+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf4109de0 con 0x7f0bf4104550 2024-04-26T21:52:46.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.470+0000 7f0be67fc640 1 -- 172.21.15.55:0/3969450511 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0bec004140 con 0x7f0bf4104550 2024-04-26T21:52:46.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.470+0000 7f0be67fc640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0be00a3c40 0x7f0be00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.470+0000 7f0bf954b640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0be00a3c40 0x7f0be00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.470+0000 7f0be67fc640 1 -- 172.21.15.55:0/3969450511 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f0bec0d02a0 con 0x7f0bf4104550 2024-04-26T21:52:46.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.471+0000 7f0bf954b640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0be00a3c40 0x7f0be00a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0be8000c00 tx=0x7f0be80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.475+0000 7f0be67fc640 1 -- 172.21.15.55:0/3969450511 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0bec0047e0 con 0x7f0bf4104550 2024-04-26T21:52:46.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:46 smithi055 ceph-mon[29221]: pgmap v345: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:46.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:46 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3890851075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:46.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:46 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/835320781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:46.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:46 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3155383827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:46.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.700+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bf402dad0 con 0x7f0bf4104550 2024-04-26T21:52:46.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.701+0000 7f0be67fc640 1 -- 172.21.15.55:0/3969450511 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11863 (secure 0 0 0) 0x7f0bec09cb00 con 0x7f0bf4104550 2024-04-26T21:52:46.702 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:46.702 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:51:01.298886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:52:46.702 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 20 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0be00a3c40 msgr2=0x7f0be00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0be00a3c40 0x7f0be00a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0be8000c00 tx=0x7f0be80314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf4104550 msgr2=0x7f0bf4075260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf4104550 0x7f0bf4075260 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009fd0 tx=0x7f0bec00b260 comp rx=0 tx=0).stop 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 shutdown_connections 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0be00a3c40 0x7f0be00a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0bf4104550 0x7f0bf4075260 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bf4103b90 0x7f0bf4074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 --2- 172.21.15.55:0/3969450511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bf41029a0 0x7f0bf40766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 >> 172.21.15.55:0/3969450511 conn(0x7f0bf40fdf30 msgr2=0x7f0bf4106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 shutdown_connections 2024-04-26T21:52:46.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.703+0000 7f0bfb7af640 1 -- 172.21.15.55:0/3969450511 wait complete. 2024-04-26T21:52:46.715 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:46.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- 172.21.15.55:0/2222631154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 msgr2=0x7f9534102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 --2- 172.21.15.55:0/2222631154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 0x7f9534102dc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f952c009900 tx=0x7f952c02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:46.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- 172.21.15.55:0/2222631154 shutdown_connections 2024-04-26T21:52:46.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 --2- 172.21.15.55:0/2222631154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9534104550 0x7f953410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 --2- 172.21.15.55:0/2222631154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9534103b90 0x7f9534104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 --2- 172.21.15.55:0/2222631154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 0x7f9534102dc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- 172.21.15.55:0/2222631154 >> 172.21.15.55:0/2222631154 conn(0x7f95340fdf30 msgr2=0x7f9534100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- 172.21.15.55:0/2222631154 shutdown_connections 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- 172.21.15.55:0/2222631154 wait complete. 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 Processor -- start 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- start start 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 0x7f9534198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9534103b90 0x7f9534199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9534104550 0x7f95341a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f953410e520 con 0x7f95341029a0 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f953410e3a0 con 0x7f9534103b90 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.816+0000 7f953ae2a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f953410e6a0 con 0x7f9534104550 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 0x7f9534198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 0x7f9534198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54142/0 (socket says 172.21.15.55:54142) 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 -- 172.21.15.55:0/4138694862 learned_addr learned my addr 172.21.15.55:0/4138694862 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:46.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f95393c7640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9534104550 0x7f95341a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 -- 172.21.15.55:0/4138694862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9534104550 msgr2=0x7f95341a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9534104550 0x7f95341a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 -- 172.21.15.55:0/4138694862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9534103b90 msgr2=0x7f9534199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9534103b90 0x7f9534199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 -- 172.21.15.55:0/4138694862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95341a0810 con 0x7f95341029a0 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9538bc6640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 0x7f9534198bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f952c006010 tx=0x7f952c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9531ffb640 1 -- 172.21.15.55:0/4138694862 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c03f030 con 0x7f95341029a0 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95341a0aa0 con 0x7f95341029a0 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9531ffb640 1 -- 172.21.15.55:0/4138694862 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952c003cb0 con 0x7f95341029a0 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f9531ffb640 1 -- 172.21.15.55:0/4138694862 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c035070 con 0x7f95341029a0 2024-04-26T21:52:46.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.817+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95341a0f90 con 0x7f95341029a0 2024-04-26T21:52:46.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.818+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9534109d30 con 0x7f95341029a0 2024-04-26T21:52:46.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.819+0000 7f9531ffb640 1 -- 172.21.15.55:0/4138694862 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f952c042030 con 0x7f95341029a0 2024-04-26T21:52:46.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.820+0000 7f9531ffb640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95200a3c40 0x7f95200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:46.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.820+0000 7f9533fff640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95200a3c40 0x7f95200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:46.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.820+0000 7f9531ffb640 1 -- 172.21.15.55:0/4138694862 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f952c0f0de0 con 0x7f95341029a0 2024-04-26T21:52:46.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.821+0000 7f9533fff640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95200a3c40 0x7f95200a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f953419a100 tx=0x7f9524009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:46.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:46.824+0000 7f9531ffb640 1 -- 172.21.15.55:0/4138694862 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f952c003e60 con 0x7f95341029a0 2024-04-26T21:52:46.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:46 smithi165 ceph-mon[33296]: pgmap v345: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:46.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:46 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3890851075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:46.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:46 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/835320781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:46.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:46 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3155383827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:47.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:46 smithi064 ceph-mon[32765]: pgmap v345: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:47.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:46 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3890851075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:47.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:46 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/835320781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:47.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:46 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3155383827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:47.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.010+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f953402dd20 con 0x7f95341029a0 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.012+0000 7f9531ffb640 1 -- 172.21.15.55:0/4138694862 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22713 (secure 0 0 0) 0x7f952c0bd5e0 con 0x7f95341029a0 2024-04-26T21:52:47.012 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:47.013 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:51:50.534100+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:47.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95200a3c40 msgr2=0x7f95200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95200a3c40 0x7f95200a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f953419a100 tx=0x7f9524009250 comp rx=0 tx=0).stop 2024-04-26T21:52:47.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 msgr2=0x7f9534198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 0x7f9534198bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f952c006010 tx=0x7f952c004430 comp rx=0 tx=0).stop 2024-04-26T21:52:47.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 shutdown_connections 2024-04-26T21:52:47.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95200a3c40 0x7f95200a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9534104550 0x7f95341a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9534103b90 0x7f9534199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 --2- 172.21.15.55:0/4138694862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95341029a0 0x7f9534198bf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 >> 172.21.15.55:0/4138694862 conn(0x7f95340fdf30 msgr2=0x7f9534106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 shutdown_connections 2024-04-26T21:52:47.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.015+0000 7f953ae2a640 1 -- 172.21.15.55:0/4138694862 wait complete. 2024-04-26T21:52:47.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:52:47.029 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:52:47.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- 172.21.15.55:0/1913278378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc103b70 msgr2=0x7f7ddc103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 --2- 172.21.15.55:0/1913278378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc103b70 0x7f7ddc103ff0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009900 tx=0x7f7dcc02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:47.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- 172.21.15.55:0/1913278378 shutdown_connections 2024-04-26T21:52:47.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 --2- 172.21.15.55:0/1913278378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ddc104530 0x7f7ddc10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 --2- 172.21.15.55:0/1913278378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc103b70 0x7f7ddc103ff0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 --2- 172.21.15.55:0/1913278378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ddc102980 0x7f7ddc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- 172.21.15.55:0/1913278378 >> 172.21.15.55:0/1913278378 conn(0x7f7ddc0fdf10 msgr2=0x7f7ddc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- 172.21.15.55:0/1913278378 shutdown_connections 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- 172.21.15.55:0/1913278378 wait complete. 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 Processor -- start 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- start start 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc102980 0x7f7ddc0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ddc103b70 0x7f7ddc074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ddc104530 0x7f7ddc075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc10e550 con 0x7f7ddc102980 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc10e3d0 con 0x7f7ddc104530 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.134+0000 7f7de343e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ddc10e6d0 con 0x7f7ddc103b70 2024-04-26T21:52:47.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc102980 0x7f7ddc0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc102980 0x7f7ddc0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54172/0 (socket says 172.21.15.55:54172) 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 -- 172.21.15.55:0/2031137342 learned_addr learned my addr 172.21.15.55:0/2031137342 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de09d9640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ddc103b70 0x7f7ddc074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de19db640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ddc104530 0x7f7ddc075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 -- 172.21.15.55:0/2031137342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ddc103b70 msgr2=0x7f7ddc074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ddc103b70 0x7f7ddc074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 -- 172.21.15.55:0/2031137342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ddc104530 msgr2=0x7f7ddc075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ddc104530 0x7f7ddc075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 -- 172.21.15.55:0/2031137342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ddc075af0 con 0x7f7ddc102980 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de09d9640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ddc103b70 0x7f7ddc074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:47.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de19db640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ddc104530 0x7f7ddc075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:47.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de11da640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc102980 0x7f7ddc0766e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400b450 tx=0x7f7dd400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7dd27fc640 1 -- 172.21.15.55:0/2031137342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4015020 con 0x7f7ddc102980 2024-04-26T21:52:47.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7dd27fc640 1 -- 172.21.15.55:0/2031137342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd4011d40 con 0x7f7ddc102980 2024-04-26T21:52:47.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ddc1a2db0 con 0x7f7ddc102980 2024-04-26T21:52:47.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7dd27fc640 1 -- 172.21.15.55:0/2031137342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400f340 con 0x7f7ddc102980 2024-04-26T21:52:47.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.135+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ddc1a3290 con 0x7f7ddc102980 2024-04-26T21:52:47.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.136+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ddc109db0 con 0x7f7ddc102980 2024-04-26T21:52:47.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.137+0000 7f7dd27fc640 1 -- 172.21.15.55:0/2031137342 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7dd4018020 con 0x7f7ddc102980 2024-04-26T21:52:47.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.138+0000 7f7dd27fc640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7dc80a3c40 0x7f7dc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.138+0000 7f7de09d9640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7dc80a3c40 0x7f7dc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.138+0000 7f7dd27fc640 1 -- 172.21.15.55:0/2031137342 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f7dd40d0040 con 0x7f7ddc102980 2024-04-26T21:52:47.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.138+0000 7f7de09d9640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7dc80a3c40 0x7f7dc80a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc076450 tx=0x7f7dcc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.142+0000 7f7dd27fc640 1 -- 172.21.15.55:0/2031137342 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7dd401f020 con 0x7f7ddc102980 2024-04-26T21:52:47.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.327+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ddc063560 con 0x7f7ddc102980 2024-04-26T21:52:47.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.328+0000 7f7dd27fc640 1 -- 172.21.15.55:0/2031137342 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f7dd4096e20 con 0x7f7ddc102980 2024-04-26T21:52:47.328 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:52:47.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.329+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7dc80a3c40 msgr2=0x7f7dc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.329+0000 7f7de343e640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7dc80a3c40 0x7f7dc80a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc076450 tx=0x7f7dcc0314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc102980 msgr2=0x7f7ddc0766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc102980 0x7f7ddc0766e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400b450 tx=0x7f7dd400b910 comp rx=0 tx=0).stop 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 shutdown_connections 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7dc80a3c40 0x7f7dc80a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ddc104530 0x7f7ddc075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ddc103b70 0x7f7ddc074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 --2- 172.21.15.55:0/2031137342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ddc102980 0x7f7ddc0766e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 >> 172.21.15.55:0/2031137342 conn(0x7f7ddc0fdf10 msgr2=0x7f7ddc106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 shutdown_connections 2024-04-26T21:52:47.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.330+0000 7f7de343e640 1 -- 172.21.15.55:0/2031137342 wait complete. 2024-04-26T21:52:47.341 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:52:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 -- 172.21.15.55:0/556964459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 msgr2=0x7f0738102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 --2- 172.21.15.55:0/556964459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 0x7f0738102db0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0728009900 tx=0x7f072802a410 comp rx=0 tx=0).stop 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 -- 172.21.15.55:0/556964459 shutdown_connections 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 --2- 172.21.15.55:0/556964459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0738104540 0x7f073810add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 --2- 172.21.15.55:0/556964459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738103b80 0x7f0738104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 --2- 172.21.15.55:0/556964459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 0x7f0738102db0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 -- 172.21.15.55:0/556964459 >> 172.21.15.55:0/556964459 conn(0x7f07380fdf00 msgr2=0x7f0738100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 -- 172.21.15.55:0/556964459 shutdown_connections 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.446+0000 7f073e39d640 1 -- 172.21.15.55:0/556964459 wait complete. 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073e39d640 1 Processor -- start 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073e39d640 1 -- start start 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073e39d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 0x7f07380766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073e39d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0738103b80 0x7f0738074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073e39d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738104540 0x7f0738075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073e39d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f073810e490 con 0x7f0738102990 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073e39d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f073810e310 con 0x7f0738103b80 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073e39d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f073810e610 con 0x7f0738104540 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f0737fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 0x7f07380766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f0737fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 0x7f07380766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54192/0 (socket says 172.21.15.55:54192) 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f0737fff640 1 -- 172.21.15.55:0/2538795610 learned_addr learned my addr 172.21.15.55:0/2538795610 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f07377fe640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0738103b80 0x7f0738074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.447+0000 7f073c93a640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738104540 0x7f0738075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f0737fff640 1 -- 172.21.15.55:0/2538795610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738104540 msgr2=0x7f0738075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f0737fff640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738104540 0x7f0738075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f0737fff640 1 -- 172.21.15.55:0/2538795610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0738103b80 msgr2=0x7f0738074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f0737fff640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0738103b80 0x7f0738074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f0737fff640 1 -- 172.21.15.55:0/2538795610 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0738075af0 con 0x7f0738102990 2024-04-26T21:52:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f0737fff640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 0x7f07380766e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0728006010 tx=0x7f07280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f07357fa640 1 -- 172.21.15.55:0/2538795610 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072803f030 con 0x7f0738102990 2024-04-26T21:52:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f07357fa640 1 -- 172.21.15.55:0/2538795610 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07280318f0 con 0x7f0738102990 2024-04-26T21:52:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07381a2c00 con 0x7f0738102990 2024-04-26T21:52:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f07357fa640 1 -- 172.21.15.55:0/2538795610 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728035070 con 0x7f0738102990 2024-04-26T21:52:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.448+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381a31a0 con 0x7f0738102990 2024-04-26T21:52:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.449+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0738109d20 con 0x7f0738102990 2024-04-26T21:52:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.450+0000 7f07357fa640 1 -- 172.21.15.55:0/2538795610 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0728042030 con 0x7f0738102990 2024-04-26T21:52:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.451+0000 7f07357fa640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07200a3cc0 0x7f07200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.451+0000 7f07377fe640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07200a3cc0 0x7f07200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.451+0000 7f07357fa640 1 -- 172.21.15.55:0/2538795610 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f07280f0ae0 con 0x7f0738102990 2024-04-26T21:52:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.451+0000 7f07377fe640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07200a3cc0 0x7f07200a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0738076450 tx=0x7f072c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.455+0000 7f07357fa640 1 -- 172.21.15.55:0/2538795610 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f07280bd2e0 con 0x7f0738102990 2024-04-26T21:52:47.617 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:47 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3969450511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.617 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:47 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4138694862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.617 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:47 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2031137342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:47.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.641+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f073802dc80 con 0x7f0738102990 2024-04-26T21:52:47.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.641+0000 7f07357fa640 1 -- 172.21.15.55:0/2538795610 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f07280bcbe0 con 0x7f0738102990 2024-04-26T21:52:47.642 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07200a3cc0 msgr2=0x7f07200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07200a3cc0 0x7f07200a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0738076450 tx=0x7f072c00a2d0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 msgr2=0x7f07380766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 0x7f07380766e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0728006010 tx=0x7f07280042f0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 shutdown_connections 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07200a3cc0 0x7f07200a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0738104540 0x7f0738075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0738103b80 0x7f0738074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 --2- 172.21.15.55:0/2538795610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0738102990 0x7f07380766e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 >> 172.21.15.55:0/2538795610 conn(0x7f07380fdf00 msgr2=0x7f0738106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 shutdown_connections 2024-04-26T21:52:47.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.643+0000 7f073e39d640 1 -- 172.21.15.55:0/2538795610 wait complete. 2024-04-26T21:52:47.655 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 -- 172.21.15.55:0/4203526919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c105d40 msgr2=0x7f080c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 --2- 172.21.15.55:0/4203526919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c105d40 0x7f080c108130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0800009900 tx=0x7f080002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 -- 172.21.15.55:0/4203526919 shutdown_connections 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 --2- 172.21.15.55:0/4203526919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f080c108670 0x7f080c10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 --2- 172.21.15.55:0/4203526919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c105d40 0x7f080c108130 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 --2- 172.21.15.55:0/4203526919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c103410 0x7f080c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 -- 172.21.15.55:0/4203526919 >> 172.21.15.55:0/4203526919 conn(0x7f080c0fce20 msgr2=0x7f080c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 -- 172.21.15.55:0/4203526919 shutdown_connections 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.759+0000 7f0813346640 1 -- 172.21.15.55:0/4203526919 wait complete. 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f0813346640 1 Processor -- start 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f0813346640 1 -- start start 2024-04-26T21:52:47.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f0813346640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c103410 0x7f080c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f0813346640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f080c105d40 0x7f080c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f0813346640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c108670 0x7f080c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f0813346640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f080c10e510 con 0x7f080c103410 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f0813346640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f080c10e390 con 0x7f080c108670 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f0813346640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f080c10e690 con 0x7f080c105d40 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08110e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c103410 0x7f080c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08110e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c103410 0x7f080c198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54198/0 (socket says 172.21.15.55:54198) 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08110e2640 1 -- 172.21.15.55:0/1925874481 learned_addr learned my addr 172.21.15.55:0/1925874481 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08108e1640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f080c105d40 0x7f080c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08118e3640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c108670 0x7f080c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08110e2640 1 -- 172.21.15.55:0/1925874481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f080c105d40 msgr2=0x7f080c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08110e2640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f080c105d40 0x7f080c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08110e2640 1 -- 172.21.15.55:0/1925874481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c108670 msgr2=0x7f080c1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08110e2640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c108670 0x7f080c1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08110e2640 1 -- 172.21.15.55:0/1925874481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c1a0890 con 0x7f080c103410 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.760+0000 7f08118e3640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c108670 0x7f080c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.761+0000 7f08110e2640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c103410 0x7f080c198cd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00eab0 tx=0x7f07fc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.761+0000 7f080a7fc640 1 -- 172.21.15.55:0/1925874481 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00ce40 con 0x7f080c103410 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.761+0000 7f080a7fc640 1 -- 172.21.15.55:0/1925874481 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07fc011920 con 0x7f080c103410 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.761+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c1a0b80 con 0x7f080c103410 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.761+0000 7f080a7fc640 1 -- 172.21.15.55:0/1925874481 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc010a30 con 0x7f080c103410 2024-04-26T21:52:47.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.761+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c1a1120 con 0x7f080c103410 2024-04-26T21:52:47.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.762+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c0fe910 con 0x7f080c103410 2024-04-26T21:52:47.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.763+0000 7f080a7fc640 1 -- 172.21.15.55:0/1925874481 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f07fc010bd0 con 0x7f080c103410 2024-04-26T21:52:47.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.764+0000 7f080a7fc640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07f40a3cc0 0x7f07f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:47.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.764+0000 7f08108e1640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07f40a3cc0 0x7f07f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:47.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.764+0000 7f080a7fc640 1 -- 172.21.15.55:0/1925874481 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f07fc0d0c30 con 0x7f080c103410 2024-04-26T21:52:47.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.764+0000 7f08108e1640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07f40a3cc0 0x7f07f40a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0800000c00 tx=0x7f08000314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:47.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.767+0000 7f080a7fc640 1 -- 172.21.15.55:0/1925874481 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f07fc016030 con 0x7f080c103410 2024-04-26T21:52:47.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:47 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3969450511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:47 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4138694862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:47.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:47 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2031137342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:47.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.954+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f080c02dd30 con 0x7f080c103410 2024-04-26T21:52:47.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.954+0000 7f080a7fc640 1 -- 172.21.15.55:0/1925874481 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f07fc021350 con 0x7f080c103410 2024-04-26T21:52:47.955 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T21:52:47.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07f40a3cc0 msgr2=0x7f07f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07f40a3cc0 0x7f07f40a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0800000c00 tx=0x7f08000314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c103410 msgr2=0x7f080c198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c103410 0x7f080c198cd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00eab0 tx=0x7f07fc00ef70 comp rx=0 tx=0).stop 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 shutdown_connections 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f07f40a3cc0 0x7f07f40a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f080c108670 0x7f080c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f080c105d40 0x7f080c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 --2- 172.21.15.55:0/1925874481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f080c103410 0x7f080c198cd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 >> 172.21.15.55:0/1925874481 conn(0x7f080c0fce20 msgr2=0x7f080c103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 shutdown_connections 2024-04-26T21:52:47.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:47.956+0000 7f0813346640 1 -- 172.21.15.55:0/1925874481 wait complete. 2024-04-26T21:52:47.968 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:48.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:47 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3969450511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:48.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:47 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4138694862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:48.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:47 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2031137342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 -- 172.21.15.55:0/2280380924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8103420 msgr2=0x7fd1c8109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 --2- 172.21.15.55:0/2280380924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8103420 0x7fd1c8109cb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009ca0 tx=0x7fd1c002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 -- 172.21.15.55:0/2280380924 shutdown_connections 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 --2- 172.21.15.55:0/2280380924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8103420 0x7fd1c8109cb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 --2- 172.21.15.55:0/2280380924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c8102a60 0x7fd1c8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 --2- 172.21.15.55:0/2280380924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8101870 0x7fd1c8101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 -- 172.21.15.55:0/2280380924 >> 172.21.15.55:0/2280380924 conn(0x7fd1c80fcde0 msgr2=0x7fd1c80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 -- 172.21.15.55:0/2280380924 shutdown_connections 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.068+0000 7fd1cd959640 1 -- 172.21.15.55:0/2280380924 wait complete. 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1cd959640 1 Processor -- start 2024-04-26T21:52:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1cd959640 1 -- start start 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1cd959640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c8101870 0x7fd1c8105800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1cd959640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8102a60 0x7fd1c8103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1cd959640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8103420 0x7fd1c8104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1cd959640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810d380 con 0x7fd1c8102a60 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1cd959640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810d200 con 0x7fd1c8103420 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1cd959640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c810d500 con 0x7fd1c8101870 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1c67fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8102a60 0x7fd1c8103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1c67fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8102a60 0x7fd1c8103e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54222/0 (socket says 172.21.15.55:54222) 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.069+0000 7fd1c67fc640 1 -- 172.21.15.55:0/1062508416 learned_addr learned my addr 172.21.15.55:0/1062508416 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c6ffd640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c8101870 0x7fd1c8105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c67fc640 1 -- 172.21.15.55:0/1062508416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c8101870 msgr2=0x7fd1c8105800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c77fe640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8103420 0x7fd1c8104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c67fc640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c8101870 0x7fd1c8105800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c67fc640 1 -- 172.21.15.55:0/1062508416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8103420 msgr2=0x7fd1c8104380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c67fc640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8103420 0x7fd1c8104380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c67fc640 1 -- 172.21.15.55:0/1062508416 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c8104c10 con 0x7fd1c8102a60 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c77fe640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8103420 0x7fd1c8104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1c67fc640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8102a60 0x7fd1c8103e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00c920 tx=0x7fd1bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1cc957640 1 -- 172.21.15.55:0/1062508416 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc007c70 con 0x7fd1c8102a60 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c81a2c00 con 0x7fd1c8102a60 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1cc957640 1 -- 172.21.15.55:0/1062508416 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc007e10 con 0x7fd1c8102a60 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1cc957640 1 -- 172.21.15.55:0/1062508416 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc018820 con 0x7fd1c8102a60 2024-04-26T21:52:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.070+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c81a31a0 con 0x7fd1c8102a60 2024-04-26T21:52:48.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.071+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c81a33d0 con 0x7fd1c8102a60 2024-04-26T21:52:48.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.072+0000 7fd1cc957640 1 -- 172.21.15.55:0/1062508416 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd1bc00f450 con 0x7fd1c8102a60 2024-04-26T21:52:48.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.073+0000 7fd1cc957640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd1b00a3c40 0x7fd1b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.073+0000 7fd1cc957640 1 -- 172.21.15.55:0/1062508416 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7fd1bc0d0ad0 con 0x7fd1c8102a60 2024-04-26T21:52:48.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.073+0000 7fd1c6ffd640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd1b00a3c40 0x7fd1b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.073+0000 7fd1c6ffd640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd1b00a3c40 0x7fd1b00a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8005db0 tx=0x7fd1b8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.076+0000 7fd1cc957640 1 -- 172.21.15.55:0/1062508416 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd1bc014030 con 0x7fd1c8102a60 2024-04-26T21:52:48.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.304+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1c81a35a0 con 0x7fd1c8102a60 2024-04-26T21:52:48.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.305+0000 7fd1cc957640 1 -- 172.21.15.55:0/1062508416 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11863 (secure 0 0 0) 0x7fd1bc016300 con 0x7fd1c8102a60 2024-04-26T21:52:48.305 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:48.305 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:51:01.298886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:52:48.306 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 20 2024-04-26T21:52:48.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd1b00a3c40 msgr2=0x7fd1b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd1b00a3c40 0x7fd1b00a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8005db0 tx=0x7fd1b8005d40 comp rx=0 tx=0).stop 2024-04-26T21:52:48.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8102a60 msgr2=0x7fd1c8103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8102a60 0x7fd1c8103e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00c920 tx=0x7fd1bc00cde0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 shutdown_connections 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd1b00a3c40 0x7fd1b00a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1c8103420 0x7fd1c8104380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1c8102a60 0x7fd1c8103e40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 --2- 172.21.15.55:0/1062508416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c8101870 0x7fd1c8105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 >> 172.21.15.55:0/1062508416 conn(0x7fd1c80fcde0 msgr2=0x7fd1c80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 shutdown_connections 2024-04-26T21:52:48.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.307+0000 7fd1cd959640 1 -- 172.21.15.55:0/1062508416 wait complete. 2024-04-26T21:52:48.319 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 -- 172.21.15.55:0/1705444224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894101870 msgr2=0x7f7894101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 --2- 172.21.15.55:0/1705444224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894101870 0x7f7894101c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7884009900 tx=0x7f788402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 -- 172.21.15.55:0/1705444224 shutdown_connections 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 --2- 172.21.15.55:0/1705444224 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7894103420 0x7f7894109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 --2- 172.21.15.55:0/1705444224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894102a60 0x7f7894102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 --2- 172.21.15.55:0/1705444224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894101870 0x7f7894101c90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 -- 172.21.15.55:0/1705444224 >> 172.21.15.55:0/1705444224 conn(0x7f78940fcde0 msgr2=0x7f78940ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 -- 172.21.15.55:0/1705444224 shutdown_connections 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 -- 172.21.15.55:0/1705444224 wait complete. 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 Processor -- start 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.422+0000 7f7899ec8640 1 -- start start 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7899ec8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7894101870 0x7f789419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7899ec8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894102a60 0x7f789419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7899ec8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894103420 0x7f78941a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7899ec8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f789410d370 con 0x7f7894103420 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7899ec8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f789410d1f0 con 0x7f7894102a60 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7899ec8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f789410d4f0 con 0x7f7894101870 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894103420 0x7f78941a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894103420 0x7f78941a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54236/0 (socket says 172.21.15.55:54236) 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 -- 172.21.15.55:0/3972775547 learned_addr learned my addr 172.21.15.55:0/3972775547 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7892ffd640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894102a60 0x7f789419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f78937fe640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7894101870 0x7f789419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 -- 172.21.15.55:0/3972775547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7894101870 msgr2=0x7f789419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7894101870 0x7f789419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 -- 172.21.15.55:0/3972775547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894102a60 msgr2=0x7f789419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894102a60 0x7f789419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 -- 172.21.15.55:0/3972775547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78941a2980 con 0x7f7894103420 2024-04-26T21:52:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7892ffd640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894102a60 0x7f789419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:48.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f78937fe640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7894101870 0x7f789419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:48.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7893fff640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894103420 0x7f78941a2310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f788c009fd0 tx=0x7f788c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7890ff9640 1 -- 172.21.15.55:0/3972775547 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c017070 con 0x7f7894103420 2024-04-26T21:52:48.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7890ff9640 1 -- 172.21.15.55:0/3972775547 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f788c003e20 con 0x7f7894103420 2024-04-26T21:52:48.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78941a2c70 con 0x7f7894103420 2024-04-26T21:52:48.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7890ff9640 1 -- 172.21.15.55:0/3972775547 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c010460 con 0x7f7894103420 2024-04-26T21:52:48.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.423+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78941a3170 con 0x7f7894103420 2024-04-26T21:52:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.425+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78941a3340 con 0x7f7894103420 2024-04-26T21:52:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.426+0000 7f7890ff9640 1 -- 172.21.15.55:0/3972775547 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f788c003970 con 0x7f7894103420 2024-04-26T21:52:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.426+0000 7f7890ff9640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f787c0a3c40 0x7f787c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.426+0000 7f78937fe640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f787c0a3c40 0x7f787c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.426+0000 7f7890ff9640 1 -- 172.21.15.55:0/3972775547 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f788c0d0420 con 0x7f7894103420 2024-04-26T21:52:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.427+0000 7f78937fe640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f787c0a3c40 0x7f787c0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7884006010 tx=0x7f7884000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.431+0000 7f7890ff9640 1 -- 172.21.15.55:0/3972775547 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f788c09cc80 con 0x7f7894103420 2024-04-26T21:52:48.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.620+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78941a36d0 con 0x7f7894103420 2024-04-26T21:52:48.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.622+0000 7f7890ff9640 1 -- 172.21.15.55:0/3972775547 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22713 (secure 0 0 0) 0x7f788c09c580 con 0x7f7894103420 2024-04-26T21:52:48.622 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:48.623 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:51:50.534100+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f787c0a3c40 msgr2=0x7f787c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f787c0a3c40 0x7f787c0a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7884006010 tx=0x7f7884000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894103420 msgr2=0x7f78941a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894103420 0x7f78941a2310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f788c009fd0 tx=0x7f788c00b260 comp rx=0 tx=0).stop 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 shutdown_connections 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f787c0a3c40 0x7f787c0a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7894103420 0x7f78941a2310 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7894102a60 0x7f789419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 --2- 172.21.15.55:0/3972775547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7894101870 0x7f789419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 >> 172.21.15.55:0/3972775547 conn(0x7f78940fcde0 msgr2=0x7f7894069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 shutdown_connections 2024-04-26T21:52:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.624+0000 7f7899ec8640 1 -- 172.21.15.55:0/3972775547 wait complete. 2024-04-26T21:52:48.634 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:48 smithi055 ceph-mon[29221]: pgmap v346: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:48.634 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:48 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2538795610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:48.634 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:48 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1925874481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:48.635 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:48 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1062508416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:48.636 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T21:50:56.719940+0000', 'modified': '2024-04-26T21:51:01.298886+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24551, 'mds_1': 24509, 'mds_2': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/125466680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 125466680}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 125466680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6835/2328208855', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 2328208855}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 2328208855}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/2954631237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2954631237}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2954631237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.165:6839/1115440142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 1115440142}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 1115440142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'h', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.64:6837/4014142220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 4014142220}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 4014142220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.64:6839/463662731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 463662731}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 463662731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24502, 24509, 24551]} 2024-04-26T21:52:48.636 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T21:52:48.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-26T21:52:48.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1427880102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 msgr2=0x7f9f50102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1427880102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 0x7f9f50102db0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009900 tx=0x7f9f4002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1427880102 shutdown_connections 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1427880102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50104540 0x7f9f5010add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1427880102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50103b80 0x7f9f50104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1427880102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 0x7f9f50102db0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1427880102 >> 172.21.15.55:0/1427880102 conn(0x7f9f500fdf00 msgr2=0x7f9f50100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1427880102 shutdown_connections 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1427880102 wait complete. 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.739+0000 7f9f55a6b640 1 Processor -- start 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f55a6b640 1 -- start start 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f55a6b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 0x7f9f500766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f55a6b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50103b80 0x7f9f50074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f55a6b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50104540 0x7f9f50075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f55a6b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010e490 con 0x7f9f50102990 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f55a6b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010e310 con 0x7f9f50103b80 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f55a6b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5010e610 con 0x7f9f50104540 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 0x7f9f500766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 0x7f9f500766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54252/0 (socket says 172.21.15.55:54252) 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/1609668492 learned_addr learned my addr 172.21.15.55:0/1609668492 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50104540 0x7f9f50075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4effd640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50103b80 0x7f9f50074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/1609668492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50104540 msgr2=0x7f9f50075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50104540 0x7f9f50075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/1609668492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50103b80 msgr2=0x7f9f50074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50103b80 0x7f9f50074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/1609668492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f50075af0 con 0x7f9f50102990 2024-04-26T21:52:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4effd640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50103b80 0x7f9f50074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.740+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 0x7f9f500766e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9f40006010 tx=0x7f9f40004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.741+0000 7f9f4cff9640 1 -- 172.21.15.55:0/1609668492 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4003f030 con 0x7f9f50102990 2024-04-26T21:52:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.741+0000 7f9f4cff9640 1 -- 172.21.15.55:0/1609668492 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f40003cb0 con 0x7f9f50102990 2024-04-26T21:52:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.741+0000 7f9f4cff9640 1 -- 172.21.15.55:0/1609668492 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40035070 con 0x7f9f50102990 2024-04-26T21:52:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.741+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1609668492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f501a2b90 con 0x7f9f50102990 2024-04-26T21:52:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.741+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1609668492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a30e0 con 0x7f9f50102990 2024-04-26T21:52:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f4cff9640 1 -- 172.21.15.55:0/1609668492 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9f40042030 con 0x7f9f50102990 2024-04-26T21:52:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f58789740 1 Processor -- start 2024-04-26T21:52:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f58789740 1 -- start start 2024-04-26T21:52:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f58789740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ad8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f58789740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528addb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f58789740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f58789740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556528a3d740 con 0x556528ad8690 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f58789740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5565289d93a0 con 0x556528ade1f0 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f58789740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556528a39700 con 0x556528ad9080 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.743+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ad8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ad8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54254/0 (socket says 172.21.15.55:54254) 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 -- 172.21.15.55:0/1223510954 learned_addr learned my addr 172.21.15.55:0/1223510954 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4cff9640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f380a3c40 0x7f9f380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f380a3c40 0x7f9f380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4effd640 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4cff9640 1 -- 172.21.15.55:0/1609668492 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9f400f0dd0 con 0x7f9f50102990 2024-04-26T21:52:48.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528addb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 -- 172.21.15.55:0/1223510954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 msgr2=0x556528addb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528addb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 -- 172.21.15.55:0/1223510954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 msgr2=0x556528ae0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 -- 172.21.15.55:0/1223510954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565289f5750 con 0x556528ad8690 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4effd640 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ad8ab0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f48012910 tx=0x7f9f48012dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc201f7d71841e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f35ffb640 1 -- 172.21.15.55:0/1223510954 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48005020 con 0x556528ad8690 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f35ffb640 1 -- 172.21.15.55:0/1223510954 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f48005670 con 0x556528ad8690 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.744+0000 7f9f35ffb640 1 -- 172.21.15.55:0/1223510954 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4801c770 con 0x556528ad8690 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 -- 172.21.15.55:0/1223510954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 msgr2=0x556528ad8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f380a3c40 0x7f9f380a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f4801f000 tx=0x7f9f4801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ad8ab0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f48012910 tx=0x7f9f48012dd0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 -- 172.21.15.55:0/1223510954 shutdown_connections 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528addb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 --2- 172.21.15.55:0/1223510954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ad8ab0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 -- 172.21.15.55:0/1223510954 >> 172.21.15.55:0/1223510954 conn(0x556528ad3d00 msgr2=0x556528ad4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 -- 172.21.15.55:0/1223510954 shutdown_connections 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 -- 172.21.15.55:0/1223510954 wait complete. 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.745+0000 7f9f58789740 1 -- start start 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f58789740 1 Processor -- start 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f58789740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ae28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f58789740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528ae2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f58789740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f58789740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556528ae11a0 con 0x556528ad8690 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f58789740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5565289c4fe0 con 0x556528ade1f0 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f58789740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556528ae1320 con 0x556528ad9080 2024-04-26T21:52:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f58789740 10 client.0 _ll_register_callbacks cb 0x5565289db150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ae28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ae28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54270/0 (socket says 172.21.15.55:54270) 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/2351734769 learned_addr learned my addr 172.21.15.55:0/2351734769 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4effd640 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528ae2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/2351734769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 msgr2=0x556528ae2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528ae2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/2351734769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 msgr2=0x556528ae9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/2351734769 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55652874f2d0 con 0x556528ad8690 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:48.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ae28d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f400f1070 tx=0x7f9f400bca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3e2a8ed9c1d5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f2effd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f2effd640 1 -- 172.21.15.55:0/2351734769 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400d5060 con 0x556528ad8690 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.746+0000 7f9f2effd640 1 -- 172.21.15.55:0/2351734769 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9f400ee950 con 0x556528ad8690 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 1 -- 172.21.15.55:0/2351734769 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f48005670 con 0x556528ad8690 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f2effd640 1 -- 172.21.15.55:0/2351734769 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9f40049050 con 0x556528ad8690 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 10 client.15192 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 1 -- 172.21.15.55:0/2351734769 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5565288196b0 con 0x556528ad8690 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f2effd640 1 -- 172.21.15.55:0/2351734769 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f48005670 con 0x556528ad8690 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f2effd640 1 -- 172.21.15.55:0/2351734769 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f9f400d6250 con 0x556528ad8690 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 10 client.15192 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 20 client.15192 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556528aef310 0x556528af0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 4 client.15192 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f58789740 1 -- 172.21.15.55:0/2351734769 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556528a4b350 con 0x556528aef310 2024-04-26T21:52:48.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.747+0000 7f9f4effd640 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556528aef310 0x556528af0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.748+0000 7f9f4effd640 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556528aef310 0x556528af0780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26b7bb67b602cd6e server_cookie=8db45e73c27ac4c8 in_seq=0 out_seq=0 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.748+0000 7f9f2effd640 10 client.15192 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.748+0000 7f9f2effd640 1 -- 172.21.15.55:0/2351734769 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556528a4b350 con 0x556528aef310 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.748+0000 7f9f2effd640 10 client.15192 handle_command_reply: tid=0 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.748+0000 7f9f58789740 1 client.15192 shutdown 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 -- 172.21.15.55:0/2351734769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 msgr2=0x556528ae28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ae28d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f400f1070 tx=0x7f9f400bca90 comp rx=0 tx=0).stop 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 -- 172.21.15.55:0/2351734769 shutdown_connections 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556528aef310 0x556528af0780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528ae2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f2effd640 0 client.15192 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 --2- 172.21.15.55:0/2351734769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528ae28d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 -- 172.21.15.55:0/2351734769 >> 172.21.15.55:0/2351734769 conn(0x556528ad3d00 msgr2=0x556528ad2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 -- 172.21.15.55:0/2351734769 shutdown_connections 2024-04-26T21:52:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 1 -- 172.21.15.55:0/2351734769 wait complete. 2024-04-26T21:52:48.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.749+0000 7f9f58789740 20 client.15192 trim_cache size 0 max 16384 2024-04-26T21:52:48.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f58789740 1 Processor -- start 2024-04-26T21:52:48.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f58789740 1 -- start start 2024-04-26T21:52:48.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f58789740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528aedfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f58789740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528aee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f58789740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f58789740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556528aeeac0 con 0x556528ad8690 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f58789740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556528aef190 con 0x556528ade1f0 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f58789740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556528aef860 con 0x556528ad9080 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528aedfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528aedfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54284/0 (socket says 172.21.15.55:54284) 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/3250060192 learned_addr learned my addr 172.21.15.55:0/3250060192 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/3250060192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 msgr2=0x556528aee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4effd640 1 --2- 172.21.15.55:0/3250060192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528aee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/3250060192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528aee4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/3250060192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 msgr2=0x556528ae9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/3250060192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 -- 172.21.15.55:0/3250060192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556528ad4a70 con 0x556528ad8690 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.767+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/3250060192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528aedfb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9f400bce60 tx=0x7f9f4003b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f61f622134e18a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f357fa640 1 -- 172.21.15.55:0/3250060192 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400d5070 con 0x556528ad8690 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f357fa640 1 -- 172.21.15.55:0/3250060192 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f400bd5d0 con 0x556528ad8690 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f357fa640 1 -- 172.21.15.55:0/3250060192 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400ee950 con 0x556528ad8690 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 -- 172.21.15.55:0/3250060192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 msgr2=0x556528aedfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 --2- 172.21.15.55:0/3250060192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528aedfb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9f400bce60 tx=0x7f9f4003b720 comp rx=0 tx=0).stop 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 -- 172.21.15.55:0/3250060192 shutdown_connections 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 --2- 172.21.15.55:0/3250060192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 --2- 172.21.15.55:0/3250060192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad9080 0x556528aee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 --2- 172.21.15.55:0/3250060192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad8690 0x556528aedfb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 -- 172.21.15.55:0/3250060192 >> 172.21.15.55:0/3250060192 conn(0x556528ad3d00 msgr2=0x556528ad40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 -- 172.21.15.55:0/3250060192 shutdown_connections 2024-04-26T21:52:48.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 -- 172.21.15.55:0/3250060192 wait complete. 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.768+0000 7f9f58789740 1 -- start start 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f58789740 1 Processor -- start 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f58789740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad8690 0x556528ae5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f58789740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad9080 0x556528ae61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f58789740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f58789740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556528af0600 con 0x556528ad9080 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f58789740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556528aeff30 con 0x556528ade1f0 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f58789740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556528af0cd0 con 0x556528ad8690 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f58789740 10 client.0 _ll_register_callbacks cb 0x7f9f40005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad9080 0x556528ae61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad9080 0x556528ae61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54300/0 (socket says 172.21.15.55:54300) 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4effd640 1 -- 172.21.15.55:0/419471678 learned_addr learned my addr 172.21.15.55:0/419471678 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad8690 0x556528ae5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4effd640 1 -- 172.21.15.55:0/419471678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad8690 msgr2=0x556528ae5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4effd640 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad8690 0x556528ae5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4effd640 1 -- 172.21.15.55:0/419471678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 msgr2=0x556528ae91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4effd640 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4effd640 1 -- 172.21.15.55:0/419471678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556528ae97e0 con 0x556528ad9080 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.769+0000 7f9f4ffff640 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f4effd640 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad9080 0x556528ae61d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9f44000c00 tx=0x7f9f44011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a5d7cd07621a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f2effd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f2effd640 1 -- 172.21.15.55:0/419471678 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4401b880 con 0x556528ad9080 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f2effd640 1 -- 172.21.15.55:0/419471678 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f9f44010550 con 0x556528ad9080 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 1 -- 172.21.15.55:0/419471678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f400bd5d0 con 0x556528ad9080 2024-04-26T21:52:48.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f2effd640 1 -- 172.21.15.55:0/419471678 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9f4402b050 con 0x556528ad9080 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 10 client.15198 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 1 -- 172.21.15.55:0/419471678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556528b55ae0 con 0x556528ad9080 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f2effd640 1 -- 172.21.15.55:0/419471678 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f400bd5d0 con 0x556528ad9080 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f2effd640 1 -- 172.21.15.55:0/419471678 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f9f440247f0 con 0x556528ad9080 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 10 client.15198 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 20 client.15198 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556528b5a690 0x556528b593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 4 client.15198 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.770+0000 7f9f58789740 1 -- 172.21.15.55:0/419471678 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "status"}) -- 0x556528a451c0 con 0x556528b5a690 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556528b5a690 0x556528b593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f4f7fe640 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556528b5a690 0x556528b593a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f6ca4e98dcc8010 server_cookie=c59351fdff8e0893 in_seq=0 out_seq=0 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f2effd640 10 client.15198 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f2effd640 1 -- 172.21.15.55:0/419471678 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556528a451c0 con 0x556528b5a690 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f2effd640 10 client.15198 handle_command_reply: tid=0 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 client.15198 shutdown 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 -- 172.21.15.55:0/419471678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad9080 msgr2=0x556528ae61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad9080 0x556528ae61d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9f44000c00 tx=0x7f9f44011a40 comp rx=0 tx=0).stop 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 -- 172.21.15.55:0/419471678 shutdown_connections 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556528b5a690 0x556528b593a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556528ade1f0 0x556528ae91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556528ad9080 0x556528ae61d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f2effd640 0 client.15198 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 --2- 172.21.15.55:0/419471678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556528ad8690 0x556528ae5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.771+0000 7f9f58789740 1 -- 172.21.15.55:0/419471678 >> 172.21.15.55:0/419471678 conn(0x556528ad3d00 msgr2=0x556528aedfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f58789740 1 -- 172.21.15.55:0/419471678 shutdown_connections 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f58789740 1 -- 172.21.15.55:0/419471678 wait complete. 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f58789740 20 client.15198 trim_cache size 0 max 16384 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 2, 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 24502, 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 108.477321178, 2024-04-26T21:52:48.779 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 20, 2024-04-26T21:52:48.780 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 87, 2024-04-26T21:52:48.780 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 85, 2024-04-26T21:52:48.780 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 126.18513484100001 2024-04-26T21:52:48.780 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f380a3c40 msgr2=0x7f9f380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f380a3c40 0x7f9f380a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f4801f000 tx=0x7f9f4801e000 comp rx=0 tx=0).stop 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 msgr2=0x7f9f500766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 0x7f9f500766e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9f40006010 tx=0x7f9f40004430 comp rx=0 tx=0).stop 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1609668492 shutdown_connections 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9f380a3c40 0x7f9f380a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50104540 0x7f9f50075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f50103b80 0x7f9f50074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 --2- 172.21.15.55:0/1609668492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f50102990 0x7f9f500766e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1609668492 >> 172.21.15.55:0/1609668492 conn(0x7f9f500fdf00 msgr2=0x7f9f50106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1609668492 shutdown_connections 2024-04-26T21:52:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.772+0000 7f9f55a6b640 1 -- 172.21.15.55:0/1609668492 wait complete. 2024-04-26T21:52:48.784 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-26T21:52:48.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:48 smithi165 ceph-mon[33296]: pgmap v346: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:48.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:48 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2538795610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:48.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:48 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1925874481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:48.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:48 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1062508416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:48.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 -- 172.21.15.55:0/634045142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 msgr2=0x7f8e8c104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 --2- 172.21.15.55:0/634045142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 0x7f8e8c104000 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009900 tx=0x7f8e8002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 -- 172.21.15.55:0/634045142 shutdown_connections 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 --2- 172.21.15.55:0/634045142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e8c104540 0x7f8e8c10add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 --2- 172.21.15.55:0/634045142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 0x7f8e8c104000 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 --2- 172.21.15.55:0/634045142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c102990 0x7f8e8c102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 -- 172.21.15.55:0/634045142 >> 172.21.15.55:0/634045142 conn(0x7f8e8c0fdf00 msgr2=0x7f8e8c100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 -- 172.21.15.55:0/634045142 shutdown_connections 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 -- 172.21.15.55:0/634045142 wait complete. 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.886+0000 7f8e91d46640 1 Processor -- start 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e91d46640 1 -- start start 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e91d46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c102990 0x7f8e8c0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e91d46640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 0x7f8e8c074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e91d46640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e8c104540 0x7f8e8c075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e91d46640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10e520 con 0x7f8e8c103b80 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e91d46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10e3a0 con 0x7f8e8c102990 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e91d46640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c10e6a0 con 0x7f8e8c104540 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 0x7f8e8c074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 0x7f8e8c074d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54312/0 (socket says 172.21.15.55:54312) 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8affd640 1 -- 172.21.15.55:0/539130873 learned_addr learned my addr 172.21.15.55:0/539130873 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c102990 0x7f8e8c0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e8c104540 0x7f8e8c075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8affd640 1 -- 172.21.15.55:0/539130873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e8c104540 msgr2=0x7f8e8c075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8affd640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e8c104540 0x7f8e8c075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8affd640 1 -- 172.21.15.55:0/539130873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c102990 msgr2=0x7f8e8c0766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8affd640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c102990 0x7f8e8c0766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.887+0000 7f8e8affd640 1 -- 172.21.15.55:0/539130873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c075af0 con 0x7f8e8c103b80 2024-04-26T21:52:48.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.888+0000 7f8e8affd640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 0x7f8e8c074d20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8e80005940 tx=0x7f8e80004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.888+0000 7f8e88ff9640 1 -- 172.21.15.55:0/539130873 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8003f030 con 0x7f8e8c103b80 2024-04-26T21:52:48.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.888+0000 7f8e91d46640 1 -- 172.21.15.55:0/539130873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c1a2d00 con 0x7f8e8c103b80 2024-04-26T21:52:48.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.888+0000 7f8e88ff9640 1 -- 172.21.15.55:0/539130873 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e80003cb0 con 0x7f8e8c103b80 2024-04-26T21:52:48.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.888+0000 7f8e88ff9640 1 -- 172.21.15.55:0/539130873 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80035070 con 0x7f8e8c103b80 2024-04-26T21:52:48.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.888+0000 7f8e91d46640 1 -- 172.21.15.55:0/539130873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a3260 con 0x7f8e8c103b80 2024-04-26T21:52:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e88ff9640 1 -- 172.21.15.55:0/539130873 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8e80042030 con 0x7f8e8c103b80 2024-04-26T21:52:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e94a69740 1 Processor -- start 2024-04-26T21:52:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e94a69740 1 -- start start 2024-04-26T21:52:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 0x55688373fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883744b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883747680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e94a69740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5568836a4740 con 0x5568837451f0 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e94a69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568836403a0 con 0x556883740080 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.890+0000 7f8e94a69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5568836a0700 con 0x55688373f690 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883747680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883747680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54326/0 (socket says 172.21.15.55:54326) 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 -- 172.21.15.55:0/3722927372 learned_addr learned my addr 172.21.15.55:0/3722927372 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883744b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 0x55688373fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e88ff9640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e740a3c40 0x7f8e740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 -- 172.21.15.55:0/3722927372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 msgr2=0x55688373fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 0x55688373fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 -- 172.21.15.55:0/3722927372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 msgr2=0x556883744b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883744b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 -- 172.21.15.55:0/3722927372 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688365c750 con 0x5568837451f0 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e740a3c40 0x7f8e740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e88ff9640 1 -- 172.21.15.55:0/539130873 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8e800f0e70 con 0x7f8e8c103b80 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8affd640 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883747680 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8e800ee740 tx=0x7f8e8008eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4749ab854eb175 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e71ffb640 1 -- 172.21.15.55:0/3722927372 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e800bc9e0 con 0x5568837451f0 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e71ffb640 1 -- 172.21.15.55:0/3722927372 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8008eb80 con 0x5568837451f0 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e740a3c40 0x7f8e740a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5568837408e0 tx=0x7f8e84004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.891+0000 7f8e71ffb640 1 -- 172.21.15.55:0/3722927372 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e800bd080 con 0x5568837451f0 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 -- 172.21.15.55:0/3722927372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 msgr2=0x556883747680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883747680 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8e800ee740 tx=0x7f8e8008eb10 comp rx=0 tx=0).stop 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 -- 172.21.15.55:0/3722927372 shutdown_connections 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883747680 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883744b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3722927372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 0x55688373fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 -- 172.21.15.55:0/3722927372 >> 172.21.15.55:0/3722927372 conn(0x55688373ad00 msgr2=0x55688373b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 -- 172.21.15.55:0/3722927372 shutdown_connections 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 -- 172.21.15.55:0/3722927372 wait complete. 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.892+0000 7f8e94a69740 1 -- start start 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e94a69740 1 Processor -- start 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55688373f690 0x5568837497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556883740080 0x556883749ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883750d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e94a69740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556883747f60 con 0x5568837451f0 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e94a69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55688362bfe0 con 0x55688373f690 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e94a69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5568837480e0 con 0x556883740080 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e94a69740 10 client.0 _ll_register_callbacks cb 0x556883642150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883750d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883750d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54336/0 (socket says 172.21.15.55:54336) 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8bfff640 1 -- 172.21.15.55:0/3973475717 learned_addr learned my addr 172.21.15.55:0/3973475717 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55688373f690 0x5568837497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8affd640 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556883740080 0x556883749ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8bfff640 1 -- 172.21.15.55:0/3973475717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556883740080 msgr2=0x556883749ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556883740080 0x556883749ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8bfff640 1 -- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55688373f690 msgr2=0x5568837497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55688373f690 0x5568837497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8bfff640 1 -- 172.21.15.55:0/3973475717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568833b62d0 con 0x5568837451f0 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.893+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55688373f690 0x5568837497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883750d20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8e84016df0 tx=0x7f8e8400d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d511100b5227f0de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e6affd640 1 -- 172.21.15.55:0/3973475717 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84027060 con 0x5568837451f0 2024-04-26T21:52:48.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e6affd640 1 -- 172.21.15.55:0/3973475717 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8e84022610 con 0x5568837451f0 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 1 -- 172.21.15.55:0/3973475717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568836b2350 con 0x5568837451f0 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e6affd640 1 -- 172.21.15.55:0/3973475717 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8e84022850 con 0x5568837451f0 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 10 client.15210 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 1 -- 172.21.15.55:0/3973475717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5568834806b0 con 0x5568837451f0 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e6affd640 1 -- 172.21.15.55:0/3973475717 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5568836b2350 con 0x5568837451f0 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e6affd640 1 -- 172.21.15.55:0/3973475717 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f8e8402d970 con 0x5568837451f0 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 10 client.15210 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 10 client.15210 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 20 client.15210 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556883756310 0x556883757780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 4 client.15210 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.894+0000 7f8e94a69740 1 -- 172.21.15.55:0/3973475717 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568836ac1c0 con 0x556883756310 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.895+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556883756310 0x556883757780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.895+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556883756310 0x556883757780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=615c80397a5be796 server_cookie=e5ec934685c363dc in_seq=0 out_seq=0 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.895+0000 7f8e6affd640 10 client.15210 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e6affd640 1 -- 172.21.15.55:0/3973475717 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568836ac1c0 con 0x556883756310 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e6affd640 10 client.15210 handle_command_reply: tid=0 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 client.15210 shutdown 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 -- 172.21.15.55:0/3973475717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 msgr2=0x556883750d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883750d20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8e84016df0 tx=0x7f8e8400d040 comp rx=0 tx=0).stop 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 -- 172.21.15.55:0/3973475717 shutdown_connections 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556883756310 0x556883757780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883750d20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e6affd640 0 client.15210 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556883740080 0x556883749ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 --2- 172.21.15.55:0/3973475717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55688373f690 0x5568837497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 -- 172.21.15.55:0/3973475717 >> 172.21.15.55:0/3973475717 conn(0x55688373ad00 msgr2=0x556883739f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 -- 172.21.15.55:0/3973475717 shutdown_connections 2024-04-26T21:52:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 1 -- 172.21.15.55:0/3973475717 wait complete. 2024-04-26T21:52:48.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.896+0000 7f8e94a69740 20 client.15210 trim_cache size 0 max 16384 2024-04-26T21:52:48.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.914+0000 7f8e94a69740 1 Processor -- start 2024-04-26T21:52:48.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.914+0000 7f8e94a69740 1 -- start start 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.914+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 0x556883736150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.914+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883750ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.914+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883749ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.914+0000 7f8e94a69740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556883755ac0 con 0x5568837451f0 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.914+0000 7f8e94a69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556883756190 con 0x556883740080 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.914+0000 7f8e94a69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556883756860 con 0x55688373f690 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883749ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883749ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54346/0 (socket says 172.21.15.55:54346) 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 -- 172.21.15.55:0/142307571 learned_addr learned my addr 172.21.15.55:0/142307571 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 0x556883736150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8affd640 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883750ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 -- 172.21.15.55:0/142307571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 msgr2=0x556883736150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 0x556883736150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 -- 172.21.15.55:0/142307571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 msgr2=0x556883750ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883750ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 -- 172.21.15.55:0/142307571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556883744ed0 con 0x5568837451f0 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8affd640 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883750ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883749ac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8e84016dc0 tx=0x7f8e84021640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0cbe620d1b8ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e717fa640 1 -- 172.21.15.55:0/142307571 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8403a070 con 0x5568837451f0 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e717fa640 1 -- 172.21.15.55:0/142307571 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8401d050 con 0x5568837451f0 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.915+0000 7f8e717fa640 1 -- 172.21.15.55:0/142307571 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84021b10 con 0x5568837451f0 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 -- 172.21.15.55:0/142307571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 msgr2=0x556883749ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883749ac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8e84016dc0 tx=0x7f8e84021640 comp rx=0 tx=0).stop 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 -- 172.21.15.55:0/142307571 shutdown_connections 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568837451f0 0x556883749ac0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x556883750ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 --2- 172.21.15.55:0/142307571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55688373f690 0x556883736150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 -- 172.21.15.55:0/142307571 >> 172.21.15.55:0/142307571 conn(0x55688373ad00 msgr2=0x5568837493b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 -- 172.21.15.55:0/142307571 shutdown_connections 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 -- 172.21.15.55:0/142307571 wait complete. 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 -- start start 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.916+0000 7f8e94a69740 1 Processor -- start 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55688373f690 0x55688374cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x55688374d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e94a69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568837451f0 0x556883750290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e94a69740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556883757600 con 0x55688373f690 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e94a69740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556883756f30 con 0x556883740080 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e94a69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556883757cd0 con 0x5568837451f0 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e94a69740 10 client.0 _ll_register_callbacks cb 0x7f8e84019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55688373f690 0x55688374cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55688373f690 0x55688374cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54348/0 (socket says 172.21.15.55:54348) 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 -- 172.21.15.55:0/2441331901 learned_addr learned my addr 172.21.15.55:0/2441331901 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8affd640 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x55688374d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8bfff640 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568837451f0 0x556883750290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 -- 172.21.15.55:0/2441331901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568837451f0 msgr2=0x556883750290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568837451f0 0x556883750290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 -- 172.21.15.55:0/2441331901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 msgr2=0x55688374d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x55688374d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 -- 172.21.15.55:0/2441331901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556883750880 con 0x55688373f690 2024-04-26T21:52:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e8b7fe640 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55688373f690 0x55688374cf10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c004650 tx=0x7f8e7c00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5e1bc60f6c12b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e6affd640 1 -- 172.21.15.55:0/2441331901 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0037f0 con 0x55688373f690 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e94a69740 1 -- 172.21.15.55:0/2441331901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e8401d050 con 0x55688373f690 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e6affd640 1 -- 172.21.15.55:0/2441331901 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f8e7c003990 con 0x55688373f690 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.917+0000 7f8e6affd640 1 -- 172.21.15.55:0/2441331901 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8e7c02f050 con 0x55688373f690 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e94a69740 10 client.15216 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e94a69740 1 -- 172.21.15.55:0/2441331901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5568837bcc90 con 0x55688373f690 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e6affd640 1 -- 172.21.15.55:0/2441331901 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8401d050 con 0x55688373f690 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e6affd640 1 -- 172.21.15.55:0/2441331901 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f8e7c011c60 con 0x55688373f690 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e94a69740 10 client.15216 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e94a69740 10 client.15216 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e94a69740 20 client.15216 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e94a69740 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5568837c18a0 0x5568837c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e94a69740 4 client.15216 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:52:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e94a69740 1 -- 172.21.15.55:0/2441331901 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "status"}) -- 0x5568836279a0 con 0x5568837c18a0 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.918+0000 7f8e8affd640 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5568837c18a0 0x5568837c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e8affd640 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5568837c18a0 0x5568837c05b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10bd02108eb15d17 server_cookie=518f4899e59324c6 in_seq=0 out_seq=0 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e6affd640 10 client.15216 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e6affd640 1 -- 172.21.15.55:0/2441331901 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5568836279a0 con 0x5568837c18a0 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e6affd640 10 client.15216 handle_command_reply: tid=0 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 client.15216 shutdown 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 -- 172.21.15.55:0/2441331901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55688373f690 msgr2=0x55688374cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55688373f690 0x55688374cf10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c004650 tx=0x7f8e7c00bd00 comp rx=0 tx=0).stop 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 -- 172.21.15.55:0/2441331901 shutdown_connections 2024-04-26T21:52:48.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5568837c18a0 0x5568837c05b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568837451f0 0x556883750290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556883740080 0x55688374d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 --2- 172.21.15.55:0/2441331901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55688373f690 0x55688374cf10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 -- 172.21.15.55:0/2441331901 >> 172.21.15.55:0/2441331901 conn(0x55688373ad00 msgr2=0x55688373da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e6affd640 0 client.15216 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 -- 172.21.15.55:0/2441331901 shutdown_connections 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 1 -- 172.21.15.55:0/2441331901 wait complete. 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.919+0000 7f8e94a69740 20 client.15216 trim_cache size 0 max 16384 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:52:48.927 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 1, 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 24509, 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 109.645603744, 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 20, 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 87, 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 85, 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 129.13147468899999 2024-04-26T21:52:48.928 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 -- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e740a3c40 msgr2=0x7f8e740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e740a3c40 0x7f8e740a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5568837408e0 tx=0x7f8e84004b50 comp rx=0 tx=0).stop 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 -- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 msgr2=0x7f8e8c074d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 0x7f8e8c074d20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8e80005940 tx=0x7f8e80004430 comp rx=0 tx=0).stop 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 -- 172.21.15.55:0/539130873 shutdown_connections 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e740a3c40 0x7f8e740a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e8c104540 0x7f8e8c075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e8c103b80 0x7f8e8c074d20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 --2- 172.21.15.55:0/539130873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c102990 0x7f8e8c0766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:48.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 -- 172.21.15.55:0/539130873 >> 172.21.15.55:0/539130873 conn(0x7f8e8c0fdf00 msgr2=0x7f8e8c106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:48.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 -- 172.21.15.55:0/539130873 shutdown_connections 2024-04-26T21:52:48.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:48.920+0000 7f8e91d46640 1 -- 172.21.15.55:0/539130873 wait complete. 2024-04-26T21:52:48.931 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T21:52:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:48 smithi064 ceph-mon[32765]: pgmap v346: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:52:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:48 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2538795610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:48 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1925874481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:49.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:48 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1062508416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:49.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 -- 172.21.15.55:0/2244709797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24103b80 msgr2=0x7f6b24104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/2244709797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24103b80 0x7f6b24104000 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009900 tx=0x7f6b1402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 -- 172.21.15.55:0/2244709797 shutdown_connections 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/2244709797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b24104540 0x7f6b2410add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/2244709797 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24103b80 0x7f6b24104000 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/2244709797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b24102990 0x7f6b24102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 -- 172.21.15.55:0/2244709797 >> 172.21.15.55:0/2244709797 conn(0x7f6b240fdf00 msgr2=0x7f6b24100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 -- 172.21.15.55:0/2244709797 shutdown_connections 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.034+0000 7f6b2b32b640 1 -- 172.21.15.55:0/2244709797 wait complete. 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b2b32b640 1 Processor -- start 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b2b32b640 1 -- start start 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b2b32b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b24102990 0x7f6b24198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b2b32b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b24103b80 0x7f6b24199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b2b32b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24104540 0x7f6b241a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b2b32b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410e4a0 con 0x7f6b24104540 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b2b32b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410e320 con 0x7f6b24102990 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b2b32b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2410e620 con 0x7f6b24103b80 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24104540 0x7f6b241a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24104540 0x7f6b241a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54360/0 (socket says 172.21.15.55:54360) 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 -- 172.21.15.55:0/1310717756 learned_addr learned my addr 172.21.15.55:0/1310717756 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b290c7640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b24102990 0x7f6b24198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b288c6640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b24103b80 0x7f6b24199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 -- 172.21.15.55:0/1310717756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b24103b80 msgr2=0x7f6b24199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b24103b80 0x7f6b24199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 -- 172.21.15.55:0/1310717756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b24102990 msgr2=0x7f6b24198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b24102990 0x7f6b24198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 -- 172.21.15.55:0/1310717756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b241a07e0 con 0x7f6b24104540 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b288c6640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b24103b80 0x7f6b24199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:49.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.035+0000 7f6b298c8640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24104540 0x7f6b241a0170 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009fd0 tx=0x7f6b1c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:49.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.036+0000 7f6b1a7fc640 1 -- 172.21.15.55:0/1310717756 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c017070 con 0x7f6b24104540 2024-04-26T21:52:49.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.036+0000 7f6b1a7fc640 1 -- 172.21.15.55:0/1310717756 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b1c003e20 con 0x7f6b24104540 2024-04-26T21:52:49.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.036+0000 7f6b2b32b640 1 -- 172.21.15.55:0/1310717756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b241a0ad0 con 0x7f6b24104540 2024-04-26T21:52:49.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.036+0000 7f6b1a7fc640 1 -- 172.21.15.55:0/1310717756 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c010460 con 0x7f6b24104540 2024-04-26T21:52:49.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.036+0000 7f6b2b32b640 1 -- 172.21.15.55:0/1310717756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b241a0fd0 con 0x7f6b24104540 2024-04-26T21:52:49.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b1a7fc640 1 -- 172.21.15.55:0/1310717756 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b1c003970 con 0x7f6b24104540 2024-04-26T21:52:49.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b2e049740 1 Processor -- start 2024-04-26T21:52:49.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b2e049740 1 -- start start 2024-04-26T21:52:49.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e07fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 0x56157e084b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e087680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b2e049740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56157dfe4740 con 0x56157e0851f0 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b2e049740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56157df803a0 con 0x56157e07f690 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b2e049740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56157dfe0700 con 0x56157e080080 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b288c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e087680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.038+0000 7f6b1a7fc640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b100a3c40 0x7f6b100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e087680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54372/0 (socket says 172.21.15.55:54372) 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 -- 172.21.15.55:0/393775555 learned_addr learned my addr 172.21.15.55:0/393775555 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b290c7640 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 0x56157e084b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b298c8640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b100a3c40 0x7f6b100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b1a7fc640 1 -- 172.21.15.55:0/1310717756 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6b1c0d0370 con 0x7f6b24104540 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e07fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 -- 172.21.15.55:0/393775555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 msgr2=0x56157e084b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 0x56157e084b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 -- 172.21.15.55:0/393775555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 msgr2=0x56157e07fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e07fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 -- 172.21.15.55:0/393775555 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157df9c750 con 0x56157e0851f0 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b288c6640 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e087680 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6b140096c0 tx=0x7f6b14000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c260374f6e9693 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b298c8640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b100a3c40 0x7f6b100a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c098ea0 tx=0x7f6b1c004480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b0f7fe640 1 -- 172.21.15.55:0/393775555 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14035040 con 0x56157e0851f0 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b0f7fe640 1 -- 172.21.15.55:0/393775555 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b140316a0 con 0x56157e0851f0 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.039+0000 7f6b0f7fe640 1 -- 172.21.15.55:0/393775555 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14003850 con 0x56157e0851f0 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 -- 172.21.15.55:0/393775555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 msgr2=0x56157e087680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e087680 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6b140096c0 tx=0x7f6b14000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 -- 172.21.15.55:0/393775555 shutdown_connections 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e087680 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 0x56157e084b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 --2- 172.21.15.55:0/393775555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e07fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 -- 172.21.15.55:0/393775555 >> 172.21.15.55:0/393775555 conn(0x56157e07ad00 msgr2=0x56157e07b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 -- 172.21.15.55:0/393775555 shutdown_connections 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 -- 172.21.15.55:0/393775555 wait complete. 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 -- start start 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.040+0000 7f6b2e049740 1 Processor -- start 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e07f690 0x56157e089b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 0x56157e08a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e0851f0 0x56157e091090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b2e049740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56157e088140 con 0x56157e07f690 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b2e049740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56157df6bfe0 con 0x56157e0851f0 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b2e049740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56157e0882c0 con 0x56157e080080 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e07f690 0x56157e089b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b2e049740 10 client.0 _ll_register_callbacks cb 0x56157df82150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e07f690 0x56157e089b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54382/0 (socket says 172.21.15.55:54382) 2024-04-26T21:52:49.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 -- 172.21.15.55:0/3381580440 learned_addr learned my addr 172.21.15.55:0/3381580440 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e0851f0 0x56157e091090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b290c7640 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 0x56157e08a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 -- 172.21.15.55:0/3381580440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 msgr2=0x56157e08a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 0x56157e08a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 -- 172.21.15.55:0/3381580440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e0851f0 msgr2=0x56157e091090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e0851f0 0x56157e091090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 -- 172.21.15.55:0/3381580440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157dcf62d0 con 0x56157e07f690 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b288c6640 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e07f690 0x56157e089b10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6b14001eb0 tx=0x7f6b14005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e1b1a9b329dc2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b0cff9640 1 -- 172.21.15.55:0/3381580440 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14035040 con 0x56157e07f690 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b0cff9640 1 -- 172.21.15.55:0/3381580440 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6b1404c900 con 0x56157e07f690 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.041+0000 7f6b2e049740 1 -- 172.21.15.55:0/3381580440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b140316a0 con 0x56157e07f690 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b0cff9640 1 -- 172.21.15.55:0/3381580440 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6b1404cb00 con 0x56157e07f690 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b2e049740 10 client.15228 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:49.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b2e049740 1 -- 172.21.15.55:0/3381580440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56157ddc06b0 con 0x56157e07f690 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b0cff9640 1 -- 172.21.15.55:0/3381580440 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b140316a0 con 0x56157e07f690 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b0cff9640 1 -- 172.21.15.55:0/3381580440 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f6b1403a050 con 0x56157e07f690 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b2e049740 10 client.15228 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b2e049740 10 client.15228 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b2e049740 20 client.15228 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b2e049740 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56157e096140 0x56157e0975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b2e049740 4 client.15228 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b2e049740 1 -- 172.21.15.55:0/3381580440 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56157dff2350 con 0x56157e096140 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.042+0000 7f6b290c7640 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56157e096140 0x56157e0975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.043+0000 7f6b0cff9640 10 client.15228 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.043+0000 7f6b290c7640 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56157e096140 0x56157e0975b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98eb268808b323b6 server_cookie=18754e2118e3d347 in_seq=0 out_seq=0 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b0cff9640 1 -- 172.21.15.55:0/3381580440 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56157dff2350 con 0x56157e096140 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b0cff9640 10 client.15228 handle_command_reply: tid=0 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 client.15228 shutdown 2024-04-26T21:52:49.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 -- 172.21.15.55:0/3381580440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e07f690 msgr2=0x56157e089b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e07f690 0x56157e089b10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6b14001eb0 tx=0x7f6b14005bd0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 -- 172.21.15.55:0/3381580440 shutdown_connections 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e0851f0 0x56157e091090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e080080 0x56157e08a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56157e096140 0x56157e0975b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 --2- 172.21.15.55:0/3381580440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e07f690 0x56157e089b10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b0cff9640 0 client.15228 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 -- 172.21.15.55:0/3381580440 >> 172.21.15.55:0/3381580440 conn(0x56157e07ad00 msgr2=0x56157e079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 -- 172.21.15.55:0/3381580440 shutdown_connections 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 1 -- 172.21.15.55:0/3381580440 wait complete. 2024-04-26T21:52:49.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.044+0000 7f6b2e049740 20 client.15228 trim_cache size 0 max 16384 2024-04-26T21:52:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b2e049740 1 Processor -- start 2024-04-26T21:52:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b2e049740 1 -- start start 2024-04-26T21:52:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e07f690 0x56157e089720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e080080 0x56157e089c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e079670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b2e049740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56157e0958f0 con 0x56157e0851f0 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b2e049740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56157e095fc0 con 0x56157e080080 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b2e049740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56157e096690 con 0x56157e07f690 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b288c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e079670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b288c6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e079670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54388/0 (socket says 172.21.15.55:54388) 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.062+0000 7f6b288c6640 1 -- 172.21.15.55:0/1728589207 learned_addr learned my addr 172.21.15.55:0/1728589207 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b290c7640 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e080080 0x56157e089c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b288c6640 1 -- 172.21.15.55:0/1728589207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e07f690 msgr2=0x56157e089720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b288c6640 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e07f690 0x56157e089720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b288c6640 1 -- 172.21.15.55:0/1728589207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e080080 msgr2=0x56157e089c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b288c6640 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e080080 0x56157e089c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b288c6640 1 -- 172.21.15.55:0/1728589207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157e091410 con 0x56157e0851f0 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b290c7640 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e080080 0x56157e089c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b288c6640 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e079670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6b140095c0 tx=0x7f6b140092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd09cbe5e863e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b0effd640 1 -- 172.21.15.55:0/1728589207 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14035070 con 0x56157e0851f0 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b0effd640 1 -- 172.21.15.55:0/1728589207 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b14052070 con 0x56157e0851f0 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.063+0000 7f6b0effd640 1 -- 172.21.15.55:0/1728589207 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1404cca0 con 0x56157e0851f0 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- 172.21.15.55:0/1728589207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 msgr2=0x56157e079670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e079670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6b140095c0 tx=0x7f6b140092d0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- 172.21.15.55:0/1728589207 shutdown_connections 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e0851f0 0x56157e079670 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e080080 0x56157e089c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 --2- 172.21.15.55:0/1728589207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e07f690 0x56157e089720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- 172.21.15.55:0/1728589207 >> 172.21.15.55:0/1728589207 conn(0x56157e07ad00 msgr2=0x56157e07b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- 172.21.15.55:0/1728589207 shutdown_connections 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- 172.21.15.55:0/1728589207 wait complete. 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- start start 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 Processor -- start 2024-04-26T21:52:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e0f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e080080 0x56157e0f8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e0851f0 0x56157e0ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56157e097430 con 0x56157e080080 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56157e096d60 con 0x56157e07f690 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56157e097b00 con 0x56157e0851f0 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b2e049740 10 client.0 _ll_register_callbacks cb 0x7f6b2000a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b290c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e080080 0x56157e0f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b290c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e080080 0x56157e0f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54390/0 (socket says 172.21.15.55:54390) 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.064+0000 7f6b290c7640 1 -- 172.21.15.55:0/2452509573 learned_addr learned my addr 172.21.15.55:0/2452509573 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b288c6640 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e0f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b288c6640 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e0851f0 0x56157e0ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b290c7640 1 -- 172.21.15.55:0/2452509573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e0851f0 msgr2=0x56157e0ff4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b290c7640 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e0851f0 0x56157e0ff4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b290c7640 1 -- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 msgr2=0x56157e0f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b290c7640 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e0f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b290c7640 1 -- 172.21.15.55:0/2452509573 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157e0ffae0 con 0x56157e080080 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b288c6640 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e0f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b290c7640 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e080080 0x56157e0f8860 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000ab90 tx=0x7f6b20010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14988cf8043fd74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b0cff9640 1 -- 172.21.15.55:0/2452509573 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20004440 con 0x56157e080080 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b0cff9640 1 -- 172.21.15.55:0/2452509573 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8269+0+0 (secure 0 0 0) 0x7f6b2001d3b0 con 0x56157e080080 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b2e049740 1 -- 172.21.15.55:0/2452509573 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b14052070 con 0x56157e080080 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b0cff9640 1 -- 172.21.15.55:0/2452509573 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6b2002d050 con 0x56157e080080 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b2e049740 10 client.15234 fetch_fsmap learned FSMap version 20 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b2e049740 1 -- 172.21.15.55:0/2452509573 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56157e100ee0 con 0x56157e080080 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b0cff9640 1 -- 172.21.15.55:0/2452509573 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b14052070 con 0x56157e080080 2024-04-26T21:52:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b0cff9640 1 -- 172.21.15.55:0/2452509573 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 20) ==== 8338+0+0 (secure 0 0 0) 0x7f6b2000f420 con 0x56157e080080 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.065+0000 7f6b2e049740 10 client.15234 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b2e049740 10 client.15234 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b2e049740 20 client.15234 populate_metadata read hostname 'smithi055' 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b2e049740 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56157e105aa0 0x56157e1025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b2e049740 4 client.15234 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b2e049740 1 -- 172.21.15.55:0/2452509573 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "status"}) -- 0x56157dfec1c0 con 0x56157e105aa0 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b288c6640 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56157e105aa0 0x56157e1025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b288c6640 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56157e105aa0 0x56157e1025b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e997d6e638df923a server_cookie=85b4fbb24320747 in_seq=0 out_seq=0 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b0cff9640 10 client.15234 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b0cff9640 1 -- 172.21.15.55:0/2452509573 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56157dfec1c0 con 0x56157e105aa0 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b0cff9640 10 client.15234 handle_command_reply: tid=0 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.066+0000 7f6b2e049740 1 client.15234 shutdown 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 -- 172.21.15.55:0/2452509573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e080080 msgr2=0x56157e0f8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e080080 0x56157e0f8860 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000ab90 tx=0x7f6b20010800 comp rx=0 tx=0).stop 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 -- 172.21.15.55:0/2452509573 shutdown_connections 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56157e0851f0 0x56157e0ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56157e105aa0 0x56157e1025b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56157e080080 0x56157e0f8860 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 --2- 172.21.15.55:0/2452509573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56157e07f690 0x56157e0f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b0cff9640 0 client.15234 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 -- 172.21.15.55:0/2452509573 >> 172.21.15.55:0/2452509573 conn(0x56157e07ad00 msgr2=0x56157e089720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 -- 172.21.15.55:0/2452509573 shutdown_connections 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 1 -- 172.21.15.55:0/2452509573 wait complete. 2024-04-26T21:52:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2e049740 20 client.15234 trim_cache size 0 max 16384 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "c84f1140-0415-11ef-bc93-c7b262605968", 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 24551, 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 111.717582083, 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 20, 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 87, 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 85, 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 117.967065539 2024-04-26T21:52:49.075 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:52:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 -- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b100a3c40 msgr2=0x7f6b100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b100a3c40 0x7f6b100a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c098ea0 tx=0x7f6b1c004480 comp rx=0 tx=0).stop 2024-04-26T21:52:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 -- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24104540 msgr2=0x7f6b241a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24104540 0x7f6b241a0170 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009fd0 tx=0x7f6b1c00b260 comp rx=0 tx=0).stop 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 -- 172.21.15.55:0/1310717756 shutdown_connections 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b100a3c40 0x7f6b100a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b24104540 0x7f6b241a0170 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b24103b80 0x7f6b24199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 --2- 172.21.15.55:0/1310717756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b24102990 0x7f6b24198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 -- 172.21.15.55:0/1310717756 >> 172.21.15.55:0/1310717756 conn(0x7f6b240fdf00 msgr2=0x7f6b24106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 -- 172.21.15.55:0/1310717756 shutdown_connections 2024-04-26T21:52:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:49.067+0000 7f6b2b32b640 1 -- 172.21.15.55:0/1310717756 wait complete. 2024-04-26T21:52:49.079 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T21:52:49.091 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T21:52:49.091 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T21:52:49.091 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T21:52:49.092 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T21:52:49.113 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T21:52:49.113 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T21:52:49.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:49.113 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-04-26T21:52:49.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:49.129 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2024-04-26T21:52:49.196 INFO:teuthology.orchestra.run.smithi055.stderr:Cannot find device "ceph-brx" 2024-04-26T21:52:49.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:52:49.197 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-26T21:52:49.198 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T21:52:49.198 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:52:49.198 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T21:52:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:52:49.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:49 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3972775547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:49.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:49 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3972775547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:49 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3972775547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:50.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:50 smithi055 ceph-mon[29221]: pgmap v347: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:50.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-26T21:52:50.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:50 smithi165 ceph-mon[33296]: pgmap v347: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-26T21:52:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:50 smithi064 ceph-mon[32765]: pgmap v347: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:52:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-26T21:52:51.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 -- 172.21.15.55:0/736405404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8102320 msgr2=0x7fcbd81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/736405404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8102320 0x7fcbd81027a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009990 tx=0x7fcbc0009ca0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 -- 172.21.15.55:0/736405404 shutdown_connections 2024-04-26T21:52:51.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/736405404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8102ce0 0x7fcbd810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/736405404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd8108390 0x7fcbd8108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/736405404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8102320 0x7fcbd81027a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 -- 172.21.15.55:0/736405404 >> 172.21.15.55:0/736405404 conn(0x7fcbd80792d0 msgr2=0x7fcbd80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 -- 172.21.15.55:0/736405404 shutdown_connections 2024-04-26T21:52:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.313+0000 7fcbdcd17700 1 -- 172.21.15.55:0/736405404 wait complete. 2024-04-26T21:52:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.314+0000 7fcbdcd17700 1 Processor -- start 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.314+0000 7fcbdcd17700 1 -- start start 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbdcd17700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8102320 0x7fcbd819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbdcd17700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd8102ce0 0x7fcbd819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbdcd17700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8108390 0x7fcbd81a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbdcd17700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81125b0 con 0x7fcbd8108390 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbdcd17700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8112430 con 0x7fcbd8102ce0 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbdcd17700 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8112730 con 0x7fcbd8102320 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8108390 0x7fcbd81a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8108390 0x7fcbd81a5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45446/0 (socket says 172.21.15.55:45446) 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd659c700 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8102320 0x7fcbd819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 -- 172.21.15.55:0/2194624522 learned_addr learned my addr 172.21.15.55:0/2194624522 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd5d9b700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd8102ce0 0x7fcbd819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 -- 172.21.15.55:0/2194624522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8102320 msgr2=0x7fcbd819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8102320 0x7fcbd819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 -- 172.21.15.55:0/2194624522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd8102ce0 msgr2=0x7fcbd819b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd8102ce0 0x7fcbd819b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 -- 172.21.15.55:0/2194624522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd81a6090 con 0x7fcbd8108390 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.315+0000 7fcbd6d9d700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8108390 0x7fcbd81a5b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b970 tx=0x7fcbd000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.316+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/2194624522 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000d860 con 0x7fcbd8108390 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.316+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd81a6260 con 0x7fcbd8108390 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.316+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/2194624522 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd000e430 con 0x7fcbd8108390 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.316+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/2194624522 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0017660 con 0x7fcbd8108390 2024-04-26T21:52:51.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.316+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd81a6770 con 0x7fcbd8108390 2024-04-26T21:52:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.317+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd810a900 con 0x7fcbd8108390 2024-04-26T21:52:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.318+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/2194624522 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcbd000da00 con 0x7fcbd8108390 2024-04-26T21:52:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.319+0000 7fcbcf7fe700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcbc40a3cd0 0x7fcbc40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.319+0000 7fcbd659c700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcbc40a3cd0 0x7fcbc40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.319+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/2194624522 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8298+0+0 (secure 0 0 0) 0x7fcbd00d0190 con 0x7fcbd8108390 2024-04-26T21:52:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.319+0000 7fcbd659c700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcbc40a3cd0 0x7fcbc40a6190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009a10 tx=0x7fcbc8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:51.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.323+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/2194624522 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcbd0013030 con 0x7fcbd8108390 2024-04-26T21:52:51.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.580+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcbd806b6a0 con 0x7fcbd8108390 2024-04-26T21:52:51.581 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-26T21:52:51.581 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:51 smithi055 ceph-mon[29221]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:52:51.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.580+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/2194624522 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7fcbd009c970 con 0x7fcbd8108390 2024-04-26T21:52:51.582 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T21:52:51.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.582+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcbc40a3cd0 msgr2=0x7fcbc40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcbc40a3cd0 0x7fcbc40a6190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009a10 tx=0x7fcbc8009190 comp rx=0 tx=0).stop 2024-04-26T21:52:51.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8108390 msgr2=0x7fcbd81a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:51.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8108390 0x7fcbd81a5b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcbd000b970 tx=0x7fcbd000bc80 comp rx=0 tx=0).stop 2024-04-26T21:52:51.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 shutdown_connections 2024-04-26T21:52:51.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbd8108390 0x7fcbd81a5b50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8102320 0x7fcbd819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcbc40a3cd0 0x7fcbc40a6190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 --2- 172.21.15.55:0/2194624522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbd8102ce0 0x7fcbd819b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:51.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 >> 172.21.15.55:0/2194624522 conn(0x7fcbd80792d0 msgr2=0x7fcbd8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:51.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 shutdown_connections 2024-04-26T21:52:51.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:51.583+0000 7fcbdcd17700 1 -- 172.21.15.55:0/2194624522 wait complete. 2024-04-26T21:52:51.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-26T21:52:51.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:51 smithi165 ceph-mon[33296]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:52:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-26T21:52:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:51 smithi064 ceph-mon[32765]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T21:52:52.331 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T21:52:52.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T21:52:52.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2024-04-26T21:52:52.331 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T21:52:52.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T21:52:52.331 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T21:52:52.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T21:52:52.332 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T21:52:52.332 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:52:52.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:52.332 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2024-04-26T21:52:52.348 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:52:52.349 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2024-04-26T21:52:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-26T21:52:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42087sec preferred_lft 42087sec 2024-04-26T21:52:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-04-26T21:52:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2024-04-26T21:52:52.350 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T21:52:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2024-04-26T21:52:52.350 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2024-04-26T21:52:52.350 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T21:52:52.350 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:52:52.351 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-04-26T21:52:52.351 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add name ceph-brx type bridge 2024-04-26T21:52:52.351 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr flush dev ceph-brx 2024-04-26T21:52:52.351 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set ceph-brx up 2024-04-26T21:52:52.351 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T21:52:52.351 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-04-26T21:52:52.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:52:52.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:52.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:52.539 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T21:52:52.553 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:52 smithi055 ceph-mon[29221]: pgmap v349: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T21:52:52.553 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:52 smithi055 ceph-mon[29221]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:52:52.553 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:52 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2194624522' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:52:52.566 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-04-26T21:52:52.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:52.568 DEBUG:teuthology.orchestra.run.smithi055:> route 2024-04-26T21:52:52.639 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2024-04-26T21:52:52.639 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T21:52:52.639 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T21:52:52.639 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T21:52:52.639 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T21:52:52.640 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:52:52.640 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-04-26T21:52:52.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T21:52:52.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T21:52:52.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T21:52:52.640 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-04-26T21:52:52.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:52:52.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:52 smithi165 ceph-mon[33296]: pgmap v349: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T21:52:52.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:52 smithi165 ceph-mon[33296]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:52:52.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:52 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2194624522' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:52:52.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:52.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:52.915 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-04-26T21:52:52.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:52.931 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2024-04-26T21:52:52.987 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:52:52.987 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-04-26T21:52:52.987 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:52:52.987 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T21:52:52.987 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-04-26T21:52:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:52 smithi064 ceph-mon[32765]: pgmap v349: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-26T21:52:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:52 smithi064 ceph-mon[32765]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T21:52:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:52 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2194624522' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T21:52:53.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:52:53.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:53.106 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T21:52:53.106 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:52:53.106 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-04-26T21:52:53.106 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T21:52:53.106 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T21:52:53.107 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T21:52:53.107 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T21:52:53.107 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T21:52:53.107 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-04-26T21:52:53.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:52:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:53.227 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T21:52:53.227 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2024-04-26T21:52:53.227 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2024-04-26T21:52:53.227 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T21:52:53.228 DEBUG:teuthology.orchestra.run.smithi055:> ') 2024-04-26T21:52:53.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:52:53.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:53.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:52:53.305 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T21:52:53.335 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T21:52:53.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T21:52:53.335 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:52:53.390 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T21:52:53.390 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T21:52:53.390 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/ac2348ca-3ce8-43e6-9f97-0188cfba123e 2024-04-26T21:52:53.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:53.390 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/ac2348ca-3ce8-43e6-9f97-0188cfba123e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-26T21:52:53.480 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-26T21:52:53.481 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-04-26T21:52:53.481 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-04-26T21:52:53.605 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-26T21:52:53.605 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-04-26T21:52:53.606 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2024-04-26T21:52:53.606 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.64:6789,172.21.15.165:6789" 2024-04-26T21:52:53.606 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.64:6789,172.21.15.165:6789:/volumes/qa/sv_1/ac2348ca-3ce8-43e6-9f97-0188cfba123e 2024-04-26T21:52:53.606 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=c84f1140-0415-11ef-bc93-c7b262605968" will pass to kernel 2024-04-26T21:52:53.606 INFO:teuthology.orchestra.run.smithi055.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T21:52:53.607 INFO:teuthology.orchestra.run.smithi055.stdout: You just mounted a file system that supports labels which does not 2024-04-26T21:52:53.607 INFO:teuthology.orchestra.run.smithi055.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T21:52:53.607 INFO:teuthology.orchestra.run.smithi055.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T21:52:53.607 INFO:teuthology.orchestra.run.smithi055.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T21:52:53.608 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T21:52:53.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:52:53.609 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T21:52:53.638 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:52:53.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:52:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:53.736 INFO:teuthology.orchestra.run.smithi055.stdout:client24916 2024-04-26T21:52:53.737 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:52:53.737 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:52:53.753 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:52:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:52:53.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:53.850 INFO:teuthology.orchestra.run.smithi055.stdout:client24916 2024-04-26T21:52:53.851 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T21:52:53.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T21:52:53.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:53.907 INFO:teuthology.orchestra.run.smithi055.stdout:client24916 2024-04-26T21:52:53.907 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c84f1140-0415-11ef-bc93-c7b262605968.client24916/status') 2024-04-26T21:52:53.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T21:52:53.966 INFO:teuthology.orchestra.run.smithi055.stdout:instance: client.24916 (3)172.21.15.55:0/1203364215 2024-04-26T21:52:53.966 INFO:teuthology.orchestra.run.smithi055.stdout:blocklisted: false 2024-04-26T21:52:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records in 2024-04-26T21:52:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:0+1 records out 2024-04-26T21:52:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:71 bytes copied, 2.3847e-05 s, 3.0 MB/s 2024-04-26T21:52:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:52:53.974 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T21:52:53.987 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T21:52:53.987 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi055.front.sepia.ceph.com 2024-04-26T21:52:53.987 DEBUG:teuthology.orchestra.run.smithi055:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-26T21:52:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.090+0000 7f2691407640 1 -- 172.21.15.55:0/2226810651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c102a80 msgr2=0x7f268c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.090+0000 7f2691407640 1 --2- 172.21.15.55:0/2226810651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c102a80 0x7f268c102f00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2674009900 tx=0x7f267402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.090+0000 7f2691407640 1 -- 172.21.15.55:0/2226810651 shutdown_connections 2024-04-26T21:52:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.090+0000 7f2691407640 1 --2- 172.21.15.55:0/2226810651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c103440 0x7f268c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.090+0000 7f2691407640 1 --2- 172.21.15.55:0/2226810651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c102a80 0x7f268c102f00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.090+0000 7f2691407640 1 --2- 172.21.15.55:0/2226810651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c101890 0x7f268c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.090+0000 7f2691407640 1 -- 172.21.15.55:0/2226810651 >> 172.21.15.55:0/2226810651 conn(0x7f268c0fce00 msgr2=0x7f268c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 -- 172.21.15.55:0/2226810651 shutdown_connections 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 -- 172.21.15.55:0/2226810651 wait complete. 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 Processor -- start 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 -- start start 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c101890 0x7f268c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c102a80 0x7f268c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c103440 0x7f268c1a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f268c10e4e0 con 0x7f268c103440 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f268c10e360 con 0x7f268c101890 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.091+0000 7f2691407640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f268c10e660 con 0x7f268c102a80 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c103440 0x7f268c1a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c103440 0x7f268c1a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45468/0 (socket says 172.21.15.55:45468) 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 -- 172.21.15.55:0/3385853024 learned_addr learned my addr 172.21.15.55:0/3385853024 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268a7fc640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c102a80 0x7f268c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268affd640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c101890 0x7f268c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 -- 172.21.15.55:0/3385853024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c102a80 msgr2=0x7f268c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c102a80 0x7f268c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 -- 172.21.15.55:0/3385853024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c101890 msgr2=0x7f268c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c101890 0x7f268c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 -- 172.21.15.55:0/3385853024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c1a06f0 con 0x7f268c103440 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268a7fc640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c102a80 0x7f268c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f268b7fe640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c103440 0x7f268c1a0080 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f267c00cce0 tx=0x7f267c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f2683fff640 1 -- 172.21.15.55:0/3385853024 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c00f450 con 0x7f268c103440 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c1a09e0 con 0x7f268c103440 2024-04-26T21:52:54.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f2683fff640 1 -- 172.21.15.55:0/3385853024 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c00faa0 con 0x7f268c103440 2024-04-26T21:52:54.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f2683fff640 1 -- 172.21.15.55:0/3385853024 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c018b70 con 0x7f268c103440 2024-04-26T21:52:54.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.092+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c1a0f30 con 0x7f268c103440 2024-04-26T21:52:54.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.093+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268c108bb0 con 0x7f268c103440 2024-04-26T21:52:54.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.095+0000 7f2683fff640 1 -- 172.21.15.55:0/3385853024 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f267c018d10 con 0x7f268c103440 2024-04-26T21:52:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.095+0000 7f2683fff640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26780a3cf0 0x7f26780a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.095+0000 7f268affd640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26780a3cf0 0x7f26780a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.095+0000 7f2683fff640 1 -- 172.21.15.55:0/3385853024 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f267c0d0af0 con 0x7f268c103440 2024-04-26T21:52:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.096+0000 7f268affd640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26780a3cf0 0x7f26780a61b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2684005fd0 tx=0x7f2684005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:54.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.100+0000 7f2683fff640 1 -- 172.21.15.55:0/3385853024 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f267c014030 con 0x7f268c103440 2024-04-26T21:52:54.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.327+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f268c02dad0 con 0x7f268c103440 2024-04-26T21:52:54.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.557+0000 7f2683fff640 1 -- 172.21.15.55:0/3385853024 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v21) ==== 117+0+0 (secure 0 0 0) 0x7f267c09d330 con 0x7f268c103440 2024-04-26T21:52:54.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26780a3cf0 msgr2=0x7f26780a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26780a3cf0 0x7f26780a61b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2684005fd0 tx=0x7f2684005850 comp rx=0 tx=0).stop 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c103440 msgr2=0x7f268c1a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c103440 0x7f268c1a0080 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f267c00cce0 tx=0x7f267c00a620 comp rx=0 tx=0).stop 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 shutdown_connections 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f26780a3cf0 0x7f26780a61b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f268c103440 0x7f268c1a0080 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c102a80 0x7f268c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 --2- 172.21.15.55:0/3385853024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c101890 0x7f268c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 >> 172.21.15.55:0/3385853024 conn(0x7f268c0fce00 msgr2=0x7f268c105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 shutdown_connections 2024-04-26T21:52:54.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.559+0000 7f2691407640 1 -- 172.21.15.55:0/3385853024 wait complete. 2024-04-26T21:52:54.571 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T21:52:54.582 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-26T21:52:54.599 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 -- 172.21.15.55:0/2250748469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178102a80 msgr2=0x7f3178102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 --2- 172.21.15.55:0/2250748469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178102a80 0x7f3178102f00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3160009900 tx=0x7f316002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 -- 172.21.15.55:0/2250748469 shutdown_connections 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 --2- 172.21.15.55:0/2250748469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3178103440 0x7f3178109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 --2- 172.21.15.55:0/2250748469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178102a80 0x7f3178102f00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 --2- 172.21.15.55:0/2250748469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3178101890 0x7f3178101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 -- 172.21.15.55:0/2250748469 >> 172.21.15.55:0/2250748469 conn(0x7f31780fce20 msgr2=0x7f31780ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 -- 172.21.15.55:0/2250748469 shutdown_connections 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.702+0000 7f317cea0640 1 -- 172.21.15.55:0/2250748469 wait complete. 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317cea0640 1 Processor -- start 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317cea0640 1 -- start start 2024-04-26T21:52:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317cea0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178101890 0x7f317819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317cea0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3178102a80 0x7f317819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317cea0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3178103440 0x7f31781a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317cea0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f317810d3f0 con 0x7f3178101890 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317cea0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f317810d270 con 0x7f3178102a80 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317cea0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f317810d570 con 0x7f3178103440 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178101890 0x7f317819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178101890 0x7f317819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45482/0 (socket says 172.21.15.55:45482) 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317659c640 1 -- 172.21.15.55:0/1213523313 learned_addr learned my addr 172.21.15.55:0/1213523313 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f3175d9b640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3178102a80 0x7f317819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f3176d9d640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3178103440 0x7f31781a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317659c640 1 -- 172.21.15.55:0/1213523313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3178103440 msgr2=0x7f31781a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317659c640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3178103440 0x7f31781a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317659c640 1 -- 172.21.15.55:0/1213523313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3178102a80 msgr2=0x7f317819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.703+0000 7f317659c640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3178102a80 0x7f317819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.704+0000 7f317659c640 1 -- 172.21.15.55:0/1213523313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31781a2a10 con 0x7f3178101890 2024-04-26T21:52:54.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.704+0000 7f3175d9b640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3178102a80 0x7f317819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:54.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.704+0000 7f317659c640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178101890 0x7f317819aeb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f317000ccb0 tx=0x7f317000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:54.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.704+0000 7f316f7fe640 1 -- 172.21.15.55:0/1213523313 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170003990 con 0x7f3178101890 2024-04-26T21:52:54.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.704+0000 7f316f7fe640 1 -- 172.21.15.55:0/1213523313 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3170004020 con 0x7f3178101890 2024-04-26T21:52:54.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.704+0000 7f316f7fe640 1 -- 172.21.15.55:0/1213523313 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317000f6b0 con 0x7f3178101890 2024-04-26T21:52:54.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.704+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31781a2d00 con 0x7f3178101890 2024-04-26T21:52:54.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.704+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31781a31c0 con 0x7f3178101890 2024-04-26T21:52:54.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.705+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31781a3450 con 0x7f3178101890 2024-04-26T21:52:54.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.706+0000 7f316f7fe640 1 -- 172.21.15.55:0/1213523313 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f31700044b0 con 0x7f3178101890 2024-04-26T21:52:54.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.707+0000 7f316f7fe640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31640a3cc0 0x7f31640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:54.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.707+0000 7f3175d9b640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31640a3cc0 0x7f31640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:54.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.707+0000 7f316f7fe640 1 -- 172.21.15.55:0/1213523313 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f31700d09b0 con 0x7f3178101890 2024-04-26T21:52:54.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.707+0000 7f3175d9b640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31640a3cc0 0x7f31640a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3160000c00 tx=0x7f31600314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:54.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.711+0000 7f316f7fe640 1 -- 172.21.15.55:0/1213523313 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3170014030 con 0x7f3178101890 2024-04-26T21:52:54.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:54 smithi165 ceph-mon[33296]: pgmap v351: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T21:52:54.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:54 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3385853024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:52:54.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.898+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31781a3940 con 0x7f3178101890 2024-04-26T21:52:54.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.900+0000 7f316f7fe640 1 -- 172.21.15.55:0/1213523313 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22760 (secure 0 0 0) 0x7f317009d1f0 con 0x7f3178101890 2024-04-26T21:52:54.901 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:54.901 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:52:51.546766+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:54.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31640a3cc0 msgr2=0x7f31640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31640a3cc0 0x7f31640a6180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3160000c00 tx=0x7f31600314c0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178101890 msgr2=0x7f317819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178101890 0x7f317819aeb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f317000ccb0 tx=0x7f317000a620 comp rx=0 tx=0).stop 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 shutdown_connections 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31640a3cc0 0x7f31640a6180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3178103440 0x7f31781a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3178102a80 0x7f317819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 --2- 172.21.15.55:0/1213523313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3178101890 0x7f317819aeb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 >> 172.21.15.55:0/1213523313 conn(0x7f31780fce20 msgr2=0x7f31780fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 shutdown_connections 2024-04-26T21:52:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:54.902+0000 7f317cea0640 1 -- 172.21.15.55:0/1213523313 wait complete. 2024-04-26T21:52:54.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:54 smithi055 ceph-mon[29221]: pgmap v351: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T21:52:54.912 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:54 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3385853024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:52:54.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:52:54.914 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:52:55.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:54 smithi064 ceph-mon[32765]: pgmap v351: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T21:52:55.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:54 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3385853024' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T21:52:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 -- 172.21.15.55:0/1077728922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8378108650 msgr2=0x7f837810aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 --2- 172.21.15.55:0/1077728922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8378108650 0x7f837810aaa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8370009ca0 tx=0x7f837002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 -- 172.21.15.55:0/1077728922 shutdown_connections 2024-04-26T21:52:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 --2- 172.21.15.55:0/1077728922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8378108650 0x7f837810aaa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 --2- 172.21.15.55:0/1077728922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8378105d20 0x7f8378108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 --2- 172.21.15.55:0/1077728922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83781033f0 0x7f83781057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 -- 172.21.15.55:0/1077728922 >> 172.21.15.55:0/1077728922 conn(0x7f83780fce20 msgr2=0x7f83780ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 -- 172.21.15.55:0/1077728922 shutdown_connections 2024-04-26T21:52:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.019+0000 7f837d085640 1 -- 172.21.15.55:0/1077728922 wait complete. 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837d085640 1 Processor -- start 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837d085640 1 -- start start 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837d085640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83781033f0 0x7f8378198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837d085640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8378105d20 0x7f83781990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837d085640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8378108650 0x7f83781a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837d085640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f837810e4c0 con 0x7f83781033f0 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837d085640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f837810e340 con 0x7f8378108650 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837d085640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f837810e640 con 0x7f8378105d20 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f8376d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83781033f0 0x7f8378198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f8376d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83781033f0 0x7f8378198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45500/0 (socket says 172.21.15.55:45500) 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f8376d9d640 1 -- 172.21.15.55:0/2554280497 learned_addr learned my addr 172.21.15.55:0/2554280497 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837759e640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8378108650 0x7f83781a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837659c640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8378105d20 0x7f83781990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f8376d9d640 1 -- 172.21.15.55:0/2554280497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8378105d20 msgr2=0x7f83781990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f8376d9d640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8378105d20 0x7f83781990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f8376d9d640 1 -- 172.21.15.55:0/2554280497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8378108650 msgr2=0x7f83781a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f8376d9d640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8378108650 0x7f83781a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f8376d9d640 1 -- 172.21.15.55:0/2554280497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83781a0790 con 0x7f83781033f0 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.020+0000 7f837759e640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8378108650 0x7f83781a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.021+0000 7f8376d9d640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83781033f0 0x7f8378198b70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f836800e9f0 tx=0x7f836800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.021+0000 7f835ffff640 1 -- 172.21.15.55:0/2554280497 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836800cd80 con 0x7f83781033f0 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.021+0000 7f835ffff640 1 -- 172.21.15.55:0/2554280497 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83680054d0 con 0x7f83781033f0 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.021+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83781a0a80 con 0x7f83781033f0 2024-04-26T21:52:55.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.021+0000 7f835ffff640 1 -- 172.21.15.55:0/2554280497 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368010690 con 0x7f83781033f0 2024-04-26T21:52:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.021+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83781a0f80 con 0x7f83781033f0 2024-04-26T21:52:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.022+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83780fe910 con 0x7f83781033f0 2024-04-26T21:52:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.023+0000 7f835ffff640 1 -- 172.21.15.55:0/2554280497 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8368004e90 con 0x7f83781033f0 2024-04-26T21:52:55.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.024+0000 7f835ffff640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f83600a3c40 0x7f83600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.024+0000 7f837659c640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f83600a3c40 0x7f83600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.024+0000 7f835ffff640 1 -- 172.21.15.55:0/2554280497 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f83680d0b20 con 0x7f83781033f0 2024-04-26T21:52:55.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.024+0000 7f837659c640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f83600a3c40 0x7f83600a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f837819a080 tx=0x7f836c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.028+0000 7f835ffff640 1 -- 172.21.15.55:0/2554280497 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f836809d360 con 0x7f83781033f0 2024-04-26T21:52:55.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.213+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f837802dd10 con 0x7f83781033f0 2024-04-26T21:52:55.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.213+0000 7f835ffff640 1 -- 172.21.15.55:0/2554280497 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f83680bbe20 con 0x7f83781033f0 2024-04-26T21:52:55.213 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f83600a3c40 msgr2=0x7f83600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f83600a3c40 0x7f83600a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f837819a080 tx=0x7f836c005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83781033f0 msgr2=0x7f8378198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83781033f0 0x7f8378198b70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f836800e9f0 tx=0x7f836800edf0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 shutdown_connections 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f83600a3c40 0x7f83600a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8378108650 0x7f83781a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8378105d20 0x7f83781990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 --2- 172.21.15.55:0/2554280497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83781033f0 0x7f8378198b70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 >> 172.21.15.55:0/2554280497 conn(0x7f83780fce20 msgr2=0x7f8378103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 shutdown_connections 2024-04-26T21:52:55.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.215+0000 7f837d085640 1 -- 172.21.15.55:0/2554280497 wait complete. 2024-04-26T21:52:55.226 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:52:55.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 -- 172.21.15.55:0/3699203909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 msgr2=0x7f1dcc101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3699203909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 0x7f1dcc101ca0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009900 tx=0x7f1dc402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:55.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 -- 172.21.15.55:0/3699203909 shutdown_connections 2024-04-26T21:52:55.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3699203909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dcc103430 0x7f1dcc109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3699203909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dcc102a70 0x7f1dcc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3699203909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 0x7f1dcc101ca0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 -- 172.21.15.55:0/3699203909 >> 172.21.15.55:0/3699203909 conn(0x7f1dcc0fcdf0 msgr2=0x7f1dcc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 -- 172.21.15.55:0/3699203909 shutdown_connections 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 -- 172.21.15.55:0/3699203909 wait complete. 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 Processor -- start 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.331+0000 7f1dd422a640 1 -- start start 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd422a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 0x7f1dcc06b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd422a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dcc102a70 0x7f1dcc19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd422a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dcc103430 0x7f1dcc19ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd422a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc10d3a0 con 0x7f1dcc101880 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd422a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc10d220 con 0x7f1dcc102a70 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd422a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc10d520 con 0x7f1dcc103430 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 0x7f1dcc06b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 0x7f1dcc06b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45510/0 (socket says 172.21.15.55:45510) 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 -- 172.21.15.55:0/3178767150 learned_addr learned my addr 172.21.15.55:0/3178767150 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd27c7640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dcc103430 0x7f1dcc19ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd17c5640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dcc102a70 0x7f1dcc19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 -- 172.21.15.55:0/3178767150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dcc103430 msgr2=0x7f1dcc19ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dcc103430 0x7f1dcc19ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 -- 172.21.15.55:0/3178767150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dcc102a70 msgr2=0x7f1dcc19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dcc102a70 0x7f1dcc19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 -- 172.21.15.55:0/3178767150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dcc1a0820 con 0x7f1dcc101880 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dd1fc6640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 0x7f1dcc06b720 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4006010 tx=0x7f1dc40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.332+0000 7f1dcaffd640 1 -- 172.21.15.55:0/3178767150 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc403f030 con 0x7f1dcc101880 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.333+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dcc1a0ab0 con 0x7f1dcc101880 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.333+0000 7f1dcaffd640 1 -- 172.21.15.55:0/3178767150 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc40318f0 con 0x7f1dcc101880 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.333+0000 7f1dcaffd640 1 -- 172.21.15.55:0/3178767150 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4035070 con 0x7f1dcc101880 2024-04-26T21:52:55.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.333+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dcc1a0fa0 con 0x7f1dcc101880 2024-04-26T21:52:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.333+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dcc108ce0 con 0x7f1dcc101880 2024-04-26T21:52:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.335+0000 7f1dcaffd640 1 -- 172.21.15.55:0/3178767150 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1dc4042030 con 0x7f1dcc101880 2024-04-26T21:52:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.335+0000 7f1dcaffd640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1db40a3c40 0x7f1db40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.335+0000 7f1dd17c5640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1db40a3c40 0x7f1db40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.335+0000 7f1dcaffd640 1 -- 172.21.15.55:0/3178767150 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1dc40f0ea0 con 0x7f1dcc101880 2024-04-26T21:52:55.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.336+0000 7f1dd17c5640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1db40a3c40 0x7f1db40a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc06b490 tx=0x7f1dc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.339+0000 7f1dcaffd640 1 -- 172.21.15.55:0/3178767150 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1dc40bd090 con 0x7f1dcc101880 2024-04-26T21:52:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.523+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1dcc02dad0 con 0x7f1dcc101880 2024-04-26T21:52:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.524+0000 7f1dcaffd640 1 -- 172.21.15.55:0/3178767150 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f1dc40c1390 con 0x7f1dcc101880 2024-04-26T21:52:55.524 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T21:52:55.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1db40a3c40 msgr2=0x7f1db40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1db40a3c40 0x7f1db40a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc06b490 tx=0x7f1dc0005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:55.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 msgr2=0x7f1dcc06b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 0x7f1dcc06b720 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4006010 tx=0x7f1dc40042f0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 shutdown_connections 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1db40a3c40 0x7f1db40a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dcc103430 0x7f1dcc19ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1dcc102a70 0x7f1dcc19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 --2- 172.21.15.55:0/3178767150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1dcc101880 0x7f1dcc06b720 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 >> 172.21.15.55:0/3178767150 conn(0x7f1dcc0fcdf0 msgr2=0x7f1dcc105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 shutdown_connections 2024-04-26T21:52:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.526+0000 7f1dd422a640 1 -- 172.21.15.55:0/3178767150 wait complete. 2024-04-26T21:52:55.537 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:52:55.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.642+0000 7f46b3339640 1 -- 172.21.15.55:0/1277519655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac108650 msgr2=0x7f46ac10aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.642+0000 7f46b3339640 1 --2- 172.21.15.55:0/1277519655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac108650 0x7f46ac10aaa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009900 tx=0x7f46a402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:55.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.642+0000 7f46b3339640 1 -- 172.21.15.55:0/1277519655 shutdown_connections 2024-04-26T21:52:55.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.642+0000 7f46b3339640 1 --2- 172.21.15.55:0/1277519655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac108650 0x7f46ac10aaa0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.642+0000 7f46b3339640 1 --2- 172.21.15.55:0/1277519655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ac105d20 0x7f46ac108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.642+0000 7f46b3339640 1 --2- 172.21.15.55:0/1277519655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46ac1033f0 0x7f46ac1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.642+0000 7f46b3339640 1 -- 172.21.15.55:0/1277519655 >> 172.21.15.55:0/1277519655 conn(0x7f46ac0fcde0 msgr2=0x7f46ac0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 -- 172.21.15.55:0/1277519655 shutdown_connections 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 -- 172.21.15.55:0/1277519655 wait complete. 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 Processor -- start 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 -- start start 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac1033f0 0x7f46ac102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ac105d20 0x7f46ac1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46ac108650 0x7f46ac1018e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac10e4a0 con 0x7f46ac1033f0 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac10e320 con 0x7f46ac105d20 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b3339640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac10e620 con 0x7f46ac108650 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b10d5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac1033f0 0x7f46ac102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.643+0000 7f46b10d5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac1033f0 0x7f46ac102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45522/0 (socket says 172.21.15.55:45522) 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b10d5640 1 -- 172.21.15.55:0/4183917011 learned_addr learned my addr 172.21.15.55:0/4183917011 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b08d4640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ac105d20 0x7f46ac1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b10d5640 1 -- 172.21.15.55:0/4183917011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46ac108650 msgr2=0x7f46ac1018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b10d5640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46ac108650 0x7f46ac1018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b10d5640 1 -- 172.21.15.55:0/4183917011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ac105d20 msgr2=0x7f46ac1013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b10d5640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ac105d20 0x7f46ac1013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b10d5640 1 -- 172.21.15.55:0/4183917011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac102170 con 0x7f46ac1033f0 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b10d5640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac1033f0 0x7f46ac102d60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f469c00cce0 tx=0x7f469c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46aa7fc640 1 -- 172.21.15.55:0/4183917011 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c004b20 con 0x7f46ac1033f0 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac1a2cf0 con 0x7f46ac1033f0 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46aa7fc640 1 -- 172.21.15.55:0/4183917011 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c004cc0 con 0x7f46ac1033f0 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac1a3180 con 0x7f46ac1033f0 2024-04-26T21:52:55.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.644+0000 7f46aa7fc640 1 -- 172.21.15.55:0/4183917011 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c01d060 con 0x7f46ac1033f0 2024-04-26T21:52:55.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.645+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac0fe8d0 con 0x7f46ac1033f0 2024-04-26T21:52:55.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.647+0000 7f46aa7fc640 1 -- 172.21.15.55:0/4183917011 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f469c005000 con 0x7f46ac1033f0 2024-04-26T21:52:55.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.648+0000 7f46aa7fc640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46940a3c40 0x7f46940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.648+0000 7f46aa7fc640 1 -- 172.21.15.55:0/4183917011 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f469c0d0a30 con 0x7f46ac1033f0 2024-04-26T21:52:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.648+0000 7f46b08d4640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46940a3c40 0x7f46940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.648+0000 7f46b08d4640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46940a3c40 0x7f46940a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46ac102ad0 tx=0x7f46a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.651+0000 7f46aa7fc640 1 -- 172.21.15.55:0/4183917011 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f469c014030 con 0x7f46ac1033f0 2024-04-26T21:52:55.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.842+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f46ac02dd30 con 0x7f46ac1033f0 2024-04-26T21:52:55.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.843+0000 7f46aa7fc640 1 -- 172.21.15.55:0/4183917011 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f469c09d210 con 0x7f46ac1033f0 2024-04-26T21:52:55.843 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T21:52:55.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46940a3c40 msgr2=0x7f46940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46940a3c40 0x7f46940a6100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46ac102ad0 tx=0x7f46a0005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:55.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac1033f0 msgr2=0x7f46ac102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac1033f0 0x7f46ac102d60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f469c00cce0 tx=0x7f469c00a620 comp rx=0 tx=0).stop 2024-04-26T21:52:55.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 shutdown_connections 2024-04-26T21:52:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f46940a3c40 0x7f46940a6100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46ac108650 0x7f46ac1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46ac105d20 0x7f46ac1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 --2- 172.21.15.55:0/4183917011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46ac1033f0 0x7f46ac102d60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 >> 172.21.15.55:0/4183917011 conn(0x7f46ac0fcde0 msgr2=0x7f46ac103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 shutdown_connections 2024-04-26T21:52:55.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.845+0000 7f46b3339640 1 -- 172.21.15.55:0/4183917011 wait complete. 2024-04-26T21:52:55.855 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3385853024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:52:55.855 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:55 smithi055 ceph-mon[29221]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:52:55.855 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1213523313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:55.855 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2554280497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:55.855 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3178767150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:55.856 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:55.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 -- 172.21.15.55:0/1833102642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 msgr2=0x7f3f34102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1833102642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 0x7f3f34102dc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009900 tx=0x7f3f2c02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:55.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 -- 172.21.15.55:0/1833102642 shutdown_connections 2024-04-26T21:52:55.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1833102642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f34104550 0x7f3f3410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1833102642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34103b90 0x7f3f34104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1833102642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 0x7f3f34102dc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 -- 172.21.15.55:0/1833102642 >> 172.21.15.55:0/1833102642 conn(0x7f3f340fdf30 msgr2=0x7f3f34100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 -- 172.21.15.55:0/1833102642 shutdown_connections 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.958+0000 7f3f38da4640 1 -- 172.21.15.55:0/1833102642 wait complete. 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f38da4640 1 Processor -- start 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f38da4640 1 -- start start 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f38da4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 0x7f3f34198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f38da4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f34103b90 0x7f3f341991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f38da4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34104550 0x7f3f341a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f38da4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410e520 con 0x7f3f341029a0 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f38da4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410e3a0 con 0x7f3f34104550 2024-04-26T21:52:55.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f38da4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410e6a0 con 0x7f3f34103b90 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 0x7f3f34198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 0x7f3f34198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45530/0 (socket says 172.21.15.55:45530) 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 -- 172.21.15.55:0/1322905615 learned_addr learned my addr 172.21.15.55:0/1322905615 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f32d9d640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34104550 0x7f3f341a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f31d9b640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f34103b90 0x7f3f341991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 -- 172.21.15.55:0/1322905615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f34103b90 msgr2=0x7f3f341991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f34103b90 0x7f3f341991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 -- 172.21.15.55:0/1322905615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34104550 msgr2=0x7f3f341a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34104550 0x7f3f341a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 -- 172.21.15.55:0/1322905615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f341a08b0 con 0x7f3f341029a0 2024-04-26T21:52:55.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.959+0000 7f3f3259c640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 0x7f3f34198c70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c006010 tx=0x7f3f2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.960+0000 7f3f2b7fe640 1 -- 172.21.15.55:0/1322905615 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c03f030 con 0x7f3f341029a0 2024-04-26T21:52:55.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.960+0000 7f3f2b7fe640 1 -- 172.21.15.55:0/1322905615 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2c003cb0 con 0x7f3f341029a0 2024-04-26T21:52:55.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.960+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f341a0b40 con 0x7f3f341029a0 2024-04-26T21:52:55.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.960+0000 7f3f2b7fe640 1 -- 172.21.15.55:0/1322905615 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c035070 con 0x7f3f341029a0 2024-04-26T21:52:55.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.960+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f341a1030 con 0x7f3f341029a0 2024-04-26T21:52:55.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.961+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f34109de0 con 0x7f3f341029a0 2024-04-26T21:52:55.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.962+0000 7f3f2b7fe640 1 -- 172.21.15.55:0/1322905615 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3f2c042030 con 0x7f3f341029a0 2024-04-26T21:52:55.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.963+0000 7f3f2b7fe640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f200a3c40 0x7f3f200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:55.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.963+0000 7f3f31d9b640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f200a3c40 0x7f3f200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:55.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.963+0000 7f3f2b7fe640 1 -- 172.21.15.55:0/1322905615 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3f2c0f0ba0 con 0x7f3f341029a0 2024-04-26T21:52:55.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.963+0000 7f3f31d9b640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f200a3c40 0x7f3f200a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f3419a170 tx=0x7f3f1c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:55.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:55.967+0000 7f3f2b7fe640 1 -- 172.21.15.55:0/1322905615 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3f2c0bcd30 con 0x7f3f341029a0 2024-04-26T21:52:56.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:55 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3385853024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:52:56.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:55 smithi165 ceph-mon[33296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:52:56.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:55 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1213523313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:56.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:55 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2554280497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:56.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:55 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3178767150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:56.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.155+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f3402dd20 con 0x7f3f341029a0 2024-04-26T21:52:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.156+0000 7f3f2b7fe640 1 -- 172.21.15.55:0/1322905615 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22760 (secure 0 0 0) 0x7f3f2c0c1030 con 0x7f3f341029a0 2024-04-26T21:52:56.158 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:56.158 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:52:51.546766+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:56.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f200a3c40 msgr2=0x7f3f200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f200a3c40 0x7f3f200a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f3419a170 tx=0x7f3f1c00b390 comp rx=0 tx=0).stop 2024-04-26T21:52:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 msgr2=0x7f3f34198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 0x7f3f34198c70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c006010 tx=0x7f3f2c0042f0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 shutdown_connections 2024-04-26T21:52:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3f200a3c40 0x7f3f200a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f34104550 0x7f3f341a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f34103b90 0x7f3f341991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 --2- 172.21.15.55:0/1322905615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f341029a0 0x7f3f34198c70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.158+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 >> 172.21.15.55:0/1322905615 conn(0x7f3f340fdf30 msgr2=0x7f3f34106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.159+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 shutdown_connections 2024-04-26T21:52:56.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.159+0000 7f3f38da4640 1 -- 172.21.15.55:0/1322905615 wait complete. 2024-04-26T21:52:56.170 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:52:56.171 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:52:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:55 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3385853024' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T21:52:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:55 smithi064 ceph-mon[32765]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:52:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:55 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1213523313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:55 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2554280497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:55 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3178767150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.276+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3298176495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 msgr2=0x7fdc34109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.276+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3298176495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 0x7fdc34109cb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009ca0 tx=0x7fdc2c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.276+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3298176495 shutdown_connections 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.276+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3298176495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 0x7fdc34109cb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.276+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3298176495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc34102a60 0x7fdc34102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.276+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3298176495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc34101870 0x7fdc34101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.276+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3298176495 >> 172.21.15.55:0/3298176495 conn(0x7fdc340fcde0 msgr2=0x7fdc340ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3298176495 shutdown_connections 2024-04-26T21:52:56.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3298176495 wait complete. 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 Processor -- start 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 -- start start 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc34101870 0x7fdc34198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc34102a60 0x7fdc34199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 0x7fdc341a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3410d380 con 0x7fdc34103420 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3410d200 con 0x7fdc34101870 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc3aa93640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3410d500 con 0x7fdc34102a60 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc39030640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 0x7fdc341a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc39030640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 0x7fdc341a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45558/0 (socket says 172.21.15.55:45558) 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.277+0000 7fdc39030640 1 -- 172.21.15.55:0/3493802271 learned_addr learned my addr 172.21.15.55:0/3493802271 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc3882f640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc34101870 0x7fdc34198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc39030640 1 -- 172.21.15.55:0/3493802271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc34102a60 msgr2=0x7fdc34199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc39030640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc34102a60 0x7fdc34199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc39030640 1 -- 172.21.15.55:0/3493802271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc34101870 msgr2=0x7fdc34198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc39030640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc34101870 0x7fdc34198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc39030640 1 -- 172.21.15.55:0/3493802271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc341a07f0 con 0x7fdc34103420 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc3882f640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc34101870 0x7fdc34198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc39030640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 0x7fdc341a0180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0097b0 tx=0x7fdc2c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc31ffb640 1 -- 172.21.15.55:0/3493802271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c004240 con 0x7fdc34103420 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc31ffb640 1 -- 172.21.15.55:0/3493802271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc2c0317c0 con 0x7fdc34103420 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc341a0a80 con 0x7fdc34103420 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc31ffb640 1 -- 172.21.15.55:0/3493802271 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c02fd80 con 0x7fdc34103420 2024-04-26T21:52:56.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.278+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc341a1020 con 0x7fdc34103420 2024-04-26T21:52:56.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.279+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc341a1250 con 0x7fdc34103420 2024-04-26T21:52:56.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.280+0000 7fdc31ffb640 1 -- 172.21.15.55:0/3493802271 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdc2c0043e0 con 0x7fdc34103420 2024-04-26T21:52:56.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.281+0000 7fdc31ffb640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdc1c0a3cc0 0x7fdc1c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.281+0000 7fdc31ffb640 1 -- 172.21.15.55:0/3493802271 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fdc2c0f1d80 con 0x7fdc34103420 2024-04-26T21:52:56.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.281+0000 7fdc3882f640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdc1c0a3cc0 0x7fdc1c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.281+0000 7fdc3882f640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdc1c0a3cc0 0x7fdc1c0a6180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc24005d00 tx=0x7fdc24005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.286+0000 7fdc31ffb640 1 -- 172.21.15.55:0/3493802271 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdc2c0be6d0 con 0x7fdc34103420 2024-04-26T21:52:56.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.473+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc341a15e0 con 0x7fdc34103420 2024-04-26T21:52:56.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.473+0000 7fdc31ffb640 1 -- 172.21.15.55:0/3493802271 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fdc2c0098b0 con 0x7fdc34103420 2024-04-26T21:52:56.474 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdc1c0a3cc0 msgr2=0x7fdc1c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdc1c0a3cc0 0x7fdc1c0a6180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc24005d00 tx=0x7fdc24005ef0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 msgr2=0x7fdc341a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 0x7fdc341a0180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0097b0 tx=0x7fdc2c003c20 comp rx=0 tx=0).stop 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 shutdown_connections 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdc1c0a3cc0 0x7fdc1c0a6180 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc34103420 0x7fdc341a0180 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc34102a60 0x7fdc34199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 --2- 172.21.15.55:0/3493802271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc34101870 0x7fdc34198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 >> 172.21.15.55:0/3493802271 conn(0x7fdc340fcde0 msgr2=0x7fdc340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 shutdown_connections 2024-04-26T21:52:56.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.475+0000 7fdc3aa93640 1 -- 172.21.15.55:0/3493802271 wait complete. 2024-04-26T21:52:56.487 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:52:56.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 -- 172.21.15.55:0/769275310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 msgr2=0x7fad28105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/769275310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 0x7fad28105800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009900 tx=0x7fad1c02a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 -- 172.21.15.55:0/769275310 shutdown_connections 2024-04-26T21:52:56.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/769275310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad28108670 0x7fad2810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/769275310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad28105d40 0x7fad28108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/769275310 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 0x7fad28105800 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 -- 172.21.15.55:0/769275310 >> 172.21.15.55:0/769275310 conn(0x7fad280fce20 msgr2=0x7fad280ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 -- 172.21.15.55:0/769275310 shutdown_connections 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.592+0000 7fad2e8ea640 1 -- 172.21.15.55:0/769275310 wait complete. 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad2e8ea640 1 Processor -- start 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad2e8ea640 1 -- start start 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad2e8ea640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 0x7fad28198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad2e8ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad28105d40 0x7fad28199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad2e8ea640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad28108670 0x7fad281a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad2e8ea640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fad2810e4c0 con 0x7fad28103410 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad2e8ea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad2810e340 con 0x7fad28105d40 2024-04-26T21:52:56.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad2e8ea640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fad2810e640 con 0x7fad28108670 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 0x7fad28198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 0x7fad28198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45566/0 (socket says 172.21.15.55:45566) 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.593+0000 7fad27fff640 1 -- 172.21.15.55:0/3989274008 learned_addr learned my addr 172.21.15.55:0/3989274008 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad277fe640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad28105d40 0x7fad28199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad2ce87640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad28108670 0x7fad281a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad27fff640 1 -- 172.21.15.55:0/3989274008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad28108670 msgr2=0x7fad281a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad27fff640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad28108670 0x7fad281a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad27fff640 1 -- 172.21.15.55:0/3989274008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad28105d40 msgr2=0x7fad28199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad27fff640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad28105d40 0x7fad28199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad27fff640 1 -- 172.21.15.55:0/3989274008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad281a07e0 con 0x7fad28103410 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad2ce87640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad28108670 0x7fad281a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad27fff640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 0x7fad28198bc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad1c005940 tx=0x7fad1c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad257fa640 1 -- 172.21.15.55:0/3989274008 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c03f030 con 0x7fad28103410 2024-04-26T21:52:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad257fa640 1 -- 172.21.15.55:0/3989274008 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad1c003cb0 con 0x7fad28103410 2024-04-26T21:52:56.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad281a0a70 con 0x7fad28103410 2024-04-26T21:52:56.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad257fa640 1 -- 172.21.15.55:0/3989274008 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c035070 con 0x7fad28103410 2024-04-26T21:52:56.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.594+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad281a0f60 con 0x7fad28103410 2024-04-26T21:52:56.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.595+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad280fe910 con 0x7fad28103410 2024-04-26T21:52:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.596+0000 7fad257fa640 1 -- 172.21.15.55:0/3989274008 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fad1c042030 con 0x7fad28103410 2024-04-26T21:52:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.597+0000 7fad257fa640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad100a3c40 0x7fad100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.597+0000 7fad257fa640 1 -- 172.21.15.55:0/3989274008 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fad1c0f11d0 con 0x7fad28103410 2024-04-26T21:52:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.597+0000 7fad277fe640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad100a3c40 0x7fad100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.598+0000 7fad277fe640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad100a3c40 0x7fad100a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fad2819a0d0 tx=0x7fad1800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.600+0000 7fad257fa640 1 -- 172.21.15.55:0/3989274008 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fad1c0bda10 con 0x7fad28103410 2024-04-26T21:52:56.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.791+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fad2802dd40 con 0x7fad28103410 2024-04-26T21:52:56.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.794+0000 7fad257fa640 1 -- 172.21.15.55:0/3989274008 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fad1c003e60 con 0x7fad28103410 2024-04-26T21:52:56.795 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T21:52:56.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad100a3c40 msgr2=0x7fad100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad100a3c40 0x7fad100a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fad2819a0d0 tx=0x7fad1800a2d0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 msgr2=0x7fad28198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 0x7fad28198bc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad1c005940 tx=0x7fad1c004430 comp rx=0 tx=0).stop 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 shutdown_connections 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fad100a3c40 0x7fad100a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad28108670 0x7fad281a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad28105d40 0x7fad28199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 --2- 172.21.15.55:0/3989274008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28103410 0x7fad28198bc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 >> 172.21.15.55:0/3989274008 conn(0x7fad280fce20 msgr2=0x7fad28103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 shutdown_connections 2024-04-26T21:52:56.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.797+0000 7fad2e8ea640 1 -- 172.21.15.55:0/3989274008 wait complete. 2024-04-26T21:52:56.808 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:52:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 -- 172.21.15.55:0/1179922515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c1011e0 msgr2=0x7f805c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 --2- 172.21.15.55:0/1179922515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c1011e0 0x7f805c103610 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f804c009ca0 tx=0x7f804c02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 -- 172.21.15.55:0/1179922515 shutdown_connections 2024-04-26T21:52:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 --2- 172.21.15.55:0/1179922515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f805c106500 0x7f805c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 --2- 172.21.15.55:0/1179922515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f805c103b50 0x7f805c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 --2- 172.21.15.55:0/1179922515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c1011e0 0x7f805c103610 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 -- 172.21.15.55:0/1179922515 >> 172.21.15.55:0/1179922515 conn(0x7f805c0fcde0 msgr2=0x7f805c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 -- 172.21.15.55:0/1179922515 shutdown_connections 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.913+0000 7f80639e7640 1 -- 172.21.15.55:0/1179922515 wait complete. 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f80639e7640 1 Processor -- start 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f80639e7640 1 -- start start 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f80639e7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f805c1011e0 0x7f805c19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f80639e7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f805c103b50 0x7f805c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f80639e7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c106500 0x7f805c1a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f80639e7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f805c114ac0 con 0x7f805c106500 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f80639e7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f805c114940 con 0x7f805c1011e0 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f80639e7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f805c114c40 con 0x7f805c103b50 2024-04-26T21:52:56.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f8061f84640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c106500 0x7f805c1a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f8061f84640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c106500 0x7f805c1a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45580/0 (socket says 172.21.15.55:45580) 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f8061f84640 1 -- 172.21.15.55:0/2767848174 learned_addr learned my addr 172.21.15.55:0/2767848174 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f8061783640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f805c1011e0 0x7f805c19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.914+0000 7f8060f82640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f805c103b50 0x7f805c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f8061f84640 1 -- 172.21.15.55:0/2767848174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f805c103b50 msgr2=0x7f805c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f8061f84640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f805c103b50 0x7f805c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f8061f84640 1 -- 172.21.15.55:0/2767848174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f805c1011e0 msgr2=0x7f805c19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f8061f84640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f805c1011e0 0x7f805c19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f8061f84640 1 -- 172.21.15.55:0/2767848174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c1a6e20 con 0x7f805c106500 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f8061f84640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c106500 0x7f805c1a67b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f805400c970 tx=0x7f805400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f805a7fc640 1 -- 172.21.15.55:0/2767848174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054007bb0 con 0x7f805c106500 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f805a7fc640 1 -- 172.21.15.55:0/2767848174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8054007d50 con 0x7f805c106500 2024-04-26T21:52:56.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f805a7fc640 1 -- 172.21.15.55:0/2767848174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054004e60 con 0x7f805c106500 2024-04-26T21:52:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c1a7110 con 0x7f805c106500 2024-04-26T21:52:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.915+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c1a7610 con 0x7f805c106500 2024-04-26T21:52:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.916+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c0fe8d0 con 0x7f805c106500 2024-04-26T21:52:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.917+0000 7f805a7fc640 1 -- 172.21.15.55:0/2767848174 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8054005020 con 0x7f805c106500 2024-04-26T21:52:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.918+0000 7f805a7fc640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80440a3c40 0x7f80440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:56.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.918+0000 7f805a7fc640 1 -- 172.21.15.55:0/2767848174 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f80540d0260 con 0x7f805c106500 2024-04-26T21:52:56.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.918+0000 7f8061783640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80440a3c40 0x7f80440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:56.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.918+0000 7f8061783640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80440a3c40 0x7f80440a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f804c005e30 tx=0x7f804c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:56.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:56.922+0000 7f805a7fc640 1 -- 172.21.15.55:0/2767848174 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8054014030 con 0x7f805c106500 2024-04-26T21:52:57.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:56 smithi165 ceph-mon[33296]: pgmap v352: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-26T21:52:57.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:56 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4183917011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:57.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:56 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1322905615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:57.078 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:56 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3493802271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:57.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.109+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f805c063530 con 0x7f805c106500 2024-04-26T21:52:57.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.110+0000 7f805a7fc640 1 -- 172.21.15.55:0/2767848174 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f805409caa0 con 0x7f805c106500 2024-04-26T21:52:57.110 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T21:52:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.111+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80440a3c40 msgr2=0x7f80440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.111+0000 7f80639e7640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80440a3c40 0x7f80440a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f804c005e30 tx=0x7f804c005dc0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c106500 msgr2=0x7f805c1a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c106500 0x7f805c1a67b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f805400c970 tx=0x7f805400ce30 comp rx=0 tx=0).stop 2024-04-26T21:52:57.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 shutdown_connections 2024-04-26T21:52:57.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80440a3c40 0x7f80440a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f805c106500 0x7f805c1a67b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f805c103b50 0x7f805c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 --2- 172.21.15.55:0/2767848174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f805c1011e0 0x7f805c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 >> 172.21.15.55:0/2767848174 conn(0x7f805c0fcde0 msgr2=0x7f805c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:57.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 shutdown_connections 2024-04-26T21:52:57.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.112+0000 7f80639e7640 1 -- 172.21.15.55:0/2767848174 wait complete. 2024-04-26T21:52:57.122 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:56 smithi055 ceph-mon[29221]: pgmap v352: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-26T21:52:57.122 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:56 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4183917011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:57.122 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:56 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1322905615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:57.122 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:56 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3493802271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:57.123 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-26T21:52:57.123 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:57.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 -- 172.21.15.55:0/4119979775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 msgr2=0x7f8cf0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 --2- 172.21.15.55:0/4119979775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 0x7f8cf0102f00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009900 tx=0x7f8ce002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:57.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 -- 172.21.15.55:0/4119979775 shutdown_connections 2024-04-26T21:52:57.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 --2- 172.21.15.55:0/4119979775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf0103440 0x7f8cf0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 --2- 172.21.15.55:0/4119979775 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 0x7f8cf0102f00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 --2- 172.21.15.55:0/4119979775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0101890 0x7f8cf0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 -- 172.21.15.55:0/4119979775 >> 172.21.15.55:0/4119979775 conn(0x7f8cf00fce20 msgr2=0x7f8cf00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:57.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 -- 172.21.15.55:0/4119979775 shutdown_connections 2024-04-26T21:52:57.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.227+0000 7f8cf6746640 1 -- 172.21.15.55:0/4119979775 wait complete. 2024-04-26T21:52:57.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cf6746640 1 Processor -- start 2024-04-26T21:52:57.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cf6746640 1 -- start start 2024-04-26T21:52:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cf6746640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf0101890 0x7f8cf0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cf6746640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 0x7f8cf0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cf6746640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103440 0x7f8cf019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cf6746640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010d3c0 con 0x7f8cf0102a80 2024-04-26T21:52:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cf6746640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010d240 con 0x7f8cf0101890 2024-04-26T21:52:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cf6746640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf010d540 con 0x7f8cf0103440 2024-04-26T21:52:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cef7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 0x7f8cf0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cef7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 0x7f8cf0198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45598/0 (socket says 172.21.15.55:45598) 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.228+0000 7f8cef7fe640 1 -- 172.21.15.55:0/2702890869 learned_addr learned my addr 172.21.15.55:0/2702890869 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8ceffff640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf0101890 0x7f8cf0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cf4ce3640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103440 0x7f8cf019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cef7fe640 1 -- 172.21.15.55:0/2702890869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103440 msgr2=0x7f8cf019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cef7fe640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103440 0x7f8cf019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cef7fe640 1 -- 172.21.15.55:0/2702890869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf0101890 msgr2=0x7f8cf0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cef7fe640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf0101890 0x7f8cf0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cef7fe640 1 -- 172.21.15.55:0/2702890869 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf019f710 con 0x7f8cf0102a80 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8ceffff640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf0101890 0x7f8cf0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cf4ce3640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103440 0x7f8cf019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cef7fe640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 0x7f8cf0198080 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005940 tx=0x7f8ce0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8ced7fa640 1 -- 172.21.15.55:0/2702890869 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce003f030 con 0x7f8cf0102a80 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf019f9a0 con 0x7f8cf0102a80 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8ced7fa640 1 -- 172.21.15.55:0/2702890869 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ce0003cb0 con 0x7f8cf0102a80 2024-04-26T21:52:57.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8ced7fa640 1 -- 172.21.15.55:0/2702890869 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0035070 con 0x7f8cf0102a80 2024-04-26T21:52:57.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.229+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf019fe90 con 0x7f8cf0102a80 2024-04-26T21:52:57.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.230+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf0108d40 con 0x7f8cf0102a80 2024-04-26T21:52:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.231+0000 7f8ced7fa640 1 -- 172.21.15.55:0/2702890869 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8ce0042030 con 0x7f8cf0102a80 2024-04-26T21:52:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.232+0000 7f8ced7fa640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8cd80a3c40 0x7f8cd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.232+0000 7f8ceffff640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8cd80a3c40 0x7f8cd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.232+0000 7f8ced7fa640 1 -- 172.21.15.55:0/2702890869 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8ce00f1250 con 0x7f8cf0102a80 2024-04-26T21:52:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.232+0000 7f8ceffff640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8cd80a3c40 0x7f8cd80a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8cf01028e0 tx=0x7f8ce4005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:57.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.235+0000 7f8ced7fa640 1 -- 172.21.15.55:0/2702890869 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8ce00bda30 con 0x7f8cf0102a80 2024-04-26T21:52:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:56 smithi064 ceph-mon[32765]: pgmap v352: 97 pgs: 97 active+clean; 629 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-26T21:52:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:56 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4183917011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:56 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1322905615' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:52:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:56 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3493802271' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:52:57.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.466+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cf002dad0 con 0x7f8cf0102a80 2024-04-26T21:52:57.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.467+0000 7f8ced7fa640 1 -- 172.21.15.55:0/2702890869 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11863 (secure 0 0 0) 0x7f8ce0003e60 con 0x7f8cf0102a80 2024-04-26T21:52:57.467 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:57.468 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:52:54.328649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24502,24509]},"id":1}]} 2024-04-26T21:52:57.468 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 21 2024-04-26T21:52:57.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.469+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8cd80a3c40 msgr2=0x7f8cd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.469+0000 7f8cf6746640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8cd80a3c40 0x7f8cd80a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8cf01028e0 tx=0x7f8ce4005e70 comp rx=0 tx=0).stop 2024-04-26T21:52:57.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.469+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 msgr2=0x7f8cf0198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.469+0000 7f8cf6746640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 0x7f8cf0198080 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005940 tx=0x7f8ce0004430 comp rx=0 tx=0).stop 2024-04-26T21:52:57.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.469+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 shutdown_connections 2024-04-26T21:52:57.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.470+0000 7f8cf6746640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8cd80a3c40 0x7f8cd80a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.470+0000 7f8cf6746640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0103440 0x7f8cf019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.470+0000 7f8cf6746640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8cf0102a80 0x7f8cf0198080 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.470+0000 7f8cf6746640 1 --2- 172.21.15.55:0/2702890869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cf0101890 0x7f8cf0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.470+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 >> 172.21.15.55:0/2702890869 conn(0x7f8cf00fce20 msgr2=0x7f8cf0105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.470+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 shutdown_connections 2024-04-26T21:52:57.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.470+0000 7f8cf6746640 1 -- 172.21.15.55:0/2702890869 wait complete. 2024-04-26T21:52:57.481 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-26T21:52:57.482 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-26T21:52:57.482 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 -- 172.21.15.55:0/2917807864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc1029a0 msgr2=0x7f64cc102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 --2- 172.21.15.55:0/2917807864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc1029a0 0x7f64cc102dc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009900 tx=0x7f64b402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 -- 172.21.15.55:0/2917807864 shutdown_connections 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 --2- 172.21.15.55:0/2917807864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64cc104550 0x7f64cc10ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 --2- 172.21.15.55:0/2917807864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64cc103b90 0x7f64cc104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 --2- 172.21.15.55:0/2917807864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc1029a0 0x7f64cc102dc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 -- 172.21.15.55:0/2917807864 >> 172.21.15.55:0/2917807864 conn(0x7f64cc0fdf30 msgr2=0x7f64cc100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 -- 172.21.15.55:0/2917807864 shutdown_connections 2024-04-26T21:52:57.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.583+0000 7f64d22c4640 1 -- 172.21.15.55:0/2917807864 wait complete. 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64d22c4640 1 Processor -- start 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64d22c4640 1 -- start start 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64d22c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64cc1029a0 0x7f64cc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64d22c4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc103b90 0x7f64cc199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64d22c4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64cc104550 0x7f64cc1a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64d22c4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc10e4a0 con 0x7f64cc103b90 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64d22c4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc10e320 con 0x7f64cc1029a0 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64d22c4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64cc10e620 con 0x7f64cc104550 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc103b90 0x7f64cc199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc103b90 0x7f64cc199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45616/0 (socket says 172.21.15.55:45616) 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.584+0000 7f64cb7fe640 1 -- 172.21.15.55:0/1963027107 learned_addr learned my addr 172.21.15.55:0/1963027107 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:57.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64d0861640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64cc104550 0x7f64cc1a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64cb7fe640 1 -- 172.21.15.55:0/1963027107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64cc104550 msgr2=0x7f64cc1a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64cb7fe640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64cc104550 0x7f64cc1a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64cb7fe640 1 -- 172.21.15.55:0/1963027107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64cc1029a0 msgr2=0x7f64cc198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64cbfff640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64cc1029a0 0x7f64cc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64cb7fe640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64cc1029a0 0x7f64cc198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64cb7fe640 1 -- 172.21.15.55:0/1963027107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64cc1a07f0 con 0x7f64cc103b90 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64cb7fe640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc103b90 0x7f64cc199110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00c890 tx=0x7f64bc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64c97fa640 1 -- 172.21.15.55:0/1963027107 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc004220 con 0x7f64cc103b90 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64cc1a0ae0 con 0x7f64cc103b90 2024-04-26T21:52:57.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64cc1a1080 con 0x7f64cc103b90 2024-04-26T21:52:57.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64c97fa640 1 -- 172.21.15.55:0/1963027107 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc0043c0 con 0x7f64cc103b90 2024-04-26T21:52:57.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.585+0000 7f64c97fa640 1 -- 172.21.15.55:0/1963027107 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc01d060 con 0x7f64cc103b90 2024-04-26T21:52:57.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.586+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ac0052c0 con 0x7f64cc103b90 2024-04-26T21:52:57.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.587+0000 7f64c97fa640 1 -- 172.21.15.55:0/1963027107 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f64bc005020 con 0x7f64cc103b90 2024-04-26T21:52:57.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.588+0000 7f64c97fa640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64b80a3c40 0x7f64b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.588+0000 7f64cbfff640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64b80a3c40 0x7f64b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.588+0000 7f64c97fa640 1 -- 172.21.15.55:0/1963027107 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f64bc0d0670 con 0x7f64cc103b90 2024-04-26T21:52:57.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.588+0000 7f64cbfff640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64b80a3c40 0x7f64b80a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f64b402a930 tx=0x7f64b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:57.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.591+0000 7f64c97fa640 1 -- 172.21.15.55:0/1963027107 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f64bc014030 con 0x7f64cc103b90 2024-04-26T21:52:57.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.775+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64ac000bf0 con 0x7f64b80a3c40 2024-04-26T21:52:57.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.782+0000 7f64c97fa640 1 -- 172.21.15.55:0/1963027107 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259015 (secure 0 0 0) 0x7f64ac000bf0 con 0x7f64b80a3c40 2024-04-26T21:52:57.785 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:57.787 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-04-26T21:52:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64b80a3c40 msgr2=0x7f64b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64b80a3c40 0x7f64b80a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f64b402a930 tx=0x7f64b4000bc0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc103b90 msgr2=0x7f64cc199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc103b90 0x7f64cc199110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00c890 tx=0x7f64bc00cd50 comp rx=0 tx=0).stop 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 shutdown_connections 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64b80a3c40 0x7f64b80a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64cc104550 0x7f64cc1a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64cc103b90 0x7f64cc199110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 --2- 172.21.15.55:0/1963027107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64cc1029a0 0x7f64cc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 >> 172.21.15.55:0/1963027107 conn(0x7f64cc0fdf30 msgr2=0x7f64cc106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 shutdown_connections 2024-04-26T21:52:57.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.785+0000 7f64d22c4640 1 -- 172.21.15.55:0/1963027107 wait complete. 2024-04-26T21:52:57.797 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-26T21:52:57.371822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644696,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":785,"num_read_kb":519,"num_write":343,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352688,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589136,"statfs":{"total":1151940427776,"available":1151579275264,"internally_reserved":0,"allocated":12079104,"data_stored":7598310,"data_compressed":250262,"data_compressed_allocated":339968,"data_compressed_original":2179072,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001934"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621845+0000","last_change":"2024-04-26T21:50:57.139486+0000","last_active":"2024-04-26T21:52:51.621845+0000","last_peered":"2024-04-26T21:52:51.621845+0000","last_clean":"2024-04-26T21:52:51.621845+0000","last_became_active":"2024-04-26T21:50:57.139133+0000","last_became_peered":"2024-04-26T21:50:57.139133+0000","last_unstale":"2024-04-26T21:52:51.621845+0000","last_undegraded":"2024-04-26T21:52:51.621845+0000","last_fullsized":"2024-04-26T21:52:51.621845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:33.211627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422137+0000","last_change":"2024-04-26T21:50:57.138519+0000","last_active":"2024-04-26T21:52:53.422137+0000","last_peered":"2024-04-26T21:52:53.422137+0000","last_clean":"2024-04-26T21:52:53.422137+0000","last_became_active":"2024-04-26T21:50:57.138131+0000","last_became_peered":"2024-04-26T21:50:57.138131+0000","last_unstale":"2024-04-26T21:52:53.422137+0000","last_undegraded":"2024-04-26T21:52:53.422137+0000","last_fullsized":"2024-04-26T21:52:53.422137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:34.349694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422059+0000","last_change":"2024-04-26T21:50:57.146584+0000","last_active":"2024-04-26T21:52:53.422059+0000","last_peered":"2024-04-26T21:52:53.422059+0000","last_clean":"2024-04-26T21:52:53.422059+0000","last_became_active":"2024-04-26T21:50:57.146262+0000","last_became_peered":"2024-04-26T21:50:57.146262+0000","last_unstale":"2024-04-26T21:52:53.422059+0000","last_undegraded":"2024-04-26T21:52:53.422059+0000","last_fullsized":"2024-04-26T21:52:53.422059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:49.946882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556609+0000","last_change":"2024-04-26T21:51:50.546008+0000","last_active":"2024-04-26T21:52:51.556609+0000","last_peered":"2024-04-26T21:52:51.556609+0000","last_clean":"2024-04-26T21:52:51.556609+0000","last_became_active":"2024-04-26T21:51:50.545739+0000","last_became_peered":"2024-04-26T21:51:50.545739+0000","last_unstale":"2024-04-26T21:52:51.556609+0000","last_undegraded":"2024-04-26T21:52:51.556609+0000","last_fullsized":"2024-04-26T21:52:51.556609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.494950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422393+0000","last_change":"2024-04-26T21:50:57.136292+0000","last_active":"2024-04-26T21:52:53.422393+0000","last_peered":"2024-04-26T21:52:53.422393+0000","last_clean":"2024-04-26T21:52:53.422393+0000","last_became_active":"2024-04-26T21:50:57.135936+0000","last_became_peered":"2024-04-26T21:50:57.135936+0000","last_unstale":"2024-04-26T21:52:53.422393+0000","last_undegraded":"2024-04-26T21:52:53.422393+0000","last_fullsized":"2024-04-26T21:52:53.422393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:35.415440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554212+0000","last_change":"2024-04-26T21:50:57.147764+0000","last_active":"2024-04-26T21:52:51.554212+0000","last_peered":"2024-04-26T21:52:51.554212+0000","last_clean":"2024-04-26T21:52:51.554212+0000","last_became_active":"2024-04-26T21:50:57.147034+0000","last_became_peered":"2024-04-26T21:50:57.147034+0000","last_unstale":"2024-04-26T21:52:51.554212+0000","last_undegraded":"2024-04-26T21:52:51.554212+0000","last_fullsized":"2024-04-26T21:52:51.554212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:44.833213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422335+0000","last_change":"2024-04-26T21:50:57.144813+0000","last_active":"2024-04-26T21:52:53.422335+0000","last_peered":"2024-04-26T21:52:53.422335+0000","last_clean":"2024-04-26T21:52:53.422335+0000","last_became_active":"2024-04-26T21:50:57.144442+0000","last_became_peered":"2024-04-26T21:50:57.144442+0000","last_unstale":"2024-04-26T21:52:53.422335+0000","last_undegraded":"2024-04-26T21:52:53.422335+0000","last_fullsized":"2024-04-26T21:52:53.422335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:59.032263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556284+0000","last_change":"2024-04-26T21:51:50.547307+0000","last_active":"2024-04-26T21:52:51.556284+0000","last_peered":"2024-04-26T21:52:51.556284+0000","last_clean":"2024-04-26T21:52:51.556284+0000","last_became_active":"2024-04-26T21:51:50.546888+0000","last_became_peered":"2024-04-26T21:51:50.546888+0000","last_unstale":"2024-04-26T21:52:51.556284+0000","last_undegraded":"2024-04-26T21:52:51.556284+0000","last_fullsized":"2024-04-26T21:52:51.556284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:36.137772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093333+0000","last_change":"2024-04-26T21:50:57.132957+0000","last_active":"2024-04-26T21:52:52.093333+0000","last_peered":"2024-04-26T21:52:52.093333+0000","last_clean":"2024-04-26T21:52:52.093333+0000","last_became_active":"2024-04-26T21:50:57.132606+0000","last_became_peered":"2024-04-26T21:50:57.132606+0000","last_unstale":"2024-04-26T21:52:52.093333+0000","last_undegraded":"2024-04-26T21:52:52.093333+0000","last_fullsized":"2024-04-26T21:52:52.093333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:00.994370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554722+0000","last_change":"2024-04-26T21:50:57.150598+0000","last_active":"2024-04-26T21:52:51.554722+0000","last_peered":"2024-04-26T21:52:51.554722+0000","last_clean":"2024-04-26T21:52:51.554722+0000","last_became_active":"2024-04-26T21:50:57.149697+0000","last_became_peered":"2024-04-26T21:50:57.149697+0000","last_unstale":"2024-04-26T21:52:51.554722+0000","last_undegraded":"2024-04-26T21:52:51.554722+0000","last_fullsized":"2024-04-26T21:52:51.554722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.621137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422578+0000","last_change":"2024-04-26T21:50:57.147264+0000","last_active":"2024-04-26T21:52:53.422578+0000","last_peered":"2024-04-26T21:52:53.422578+0000","last_clean":"2024-04-26T21:52:53.422578+0000","last_became_active":"2024-04-26T21:50:57.146888+0000","last_became_peered":"2024-04-26T21:50:57.146888+0000","last_unstale":"2024-04-26T21:52:53.422578+0000","last_undegraded":"2024-04-26T21:52:53.422578+0000","last_fullsized":"2024-04-26T21:52:53.422578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:49.584327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.118994+0000","last_change":"2024-04-26T21:50:57.142005+0000","last_active":"2024-04-26T21:52:52.118994+0000","last_peered":"2024-04-26T21:52:52.118994+0000","last_clean":"2024-04-26T21:52:52.118994+0000","last_became_active":"2024-04-26T21:50:57.141605+0000","last_became_peered":"2024-04-26T21:50:57.141605+0000","last_unstale":"2024-04-26T21:52:52.118994+0000","last_undegraded":"2024-04-26T21:52:52.118994+0000","last_fullsized":"2024-04-26T21:52:52.118994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:43.485295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529452+0000","last_change":"2024-04-26T21:50:57.138049+0000","last_active":"2024-04-26T21:52:52.529452+0000","last_peered":"2024-04-26T21:52:52.529452+0000","last_clean":"2024-04-26T21:52:52.529452+0000","last_became_active":"2024-04-26T21:50:57.137717+0000","last_became_peered":"2024-04-26T21:50:57.137717+0000","last_unstale":"2024-04-26T21:52:52.529452+0000","last_undegraded":"2024-04-26T21:52:52.529452+0000","last_fullsized":"2024-04-26T21:52:52.529452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:28.918224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421875+0000","last_change":"2024-04-26T21:50:57.139183+0000","last_active":"2024-04-26T21:52:53.421875+0000","last_peered":"2024-04-26T21:52:53.421875+0000","last_clean":"2024-04-26T21:52:53.421875+0000","last_became_active":"2024-04-26T21:50:57.138759+0000","last_became_peered":"2024-04-26T21:50:57.138759+0000","last_unstale":"2024-04-26T21:52:53.421875+0000","last_undegraded":"2024-04-26T21:52:53.421875+0000","last_fullsized":"2024-04-26T21:52:53.421875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:12.003122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421842+0000","last_change":"2024-04-26T21:50:57.143110+0000","last_active":"2024-04-26T21:52:53.421842+0000","last_peered":"2024-04-26T21:52:53.421842+0000","last_clean":"2024-04-26T21:52:53.421842+0000","last_became_active":"2024-04-26T21:50:57.142754+0000","last_became_peered":"2024-04-26T21:50:57.142754+0000","last_unstale":"2024-04-26T21:52:53.421842+0000","last_undegraded":"2024-04-26T21:52:53.421842+0000","last_fullsized":"2024-04-26T21:52:53.421842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:08.321000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092719+0000","last_change":"2024-04-26T21:50:57.146514+0000","last_active":"2024-04-26T21:52:52.092719+0000","last_peered":"2024-04-26T21:52:52.092719+0000","last_clean":"2024-04-26T21:52:52.092719+0000","last_became_active":"2024-04-26T21:50:57.146046+0000","last_became_peered":"2024-04-26T21:50:57.146046+0000","last_unstale":"2024-04-26T21:52:52.092719+0000","last_undegraded":"2024-04-26T21:52:52.092719+0000","last_fullsized":"2024-04-26T21:52:52.092719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:21.130220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621947+0000","last_change":"2024-04-26T21:50:57.140492+0000","last_active":"2024-04-26T21:52:51.621947+0000","last_peered":"2024-04-26T21:52:51.621947+0000","last_clean":"2024-04-26T21:52:51.621947+0000","last_became_active":"2024-04-26T21:50:57.140147+0000","last_became_peered":"2024-04-26T21:50:57.140147+0000","last_unstale":"2024-04-26T21:52:51.621947+0000","last_undegraded":"2024-04-26T21:52:51.621947+0000","last_fullsized":"2024-04-26T21:52:51.621947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:21.679202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556637+0000","last_change":"2024-04-26T21:50:57.140992+0000","last_active":"2024-04-26T21:52:51.556637+0000","last_peered":"2024-04-26T21:52:51.556637+0000","last_clean":"2024-04-26T21:52:51.556637+0000","last_became_active":"2024-04-26T21:50:57.140644+0000","last_became_peered":"2024-04-26T21:50:57.140644+0000","last_unstale":"2024-04-26T21:52:51.556637+0000","last_undegraded":"2024-04-26T21:52:51.556637+0000","last_fullsized":"2024-04-26T21:52:51.556637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554651+0000","last_change":"2024-04-26T21:50:57.141242+0000","last_active":"2024-04-26T21:52:51.554651+0000","last_peered":"2024-04-26T21:52:51.554651+0000","last_clean":"2024-04-26T21:52:51.554651+0000","last_became_active":"2024-04-26T21:50:57.140909+0000","last_became_peered":"2024-04-26T21:50:57.140909+0000","last_unstale":"2024-04-26T21:52:51.554651+0000","last_undegraded":"2024-04-26T21:52:51.554651+0000","last_fullsized":"2024-04-26T21:52:51.554651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:21.996984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556570+0000","last_change":"2024-04-26T21:50:56.371876+0000","last_active":"2024-04-26T21:52:51.556570+0000","last_peered":"2024-04-26T21:52:51.556570+0000","last_clean":"2024-04-26T21:52:51.556570+0000","last_became_active":"2024-04-26T21:50:56.371509+0000","last_became_peered":"2024-04-26T21:50:56.371509+0000","last_unstale":"2024-04-26T21:52:51.556570+0000","last_undegraded":"2024-04-26T21:52:51.556570+0000","last_fullsized":"2024-04-26T21:52:51.556570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:50.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400577+0000","last_change":"2024-04-26T21:50:57.145654+0000","last_active":"2024-04-26T21:52:52.400577+0000","last_peered":"2024-04-26T21:52:52.400577+0000","last_clean":"2024-04-26T21:52:52.400577+0000","last_became_active":"2024-04-26T21:50:57.145355+0000","last_became_peered":"2024-04-26T21:50:57.145355+0000","last_unstale":"2024-04-26T21:52:52.400577+0000","last_undegraded":"2024-04-26T21:52:52.400577+0000","last_fullsized":"2024-04-26T21:52:52.400577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:51.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.401003+0000","last_change":"2024-04-26T21:50:56.362758+0000","last_active":"2024-04-26T21:52:52.401003+0000","last_peered":"2024-04-26T21:52:52.401003+0000","last_clean":"2024-04-26T21:52:52.401003+0000","last_became_active":"2024-04-26T21:50:56.361338+0000","last_became_peered":"2024-04-26T21:50:56.361338+0000","last_unstale":"2024-04-26T21:52:52.401003+0000","last_undegraded":"2024-04-26T21:52:52.401003+0000","last_fullsized":"2024-04-26T21:52:52.401003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:44.603426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422204+0000","last_change":"2024-04-26T21:50:57.150701+0000","last_active":"2024-04-26T21:52:53.422204+0000","last_peered":"2024-04-26T21:52:53.422204+0000","last_clean":"2024-04-26T21:52:53.422204+0000","last_became_active":"2024-04-26T21:50:57.150388+0000","last_became_peered":"2024-04-26T21:50:57.150388+0000","last_unstale":"2024-04-26T21:52:53.422204+0000","last_undegraded":"2024-04-26T21:52:53.422204+0000","last_fullsized":"2024-04-26T21:52:53.422204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:43:07.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422749+0000","last_change":"2024-04-26T21:50:56.371379+0000","last_active":"2024-04-26T21:52:53.422749+0000","last_peered":"2024-04-26T21:52:53.422749+0000","last_clean":"2024-04-26T21:52:53.422749+0000","last_became_active":"2024-04-26T21:50:56.371025+0000","last_became_peered":"2024-04-26T21:50:56.371025+0000","last_unstale":"2024-04-26T21:52:53.422749+0000","last_undegraded":"2024-04-26T21:52:53.422749+0000","last_fullsized":"2024-04-26T21:52:53.422749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:00.633393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092905+0000","last_change":"2024-04-26T21:50:57.144777+0000","last_active":"2024-04-26T21:52:52.092905+0000","last_peered":"2024-04-26T21:52:52.092905+0000","last_clean":"2024-04-26T21:52:52.092905+0000","last_became_active":"2024-04-26T21:50:57.144431+0000","last_became_peered":"2024-04-26T21:50:57.144431+0000","last_unstale":"2024-04-26T21:52:52.092905+0000","last_undegraded":"2024-04-26T21:52:52.092905+0000","last_fullsized":"2024-04-26T21:52:52.092905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:42.164240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422585+0000","last_change":"2024-04-26T21:50:56.360843+0000","last_active":"2024-04-26T21:52:53.422585+0000","last_peered":"2024-04-26T21:52:53.422585+0000","last_clean":"2024-04-26T21:52:53.422585+0000","last_became_active":"2024-04-26T21:50:56.360440+0000","last_became_peered":"2024-04-26T21:50:56.360440+0000","last_unstale":"2024-04-26T21:52:53.422585+0000","last_undegraded":"2024-04-26T21:52:53.422585+0000","last_fullsized":"2024-04-26T21:52:53.422585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:47.585306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.120777+0000","last_change":"2024-04-26T21:52:52.120301+0000","last_active":"2024-04-26T21:52:52.120777+0000","last_peered":"2024-04-26T21:52:52.120777+0000","last_clean":"2024-04-26T21:52:52.120777+0000","last_became_active":"2024-04-26T21:52:52.119782+0000","last_became_peered":"2024-04-26T21:52:52.119782+0000","last_unstale":"2024-04-26T21:52:52.120777+0000","last_undegraded":"2024-04-26T21:52:52.120777+0000","last_fullsized":"2024-04-26T21:52:52.120777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:18.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.643628+0000","last_change":"2024-04-26T21:51:50.551697+0000","last_active":"2024-04-26T21:52:54.643628+0000","last_peered":"2024-04-26T21:52:54.643628+0000","last_clean":"2024-04-26T21:52:54.643628+0000","last_became_active":"2024-04-26T21:51:50.546274+0000","last_became_peered":"2024-04-26T21:51:50.546274+0000","last_unstale":"2024-04-26T21:52:54.643628+0000","last_undegraded":"2024-04-26T21:52:54.643628+0000","last_fullsized":"2024-04-26T21:52:54.643628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:44.549374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":1,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422319+0000","last_change":"2024-04-26T21:50:57.138817+0000","last_active":"2024-04-26T21:52:53.422319+0000","last_peered":"2024-04-26T21:52:53.422319+0000","last_clean":"2024-04-26T21:52:53.422319+0000","last_became_active":"2024-04-26T21:50:57.138489+0000","last_became_peered":"2024-04-26T21:50:57.138489+0000","last_unstale":"2024-04-26T21:52:53.422319+0000","last_undegraded":"2024-04-26T21:52:53.422319+0000","last_fullsized":"2024-04-26T21:52:53.422319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:56.072063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554203+0000","last_change":"2024-04-26T21:51:50.941723+0000","last_active":"2024-04-26T21:52:51.554203+0000","last_peered":"2024-04-26T21:52:51.554203+0000","last_clean":"2024-04-26T21:52:51.554203+0000","last_became_active":"2024-04-26T21:51:50.938689+0000","last_became_peered":"2024-04-26T21:51:50.938689+0000","last_unstale":"2024-04-26T21:52:51.554203+0000","last_undegraded":"2024-04-26T21:52:51.554203+0000","last_fullsized":"2024-04-26T21:52:51.554203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:12.119245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587067+0000","last_change":"2024-04-26T21:51:50.595771+0000","last_active":"2024-04-26T21:52:52.587067+0000","last_peered":"2024-04-26T21:52:52.587067+0000","last_clean":"2024-04-26T21:52:52.587067+0000","last_became_active":"2024-04-26T21:51:50.595441+0000","last_became_peered":"2024-04-26T21:51:50.595441+0000","last_unstale":"2024-04-26T21:52:52.587067+0000","last_undegraded":"2024-04-26T21:52:52.587067+0000","last_fullsized":"2024-04-26T21:52:52.587067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:53.753035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529164+0000","last_change":"2024-04-26T21:50:56.362272+0000","last_active":"2024-04-26T21:52:52.529164+0000","last_peered":"2024-04-26T21:52:52.529164+0000","last_clean":"2024-04-26T21:52:52.529164+0000","last_became_active":"2024-04-26T21:50:56.361941+0000","last_became_peered":"2024-04-26T21:50:56.361941+0000","last_unstale":"2024-04-26T21:52:52.529164+0000","last_undegraded":"2024-04-26T21:52:52.529164+0000","last_fullsized":"2024-04-26T21:52:52.529164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400439+0000","last_change":"2024-04-26T21:50:57.135566+0000","last_active":"2024-04-26T21:52:52.400439+0000","last_peered":"2024-04-26T21:52:52.400439+0000","last_clean":"2024-04-26T21:52:52.400439+0000","last_became_active":"2024-04-26T21:50:57.135231+0000","last_became_peered":"2024-04-26T21:50:57.135231+0000","last_unstale":"2024-04-26T21:52:52.400439+0000","last_undegraded":"2024-04-26T21:52:52.400439+0000","last_fullsized":"2024-04-26T21:52:52.400439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:09.381936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.586871+0000","last_change":"2024-04-26T21:50:56.372420+0000","last_active":"2024-04-26T21:52:52.586871+0000","last_peered":"2024-04-26T21:52:52.586871+0000","last_clean":"2024-04-26T21:52:52.586871+0000","last_became_active":"2024-04-26T21:50:56.372096+0000","last_became_peered":"2024-04-26T21:50:56.372096+0000","last_unstale":"2024-04-26T21:52:52.586871+0000","last_undegraded":"2024-04-26T21:52:52.586871+0000","last_fullsized":"2024-04-26T21:52:52.586871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:03.515804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556475+0000","last_change":"2024-04-26T21:50:57.140071+0000","last_active":"2024-04-26T21:52:51.556475+0000","last_peered":"2024-04-26T21:52:51.556475+0000","last_clean":"2024-04-26T21:52:51.556475+0000","last_became_active":"2024-04-26T21:50:57.139693+0000","last_became_peered":"2024-04-26T21:50:57.139693+0000","last_unstale":"2024-04-26T21:52:51.556475+0000","last_undegraded":"2024-04-26T21:52:51.556475+0000","last_fullsized":"2024-04-26T21:52:51.556475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:17.963966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":140,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.609221+0000","last_change":"2024-04-26T21:50:56.363748+0000","last_active":"2024-04-26T21:52:53.609221+0000","last_peered":"2024-04-26T21:52:53.609221+0000","last_clean":"2024-04-26T21:52:53.609221+0000","last_became_active":"2024-04-26T21:50:56.363328+0000","last_became_peered":"2024-04-26T21:50:56.363328+0000","last_unstale":"2024-04-26T21:52:53.609221+0000","last_undegraded":"2024-04-26T21:52:53.609221+0000","last_fullsized":"2024-04-26T21:52:53.609221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:24.450981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093241+0000","last_change":"2024-04-26T21:50:57.143064+0000","last_active":"2024-04-26T21:52:52.093241+0000","last_peered":"2024-04-26T21:52:52.093241+0000","last_clean":"2024-04-26T21:52:52.093241+0000","last_became_active":"2024-04-26T21:50:57.142715+0000","last_became_peered":"2024-04-26T21:50:57.142715+0000","last_unstale":"2024-04-26T21:52:52.093241+0000","last_undegraded":"2024-04-26T21:52:52.093241+0000","last_fullsized":"2024-04-26T21:52:52.093241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:35.901855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422759+0000","last_change":"2024-04-26T21:50:56.369898+0000","last_active":"2024-04-26T21:52:53.422759+0000","last_peered":"2024-04-26T21:52:53.422759+0000","last_clean":"2024-04-26T21:52:53.422759+0000","last_became_active":"2024-04-26T21:50:56.369349+0000","last_became_peered":"2024-04-26T21:50:56.369349+0000","last_unstale":"2024-04-26T21:52:53.422759+0000","last_undegraded":"2024-04-26T21:52:53.422759+0000","last_fullsized":"2024-04-26T21:52:53.422759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:41.378193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422210+0000","last_change":"2024-04-26T21:50:57.141417+0000","last_active":"2024-04-26T21:52:53.422210+0000","last_peered":"2024-04-26T21:52:53.422210+0000","last_clean":"2024-04-26T21:52:53.422210+0000","last_became_active":"2024-04-26T21:50:57.141091+0000","last_became_peered":"2024-04-26T21:50:57.141091+0000","last_unstale":"2024-04-26T21:52:53.422210+0000","last_undegraded":"2024-04-26T21:52:53.422210+0000","last_fullsized":"2024-04-26T21:52:53.422210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:45.357827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587434+0000","last_change":"2024-04-26T21:50:56.364439+0000","last_active":"2024-04-26T21:52:52.587434+0000","last_peered":"2024-04-26T21:52:52.587434+0000","last_clean":"2024-04-26T21:52:52.587434+0000","last_became_active":"2024-04-26T21:50:56.364090+0000","last_became_peered":"2024-04-26T21:50:56.364090+0000","last_unstale":"2024-04-26T21:52:52.587434+0000","last_undegraded":"2024-04-26T21:52:52.587434+0000","last_fullsized":"2024-04-26T21:52:52.587434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:43.562189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093083+0000","last_change":"2024-04-26T21:50:57.145976+0000","last_active":"2024-04-26T21:52:52.093083+0000","last_peered":"2024-04-26T21:52:52.093083+0000","last_clean":"2024-04-26T21:52:52.093083+0000","last_became_active":"2024-04-26T21:50:57.145620+0000","last_became_peered":"2024-04-26T21:50:57.145620+0000","last_unstale":"2024-04-26T21:52:52.093083+0000","last_undegraded":"2024-04-26T21:52:52.093083+0000","last_fullsized":"2024-04-26T21:52:52.093083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:26.351362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'27","reported_seq":283,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.640672+0000","last_change":"2024-04-26T21:50:56.365786+0000","last_active":"2024-04-26T21:52:53.640672+0000","last_peered":"2024-04-26T21:52:53.640672+0000","last_clean":"2024-04-26T21:52:53.640672+0000","last_became_active":"2024-04-26T21:50:56.365410+0000","last_became_peered":"2024-04-26T21:50:56.365410+0000","last_unstale":"2024-04-26T21:52:53.640672+0000","last_undegraded":"2024-04-26T21:52:53.640672+0000","last_fullsized":"2024-04-26T21:52:53.640672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:01.321661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49785,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":233,"num_read_kb":164,"num_write":28,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422593+0000","last_change":"2024-04-26T21:50:57.137333+0000","last_active":"2024-04-26T21:52:53.422593+0000","last_peered":"2024-04-26T21:52:53.422593+0000","last_clean":"2024-04-26T21:52:53.422593+0000","last_became_active":"2024-04-26T21:50:57.136983+0000","last_became_peered":"2024-04-26T21:50:57.136983+0000","last_unstale":"2024-04-26T21:52:53.422593+0000","last_undegraded":"2024-04-26T21:52:53.422593+0000","last_fullsized":"2024-04-26T21:52:53.422593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:18.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554581+0000","last_change":"2024-04-26T21:50:56.358340+0000","last_active":"2024-04-26T21:52:51.554581+0000","last_peered":"2024-04-26T21:52:51.554581+0000","last_clean":"2024-04-26T21:52:51.554581+0000","last_became_active":"2024-04-26T21:50:56.357972+0000","last_became_peered":"2024-04-26T21:50:56.357972+0000","last_unstale":"2024-04-26T21:52:51.554581+0000","last_undegraded":"2024-04-26T21:52:51.554581+0000","last_fullsized":"2024-04-26T21:52:51.554581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:52.256528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529185+0000","last_change":"2024-04-26T21:50:57.137027+0000","last_active":"2024-04-26T21:52:52.529185+0000","last_peered":"2024-04-26T21:52:52.529185+0000","last_clean":"2024-04-26T21:52:52.529185+0000","last_became_active":"2024-04-26T21:50:57.136400+0000","last_became_peered":"2024-04-26T21:50:57.136400+0000","last_unstale":"2024-04-26T21:52:52.529185+0000","last_undegraded":"2024-04-26T21:52:52.529185+0000","last_fullsized":"2024-04-26T21:52:52.529185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:45.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587015+0000","last_change":"2024-04-26T21:50:56.369784+0000","last_active":"2024-04-26T21:52:52.587015+0000","last_peered":"2024-04-26T21:52:52.587015+0000","last_clean":"2024-04-26T21:52:52.587015+0000","last_became_active":"2024-04-26T21:50:56.369093+0000","last_became_peered":"2024-04-26T21:50:56.369093+0000","last_unstale":"2024-04-26T21:52:52.587015+0000","last_undegraded":"2024-04-26T21:52:52.587015+0000","last_fullsized":"2024-04-26T21:52:52.587015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:27.595146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422543+0000","last_change":"2024-04-26T21:50:57.141404+0000","last_active":"2024-04-26T21:52:53.422543+0000","last_peered":"2024-04-26T21:52:53.422543+0000","last_clean":"2024-04-26T21:52:53.422543+0000","last_became_active":"2024-04-26T21:50:57.140973+0000","last_became_peered":"2024-04-26T21:50:57.140973+0000","last_unstale":"2024-04-26T21:52:53.422543+0000","last_undegraded":"2024-04-26T21:52:53.422543+0000","last_fullsized":"2024-04-26T21:52:53.422543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:09.405982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.532387+0000","last_change":"2024-04-26T21:50:56.370727+0000","last_active":"2024-04-26T21:52:53.532387+0000","last_peered":"2024-04-26T21:52:53.532387+0000","last_clean":"2024-04-26T21:52:53.532387+0000","last_became_active":"2024-04-26T21:50:56.370373+0000","last_became_peered":"2024-04-26T21:50:56.370373+0000","last_unstale":"2024-04-26T21:52:53.532387+0000","last_undegraded":"2024-04-26T21:52:53.532387+0000","last_fullsized":"2024-04-26T21:52:53.532387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:12.689699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422251+0000","last_change":"2024-04-26T21:50:57.145771+0000","last_active":"2024-04-26T21:52:53.422251+0000","last_peered":"2024-04-26T21:52:53.422251+0000","last_clean":"2024-04-26T21:52:53.422251+0000","last_became_active":"2024-04-26T21:50:57.145424+0000","last_became_peered":"2024-04-26T21:50:57.145424+0000","last_unstale":"2024-04-26T21:52:53.422251+0000","last_undegraded":"2024-04-26T21:52:53.422251+0000","last_fullsized":"2024-04-26T21:52:53.422251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:10.485045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422292+0000","last_change":"2024-04-26T21:51:50.560040+0000","last_active":"2024-04-26T21:52:53.422292+0000","last_peered":"2024-04-26T21:52:53.422292+0000","last_clean":"2024-04-26T21:52:53.422292+0000","last_became_active":"2024-04-26T21:51:50.546278+0000","last_became_peered":"2024-04-26T21:51:50.546278+0000","last_unstale":"2024-04-26T21:52:53.422292+0000","last_undegraded":"2024-04-26T21:52:53.422292+0000","last_fullsized":"2024-04-26T21:52:53.422292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:59.851719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.586948+0000","last_change":"2024-04-26T21:51:50.938948+0000","last_active":"2024-04-26T21:52:52.586948+0000","last_peered":"2024-04-26T21:52:52.586948+0000","last_clean":"2024-04-26T21:52:52.586948+0000","last_became_active":"2024-04-26T21:51:50.938501+0000","last_became_peered":"2024-04-26T21:51:50.938501+0000","last_unstale":"2024-04-26T21:52:52.586948+0000","last_undegraded":"2024-04-26T21:52:52.586948+0000","last_fullsized":"2024-04-26T21:52:52.586948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:44.936013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422714+0000","last_change":"2024-04-26T21:50:56.358270+0000","last_active":"2024-04-26T21:52:53.422714+0000","last_peered":"2024-04-26T21:52:53.422714+0000","last_clean":"2024-04-26T21:52:53.422714+0000","last_became_active":"2024-04-26T21:50:56.357909+0000","last_became_peered":"2024-04-26T21:50:56.357909+0000","last_unstale":"2024-04-26T21:52:53.422714+0000","last_undegraded":"2024-04-26T21:52:53.422714+0000","last_fullsized":"2024-04-26T21:52:53.422714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:45.627352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621736+0000","last_change":"2024-04-26T21:50:57.142181+0000","last_active":"2024-04-26T21:52:51.621736+0000","last_peered":"2024-04-26T21:52:51.621736+0000","last_clean":"2024-04-26T21:52:51.621736+0000","last_became_active":"2024-04-26T21:50:57.141837+0000","last_became_peered":"2024-04-26T21:50:57.141837+0000","last_unstale":"2024-04-26T21:52:51.621736+0000","last_undegraded":"2024-04-26T21:52:51.621736+0000","last_fullsized":"2024-04-26T21:52:51.621736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:42.150522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556519+0000","last_change":"2024-04-26T21:51:50.547619+0000","last_active":"2024-04-26T21:52:51.556519+0000","last_peered":"2024-04-26T21:52:51.556519+0000","last_clean":"2024-04-26T21:52:51.556519+0000","last_became_active":"2024-04-26T21:51:50.547180+0000","last_became_peered":"2024-04-26T21:51:50.547180+0000","last_unstale":"2024-04-26T21:52:51.556519+0000","last_undegraded":"2024-04-26T21:52:51.556519+0000","last_fullsized":"2024-04-26T21:52:51.556519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:17.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554589+0000","last_change":"2024-04-26T21:50:57.146682+0000","last_active":"2024-04-26T21:52:51.554589+0000","last_peered":"2024-04-26T21:52:51.554589+0000","last_clean":"2024-04-26T21:52:51.554589+0000","last_became_active":"2024-04-26T21:50:57.146329+0000","last_became_peered":"2024-04-26T21:50:57.146329+0000","last_unstale":"2024-04-26T21:52:51.554589+0000","last_undegraded":"2024-04-26T21:52:51.554589+0000","last_fullsized":"2024-04-26T21:52:51.554589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:41.035526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422540+0000","last_change":"2024-04-26T21:50:56.370533+0000","last_active":"2024-04-26T21:52:53.422540+0000","last_peered":"2024-04-26T21:52:53.422540+0000","last_clean":"2024-04-26T21:52:53.422540+0000","last_became_active":"2024-04-26T21:50:56.370029+0000","last_became_peered":"2024-04-26T21:50:56.370029+0000","last_unstale":"2024-04-26T21:52:53.422540+0000","last_undegraded":"2024-04-26T21:52:53.422540+0000","last_fullsized":"2024-04-26T21:52:53.422540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.005405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554152+0000","last_change":"2024-04-26T21:50:57.142420+0000","last_active":"2024-04-26T21:52:51.554152+0000","last_peered":"2024-04-26T21:52:51.554152+0000","last_clean":"2024-04-26T21:52:51.554152+0000","last_became_active":"2024-04-26T21:50:57.142023+0000","last_became_peered":"2024-04-26T21:50:57.142023+0000","last_unstale":"2024-04-26T21:52:51.554152+0000","last_undegraded":"2024-04-26T21:52:51.554152+0000","last_fullsized":"2024-04-26T21:52:51.554152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:31.567136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587065+0000","last_change":"2024-04-26T21:50:57.139924+0000","last_active":"2024-04-26T21:52:52.587065+0000","last_peered":"2024-04-26T21:52:52.587065+0000","last_clean":"2024-04-26T21:52:52.587065+0000","last_became_active":"2024-04-26T21:50:57.139561+0000","last_became_peered":"2024-04-26T21:50:57.139561+0000","last_unstale":"2024-04-26T21:52:52.587065+0000","last_undegraded":"2024-04-26T21:52:52.587065+0000","last_fullsized":"2024-04-26T21:52:52.587065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:52.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.533579+0000","last_change":"2024-04-26T21:50:56.365256+0000","last_active":"2024-04-26T21:52:54.533579+0000","last_peered":"2024-04-26T21:52:54.533579+0000","last_clean":"2024-04-26T21:52:54.533579+0000","last_became_active":"2024-04-26T21:50:56.364516+0000","last_became_peered":"2024-04-26T21:50:56.364516+0000","last_unstale":"2024-04-26T21:52:54.533579+0000","last_undegraded":"2024-04-26T21:52:54.533579+0000","last_fullsized":"2024-04-26T21:52:54.533579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:13.550129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400745+0000","last_change":"2024-04-26T21:50:57.138177+0000","last_active":"2024-04-26T21:52:52.400745+0000","last_peered":"2024-04-26T21:52:52.400745+0000","last_clean":"2024-04-26T21:52:52.400745+0000","last_became_active":"2024-04-26T21:50:57.137725+0000","last_became_peered":"2024-04-26T21:50:57.137725+0000","last_unstale":"2024-04-26T21:52:52.400745+0000","last_undegraded":"2024-04-26T21:52:52.400745+0000","last_fullsized":"2024-04-26T21:52:52.400745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:14.113935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.118868+0000","last_change":"2024-04-26T21:50:56.358280+0000","last_active":"2024-04-26T21:52:52.118868+0000","last_peered":"2024-04-26T21:52:52.118868+0000","last_clean":"2024-04-26T21:52:52.118868+0000","last_became_active":"2024-04-26T21:50:56.357854+0000","last_became_peered":"2024-04-26T21:50:56.357854+0000","last_unstale":"2024-04-26T21:52:52.118868+0000","last_undegraded":"2024-04-26T21:52:52.118868+0000","last_fullsized":"2024-04-26T21:52:52.118868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.499124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422416+0000","last_change":"2024-04-26T21:49:42.292617+0000","last_active":"2024-04-26T21:52:53.422416+0000","last_peered":"2024-04-26T21:52:53.422416+0000","last_clean":"2024-04-26T21:52:53.422416+0000","last_became_active":"2024-04-26T21:49:42.292160+0000","last_became_peered":"2024-04-26T21:49:42.292160+0000","last_unstale":"2024-04-26T21:52:53.422416+0000","last_undegraded":"2024-04-26T21:52:53.422416+0000","last_fullsized":"2024-04-26T21:52:53.422416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_clean_scrub_stamp":"2024-04-26T21:46:13.787665+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:24.663652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529051+0000","last_change":"2024-04-26T21:50:57.145641+0000","last_active":"2024-04-26T21:52:52.529051+0000","last_peered":"2024-04-26T21:52:52.529051+0000","last_clean":"2024-04-26T21:52:52.529051+0000","last_became_active":"2024-04-26T21:50:57.145304+0000","last_became_peered":"2024-04-26T21:50:57.145304+0000","last_unstale":"2024-04-26T21:52:52.529051+0000","last_undegraded":"2024-04-26T21:52:52.529051+0000","last_fullsized":"2024-04-26T21:52:52.529051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:11.389728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422632+0000","last_change":"2024-04-26T21:51:50.549862+0000","last_active":"2024-04-26T21:52:53.422632+0000","last_peered":"2024-04-26T21:52:53.422632+0000","last_clean":"2024-04-26T21:52:53.422632+0000","last_became_active":"2024-04-26T21:51:50.546054+0000","last_became_peered":"2024-04-26T21:51:50.546054+0000","last_unstale":"2024-04-26T21:52:53.422632+0000","last_undegraded":"2024-04-26T21:52:53.422632+0000","last_fullsized":"2024-04-26T21:52:53.422632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:33.817688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422621+0000","last_change":"2024-04-26T21:50:56.364074+0000","last_active":"2024-04-26T21:52:53.422621+0000","last_peered":"2024-04-26T21:52:53.422621+0000","last_clean":"2024-04-26T21:52:53.422621+0000","last_became_active":"2024-04-26T21:50:56.363688+0000","last_became_peered":"2024-04-26T21:50:56.363688+0000","last_unstale":"2024-04-26T21:52:53.422621+0000","last_undegraded":"2024-04-26T21:52:53.422621+0000","last_fullsized":"2024-04-26T21:52:53.422621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:35.526125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529330+0000","last_change":"2024-04-26T21:50:57.136811+0000","last_active":"2024-04-26T21:52:52.529330+0000","last_peered":"2024-04-26T21:52:52.529330+0000","last_clean":"2024-04-26T21:52:52.529330+0000","last_became_active":"2024-04-26T21:50:57.136252+0000","last_became_peered":"2024-04-26T21:50:57.136252+0000","last_unstale":"2024-04-26T21:52:52.529330+0000","last_undegraded":"2024-04-26T21:52:52.529330+0000","last_fullsized":"2024-04-26T21:52:52.529330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:01.061283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556684+0000","last_change":"2024-04-26T21:50:56.364317+0000","last_active":"2024-04-26T21:52:51.556684+0000","last_peered":"2024-04-26T21:52:51.556684+0000","last_clean":"2024-04-26T21:52:51.556684+0000","last_became_active":"2024-04-26T21:50:56.363882+0000","last_became_peered":"2024-04-26T21:50:56.363882+0000","last_unstale":"2024-04-26T21:52:51.556684+0000","last_undegraded":"2024-04-26T21:52:51.556684+0000","last_fullsized":"2024-04-26T21:52:51.556684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:12.994293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.401010+0000","last_change":"2024-04-26T21:50:57.144856+0000","last_active":"2024-04-26T21:52:52.401010+0000","last_peered":"2024-04-26T21:52:52.401010+0000","last_clean":"2024-04-26T21:52:52.401010+0000","last_became_active":"2024-04-26T21:50:57.143717+0000","last_became_peered":"2024-04-26T21:50:57.143717+0000","last_unstale":"2024-04-26T21:52:52.401010+0000","last_undegraded":"2024-04-26T21:52:52.401010+0000","last_fullsized":"2024-04-26T21:52:52.401010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:53.049243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.119199+0000","last_change":"2024-04-26T21:50:56.370284+0000","last_active":"2024-04-26T21:52:52.119199+0000","last_peered":"2024-04-26T21:52:52.119199+0000","last_clean":"2024-04-26T21:52:52.119199+0000","last_became_active":"2024-04-26T21:50:56.369738+0000","last_became_peered":"2024-04-26T21:50:56.369738+0000","last_unstale":"2024-04-26T21:52:52.119199+0000","last_undegraded":"2024-04-26T21:52:52.119199+0000","last_fullsized":"2024-04-26T21:52:52.119199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:23.390553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422256+0000","last_change":"2024-04-26T21:50:57.142687+0000","last_active":"2024-04-26T21:52:53.422256+0000","last_peered":"2024-04-26T21:52:53.422256+0000","last_clean":"2024-04-26T21:52:53.422256+0000","last_became_active":"2024-04-26T21:50:57.142365+0000","last_became_peered":"2024-04-26T21:50:57.142365+0000","last_unstale":"2024-04-26T21:52:53.422256+0000","last_undegraded":"2024-04-26T21:52:53.422256+0000","last_fullsized":"2024-04-26T21:52:53.422256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:14.261741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092789+0000","last_change":"2024-04-26T21:50:56.364163+0000","last_active":"2024-04-26T21:52:52.092789+0000","last_peered":"2024-04-26T21:52:52.092789+0000","last_clean":"2024-04-26T21:52:52.092789+0000","last_became_active":"2024-04-26T21:50:56.363729+0000","last_became_peered":"2024-04-26T21:50:56.363729+0000","last_unstale":"2024-04-26T21:52:52.092789+0000","last_undegraded":"2024-04-26T21:52:52.092789+0000","last_fullsized":"2024-04-26T21:52:52.092789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:54.620884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422328+0000","last_change":"2024-04-26T21:50:57.142692+0000","last_active":"2024-04-26T21:52:53.422328+0000","last_peered":"2024-04-26T21:52:53.422328+0000","last_clean":"2024-04-26T21:52:53.422328+0000","last_became_active":"2024-04-26T21:50:57.142137+0000","last_became_peered":"2024-04-26T21:50:57.142137+0000","last_unstale":"2024-04-26T21:52:53.422328+0000","last_undegraded":"2024-04-26T21:52:53.422328+0000","last_fullsized":"2024-04-26T21:52:53.422328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.996485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400523+0000","last_change":"2024-04-26T21:50:56.366374+0000","last_active":"2024-04-26T21:52:52.400523+0000","last_peered":"2024-04-26T21:52:52.400523+0000","last_clean":"2024-04-26T21:52:52.400523+0000","last_became_active":"2024-04-26T21:50:56.366014+0000","last_became_peered":"2024-04-26T21:50:56.366014+0000","last_unstale":"2024-04-26T21:52:52.400523+0000","last_undegraded":"2024-04-26T21:52:52.400523+0000","last_fullsized":"2024-04-26T21:52:52.400523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:41.867792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621428+0000","last_change":"2024-04-26T21:50:57.142833+0000","last_active":"2024-04-26T21:52:51.621428+0000","last_peered":"2024-04-26T21:52:51.621428+0000","last_clean":"2024-04-26T21:52:51.621428+0000","last_became_active":"2024-04-26T21:50:57.142061+0000","last_became_peered":"2024-04-26T21:50:57.142061+0000","last_unstale":"2024-04-26T21:52:51.621428+0000","last_undegraded":"2024-04-26T21:52:51.621428+0000","last_fullsized":"2024-04-26T21:52:51.621428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:33.023159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554453+0000","last_change":"2024-04-26T21:50:56.363172+0000","last_active":"2024-04-26T21:52:51.554453+0000","last_peered":"2024-04-26T21:52:51.554453+0000","last_clean":"2024-04-26T21:52:51.554453+0000","last_became_active":"2024-04-26T21:50:56.362782+0000","last_became_peered":"2024-04-26T21:50:56.362782+0000","last_unstale":"2024-04-26T21:52:51.554453+0000","last_undegraded":"2024-04-26T21:52:51.554453+0000","last_fullsized":"2024-04-26T21:52:51.554453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:37.183497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400740+0000","last_change":"2024-04-26T21:50:57.142088+0000","last_active":"2024-04-26T21:52:52.400740+0000","last_peered":"2024-04-26T21:52:52.400740+0000","last_clean":"2024-04-26T21:52:52.400740+0000","last_became_active":"2024-04-26T21:50:57.141685+0000","last_became_peered":"2024-04-26T21:50:57.141685+0000","last_unstale":"2024-04-26T21:52:52.400740+0000","last_undegraded":"2024-04-26T21:52:52.400740+0000","last_fullsized":"2024-04-26T21:52:52.400740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:24.033787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422419+0000","last_change":"2024-04-26T21:50:56.371717+0000","last_active":"2024-04-26T21:52:53.422419+0000","last_peered":"2024-04-26T21:52:53.422419+0000","last_clean":"2024-04-26T21:52:53.422419+0000","last_became_active":"2024-04-26T21:50:56.370930+0000","last_became_peered":"2024-04-26T21:50:56.370930+0000","last_unstale":"2024-04-26T21:52:53.422419+0000","last_undegraded":"2024-04-26T21:52:53.422419+0000","last_fullsized":"2024-04-26T21:52:53.422419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.977392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422495+0000","last_change":"2024-04-26T21:50:57.139671+0000","last_active":"2024-04-26T21:52:53.422495+0000","last_peered":"2024-04-26T21:52:53.422495+0000","last_clean":"2024-04-26T21:52:53.422495+0000","last_became_active":"2024-04-26T21:50:57.139245+0000","last_became_peered":"2024-04-26T21:50:57.139245+0000","last_unstale":"2024-04-26T21:52:53.422495+0000","last_undegraded":"2024-04-26T21:52:53.422495+0000","last_fullsized":"2024-04-26T21:52:53.422495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:26.903905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554682+0000","last_change":"2024-04-26T21:50:56.369836+0000","last_active":"2024-04-26T21:52:51.554682+0000","last_peered":"2024-04-26T21:52:51.554682+0000","last_clean":"2024-04-26T21:52:51.554682+0000","last_became_active":"2024-04-26T21:50:56.369522+0000","last_became_peered":"2024-04-26T21:50:56.369522+0000","last_unstale":"2024-04-26T21:52:51.554682+0000","last_undegraded":"2024-04-26T21:52:51.554682+0000","last_fullsized":"2024-04-26T21:52:51.554682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:40.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422787+0000","last_change":"2024-04-26T21:50:57.138551+0000","last_active":"2024-04-26T21:52:53.422787+0000","last_peered":"2024-04-26T21:52:53.422787+0000","last_clean":"2024-04-26T21:52:53.422787+0000","last_became_active":"2024-04-26T21:50:57.138187+0000","last_became_peered":"2024-04-26T21:50:57.138187+0000","last_unstale":"2024-04-26T21:52:53.422787+0000","last_undegraded":"2024-04-26T21:52:53.422787+0000","last_fullsized":"2024-04-26T21:52:53.422787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:46.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587331+0000","last_change":"2024-04-26T21:50:56.362009+0000","last_active":"2024-04-26T21:52:52.587331+0000","last_peered":"2024-04-26T21:52:52.587331+0000","last_clean":"2024-04-26T21:52:52.587331+0000","last_became_active":"2024-04-26T21:50:56.361646+0000","last_became_peered":"2024-04-26T21:50:56.361646+0000","last_unstale":"2024-04-26T21:52:52.587331+0000","last_undegraded":"2024-04-26T21:52:52.587331+0000","last_fullsized":"2024-04-26T21:52:52.587331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:25.316195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422529+0000","last_change":"2024-04-26T21:50:57.139983+0000","last_active":"2024-04-26T21:52:53.422529+0000","last_peered":"2024-04-26T21:52:53.422529+0000","last_clean":"2024-04-26T21:52:53.422529+0000","last_became_active":"2024-04-26T21:50:57.138426+0000","last_became_peered":"2024-04-26T21:50:57.138426+0000","last_unstale":"2024-04-26T21:52:53.422529+0000","last_undegraded":"2024-04-26T21:52:53.422529+0000","last_fullsized":"2024-04-26T21:52:53.422529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:53.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529527+0000","last_change":"2024-04-26T21:50:56.369527+0000","last_active":"2024-04-26T21:52:52.529527+0000","last_peered":"2024-04-26T21:52:52.529527+0000","last_clean":"2024-04-26T21:52:52.529527+0000","last_became_active":"2024-04-26T21:50:56.369168+0000","last_became_peered":"2024-04-26T21:50:56.369168+0000","last_unstale":"2024-04-26T21:52:52.529527+0000","last_undegraded":"2024-04-26T21:52:52.529527+0000","last_fullsized":"2024-04-26T21:52:52.529527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:38.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422889+0000","last_change":"2024-04-26T21:50:57.137957+0000","last_active":"2024-04-26T21:52:53.422889+0000","last_peered":"2024-04-26T21:52:53.422889+0000","last_clean":"2024-04-26T21:52:53.422889+0000","last_became_active":"2024-04-26T21:50:57.137595+0000","last_became_peered":"2024-04-26T21:50:57.137595+0000","last_unstale":"2024-04-26T21:52:53.422889+0000","last_undegraded":"2024-04-26T21:52:53.422889+0000","last_fullsized":"2024-04-26T21:52:53.422889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:09.355937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621530+0000","last_change":"2024-04-26T21:50:57.143098+0000","last_active":"2024-04-26T21:52:51.621530+0000","last_peered":"2024-04-26T21:52:51.621530+0000","last_clean":"2024-04-26T21:52:51.621530+0000","last_became_active":"2024-04-26T21:50:57.142524+0000","last_became_peered":"2024-04-26T21:50:57.142524+0000","last_unstale":"2024-04-26T21:52:51.621530+0000","last_undegraded":"2024-04-26T21:52:51.621530+0000","last_fullsized":"2024-04-26T21:52:51.621530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:33.333382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554379+0000","last_change":"2024-04-26T21:50:57.143123+0000","last_active":"2024-04-26T21:52:51.554379+0000","last_peered":"2024-04-26T21:52:51.554379+0000","last_clean":"2024-04-26T21:52:51.554379+0000","last_became_active":"2024-04-26T21:50:57.142805+0000","last_became_peered":"2024-04-26T21:50:57.142805+0000","last_unstale":"2024-04-26T21:52:51.554379+0000","last_undegraded":"2024-04-26T21:52:51.554379+0000","last_fullsized":"2024-04-26T21:52:51.554379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:19.330202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554320+0000","last_change":"2024-04-26T21:50:57.140268+0000","last_active":"2024-04-26T21:52:51.554320+0000","last_peered":"2024-04-26T21:52:51.554320+0000","last_clean":"2024-04-26T21:52:51.554320+0000","last_became_active":"2024-04-26T21:50:57.139927+0000","last_became_peered":"2024-04-26T21:50:57.139927+0000","last_unstale":"2024-04-26T21:52:51.554320+0000","last_undegraded":"2024-04-26T21:52:51.554320+0000","last_fullsized":"2024-04-26T21:52:51.554320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:01.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421994+0000","last_change":"2024-04-26T21:50:57.143564+0000","last_active":"2024-04-26T21:52:53.421994+0000","last_peered":"2024-04-26T21:52:53.421994+0000","last_clean":"2024-04-26T21:52:53.421994+0000","last_became_active":"2024-04-26T21:50:57.142740+0000","last_became_peered":"2024-04-26T21:50:57.142740+0000","last_unstale":"2024-04-26T21:52:53.421994+0000","last_undegraded":"2024-04-26T21:52:53.421994+0000","last_fullsized":"2024-04-26T21:52:53.421994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:58.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529451+0000","last_change":"2024-04-26T21:50:57.146883+0000","last_active":"2024-04-26T21:52:52.529451+0000","last_peered":"2024-04-26T21:52:52.529451+0000","last_clean":"2024-04-26T21:52:52.529451+0000","last_became_active":"2024-04-26T21:50:57.146540+0000","last_became_peered":"2024-04-26T21:50:57.146540+0000","last_unstale":"2024-04-26T21:52:52.529451+0000","last_undegraded":"2024-04-26T21:52:52.529451+0000","last_fullsized":"2024-04-26T21:52:52.529451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.787618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621730+0000","last_change":"2024-04-26T21:50:57.136938+0000","last_active":"2024-04-26T21:52:51.621730+0000","last_peered":"2024-04-26T21:52:51.621730+0000","last_clean":"2024-04-26T21:52:51.621730+0000","last_became_active":"2024-04-26T21:50:57.136474+0000","last_became_peered":"2024-04-26T21:50:57.136474+0000","last_unstale":"2024-04-26T21:52:51.621730+0000","last_undegraded":"2024-04-26T21:52:51.621730+0000","last_fullsized":"2024-04-26T21:52:51.621730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:42.534342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554864+0000","last_change":"2024-04-26T21:50:57.150958+0000","last_active":"2024-04-26T21:52:51.554864+0000","last_peered":"2024-04-26T21:52:51.554864+0000","last_clean":"2024-04-26T21:52:51.554864+0000","last_became_active":"2024-04-26T21:50:57.150593+0000","last_became_peered":"2024-04-26T21:50:57.150593+0000","last_unstale":"2024-04-26T21:52:51.554864+0000","last_undegraded":"2024-04-26T21:52:51.554864+0000","last_fullsized":"2024-04-26T21:52:51.554864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:26.583983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400991+0000","last_change":"2024-04-26T21:50:57.147766+0000","last_active":"2024-04-26T21:52:52.400991+0000","last_peered":"2024-04-26T21:52:52.400991+0000","last_clean":"2024-04-26T21:52:52.400991+0000","last_became_active":"2024-04-26T21:50:57.147442+0000","last_became_peered":"2024-04-26T21:50:57.147442+0000","last_unstale":"2024-04-26T21:52:52.400991+0000","last_undegraded":"2024-04-26T21:52:52.400991+0000","last_fullsized":"2024-04-26T21:52:52.400991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.489117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422382+0000","last_change":"2024-04-26T21:51:50.595554+0000","last_active":"2024-04-26T21:52:53.422382+0000","last_peered":"2024-04-26T21:52:53.422382+0000","last_clean":"2024-04-26T21:52:53.422382+0000","last_became_active":"2024-04-26T21:51:50.595248+0000","last_became_peered":"2024-04-26T21:51:50.595248+0000","last_unstale":"2024-04-26T21:52:53.422382+0000","last_undegraded":"2024-04-26T21:52:53.422382+0000","last_fullsized":"2024-04-26T21:52:53.422382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:36.853146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400448+0000","last_change":"2024-04-26T21:50:57.142804+0000","last_active":"2024-04-26T21:52:52.400448+0000","last_peered":"2024-04-26T21:52:52.400448+0000","last_clean":"2024-04-26T21:52:52.400448+0000","last_became_active":"2024-04-26T21:50:57.141902+0000","last_became_peered":"2024-04-26T21:50:57.141902+0000","last_unstale":"2024-04-26T21:52:52.400448+0000","last_undegraded":"2024-04-26T21:52:52.400448+0000","last_fullsized":"2024-04-26T21:52:52.400448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:37.590730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093034+0000","last_change":"2024-04-26T21:50:57.138637+0000","last_active":"2024-04-26T21:52:52.093034+0000","last_peered":"2024-04-26T21:52:52.093034+0000","last_clean":"2024-04-26T21:52:52.093034+0000","last_became_active":"2024-04-26T21:50:57.138295+0000","last_became_peered":"2024-04-26T21:50:57.138295+0000","last_unstale":"2024-04-26T21:52:52.093034+0000","last_undegraded":"2024-04-26T21:52:52.093034+0000","last_fullsized":"2024-04-26T21:52:52.093034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:19.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529384+0000","last_change":"2024-04-26T21:50:57.137377+0000","last_active":"2024-04-26T21:52:52.529384+0000","last_peered":"2024-04-26T21:52:52.529384+0000","last_clean":"2024-04-26T21:52:52.529384+0000","last_became_active":"2024-04-26T21:50:57.136893+0000","last_became_peered":"2024-04-26T21:50:57.136893+0000","last_unstale":"2024-04-26T21:52:52.529384+0000","last_undegraded":"2024-04-26T21:52:52.529384+0000","last_fullsized":"2024-04-26T21:52:52.529384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:44.257438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621602+0000","last_change":"2024-04-26T21:50:57.141439+0000","last_active":"2024-04-26T21:52:51.621602+0000","last_peered":"2024-04-26T21:52:51.621602+0000","last_clean":"2024-04-26T21:52:51.621602+0000","last_became_active":"2024-04-26T21:50:57.141060+0000","last_became_peered":"2024-04-26T21:50:57.141060+0000","last_unstale":"2024-04-26T21:52:51.621602+0000","last_undegraded":"2024-04-26T21:52:51.621602+0000","last_fullsized":"2024-04-26T21:52:51.621602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:27.174348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250381,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54094,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":410,"num_write":160,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":161666,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":474342,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1101824,"data_stored":1114761,"data_compressed":49838,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":473648,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":473568,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1065806,"data_compressed":47984,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":473037,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":522307,"data_compressed":13369,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":474308,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":523545,"data_compressed":13369,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":464842,"data_compressed":8628,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":473977,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1064169,"data_compressed":47984,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":49785,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":50715,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51754,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:52:57.801 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 -- 172.21.15.55:0/2295655755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c105d40 msgr2=0x7f153c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 --2- 172.21.15.55:0/2295655755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c105d40 0x7f153c108130 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f152c009900 tx=0x7f152c02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 -- 172.21.15.55:0/2295655755 shutdown_connections 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 --2- 172.21.15.55:0/2295655755 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f153c108670 0x7f153c10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 --2- 172.21.15.55:0/2295655755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c105d40 0x7f153c108130 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 --2- 172.21.15.55:0/2295655755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f153c103410 0x7f153c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 -- 172.21.15.55:0/2295655755 >> 172.21.15.55:0/2295655755 conn(0x7f153c0fce20 msgr2=0x7f153c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 -- 172.21.15.55:0/2295655755 shutdown_connections 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.939+0000 7f15422f9640 1 -- 172.21.15.55:0/2295655755 wait complete. 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f15422f9640 1 Processor -- start 2024-04-26T21:52:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f15422f9640 1 -- start start 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f15422f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f153c103410 0x7f153c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f15422f9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f153c105d40 0x7f153c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f15422f9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c108670 0x7f153c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f15422f9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f153c10e560 con 0x7f153c108670 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f15422f9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f153c10e3e0 con 0x7f153c103410 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f15422f9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f153c10e6e0 con 0x7f153c105d40 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f1540896640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c108670 0x7f153c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f1540896640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c108670 0x7f153c1a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45626/0 (socket says 172.21.15.55:45626) 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f1540896640 1 -- 172.21.15.55:0/1776361435 learned_addr learned my addr 172.21.15.55:0/1776361435 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f153b7fe640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f153c105d40 0x7f153c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.940+0000 7f153bfff640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f153c103410 0x7f153c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f1540896640 1 -- 172.21.15.55:0/1776361435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f153c105d40 msgr2=0x7f153c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f1540896640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f153c105d40 0x7f153c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f1540896640 1 -- 172.21.15.55:0/1776361435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f153c103410 msgr2=0x7f153c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f1540896640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f153c103410 0x7f153c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:57.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f1540896640 1 -- 172.21.15.55:0/1776361435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c1a0890 con 0x7f153c108670 2024-04-26T21:52:57.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f1540896640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c108670 0x7f153c1a01f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f153400b970 tx=0x7f153400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:57.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f15397fa640 1 -- 172.21.15.55:0/1776361435 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534019070 con 0x7f153c108670 2024-04-26T21:52:57.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c1a0b80 con 0x7f153c108670 2024-04-26T21:52:57.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f15397fa640 1 -- 172.21.15.55:0/1776361435 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1534014440 con 0x7f153c108670 2024-04-26T21:52:57.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f15397fa640 1 -- 172.21.15.55:0/1776361435 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15340075d0 con 0x7f153c108670 2024-04-26T21:52:57.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.941+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c1a1080 con 0x7f153c108670 2024-04-26T21:52:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.942+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c0fe910 con 0x7f153c108670 2024-04-26T21:52:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.943+0000 7f15397fa640 1 -- 172.21.15.55:0/1776361435 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f153401c070 con 0x7f153c108670 2024-04-26T21:52:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.944+0000 7f15397fa640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15280a3c40 0x7f15280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.944+0000 7f153bfff640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15280a3c40 0x7f15280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.944+0000 7f15397fa640 1 -- 172.21.15.55:0/1776361435 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f15340cfaf0 con 0x7f153c108670 2024-04-26T21:52:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.945+0000 7f153bfff640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15280a3c40 0x7f15280a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1524005fd0 tx=0x7f1524005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:57.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:57.949+0000 7f15397fa640 1 -- 172.21.15.55:0/1776361435 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f153409bdf0 con 0x7f153c108670 2024-04-26T21:52:58.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:57 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3989274008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:58.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:57 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2767848174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:58.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:57 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2702890869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:58.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.132+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f153c062d70 con 0x7f15280a3c40 2024-04-26T21:52:58.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.139+0000 7f15397fa640 1 -- 172.21.15.55:0/1776361435 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259015 (secure 0 0 0) 0x7f153c062d70 con 0x7f15280a3c40 2024-04-26T21:52:58.142 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:58.144 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-04-26T21:52:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15280a3c40 msgr2=0x7f15280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15280a3c40 0x7f15280a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1524005fd0 tx=0x7f1524005850 comp rx=0 tx=0).stop 2024-04-26T21:52:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c108670 msgr2=0x7f153c1a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c108670 0x7f153c1a01f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f153400b970 tx=0x7f153400be30 comp rx=0 tx=0).stop 2024-04-26T21:52:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 shutdown_connections 2024-04-26T21:52:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15280a3c40 0x7f15280a6100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f153c108670 0x7f153c1a01f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f153c105d40 0x7f153c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 --2- 172.21.15.55:0/1776361435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f153c103410 0x7f153c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 >> 172.21.15.55:0/1776361435 conn(0x7f153c0fce20 msgr2=0x7f153c103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 shutdown_connections 2024-04-26T21:52:58.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.142+0000 7f15422f9640 1 -- 172.21.15.55:0/1776361435 wait complete. 2024-04-26T21:52:58.153 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:57 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3989274008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:58.153 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:57 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2767848174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:58.153 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:57 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2702890869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:58.154 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-26T21:52:57.371822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644696,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":785,"num_read_kb":519,"num_write":343,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352688,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589136,"statfs":{"total":1151940427776,"available":1151579275264,"internally_reserved":0,"allocated":12079104,"data_stored":7598310,"data_compressed":250262,"data_compressed_allocated":339968,"data_compressed_original":2179072,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001934"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621845+0000","last_change":"2024-04-26T21:50:57.139486+0000","last_active":"2024-04-26T21:52:51.621845+0000","last_peered":"2024-04-26T21:52:51.621845+0000","last_clean":"2024-04-26T21:52:51.621845+0000","last_became_active":"2024-04-26T21:50:57.139133+0000","last_became_peered":"2024-04-26T21:50:57.139133+0000","last_unstale":"2024-04-26T21:52:51.621845+0000","last_undegraded":"2024-04-26T21:52:51.621845+0000","last_fullsized":"2024-04-26T21:52:51.621845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:33.211627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422137+0000","last_change":"2024-04-26T21:50:57.138519+0000","last_active":"2024-04-26T21:52:53.422137+0000","last_peered":"2024-04-26T21:52:53.422137+0000","last_clean":"2024-04-26T21:52:53.422137+0000","last_became_active":"2024-04-26T21:50:57.138131+0000","last_became_peered":"2024-04-26T21:50:57.138131+0000","last_unstale":"2024-04-26T21:52:53.422137+0000","last_undegraded":"2024-04-26T21:52:53.422137+0000","last_fullsized":"2024-04-26T21:52:53.422137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:34.349694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422059+0000","last_change":"2024-04-26T21:50:57.146584+0000","last_active":"2024-04-26T21:52:53.422059+0000","last_peered":"2024-04-26T21:52:53.422059+0000","last_clean":"2024-04-26T21:52:53.422059+0000","last_became_active":"2024-04-26T21:50:57.146262+0000","last_became_peered":"2024-04-26T21:50:57.146262+0000","last_unstale":"2024-04-26T21:52:53.422059+0000","last_undegraded":"2024-04-26T21:52:53.422059+0000","last_fullsized":"2024-04-26T21:52:53.422059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:49.946882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556609+0000","last_change":"2024-04-26T21:51:50.546008+0000","last_active":"2024-04-26T21:52:51.556609+0000","last_peered":"2024-04-26T21:52:51.556609+0000","last_clean":"2024-04-26T21:52:51.556609+0000","last_became_active":"2024-04-26T21:51:50.545739+0000","last_became_peered":"2024-04-26T21:51:50.545739+0000","last_unstale":"2024-04-26T21:52:51.556609+0000","last_undegraded":"2024-04-26T21:52:51.556609+0000","last_fullsized":"2024-04-26T21:52:51.556609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.494950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422393+0000","last_change":"2024-04-26T21:50:57.136292+0000","last_active":"2024-04-26T21:52:53.422393+0000","last_peered":"2024-04-26T21:52:53.422393+0000","last_clean":"2024-04-26T21:52:53.422393+0000","last_became_active":"2024-04-26T21:50:57.135936+0000","last_became_peered":"2024-04-26T21:50:57.135936+0000","last_unstale":"2024-04-26T21:52:53.422393+0000","last_undegraded":"2024-04-26T21:52:53.422393+0000","last_fullsized":"2024-04-26T21:52:53.422393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:35.415440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554212+0000","last_change":"2024-04-26T21:50:57.147764+0000","last_active":"2024-04-26T21:52:51.554212+0000","last_peered":"2024-04-26T21:52:51.554212+0000","last_clean":"2024-04-26T21:52:51.554212+0000","last_became_active":"2024-04-26T21:50:57.147034+0000","last_became_peered":"2024-04-26T21:50:57.147034+0000","last_unstale":"2024-04-26T21:52:51.554212+0000","last_undegraded":"2024-04-26T21:52:51.554212+0000","last_fullsized":"2024-04-26T21:52:51.554212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:44.833213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422335+0000","last_change":"2024-04-26T21:50:57.144813+0000","last_active":"2024-04-26T21:52:53.422335+0000","last_peered":"2024-04-26T21:52:53.422335+0000","last_clean":"2024-04-26T21:52:53.422335+0000","last_became_active":"2024-04-26T21:50:57.144442+0000","last_became_peered":"2024-04-26T21:50:57.144442+0000","last_unstale":"2024-04-26T21:52:53.422335+0000","last_undegraded":"2024-04-26T21:52:53.422335+0000","last_fullsized":"2024-04-26T21:52:53.422335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:59.032263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556284+0000","last_change":"2024-04-26T21:51:50.547307+0000","last_active":"2024-04-26T21:52:51.556284+0000","last_peered":"2024-04-26T21:52:51.556284+0000","last_clean":"2024-04-26T21:52:51.556284+0000","last_became_active":"2024-04-26T21:51:50.546888+0000","last_became_peered":"2024-04-26T21:51:50.546888+0000","last_unstale":"2024-04-26T21:52:51.556284+0000","last_undegraded":"2024-04-26T21:52:51.556284+0000","last_fullsized":"2024-04-26T21:52:51.556284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:36.137772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093333+0000","last_change":"2024-04-26T21:50:57.132957+0000","last_active":"2024-04-26T21:52:52.093333+0000","last_peered":"2024-04-26T21:52:52.093333+0000","last_clean":"2024-04-26T21:52:52.093333+0000","last_became_active":"2024-04-26T21:50:57.132606+0000","last_became_peered":"2024-04-26T21:50:57.132606+0000","last_unstale":"2024-04-26T21:52:52.093333+0000","last_undegraded":"2024-04-26T21:52:52.093333+0000","last_fullsized":"2024-04-26T21:52:52.093333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:00.994370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554722+0000","last_change":"2024-04-26T21:50:57.150598+0000","last_active":"2024-04-26T21:52:51.554722+0000","last_peered":"2024-04-26T21:52:51.554722+0000","last_clean":"2024-04-26T21:52:51.554722+0000","last_became_active":"2024-04-26T21:50:57.149697+0000","last_became_peered":"2024-04-26T21:50:57.149697+0000","last_unstale":"2024-04-26T21:52:51.554722+0000","last_undegraded":"2024-04-26T21:52:51.554722+0000","last_fullsized":"2024-04-26T21:52:51.554722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.621137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422578+0000","last_change":"2024-04-26T21:50:57.147264+0000","last_active":"2024-04-26T21:52:53.422578+0000","last_peered":"2024-04-26T21:52:53.422578+0000","last_clean":"2024-04-26T21:52:53.422578+0000","last_became_active":"2024-04-26T21:50:57.146888+0000","last_became_peered":"2024-04-26T21:50:57.146888+0000","last_unstale":"2024-04-26T21:52:53.422578+0000","last_undegraded":"2024-04-26T21:52:53.422578+0000","last_fullsized":"2024-04-26T21:52:53.422578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:49.584327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.118994+0000","last_change":"2024-04-26T21:50:57.142005+0000","last_active":"2024-04-26T21:52:52.118994+0000","last_peered":"2024-04-26T21:52:52.118994+0000","last_clean":"2024-04-26T21:52:52.118994+0000","last_became_active":"2024-04-26T21:50:57.141605+0000","last_became_peered":"2024-04-26T21:50:57.141605+0000","last_unstale":"2024-04-26T21:52:52.118994+0000","last_undegraded":"2024-04-26T21:52:52.118994+0000","last_fullsized":"2024-04-26T21:52:52.118994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:43.485295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529452+0000","last_change":"2024-04-26T21:50:57.138049+0000","last_active":"2024-04-26T21:52:52.529452+0000","last_peered":"2024-04-26T21:52:52.529452+0000","last_clean":"2024-04-26T21:52:52.529452+0000","last_became_active":"2024-04-26T21:50:57.137717+0000","last_became_peered":"2024-04-26T21:50:57.137717+0000","last_unstale":"2024-04-26T21:52:52.529452+0000","last_undegraded":"2024-04-26T21:52:52.529452+0000","last_fullsized":"2024-04-26T21:52:52.529452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:28.918224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421875+0000","last_change":"2024-04-26T21:50:57.139183+0000","last_active":"2024-04-26T21:52:53.421875+0000","last_peered":"2024-04-26T21:52:53.421875+0000","last_clean":"2024-04-26T21:52:53.421875+0000","last_became_active":"2024-04-26T21:50:57.138759+0000","last_became_peered":"2024-04-26T21:50:57.138759+0000","last_unstale":"2024-04-26T21:52:53.421875+0000","last_undegraded":"2024-04-26T21:52:53.421875+0000","last_fullsized":"2024-04-26T21:52:53.421875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:12.003122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421842+0000","last_change":"2024-04-26T21:50:57.143110+0000","last_active":"2024-04-26T21:52:53.421842+0000","last_peered":"2024-04-26T21:52:53.421842+0000","last_clean":"2024-04-26T21:52:53.421842+0000","last_became_active":"2024-04-26T21:50:57.142754+0000","last_became_peered":"2024-04-26T21:50:57.142754+0000","last_unstale":"2024-04-26T21:52:53.421842+0000","last_undegraded":"2024-04-26T21:52:53.421842+0000","last_fullsized":"2024-04-26T21:52:53.421842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:08.321000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092719+0000","last_change":"2024-04-26T21:50:57.146514+0000","last_active":"2024-04-26T21:52:52.092719+0000","last_peered":"2024-04-26T21:52:52.092719+0000","last_clean":"2024-04-26T21:52:52.092719+0000","last_became_active":"2024-04-26T21:50:57.146046+0000","last_became_peered":"2024-04-26T21:50:57.146046+0000","last_unstale":"2024-04-26T21:52:52.092719+0000","last_undegraded":"2024-04-26T21:52:52.092719+0000","last_fullsized":"2024-04-26T21:52:52.092719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:21.130220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621947+0000","last_change":"2024-04-26T21:50:57.140492+0000","last_active":"2024-04-26T21:52:51.621947+0000","last_peered":"2024-04-26T21:52:51.621947+0000","last_clean":"2024-04-26T21:52:51.621947+0000","last_became_active":"2024-04-26T21:50:57.140147+0000","last_became_peered":"2024-04-26T21:50:57.140147+0000","last_unstale":"2024-04-26T21:52:51.621947+0000","last_undegraded":"2024-04-26T21:52:51.621947+0000","last_fullsized":"2024-04-26T21:52:51.621947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:21.679202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556637+0000","last_change":"2024-04-26T21:50:57.140992+0000","last_active":"2024-04-26T21:52:51.556637+0000","last_peered":"2024-04-26T21:52:51.556637+0000","last_clean":"2024-04-26T21:52:51.556637+0000","last_became_active":"2024-04-26T21:50:57.140644+0000","last_became_peered":"2024-04-26T21:50:57.140644+0000","last_unstale":"2024-04-26T21:52:51.556637+0000","last_undegraded":"2024-04-26T21:52:51.556637+0000","last_fullsized":"2024-04-26T21:52:51.556637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554651+0000","last_change":"2024-04-26T21:50:57.141242+0000","last_active":"2024-04-26T21:52:51.554651+0000","last_peered":"2024-04-26T21:52:51.554651+0000","last_clean":"2024-04-26T21:52:51.554651+0000","last_became_active":"2024-04-26T21:50:57.140909+0000","last_became_peered":"2024-04-26T21:50:57.140909+0000","last_unstale":"2024-04-26T21:52:51.554651+0000","last_undegraded":"2024-04-26T21:52:51.554651+0000","last_fullsized":"2024-04-26T21:52:51.554651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:21.996984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556570+0000","last_change":"2024-04-26T21:50:56.371876+0000","last_active":"2024-04-26T21:52:51.556570+0000","last_peered":"2024-04-26T21:52:51.556570+0000","last_clean":"2024-04-26T21:52:51.556570+0000","last_became_active":"2024-04-26T21:50:56.371509+0000","last_became_peered":"2024-04-26T21:50:56.371509+0000","last_unstale":"2024-04-26T21:52:51.556570+0000","last_undegraded":"2024-04-26T21:52:51.556570+0000","last_fullsized":"2024-04-26T21:52:51.556570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:50.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400577+0000","last_change":"2024-04-26T21:50:57.145654+0000","last_active":"2024-04-26T21:52:52.400577+0000","last_peered":"2024-04-26T21:52:52.400577+0000","last_clean":"2024-04-26T21:52:52.400577+0000","last_became_active":"2024-04-26T21:50:57.145355+0000","last_became_peered":"2024-04-26T21:50:57.145355+0000","last_unstale":"2024-04-26T21:52:52.400577+0000","last_undegraded":"2024-04-26T21:52:52.400577+0000","last_fullsized":"2024-04-26T21:52:52.400577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:51.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.401003+0000","last_change":"2024-04-26T21:50:56.362758+0000","last_active":"2024-04-26T21:52:52.401003+0000","last_peered":"2024-04-26T21:52:52.401003+0000","last_clean":"2024-04-26T21:52:52.401003+0000","last_became_active":"2024-04-26T21:50:56.361338+0000","last_became_peered":"2024-04-26T21:50:56.361338+0000","last_unstale":"2024-04-26T21:52:52.401003+0000","last_undegraded":"2024-04-26T21:52:52.401003+0000","last_fullsized":"2024-04-26T21:52:52.401003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:44.603426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422204+0000","last_change":"2024-04-26T21:50:57.150701+0000","last_active":"2024-04-26T21:52:53.422204+0000","last_peered":"2024-04-26T21:52:53.422204+0000","last_clean":"2024-04-26T21:52:53.422204+0000","last_became_active":"2024-04-26T21:50:57.150388+0000","last_became_peered":"2024-04-26T21:50:57.150388+0000","last_unstale":"2024-04-26T21:52:53.422204+0000","last_undegraded":"2024-04-26T21:52:53.422204+0000","last_fullsized":"2024-04-26T21:52:53.422204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:43:07.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422749+0000","last_change":"2024-04-26T21:50:56.371379+0000","last_active":"2024-04-26T21:52:53.422749+0000","last_peered":"2024-04-26T21:52:53.422749+0000","last_clean":"2024-04-26T21:52:53.422749+0000","last_became_active":"2024-04-26T21:50:56.371025+0000","last_became_peered":"2024-04-26T21:50:56.371025+0000","last_unstale":"2024-04-26T21:52:53.422749+0000","last_undegraded":"2024-04-26T21:52:53.422749+0000","last_fullsized":"2024-04-26T21:52:53.422749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:00.633393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092905+0000","last_change":"2024-04-26T21:50:57.144777+0000","last_active":"2024-04-26T21:52:52.092905+0000","last_peered":"2024-04-26T21:52:52.092905+0000","last_clean":"2024-04-26T21:52:52.092905+0000","last_became_active":"2024-04-26T21:50:57.144431+0000","last_became_peered":"2024-04-26T21:50:57.144431+0000","last_unstale":"2024-04-26T21:52:52.092905+0000","last_undegraded":"2024-04-26T21:52:52.092905+0000","last_fullsized":"2024-04-26T21:52:52.092905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:42.164240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422585+0000","last_change":"2024-04-26T21:50:56.360843+0000","last_active":"2024-04-26T21:52:53.422585+0000","last_peered":"2024-04-26T21:52:53.422585+0000","last_clean":"2024-04-26T21:52:53.422585+0000","last_became_active":"2024-04-26T21:50:56.360440+0000","last_became_peered":"2024-04-26T21:50:56.360440+0000","last_unstale":"2024-04-26T21:52:53.422585+0000","last_undegraded":"2024-04-26T21:52:53.422585+0000","last_fullsized":"2024-04-26T21:52:53.422585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:47.585306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.120777+0000","last_change":"2024-04-26T21:52:52.120301+0000","last_active":"2024-04-26T21:52:52.120777+0000","last_peered":"2024-04-26T21:52:52.120777+0000","last_clean":"2024-04-26T21:52:52.120777+0000","last_became_active":"2024-04-26T21:52:52.119782+0000","last_became_peered":"2024-04-26T21:52:52.119782+0000","last_unstale":"2024-04-26T21:52:52.120777+0000","last_undegraded":"2024-04-26T21:52:52.120777+0000","last_fullsized":"2024-04-26T21:52:52.120777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:18.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.643628+0000","last_change":"2024-04-26T21:51:50.551697+0000","last_active":"2024-04-26T21:52:54.643628+0000","last_peered":"2024-04-26T21:52:54.643628+0000","last_clean":"2024-04-26T21:52:54.643628+0000","last_became_active":"2024-04-26T21:51:50.546274+0000","last_became_peered":"2024-04-26T21:51:50.546274+0000","last_unstale":"2024-04-26T21:52:54.643628+0000","last_undegraded":"2024-04-26T21:52:54.643628+0000","last_fullsized":"2024-04-26T21:52:54.643628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:44.549374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":1,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422319+0000","last_change":"2024-04-26T21:50:57.138817+0000","last_active":"2024-04-26T21:52:53.422319+0000","last_peered":"2024-04-26T21:52:53.422319+0000","last_clean":"2024-04-26T21:52:53.422319+0000","last_became_active":"2024-04-26T21:50:57.138489+0000","last_became_peered":"2024-04-26T21:50:57.138489+0000","last_unstale":"2024-04-26T21:52:53.422319+0000","last_undegraded":"2024-04-26T21:52:53.422319+0000","last_fullsized":"2024-04-26T21:52:53.422319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:56.072063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554203+0000","last_change":"2024-04-26T21:51:50.941723+0000","last_active":"2024-04-26T21:52:51.554203+0000","last_peered":"2024-04-26T21:52:51.554203+0000","last_clean":"2024-04-26T21:52:51.554203+0000","last_became_active":"2024-04-26T21:51:50.938689+0000","last_became_peered":"2024-04-26T21:51:50.938689+0000","last_unstale":"2024-04-26T21:52:51.554203+0000","last_undegraded":"2024-04-26T21:52:51.554203+0000","last_fullsized":"2024-04-26T21:52:51.554203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:12.119245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587067+0000","last_change":"2024-04-26T21:51:50.595771+0000","last_active":"2024-04-26T21:52:52.587067+0000","last_peered":"2024-04-26T21:52:52.587067+0000","last_clean":"2024-04-26T21:52:52.587067+0000","last_became_active":"2024-04-26T21:51:50.595441+0000","last_became_peered":"2024-04-26T21:51:50.595441+0000","last_unstale":"2024-04-26T21:52:52.587067+0000","last_undegraded":"2024-04-26T21:52:52.587067+0000","last_fullsized":"2024-04-26T21:52:52.587067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:53.753035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529164+0000","last_change":"2024-04-26T21:50:56.362272+0000","last_active":"2024-04-26T21:52:52.529164+0000","last_peered":"2024-04-26T21:52:52.529164+0000","last_clean":"2024-04-26T21:52:52.529164+0000","last_became_active":"2024-04-26T21:50:56.361941+0000","last_became_peered":"2024-04-26T21:50:56.361941+0000","last_unstale":"2024-04-26T21:52:52.529164+0000","last_undegraded":"2024-04-26T21:52:52.529164+0000","last_fullsized":"2024-04-26T21:52:52.529164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400439+0000","last_change":"2024-04-26T21:50:57.135566+0000","last_active":"2024-04-26T21:52:52.400439+0000","last_peered":"2024-04-26T21:52:52.400439+0000","last_clean":"2024-04-26T21:52:52.400439+0000","last_became_active":"2024-04-26T21:50:57.135231+0000","last_became_peered":"2024-04-26T21:50:57.135231+0000","last_unstale":"2024-04-26T21:52:52.400439+0000","last_undegraded":"2024-04-26T21:52:52.400439+0000","last_fullsized":"2024-04-26T21:52:52.400439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:09.381936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.586871+0000","last_change":"2024-04-26T21:50:56.372420+0000","last_active":"2024-04-26T21:52:52.586871+0000","last_peered":"2024-04-26T21:52:52.586871+0000","last_clean":"2024-04-26T21:52:52.586871+0000","last_became_active":"2024-04-26T21:50:56.372096+0000","last_became_peered":"2024-04-26T21:50:56.372096+0000","last_unstale":"2024-04-26T21:52:52.586871+0000","last_undegraded":"2024-04-26T21:52:52.586871+0000","last_fullsized":"2024-04-26T21:52:52.586871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:03.515804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556475+0000","last_change":"2024-04-26T21:50:57.140071+0000","last_active":"2024-04-26T21:52:51.556475+0000","last_peered":"2024-04-26T21:52:51.556475+0000","last_clean":"2024-04-26T21:52:51.556475+0000","last_became_active":"2024-04-26T21:50:57.139693+0000","last_became_peered":"2024-04-26T21:50:57.139693+0000","last_unstale":"2024-04-26T21:52:51.556475+0000","last_undegraded":"2024-04-26T21:52:51.556475+0000","last_fullsized":"2024-04-26T21:52:51.556475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:17.963966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":140,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.609221+0000","last_change":"2024-04-26T21:50:56.363748+0000","last_active":"2024-04-26T21:52:53.609221+0000","last_peered":"2024-04-26T21:52:53.609221+0000","last_clean":"2024-04-26T21:52:53.609221+0000","last_became_active":"2024-04-26T21:50:56.363328+0000","last_became_peered":"2024-04-26T21:50:56.363328+0000","last_unstale":"2024-04-26T21:52:53.609221+0000","last_undegraded":"2024-04-26T21:52:53.609221+0000","last_fullsized":"2024-04-26T21:52:53.609221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:24.450981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093241+0000","last_change":"2024-04-26T21:50:57.143064+0000","last_active":"2024-04-26T21:52:52.093241+0000","last_peered":"2024-04-26T21:52:52.093241+0000","last_clean":"2024-04-26T21:52:52.093241+0000","last_became_active":"2024-04-26T21:50:57.142715+0000","last_became_peered":"2024-04-26T21:50:57.142715+0000","last_unstale":"2024-04-26T21:52:52.093241+0000","last_undegraded":"2024-04-26T21:52:52.093241+0000","last_fullsized":"2024-04-26T21:52:52.093241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:35.901855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422759+0000","last_change":"2024-04-26T21:50:56.369898+0000","last_active":"2024-04-26T21:52:53.422759+0000","last_peered":"2024-04-26T21:52:53.422759+0000","last_clean":"2024-04-26T21:52:53.422759+0000","last_became_active":"2024-04-26T21:50:56.369349+0000","last_became_peered":"2024-04-26T21:50:56.369349+0000","last_unstale":"2024-04-26T21:52:53.422759+0000","last_undegraded":"2024-04-26T21:52:53.422759+0000","last_fullsized":"2024-04-26T21:52:53.422759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:41.378193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422210+0000","last_change":"2024-04-26T21:50:57.141417+0000","last_active":"2024-04-26T21:52:53.422210+0000","last_peered":"2024-04-26T21:52:53.422210+0000","last_clean":"2024-04-26T21:52:53.422210+0000","last_became_active":"2024-04-26T21:50:57.141091+0000","last_became_peered":"2024-04-26T21:50:57.141091+0000","last_unstale":"2024-04-26T21:52:53.422210+0000","last_undegraded":"2024-04-26T21:52:53.422210+0000","last_fullsized":"2024-04-26T21:52:53.422210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:45.357827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587434+0000","last_change":"2024-04-26T21:50:56.364439+0000","last_active":"2024-04-26T21:52:52.587434+0000","last_peered":"2024-04-26T21:52:52.587434+0000","last_clean":"2024-04-26T21:52:52.587434+0000","last_became_active":"2024-04-26T21:50:56.364090+0000","last_became_peered":"2024-04-26T21:50:56.364090+0000","last_unstale":"2024-04-26T21:52:52.587434+0000","last_undegraded":"2024-04-26T21:52:52.587434+0000","last_fullsized":"2024-04-26T21:52:52.587434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:43.562189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093083+0000","last_change":"2024-04-26T21:50:57.145976+0000","last_active":"2024-04-26T21:52:52.093083+0000","last_peered":"2024-04-26T21:52:52.093083+0000","last_clean":"2024-04-26T21:52:52.093083+0000","last_became_active":"2024-04-26T21:50:57.145620+0000","last_became_peered":"2024-04-26T21:50:57.145620+0000","last_unstale":"2024-04-26T21:52:52.093083+0000","last_undegraded":"2024-04-26T21:52:52.093083+0000","last_fullsized":"2024-04-26T21:52:52.093083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:26.351362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'27","reported_seq":283,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.640672+0000","last_change":"2024-04-26T21:50:56.365786+0000","last_active":"2024-04-26T21:52:53.640672+0000","last_peered":"2024-04-26T21:52:53.640672+0000","last_clean":"2024-04-26T21:52:53.640672+0000","last_became_active":"2024-04-26T21:50:56.365410+0000","last_became_peered":"2024-04-26T21:50:56.365410+0000","last_unstale":"2024-04-26T21:52:53.640672+0000","last_undegraded":"2024-04-26T21:52:53.640672+0000","last_fullsized":"2024-04-26T21:52:53.640672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:01.321661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49785,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":233,"num_read_kb":164,"num_write":28,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422593+0000","last_change":"2024-04-26T21:50:57.137333+0000","last_active":"2024-04-26T21:52:53.422593+0000","last_peered":"2024-04-26T21:52:53.422593+0000","last_clean":"2024-04-26T21:52:53.422593+0000","last_became_active":"2024-04-26T21:50:57.136983+0000","last_became_peered":"2024-04-26T21:50:57.136983+0000","last_unstale":"2024-04-26T21:52:53.422593+0000","last_undegraded":"2024-04-26T21:52:53.422593+0000","last_fullsized":"2024-04-26T21:52:53.422593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:18.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554581+0000","last_change":"2024-04-26T21:50:56.358340+0000","last_active":"2024-04-26T21:52:51.554581+0000","last_peered":"2024-04-26T21:52:51.554581+0000","last_clean":"2024-04-26T21:52:51.554581+0000","last_became_active":"2024-04-26T21:50:56.357972+0000","last_became_peered":"2024-04-26T21:50:56.357972+0000","last_unstale":"2024-04-26T21:52:51.554581+0000","last_undegraded":"2024-04-26T21:52:51.554581+0000","last_fullsized":"2024-04-26T21:52:51.554581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:52.256528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529185+0000","last_change":"2024-04-26T21:50:57.137027+0000","last_active":"2024-04-26T21:52:52.529185+0000","last_peered":"2024-04-26T21:52:52.529185+0000","last_clean":"2024-04-26T21:52:52.529185+0000","last_became_active":"2024-04-26T21:50:57.136400+0000","last_became_peered":"2024-04-26T21:50:57.136400+0000","last_unstale":"2024-04-26T21:52:52.529185+0000","last_undegraded":"2024-04-26T21:52:52.529185+0000","last_fullsized":"2024-04-26T21:52:52.529185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:45.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587015+0000","last_change":"2024-04-26T21:50:56.369784+0000","last_active":"2024-04-26T21:52:52.587015+0000","last_peered":"2024-04-26T21:52:52.587015+0000","last_clean":"2024-04-26T21:52:52.587015+0000","last_became_active":"2024-04-26T21:50:56.369093+0000","last_became_peered":"2024-04-26T21:50:56.369093+0000","last_unstale":"2024-04-26T21:52:52.587015+0000","last_undegraded":"2024-04-26T21:52:52.587015+0000","last_fullsized":"2024-04-26T21:52:52.587015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:27.595146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422543+0000","last_change":"2024-04-26T21:50:57.141404+0000","last_active":"2024-04-26T21:52:53.422543+0000","last_peered":"2024-04-26T21:52:53.422543+0000","last_clean":"2024-04-26T21:52:53.422543+0000","last_became_active":"2024-04-26T21:50:57.140973+0000","last_became_peered":"2024-04-26T21:50:57.140973+0000","last_unstale":"2024-04-26T21:52:53.422543+0000","last_undegraded":"2024-04-26T21:52:53.422543+0000","last_fullsized":"2024-04-26T21:52:53.422543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:09.405982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.532387+0000","last_change":"2024-04-26T21:50:56.370727+0000","last_active":"2024-04-26T21:52:53.532387+0000","last_peered":"2024-04-26T21:52:53.532387+0000","last_clean":"2024-04-26T21:52:53.532387+0000","last_became_active":"2024-04-26T21:50:56.370373+0000","last_became_peered":"2024-04-26T21:50:56.370373+0000","last_unstale":"2024-04-26T21:52:53.532387+0000","last_undegraded":"2024-04-26T21:52:53.532387+0000","last_fullsized":"2024-04-26T21:52:53.532387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:12.689699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422251+0000","last_change":"2024-04-26T21:50:57.145771+0000","last_active":"2024-04-26T21:52:53.422251+0000","last_peered":"2024-04-26T21:52:53.422251+0000","last_clean":"2024-04-26T21:52:53.422251+0000","last_became_active":"2024-04-26T21:50:57.145424+0000","last_became_peered":"2024-04-26T21:50:57.145424+0000","last_unstale":"2024-04-26T21:52:53.422251+0000","last_undegraded":"2024-04-26T21:52:53.422251+0000","last_fullsized":"2024-04-26T21:52:53.422251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:10.485045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422292+0000","last_change":"2024-04-26T21:51:50.560040+0000","last_active":"2024-04-26T21:52:53.422292+0000","last_peered":"2024-04-26T21:52:53.422292+0000","last_clean":"2024-04-26T21:52:53.422292+0000","last_became_active":"2024-04-26T21:51:50.546278+0000","last_became_peered":"2024-04-26T21:51:50.546278+0000","last_unstale":"2024-04-26T21:52:53.422292+0000","last_undegraded":"2024-04-26T21:52:53.422292+0000","last_fullsized":"2024-04-26T21:52:53.422292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:59.851719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.586948+0000","last_change":"2024-04-26T21:51:50.938948+0000","last_active":"2024-04-26T21:52:52.586948+0000","last_peered":"2024-04-26T21:52:52.586948+0000","last_clean":"2024-04-26T21:52:52.586948+0000","last_became_active":"2024-04-26T21:51:50.938501+0000","last_became_peered":"2024-04-26T21:51:50.938501+0000","last_unstale":"2024-04-26T21:52:52.586948+0000","last_undegraded":"2024-04-26T21:52:52.586948+0000","last_fullsized":"2024-04-26T21:52:52.586948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:44.936013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422714+0000","last_change":"2024-04-26T21:50:56.358270+0000","last_active":"2024-04-26T21:52:53.422714+0000","last_peered":"2024-04-26T21:52:53.422714+0000","last_clean":"2024-04-26T21:52:53.422714+0000","last_became_active":"2024-04-26T21:50:56.357909+0000","last_became_peered":"2024-04-26T21:50:56.357909+0000","last_unstale":"2024-04-26T21:52:53.422714+0000","last_undegraded":"2024-04-26T21:52:53.422714+0000","last_fullsized":"2024-04-26T21:52:53.422714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:45.627352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621736+0000","last_change":"2024-04-26T21:50:57.142181+0000","last_active":"2024-04-26T21:52:51.621736+0000","last_peered":"2024-04-26T21:52:51.621736+0000","last_clean":"2024-04-26T21:52:51.621736+0000","last_became_active":"2024-04-26T21:50:57.141837+0000","last_became_peered":"2024-04-26T21:50:57.141837+0000","last_unstale":"2024-04-26T21:52:51.621736+0000","last_undegraded":"2024-04-26T21:52:51.621736+0000","last_fullsized":"2024-04-26T21:52:51.621736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:42.150522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556519+0000","last_change":"2024-04-26T21:51:50.547619+0000","last_active":"2024-04-26T21:52:51.556519+0000","last_peered":"2024-04-26T21:52:51.556519+0000","last_clean":"2024-04-26T21:52:51.556519+0000","last_became_active":"2024-04-26T21:51:50.547180+0000","last_became_peered":"2024-04-26T21:51:50.547180+0000","last_unstale":"2024-04-26T21:52:51.556519+0000","last_undegraded":"2024-04-26T21:52:51.556519+0000","last_fullsized":"2024-04-26T21:52:51.556519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:17.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554589+0000","last_change":"2024-04-26T21:50:57.146682+0000","last_active":"2024-04-26T21:52:51.554589+0000","last_peered":"2024-04-26T21:52:51.554589+0000","last_clean":"2024-04-26T21:52:51.554589+0000","last_became_active":"2024-04-26T21:50:57.146329+0000","last_became_peered":"2024-04-26T21:50:57.146329+0000","last_unstale":"2024-04-26T21:52:51.554589+0000","last_undegraded":"2024-04-26T21:52:51.554589+0000","last_fullsized":"2024-04-26T21:52:51.554589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:41.035526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422540+0000","last_change":"2024-04-26T21:50:56.370533+0000","last_active":"2024-04-26T21:52:53.422540+0000","last_peered":"2024-04-26T21:52:53.422540+0000","last_clean":"2024-04-26T21:52:53.422540+0000","last_became_active":"2024-04-26T21:50:56.370029+0000","last_became_peered":"2024-04-26T21:50:56.370029+0000","last_unstale":"2024-04-26T21:52:53.422540+0000","last_undegraded":"2024-04-26T21:52:53.422540+0000","last_fullsized":"2024-04-26T21:52:53.422540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.005405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554152+0000","last_change":"2024-04-26T21:50:57.142420+0000","last_active":"2024-04-26T21:52:51.554152+0000","last_peered":"2024-04-26T21:52:51.554152+0000","last_clean":"2024-04-26T21:52:51.554152+0000","last_became_active":"2024-04-26T21:50:57.142023+0000","last_became_peered":"2024-04-26T21:50:57.142023+0000","last_unstale":"2024-04-26T21:52:51.554152+0000","last_undegraded":"2024-04-26T21:52:51.554152+0000","last_fullsized":"2024-04-26T21:52:51.554152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:31.567136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587065+0000","last_change":"2024-04-26T21:50:57.139924+0000","last_active":"2024-04-26T21:52:52.587065+0000","last_peered":"2024-04-26T21:52:52.587065+0000","last_clean":"2024-04-26T21:52:52.587065+0000","last_became_active":"2024-04-26T21:50:57.139561+0000","last_became_peered":"2024-04-26T21:50:57.139561+0000","last_unstale":"2024-04-26T21:52:52.587065+0000","last_undegraded":"2024-04-26T21:52:52.587065+0000","last_fullsized":"2024-04-26T21:52:52.587065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:52.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.533579+0000","last_change":"2024-04-26T21:50:56.365256+0000","last_active":"2024-04-26T21:52:54.533579+0000","last_peered":"2024-04-26T21:52:54.533579+0000","last_clean":"2024-04-26T21:52:54.533579+0000","last_became_active":"2024-04-26T21:50:56.364516+0000","last_became_peered":"2024-04-26T21:50:56.364516+0000","last_unstale":"2024-04-26T21:52:54.533579+0000","last_undegraded":"2024-04-26T21:52:54.533579+0000","last_fullsized":"2024-04-26T21:52:54.533579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:13.550129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400745+0000","last_change":"2024-04-26T21:50:57.138177+0000","last_active":"2024-04-26T21:52:52.400745+0000","last_peered":"2024-04-26T21:52:52.400745+0000","last_clean":"2024-04-26T21:52:52.400745+0000","last_became_active":"2024-04-26T21:50:57.137725+0000","last_became_peered":"2024-04-26T21:50:57.137725+0000","last_unstale":"2024-04-26T21:52:52.400745+0000","last_undegraded":"2024-04-26T21:52:52.400745+0000","last_fullsized":"2024-04-26T21:52:52.400745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:14.113935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.118868+0000","last_change":"2024-04-26T21:50:56.358280+0000","last_active":"2024-04-26T21:52:52.118868+0000","last_peered":"2024-04-26T21:52:52.118868+0000","last_clean":"2024-04-26T21:52:52.118868+0000","last_became_active":"2024-04-26T21:50:56.357854+0000","last_became_peered":"2024-04-26T21:50:56.357854+0000","last_unstale":"2024-04-26T21:52:52.118868+0000","last_undegraded":"2024-04-26T21:52:52.118868+0000","last_fullsized":"2024-04-26T21:52:52.118868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.499124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422416+0000","last_change":"2024-04-26T21:49:42.292617+0000","last_active":"2024-04-26T21:52:53.422416+0000","last_peered":"2024-04-26T21:52:53.422416+0000","last_clean":"2024-04-26T21:52:53.422416+0000","last_became_active":"2024-04-26T21:49:42.292160+0000","last_became_peered":"2024-04-26T21:49:42.292160+0000","last_unstale":"2024-04-26T21:52:53.422416+0000","last_undegraded":"2024-04-26T21:52:53.422416+0000","last_fullsized":"2024-04-26T21:52:53.422416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_clean_scrub_stamp":"2024-04-26T21:46:13.787665+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:24.663652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529051+0000","last_change":"2024-04-26T21:50:57.145641+0000","last_active":"2024-04-26T21:52:52.529051+0000","last_peered":"2024-04-26T21:52:52.529051+0000","last_clean":"2024-04-26T21:52:52.529051+0000","last_became_active":"2024-04-26T21:50:57.145304+0000","last_became_peered":"2024-04-26T21:50:57.145304+0000","last_unstale":"2024-04-26T21:52:52.529051+0000","last_undegraded":"2024-04-26T21:52:52.529051+0000","last_fullsized":"2024-04-26T21:52:52.529051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:11.389728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422632+0000","last_change":"2024-04-26T21:51:50.549862+0000","last_active":"2024-04-26T21:52:53.422632+0000","last_peered":"2024-04-26T21:52:53.422632+0000","last_clean":"2024-04-26T21:52:53.422632+0000","last_became_active":"2024-04-26T21:51:50.546054+0000","last_became_peered":"2024-04-26T21:51:50.546054+0000","last_unstale":"2024-04-26T21:52:53.422632+0000","last_undegraded":"2024-04-26T21:52:53.422632+0000","last_fullsized":"2024-04-26T21:52:53.422632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:33.817688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422621+0000","last_change":"2024-04-26T21:50:56.364074+0000","last_active":"2024-04-26T21:52:53.422621+0000","last_peered":"2024-04-26T21:52:53.422621+0000","last_clean":"2024-04-26T21:52:53.422621+0000","last_became_active":"2024-04-26T21:50:56.363688+0000","last_became_peered":"2024-04-26T21:50:56.363688+0000","last_unstale":"2024-04-26T21:52:53.422621+0000","last_undegraded":"2024-04-26T21:52:53.422621+0000","last_fullsized":"2024-04-26T21:52:53.422621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:35.526125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529330+0000","last_change":"2024-04-26T21:50:57.136811+0000","last_active":"2024-04-26T21:52:52.529330+0000","last_peered":"2024-04-26T21:52:52.529330+0000","last_clean":"2024-04-26T21:52:52.529330+0000","last_became_active":"2024-04-26T21:50:57.136252+0000","last_became_peered":"2024-04-26T21:50:57.136252+0000","last_unstale":"2024-04-26T21:52:52.529330+0000","last_undegraded":"2024-04-26T21:52:52.529330+0000","last_fullsized":"2024-04-26T21:52:52.529330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:01.061283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556684+0000","last_change":"2024-04-26T21:50:56.364317+0000","last_active":"2024-04-26T21:52:51.556684+0000","last_peered":"2024-04-26T21:52:51.556684+0000","last_clean":"2024-04-26T21:52:51.556684+0000","last_became_active":"2024-04-26T21:50:56.363882+0000","last_became_peered":"2024-04-26T21:50:56.363882+0000","last_unstale":"2024-04-26T21:52:51.556684+0000","last_undegraded":"2024-04-26T21:52:51.556684+0000","last_fullsized":"2024-04-26T21:52:51.556684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:12.994293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.401010+0000","last_change":"2024-04-26T21:50:57.144856+0000","last_active":"2024-04-26T21:52:52.401010+0000","last_peered":"2024-04-26T21:52:52.401010+0000","last_clean":"2024-04-26T21:52:52.401010+0000","last_became_active":"2024-04-26T21:50:57.143717+0000","last_became_peered":"2024-04-26T21:50:57.143717+0000","last_unstale":"2024-04-26T21:52:52.401010+0000","last_undegraded":"2024-04-26T21:52:52.401010+0000","last_fullsized":"2024-04-26T21:52:52.401010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:53.049243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.119199+0000","last_change":"2024-04-26T21:50:56.370284+0000","last_active":"2024-04-26T21:52:52.119199+0000","last_peered":"2024-04-26T21:52:52.119199+0000","last_clean":"2024-04-26T21:52:52.119199+0000","last_became_active":"2024-04-26T21:50:56.369738+0000","last_became_peered":"2024-04-26T21:50:56.369738+0000","last_unstale":"2024-04-26T21:52:52.119199+0000","last_undegraded":"2024-04-26T21:52:52.119199+0000","last_fullsized":"2024-04-26T21:52:52.119199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:23.390553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422256+0000","last_change":"2024-04-26T21:50:57.142687+0000","last_active":"2024-04-26T21:52:53.422256+0000","last_peered":"2024-04-26T21:52:53.422256+0000","last_clean":"2024-04-26T21:52:53.422256+0000","last_became_active":"2024-04-26T21:50:57.142365+0000","last_became_peered":"2024-04-26T21:50:57.142365+0000","last_unstale":"2024-04-26T21:52:53.422256+0000","last_undegraded":"2024-04-26T21:52:53.422256+0000","last_fullsized":"2024-04-26T21:52:53.422256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:14.261741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092789+0000","last_change":"2024-04-26T21:50:56.364163+0000","last_active":"2024-04-26T21:52:52.092789+0000","last_peered":"2024-04-26T21:52:52.092789+0000","last_clean":"2024-04-26T21:52:52.092789+0000","last_became_active":"2024-04-26T21:50:56.363729+0000","last_became_peered":"2024-04-26T21:50:56.363729+0000","last_unstale":"2024-04-26T21:52:52.092789+0000","last_undegraded":"2024-04-26T21:52:52.092789+0000","last_fullsized":"2024-04-26T21:52:52.092789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:54.620884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422328+0000","last_change":"2024-04-26T21:50:57.142692+0000","last_active":"2024-04-26T21:52:53.422328+0000","last_peered":"2024-04-26T21:52:53.422328+0000","last_clean":"2024-04-26T21:52:53.422328+0000","last_became_active":"2024-04-26T21:50:57.142137+0000","last_became_peered":"2024-04-26T21:50:57.142137+0000","last_unstale":"2024-04-26T21:52:53.422328+0000","last_undegraded":"2024-04-26T21:52:53.422328+0000","last_fullsized":"2024-04-26T21:52:53.422328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.996485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400523+0000","last_change":"2024-04-26T21:50:56.366374+0000","last_active":"2024-04-26T21:52:52.400523+0000","last_peered":"2024-04-26T21:52:52.400523+0000","last_clean":"2024-04-26T21:52:52.400523+0000","last_became_active":"2024-04-26T21:50:56.366014+0000","last_became_peered":"2024-04-26T21:50:56.366014+0000","last_unstale":"2024-04-26T21:52:52.400523+0000","last_undegraded":"2024-04-26T21:52:52.400523+0000","last_fullsized":"2024-04-26T21:52:52.400523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:41.867792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621428+0000","last_change":"2024-04-26T21:50:57.142833+0000","last_active":"2024-04-26T21:52:51.621428+0000","last_peered":"2024-04-26T21:52:51.621428+0000","last_clean":"2024-04-26T21:52:51.621428+0000","last_became_active":"2024-04-26T21:50:57.142061+0000","last_became_peered":"2024-04-26T21:50:57.142061+0000","last_unstale":"2024-04-26T21:52:51.621428+0000","last_undegraded":"2024-04-26T21:52:51.621428+0000","last_fullsized":"2024-04-26T21:52:51.621428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:33.023159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554453+0000","last_change":"2024-04-26T21:50:56.363172+0000","last_active":"2024-04-26T21:52:51.554453+0000","last_peered":"2024-04-26T21:52:51.554453+0000","last_clean":"2024-04-26T21:52:51.554453+0000","last_became_active":"2024-04-26T21:50:56.362782+0000","last_became_peered":"2024-04-26T21:50:56.362782+0000","last_unstale":"2024-04-26T21:52:51.554453+0000","last_undegraded":"2024-04-26T21:52:51.554453+0000","last_fullsized":"2024-04-26T21:52:51.554453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:37.183497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400740+0000","last_change":"2024-04-26T21:50:57.142088+0000","last_active":"2024-04-26T21:52:52.400740+0000","last_peered":"2024-04-26T21:52:52.400740+0000","last_clean":"2024-04-26T21:52:52.400740+0000","last_became_active":"2024-04-26T21:50:57.141685+0000","last_became_peered":"2024-04-26T21:50:57.141685+0000","last_unstale":"2024-04-26T21:52:52.400740+0000","last_undegraded":"2024-04-26T21:52:52.400740+0000","last_fullsized":"2024-04-26T21:52:52.400740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:24.033787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422419+0000","last_change":"2024-04-26T21:50:56.371717+0000","last_active":"2024-04-26T21:52:53.422419+0000","last_peered":"2024-04-26T21:52:53.422419+0000","last_clean":"2024-04-26T21:52:53.422419+0000","last_became_active":"2024-04-26T21:50:56.370930+0000","last_became_peered":"2024-04-26T21:50:56.370930+0000","last_unstale":"2024-04-26T21:52:53.422419+0000","last_undegraded":"2024-04-26T21:52:53.422419+0000","last_fullsized":"2024-04-26T21:52:53.422419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.977392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422495+0000","last_change":"2024-04-26T21:50:57.139671+0000","last_active":"2024-04-26T21:52:53.422495+0000","last_peered":"2024-04-26T21:52:53.422495+0000","last_clean":"2024-04-26T21:52:53.422495+0000","last_became_active":"2024-04-26T21:50:57.139245+0000","last_became_peered":"2024-04-26T21:50:57.139245+0000","last_unstale":"2024-04-26T21:52:53.422495+0000","last_undegraded":"2024-04-26T21:52:53.422495+0000","last_fullsized":"2024-04-26T21:52:53.422495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:26.903905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554682+0000","last_change":"2024-04-26T21:50:56.369836+0000","last_active":"2024-04-26T21:52:51.554682+0000","last_peered":"2024-04-26T21:52:51.554682+0000","last_clean":"2024-04-26T21:52:51.554682+0000","last_became_active":"2024-04-26T21:50:56.369522+0000","last_became_peered":"2024-04-26T21:50:56.369522+0000","last_unstale":"2024-04-26T21:52:51.554682+0000","last_undegraded":"2024-04-26T21:52:51.554682+0000","last_fullsized":"2024-04-26T21:52:51.554682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:40.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422787+0000","last_change":"2024-04-26T21:50:57.138551+0000","last_active":"2024-04-26T21:52:53.422787+0000","last_peered":"2024-04-26T21:52:53.422787+0000","last_clean":"2024-04-26T21:52:53.422787+0000","last_became_active":"2024-04-26T21:50:57.138187+0000","last_became_peered":"2024-04-26T21:50:57.138187+0000","last_unstale":"2024-04-26T21:52:53.422787+0000","last_undegraded":"2024-04-26T21:52:53.422787+0000","last_fullsized":"2024-04-26T21:52:53.422787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:46.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587331+0000","last_change":"2024-04-26T21:50:56.362009+0000","last_active":"2024-04-26T21:52:52.587331+0000","last_peered":"2024-04-26T21:52:52.587331+0000","last_clean":"2024-04-26T21:52:52.587331+0000","last_became_active":"2024-04-26T21:50:56.361646+0000","last_became_peered":"2024-04-26T21:50:56.361646+0000","last_unstale":"2024-04-26T21:52:52.587331+0000","last_undegraded":"2024-04-26T21:52:52.587331+0000","last_fullsized":"2024-04-26T21:52:52.587331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:25.316195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422529+0000","last_change":"2024-04-26T21:50:57.139983+0000","last_active":"2024-04-26T21:52:53.422529+0000","last_peered":"2024-04-26T21:52:53.422529+0000","last_clean":"2024-04-26T21:52:53.422529+0000","last_became_active":"2024-04-26T21:50:57.138426+0000","last_became_peered":"2024-04-26T21:50:57.138426+0000","last_unstale":"2024-04-26T21:52:53.422529+0000","last_undegraded":"2024-04-26T21:52:53.422529+0000","last_fullsized":"2024-04-26T21:52:53.422529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:53.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529527+0000","last_change":"2024-04-26T21:50:56.369527+0000","last_active":"2024-04-26T21:52:52.529527+0000","last_peered":"2024-04-26T21:52:52.529527+0000","last_clean":"2024-04-26T21:52:52.529527+0000","last_became_active":"2024-04-26T21:50:56.369168+0000","last_became_peered":"2024-04-26T21:50:56.369168+0000","last_unstale":"2024-04-26T21:52:52.529527+0000","last_undegraded":"2024-04-26T21:52:52.529527+0000","last_fullsized":"2024-04-26T21:52:52.529527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:38.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422889+0000","last_change":"2024-04-26T21:50:57.137957+0000","last_active":"2024-04-26T21:52:53.422889+0000","last_peered":"2024-04-26T21:52:53.422889+0000","last_clean":"2024-04-26T21:52:53.422889+0000","last_became_active":"2024-04-26T21:50:57.137595+0000","last_became_peered":"2024-04-26T21:50:57.137595+0000","last_unstale":"2024-04-26T21:52:53.422889+0000","last_undegraded":"2024-04-26T21:52:53.422889+0000","last_fullsized":"2024-04-26T21:52:53.422889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:09.355937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621530+0000","last_change":"2024-04-26T21:50:57.143098+0000","last_active":"2024-04-26T21:52:51.621530+0000","last_peered":"2024-04-26T21:52:51.621530+0000","last_clean":"2024-04-26T21:52:51.621530+0000","last_became_active":"2024-04-26T21:50:57.142524+0000","last_became_peered":"2024-04-26T21:50:57.142524+0000","last_unstale":"2024-04-26T21:52:51.621530+0000","last_undegraded":"2024-04-26T21:52:51.621530+0000","last_fullsized":"2024-04-26T21:52:51.621530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:33.333382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554379+0000","last_change":"2024-04-26T21:50:57.143123+0000","last_active":"2024-04-26T21:52:51.554379+0000","last_peered":"2024-04-26T21:52:51.554379+0000","last_clean":"2024-04-26T21:52:51.554379+0000","last_became_active":"2024-04-26T21:50:57.142805+0000","last_became_peered":"2024-04-26T21:50:57.142805+0000","last_unstale":"2024-04-26T21:52:51.554379+0000","last_undegraded":"2024-04-26T21:52:51.554379+0000","last_fullsized":"2024-04-26T21:52:51.554379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:19.330202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554320+0000","last_change":"2024-04-26T21:50:57.140268+0000","last_active":"2024-04-26T21:52:51.554320+0000","last_peered":"2024-04-26T21:52:51.554320+0000","last_clean":"2024-04-26T21:52:51.554320+0000","last_became_active":"2024-04-26T21:50:57.139927+0000","last_became_peered":"2024-04-26T21:50:57.139927+0000","last_unstale":"2024-04-26T21:52:51.554320+0000","last_undegraded":"2024-04-26T21:52:51.554320+0000","last_fullsized":"2024-04-26T21:52:51.554320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:01.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421994+0000","last_change":"2024-04-26T21:50:57.143564+0000","last_active":"2024-04-26T21:52:53.421994+0000","last_peered":"2024-04-26T21:52:53.421994+0000","last_clean":"2024-04-26T21:52:53.421994+0000","last_became_active":"2024-04-26T21:50:57.142740+0000","last_became_peered":"2024-04-26T21:50:57.142740+0000","last_unstale":"2024-04-26T21:52:53.421994+0000","last_undegraded":"2024-04-26T21:52:53.421994+0000","last_fullsized":"2024-04-26T21:52:53.421994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:58.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529451+0000","last_change":"2024-04-26T21:50:57.146883+0000","last_active":"2024-04-26T21:52:52.529451+0000","last_peered":"2024-04-26T21:52:52.529451+0000","last_clean":"2024-04-26T21:52:52.529451+0000","last_became_active":"2024-04-26T21:50:57.146540+0000","last_became_peered":"2024-04-26T21:50:57.146540+0000","last_unstale":"2024-04-26T21:52:52.529451+0000","last_undegraded":"2024-04-26T21:52:52.529451+0000","last_fullsized":"2024-04-26T21:52:52.529451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.787618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621730+0000","last_change":"2024-04-26T21:50:57.136938+0000","last_active":"2024-04-26T21:52:51.621730+0000","last_peered":"2024-04-26T21:52:51.621730+0000","last_clean":"2024-04-26T21:52:51.621730+0000","last_became_active":"2024-04-26T21:50:57.136474+0000","last_became_peered":"2024-04-26T21:50:57.136474+0000","last_unstale":"2024-04-26T21:52:51.621730+0000","last_undegraded":"2024-04-26T21:52:51.621730+0000","last_fullsized":"2024-04-26T21:52:51.621730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:42.534342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554864+0000","last_change":"2024-04-26T21:50:57.150958+0000","last_active":"2024-04-26T21:52:51.554864+0000","last_peered":"2024-04-26T21:52:51.554864+0000","last_clean":"2024-04-26T21:52:51.554864+0000","last_became_active":"2024-04-26T21:50:57.150593+0000","last_became_peered":"2024-04-26T21:50:57.150593+0000","last_unstale":"2024-04-26T21:52:51.554864+0000","last_undegraded":"2024-04-26T21:52:51.554864+0000","last_fullsized":"2024-04-26T21:52:51.554864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:26.583983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400991+0000","last_change":"2024-04-26T21:50:57.147766+0000","last_active":"2024-04-26T21:52:52.400991+0000","last_peered":"2024-04-26T21:52:52.400991+0000","last_clean":"2024-04-26T21:52:52.400991+0000","last_became_active":"2024-04-26T21:50:57.147442+0000","last_became_peered":"2024-04-26T21:50:57.147442+0000","last_unstale":"2024-04-26T21:52:52.400991+0000","last_undegraded":"2024-04-26T21:52:52.400991+0000","last_fullsized":"2024-04-26T21:52:52.400991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.489117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422382+0000","last_change":"2024-04-26T21:51:50.595554+0000","last_active":"2024-04-26T21:52:53.422382+0000","last_peered":"2024-04-26T21:52:53.422382+0000","last_clean":"2024-04-26T21:52:53.422382+0000","last_became_active":"2024-04-26T21:51:50.595248+0000","last_became_peered":"2024-04-26T21:51:50.595248+0000","last_unstale":"2024-04-26T21:52:53.422382+0000","last_undegraded":"2024-04-26T21:52:53.422382+0000","last_fullsized":"2024-04-26T21:52:53.422382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:36.853146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400448+0000","last_change":"2024-04-26T21:50:57.142804+0000","last_active":"2024-04-26T21:52:52.400448+0000","last_peered":"2024-04-26T21:52:52.400448+0000","last_clean":"2024-04-26T21:52:52.400448+0000","last_became_active":"2024-04-26T21:50:57.141902+0000","last_became_peered":"2024-04-26T21:50:57.141902+0000","last_unstale":"2024-04-26T21:52:52.400448+0000","last_undegraded":"2024-04-26T21:52:52.400448+0000","last_fullsized":"2024-04-26T21:52:52.400448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:37.590730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093034+0000","last_change":"2024-04-26T21:50:57.138637+0000","last_active":"2024-04-26T21:52:52.093034+0000","last_peered":"2024-04-26T21:52:52.093034+0000","last_clean":"2024-04-26T21:52:52.093034+0000","last_became_active":"2024-04-26T21:50:57.138295+0000","last_became_peered":"2024-04-26T21:50:57.138295+0000","last_unstale":"2024-04-26T21:52:52.093034+0000","last_undegraded":"2024-04-26T21:52:52.093034+0000","last_fullsized":"2024-04-26T21:52:52.093034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:19.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529384+0000","last_change":"2024-04-26T21:50:57.137377+0000","last_active":"2024-04-26T21:52:52.529384+0000","last_peered":"2024-04-26T21:52:52.529384+0000","last_clean":"2024-04-26T21:52:52.529384+0000","last_became_active":"2024-04-26T21:50:57.136893+0000","last_became_peered":"2024-04-26T21:50:57.136893+0000","last_unstale":"2024-04-26T21:52:52.529384+0000","last_undegraded":"2024-04-26T21:52:52.529384+0000","last_fullsized":"2024-04-26T21:52:52.529384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:44.257438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621602+0000","last_change":"2024-04-26T21:50:57.141439+0000","last_active":"2024-04-26T21:52:51.621602+0000","last_peered":"2024-04-26T21:52:51.621602+0000","last_clean":"2024-04-26T21:52:51.621602+0000","last_became_active":"2024-04-26T21:50:57.141060+0000","last_became_peered":"2024-04-26T21:50:57.141060+0000","last_unstale":"2024-04-26T21:52:51.621602+0000","last_undegraded":"2024-04-26T21:52:51.621602+0000","last_fullsized":"2024-04-26T21:52:51.621602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:27.174348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250381,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54094,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":410,"num_write":160,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":161666,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":474342,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1101824,"data_stored":1114761,"data_compressed":49838,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":473648,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":473568,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1065806,"data_compressed":47984,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":473037,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":522307,"data_compressed":13369,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":474308,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":523545,"data_compressed":13369,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":464842,"data_compressed":8628,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":473977,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1064169,"data_compressed":47984,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":49785,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":50715,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51754,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:52:58.158 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-26T21:52:58.158 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:52:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:57 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3989274008' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:52:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:57 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2767848174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:52:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:57 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2702890869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:52:58.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 -- 172.21.15.55:0/3569128920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0108670 msgr2=0x7f10f010aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 --2- 172.21.15.55:0/3569128920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0108670 0x7f10f010aac0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009ca0 tx=0x7f10e802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 -- 172.21.15.55:0/3569128920 shutdown_connections 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 --2- 172.21.15.55:0/3569128920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0108670 0x7f10f010aac0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 --2- 172.21.15.55:0/3569128920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f0105d40 0x7f10f0108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 --2- 172.21.15.55:0/3569128920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0103410 0x7f10f0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 -- 172.21.15.55:0/3569128920 >> 172.21.15.55:0/3569128920 conn(0x7f10f00fce20 msgr2=0x7f10f00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 -- 172.21.15.55:0/3569128920 shutdown_connections 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.296+0000 7f10f51e2640 1 -- 172.21.15.55:0/3569128920 wait complete. 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10f51e2640 1 Processor -- start 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10f51e2640 1 -- start start 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10f51e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f0103410 0x7f10f0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10f51e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0105d40 0x7f10f01990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10f51e2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0108670 0x7f10f01a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10f51e2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10f010e4c0 con 0x7f10f0105d40 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10f51e2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10f010e340 con 0x7f10f0103410 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10f51e2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f10f010e640 con 0x7f10f0108670 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10e7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0105d40 0x7f10f01990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10e7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0105d40 0x7f10f01990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45634/0 (socket says 172.21.15.55:45634) 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10e7fff640 1 -- 172.21.15.55:0/2235531182 learned_addr learned my addr 172.21.15.55:0/2235531182 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10eed9d640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f0103410 0x7f10f0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10ef59e640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0108670 0x7f10f01a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10e7fff640 1 -- 172.21.15.55:0/2235531182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0108670 msgr2=0x7f10f01a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10e7fff640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0108670 0x7f10f01a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10e7fff640 1 -- 172.21.15.55:0/2235531182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f0103410 msgr2=0x7f10f0198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10e7fff640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f0103410 0x7f10f0198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.297+0000 7f10e7fff640 1 -- 172.21.15.55:0/2235531182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f01a07b0 con 0x7f10f0105d40 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.298+0000 7f10e7fff640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0105d40 0x7f10f01990d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f10d800c970 tx=0x7f10d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.298+0000 7f10ecd99640 1 -- 172.21.15.55:0/2235531182 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8007bb0 con 0x7f10f0105d40 2024-04-26T21:52:58.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.298+0000 7f10ecd99640 1 -- 172.21.15.55:0/2235531182 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d8007d50 con 0x7f10f0105d40 2024-04-26T21:52:58.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.298+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f01a0aa0 con 0x7f10f0105d40 2024-04-26T21:52:58.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.298+0000 7f10ecd99640 1 -- 172.21.15.55:0/2235531182 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d800f450 con 0x7f10f0105d40 2024-04-26T21:52:58.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.298+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f01a1040 con 0x7f10f0105d40 2024-04-26T21:52:58.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.299+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f00fe910 con 0x7f10f0105d40 2024-04-26T21:52:58.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.300+0000 7f10ecd99640 1 -- 172.21.15.55:0/2235531182 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f10d8004f20 con 0x7f10f0105d40 2024-04-26T21:52:58.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.301+0000 7f10ecd99640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f10dc0a3cc0 0x7f10dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.301+0000 7f10eed9d640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f10dc0a3cc0 0x7f10dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.301+0000 7f10ecd99640 1 -- 172.21.15.55:0/2235531182 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f10d80d05b0 con 0x7f10f0105d40 2024-04-26T21:52:58.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.301+0000 7f10eed9d640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f10dc0a3cc0 0x7f10dc0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f10e00058c0 tx=0x7f10e000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:58.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.305+0000 7f10ecd99640 1 -- 172.21.15.55:0/2235531182 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f10d8014030 con 0x7f10f0105d40 2024-04-26T21:52:58.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.489+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 --> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10f0062d70 con 0x7f10dc0a3cc0 2024-04-26T21:52:58.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.496+0000 7f10ecd99640 1 -- 172.21.15.55:0/2235531182 <== mgr.14152 v2:172.21.15.55:6800/3284422558 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259015 (secure 0 0 0) 0x7f10f0062d70 con 0x7f10dc0a3cc0 2024-04-26T21:52:58.499 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:58.500 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-04-26T21:52:58.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f10dc0a3cc0 msgr2=0x7f10dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f10dc0a3cc0 0x7f10dc0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f10e00058c0 tx=0x7f10e000a2d0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0105d40 msgr2=0x7f10f01990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0105d40 0x7f10f01990d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f10d800c970 tx=0x7f10d800ce30 comp rx=0 tx=0).stop 2024-04-26T21:52:58.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 shutdown_connections 2024-04-26T21:52:58.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f10dc0a3cc0 0x7f10dc0a6180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0108670 0x7f10f01a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0105d40 0x7f10f01990d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 --2- 172.21.15.55:0/2235531182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10f0103410 0x7f10f0198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 >> 172.21.15.55:0/2235531182 conn(0x7f10f00fce20 msgr2=0x7f10f0103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:58.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 shutdown_connections 2024-04-26T21:52:58.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.499+0000 7f10f51e2640 1 -- 172.21.15.55:0/2235531182 wait complete. 2024-04-26T21:52:58.511 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-26T21:52:57.371822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644696,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":785,"num_read_kb":519,"num_write":343,"num_write_kb":3040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352688,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589136,"statfs":{"total":1151940427776,"available":1151579275264,"internally_reserved":0,"allocated":12079104,"data_stored":7598310,"data_compressed":250262,"data_compressed_allocated":339968,"data_compressed_original":2179072,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001934"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621845+0000","last_change":"2024-04-26T21:50:57.139486+0000","last_active":"2024-04-26T21:52:51.621845+0000","last_peered":"2024-04-26T21:52:51.621845+0000","last_clean":"2024-04-26T21:52:51.621845+0000","last_became_active":"2024-04-26T21:50:57.139133+0000","last_became_peered":"2024-04-26T21:50:57.139133+0000","last_unstale":"2024-04-26T21:52:51.621845+0000","last_undegraded":"2024-04-26T21:52:51.621845+0000","last_fullsized":"2024-04-26T21:52:51.621845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:33.211627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422137+0000","last_change":"2024-04-26T21:50:57.138519+0000","last_active":"2024-04-26T21:52:53.422137+0000","last_peered":"2024-04-26T21:52:53.422137+0000","last_clean":"2024-04-26T21:52:53.422137+0000","last_became_active":"2024-04-26T21:50:57.138131+0000","last_became_peered":"2024-04-26T21:50:57.138131+0000","last_unstale":"2024-04-26T21:52:53.422137+0000","last_undegraded":"2024-04-26T21:52:53.422137+0000","last_fullsized":"2024-04-26T21:52:53.422137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:34.349694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422059+0000","last_change":"2024-04-26T21:50:57.146584+0000","last_active":"2024-04-26T21:52:53.422059+0000","last_peered":"2024-04-26T21:52:53.422059+0000","last_clean":"2024-04-26T21:52:53.422059+0000","last_became_active":"2024-04-26T21:50:57.146262+0000","last_became_peered":"2024-04-26T21:50:57.146262+0000","last_unstale":"2024-04-26T21:52:53.422059+0000","last_undegraded":"2024-04-26T21:52:53.422059+0000","last_fullsized":"2024-04-26T21:52:53.422059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:49.946882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556609+0000","last_change":"2024-04-26T21:51:50.546008+0000","last_active":"2024-04-26T21:52:51.556609+0000","last_peered":"2024-04-26T21:52:51.556609+0000","last_clean":"2024-04-26T21:52:51.556609+0000","last_became_active":"2024-04-26T21:51:50.545739+0000","last_became_peered":"2024-04-26T21:51:50.545739+0000","last_unstale":"2024-04-26T21:52:51.556609+0000","last_undegraded":"2024-04-26T21:52:51.556609+0000","last_fullsized":"2024-04-26T21:52:51.556609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:17.494950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422393+0000","last_change":"2024-04-26T21:50:57.136292+0000","last_active":"2024-04-26T21:52:53.422393+0000","last_peered":"2024-04-26T21:52:53.422393+0000","last_clean":"2024-04-26T21:52:53.422393+0000","last_became_active":"2024-04-26T21:50:57.135936+0000","last_became_peered":"2024-04-26T21:50:57.135936+0000","last_unstale":"2024-04-26T21:52:53.422393+0000","last_undegraded":"2024-04-26T21:52:53.422393+0000","last_fullsized":"2024-04-26T21:52:53.422393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:03:35.415440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554212+0000","last_change":"2024-04-26T21:50:57.147764+0000","last_active":"2024-04-26T21:52:51.554212+0000","last_peered":"2024-04-26T21:52:51.554212+0000","last_clean":"2024-04-26T21:52:51.554212+0000","last_became_active":"2024-04-26T21:50:57.147034+0000","last_became_peered":"2024-04-26T21:50:57.147034+0000","last_unstale":"2024-04-26T21:52:51.554212+0000","last_undegraded":"2024-04-26T21:52:51.554212+0000","last_fullsized":"2024-04-26T21:52:51.554212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:44.833213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422335+0000","last_change":"2024-04-26T21:50:57.144813+0000","last_active":"2024-04-26T21:52:53.422335+0000","last_peered":"2024-04-26T21:52:53.422335+0000","last_clean":"2024-04-26T21:52:53.422335+0000","last_became_active":"2024-04-26T21:50:57.144442+0000","last_became_peered":"2024-04-26T21:50:57.144442+0000","last_unstale":"2024-04-26T21:52:53.422335+0000","last_undegraded":"2024-04-26T21:52:53.422335+0000","last_fullsized":"2024-04-26T21:52:53.422335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:59.032263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556284+0000","last_change":"2024-04-26T21:51:50.547307+0000","last_active":"2024-04-26T21:52:51.556284+0000","last_peered":"2024-04-26T21:52:51.556284+0000","last_clean":"2024-04-26T21:52:51.556284+0000","last_became_active":"2024-04-26T21:51:50.546888+0000","last_became_peered":"2024-04-26T21:51:50.546888+0000","last_unstale":"2024-04-26T21:52:51.556284+0000","last_undegraded":"2024-04-26T21:52:51.556284+0000","last_fullsized":"2024-04-26T21:52:51.556284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:36.137772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093333+0000","last_change":"2024-04-26T21:50:57.132957+0000","last_active":"2024-04-26T21:52:52.093333+0000","last_peered":"2024-04-26T21:52:52.093333+0000","last_clean":"2024-04-26T21:52:52.093333+0000","last_became_active":"2024-04-26T21:50:57.132606+0000","last_became_peered":"2024-04-26T21:50:57.132606+0000","last_unstale":"2024-04-26T21:52:52.093333+0000","last_undegraded":"2024-04-26T21:52:52.093333+0000","last_fullsized":"2024-04-26T21:52:52.093333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:00.994370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554722+0000","last_change":"2024-04-26T21:50:57.150598+0000","last_active":"2024-04-26T21:52:51.554722+0000","last_peered":"2024-04-26T21:52:51.554722+0000","last_clean":"2024-04-26T21:52:51.554722+0000","last_became_active":"2024-04-26T21:50:57.149697+0000","last_became_peered":"2024-04-26T21:50:57.149697+0000","last_unstale":"2024-04-26T21:52:51.554722+0000","last_undegraded":"2024-04-26T21:52:51.554722+0000","last_fullsized":"2024-04-26T21:52:51.554722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.621137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422578+0000","last_change":"2024-04-26T21:50:57.147264+0000","last_active":"2024-04-26T21:52:53.422578+0000","last_peered":"2024-04-26T21:52:53.422578+0000","last_clean":"2024-04-26T21:52:53.422578+0000","last_became_active":"2024-04-26T21:50:57.146888+0000","last_became_peered":"2024-04-26T21:50:57.146888+0000","last_unstale":"2024-04-26T21:52:53.422578+0000","last_undegraded":"2024-04-26T21:52:53.422578+0000","last_fullsized":"2024-04-26T21:52:53.422578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:26:49.584327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.118994+0000","last_change":"2024-04-26T21:50:57.142005+0000","last_active":"2024-04-26T21:52:52.118994+0000","last_peered":"2024-04-26T21:52:52.118994+0000","last_clean":"2024-04-26T21:52:52.118994+0000","last_became_active":"2024-04-26T21:50:57.141605+0000","last_became_peered":"2024-04-26T21:50:57.141605+0000","last_unstale":"2024-04-26T21:52:52.118994+0000","last_undegraded":"2024-04-26T21:52:52.118994+0000","last_fullsized":"2024-04-26T21:52:52.118994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:43.485295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529452+0000","last_change":"2024-04-26T21:50:57.138049+0000","last_active":"2024-04-26T21:52:52.529452+0000","last_peered":"2024-04-26T21:52:52.529452+0000","last_clean":"2024-04-26T21:52:52.529452+0000","last_became_active":"2024-04-26T21:50:57.137717+0000","last_became_peered":"2024-04-26T21:50:57.137717+0000","last_unstale":"2024-04-26T21:52:52.529452+0000","last_undegraded":"2024-04-26T21:52:52.529452+0000","last_fullsized":"2024-04-26T21:52:52.529452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:28.918224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421875+0000","last_change":"2024-04-26T21:50:57.139183+0000","last_active":"2024-04-26T21:52:53.421875+0000","last_peered":"2024-04-26T21:52:53.421875+0000","last_clean":"2024-04-26T21:52:53.421875+0000","last_became_active":"2024-04-26T21:50:57.138759+0000","last_became_peered":"2024-04-26T21:50:57.138759+0000","last_unstale":"2024-04-26T21:52:53.421875+0000","last_undegraded":"2024-04-26T21:52:53.421875+0000","last_fullsized":"2024-04-26T21:52:53.421875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:12.003122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421842+0000","last_change":"2024-04-26T21:50:57.143110+0000","last_active":"2024-04-26T21:52:53.421842+0000","last_peered":"2024-04-26T21:52:53.421842+0000","last_clean":"2024-04-26T21:52:53.421842+0000","last_became_active":"2024-04-26T21:50:57.142754+0000","last_became_peered":"2024-04-26T21:50:57.142754+0000","last_unstale":"2024-04-26T21:52:53.421842+0000","last_undegraded":"2024-04-26T21:52:53.421842+0000","last_fullsized":"2024-04-26T21:52:53.421842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:08.321000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092719+0000","last_change":"2024-04-26T21:50:57.146514+0000","last_active":"2024-04-26T21:52:52.092719+0000","last_peered":"2024-04-26T21:52:52.092719+0000","last_clean":"2024-04-26T21:52:52.092719+0000","last_became_active":"2024-04-26T21:50:57.146046+0000","last_became_peered":"2024-04-26T21:50:57.146046+0000","last_unstale":"2024-04-26T21:52:52.092719+0000","last_undegraded":"2024-04-26T21:52:52.092719+0000","last_fullsized":"2024-04-26T21:52:52.092719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:11:21.130220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621947+0000","last_change":"2024-04-26T21:50:57.140492+0000","last_active":"2024-04-26T21:52:51.621947+0000","last_peered":"2024-04-26T21:52:51.621947+0000","last_clean":"2024-04-26T21:52:51.621947+0000","last_became_active":"2024-04-26T21:50:57.140147+0000","last_became_peered":"2024-04-26T21:50:57.140147+0000","last_unstale":"2024-04-26T21:52:51.621947+0000","last_undegraded":"2024-04-26T21:52:51.621947+0000","last_fullsized":"2024-04-26T21:52:51.621947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:21.679202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556637+0000","last_change":"2024-04-26T21:50:57.140992+0000","last_active":"2024-04-26T21:52:51.556637+0000","last_peered":"2024-04-26T21:52:51.556637+0000","last_clean":"2024-04-26T21:52:51.556637+0000","last_became_active":"2024-04-26T21:50:57.140644+0000","last_became_peered":"2024-04-26T21:50:57.140644+0000","last_unstale":"2024-04-26T21:52:51.556637+0000","last_undegraded":"2024-04-26T21:52:51.556637+0000","last_fullsized":"2024-04-26T21:52:51.556637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:30:46.239137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554651+0000","last_change":"2024-04-26T21:50:57.141242+0000","last_active":"2024-04-26T21:52:51.554651+0000","last_peered":"2024-04-26T21:52:51.554651+0000","last_clean":"2024-04-26T21:52:51.554651+0000","last_became_active":"2024-04-26T21:50:57.140909+0000","last_became_peered":"2024-04-26T21:50:57.140909+0000","last_unstale":"2024-04-26T21:52:51.554651+0000","last_undegraded":"2024-04-26T21:52:51.554651+0000","last_fullsized":"2024-04-26T21:52:51.554651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:21.996984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556570+0000","last_change":"2024-04-26T21:50:56.371876+0000","last_active":"2024-04-26T21:52:51.556570+0000","last_peered":"2024-04-26T21:52:51.556570+0000","last_clean":"2024-04-26T21:52:51.556570+0000","last_became_active":"2024-04-26T21:50:56.371509+0000","last_became_peered":"2024-04-26T21:50:56.371509+0000","last_unstale":"2024-04-26T21:52:51.556570+0000","last_undegraded":"2024-04-26T21:52:51.556570+0000","last_fullsized":"2024-04-26T21:52:51.556570+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:50.161285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400577+0000","last_change":"2024-04-26T21:50:57.145654+0000","last_active":"2024-04-26T21:52:52.400577+0000","last_peered":"2024-04-26T21:52:52.400577+0000","last_clean":"2024-04-26T21:52:52.400577+0000","last_became_active":"2024-04-26T21:50:57.145355+0000","last_became_peered":"2024-04-26T21:50:57.145355+0000","last_unstale":"2024-04-26T21:52:52.400577+0000","last_undegraded":"2024-04-26T21:52:52.400577+0000","last_fullsized":"2024-04-26T21:52:52.400577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:51.686023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.401003+0000","last_change":"2024-04-26T21:50:56.362758+0000","last_active":"2024-04-26T21:52:52.401003+0000","last_peered":"2024-04-26T21:52:52.401003+0000","last_clean":"2024-04-26T21:52:52.401003+0000","last_became_active":"2024-04-26T21:50:56.361338+0000","last_became_peered":"2024-04-26T21:50:56.361338+0000","last_unstale":"2024-04-26T21:52:52.401003+0000","last_undegraded":"2024-04-26T21:52:52.401003+0000","last_fullsized":"2024-04-26T21:52:52.401003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:44.603426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422204+0000","last_change":"2024-04-26T21:50:57.150701+0000","last_active":"2024-04-26T21:52:53.422204+0000","last_peered":"2024-04-26T21:52:53.422204+0000","last_clean":"2024-04-26T21:52:53.422204+0000","last_became_active":"2024-04-26T21:50:57.150388+0000","last_became_peered":"2024-04-26T21:50:57.150388+0000","last_unstale":"2024-04-26T21:52:53.422204+0000","last_undegraded":"2024-04-26T21:52:53.422204+0000","last_fullsized":"2024-04-26T21:52:53.422204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:43:07.278964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422749+0000","last_change":"2024-04-26T21:50:56.371379+0000","last_active":"2024-04-26T21:52:53.422749+0000","last_peered":"2024-04-26T21:52:53.422749+0000","last_clean":"2024-04-26T21:52:53.422749+0000","last_became_active":"2024-04-26T21:50:56.371025+0000","last_became_peered":"2024-04-26T21:50:56.371025+0000","last_unstale":"2024-04-26T21:52:53.422749+0000","last_undegraded":"2024-04-26T21:52:53.422749+0000","last_fullsized":"2024-04-26T21:52:53.422749+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:46:00.633393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092905+0000","last_change":"2024-04-26T21:50:57.144777+0000","last_active":"2024-04-26T21:52:52.092905+0000","last_peered":"2024-04-26T21:52:52.092905+0000","last_clean":"2024-04-26T21:52:52.092905+0000","last_became_active":"2024-04-26T21:50:57.144431+0000","last_became_peered":"2024-04-26T21:50:57.144431+0000","last_unstale":"2024-04-26T21:52:52.092905+0000","last_undegraded":"2024-04-26T21:52:52.092905+0000","last_fullsized":"2024-04-26T21:52:52.092905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:42.164240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422585+0000","last_change":"2024-04-26T21:50:56.360843+0000","last_active":"2024-04-26T21:52:53.422585+0000","last_peered":"2024-04-26T21:52:53.422585+0000","last_clean":"2024-04-26T21:52:53.422585+0000","last_became_active":"2024-04-26T21:50:56.360440+0000","last_became_peered":"2024-04-26T21:50:56.360440+0000","last_unstale":"2024-04-26T21:52:53.422585+0000","last_undegraded":"2024-04-26T21:52:53.422585+0000","last_fullsized":"2024-04-26T21:52:53.422585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:47.585306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.120777+0000","last_change":"2024-04-26T21:52:52.120301+0000","last_active":"2024-04-26T21:52:52.120777+0000","last_peered":"2024-04-26T21:52:52.120777+0000","last_clean":"2024-04-26T21:52:52.120777+0000","last_became_active":"2024-04-26T21:52:52.119782+0000","last_became_peered":"2024-04-26T21:52:52.119782+0000","last_unstale":"2024-04-26T21:52:52.120777+0000","last_undegraded":"2024-04-26T21:52:52.120777+0000","last_fullsized":"2024-04-26T21:52:52.120777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:18.876998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":89,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.643628+0000","last_change":"2024-04-26T21:51:50.551697+0000","last_active":"2024-04-26T21:52:54.643628+0000","last_peered":"2024-04-26T21:52:54.643628+0000","last_clean":"2024-04-26T21:52:54.643628+0000","last_became_active":"2024-04-26T21:51:50.546274+0000","last_became_peered":"2024-04-26T21:51:50.546274+0000","last_unstale":"2024-04-26T21:52:54.643628+0000","last_undegraded":"2024-04-26T21:52:54.643628+0000","last_fullsized":"2024-04-26T21:52:54.643628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:44.549374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":1,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422319+0000","last_change":"2024-04-26T21:50:57.138817+0000","last_active":"2024-04-26T21:52:53.422319+0000","last_peered":"2024-04-26T21:52:53.422319+0000","last_clean":"2024-04-26T21:52:53.422319+0000","last_became_active":"2024-04-26T21:50:57.138489+0000","last_became_peered":"2024-04-26T21:50:57.138489+0000","last_unstale":"2024-04-26T21:52:53.422319+0000","last_undegraded":"2024-04-26T21:52:53.422319+0000","last_fullsized":"2024-04-26T21:52:53.422319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:58:56.072063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554203+0000","last_change":"2024-04-26T21:51:50.941723+0000","last_active":"2024-04-26T21:52:51.554203+0000","last_peered":"2024-04-26T21:52:51.554203+0000","last_clean":"2024-04-26T21:52:51.554203+0000","last_became_active":"2024-04-26T21:51:50.938689+0000","last_became_peered":"2024-04-26T21:51:50.938689+0000","last_unstale":"2024-04-26T21:52:51.554203+0000","last_undegraded":"2024-04-26T21:52:51.554203+0000","last_fullsized":"2024-04-26T21:52:51.554203+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:23:12.119245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587067+0000","last_change":"2024-04-26T21:51:50.595771+0000","last_active":"2024-04-26T21:52:52.587067+0000","last_peered":"2024-04-26T21:52:52.587067+0000","last_clean":"2024-04-26T21:52:52.587067+0000","last_became_active":"2024-04-26T21:51:50.595441+0000","last_became_peered":"2024-04-26T21:51:50.595441+0000","last_unstale":"2024-04-26T21:52:52.587067+0000","last_undegraded":"2024-04-26T21:52:52.587067+0000","last_fullsized":"2024-04-26T21:52:52.587067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:53.753035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529164+0000","last_change":"2024-04-26T21:50:56.362272+0000","last_active":"2024-04-26T21:52:52.529164+0000","last_peered":"2024-04-26T21:52:52.529164+0000","last_clean":"2024-04-26T21:52:52.529164+0000","last_became_active":"2024-04-26T21:50:56.361941+0000","last_became_peered":"2024-04-26T21:50:56.361941+0000","last_unstale":"2024-04-26T21:52:52.529164+0000","last_undegraded":"2024-04-26T21:52:52.529164+0000","last_fullsized":"2024-04-26T21:52:52.529164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:39.309222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400439+0000","last_change":"2024-04-26T21:50:57.135566+0000","last_active":"2024-04-26T21:52:52.400439+0000","last_peered":"2024-04-26T21:52:52.400439+0000","last_clean":"2024-04-26T21:52:52.400439+0000","last_became_active":"2024-04-26T21:50:57.135231+0000","last_became_peered":"2024-04-26T21:50:57.135231+0000","last_unstale":"2024-04-26T21:52:52.400439+0000","last_undegraded":"2024-04-26T21:52:52.400439+0000","last_fullsized":"2024-04-26T21:52:52.400439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:09.381936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.586871+0000","last_change":"2024-04-26T21:50:56.372420+0000","last_active":"2024-04-26T21:52:52.586871+0000","last_peered":"2024-04-26T21:52:52.586871+0000","last_clean":"2024-04-26T21:52:52.586871+0000","last_became_active":"2024-04-26T21:50:56.372096+0000","last_became_peered":"2024-04-26T21:50:56.372096+0000","last_unstale":"2024-04-26T21:52:52.586871+0000","last_undegraded":"2024-04-26T21:52:52.586871+0000","last_fullsized":"2024-04-26T21:52:52.586871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:22:03.515804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556475+0000","last_change":"2024-04-26T21:50:57.140071+0000","last_active":"2024-04-26T21:52:51.556475+0000","last_peered":"2024-04-26T21:52:51.556475+0000","last_clean":"2024-04-26T21:52:51.556475+0000","last_became_active":"2024-04-26T21:50:57.139693+0000","last_became_peered":"2024-04-26T21:50:57.139693+0000","last_unstale":"2024-04-26T21:52:51.556475+0000","last_undegraded":"2024-04-26T21:52:51.556475+0000","last_fullsized":"2024-04-26T21:52:51.556475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:17.963966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":140,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.609221+0000","last_change":"2024-04-26T21:50:56.363748+0000","last_active":"2024-04-26T21:52:53.609221+0000","last_peered":"2024-04-26T21:52:53.609221+0000","last_clean":"2024-04-26T21:52:53.609221+0000","last_became_active":"2024-04-26T21:50:56.363328+0000","last_became_peered":"2024-04-26T21:50:56.363328+0000","last_unstale":"2024-04-26T21:52:53.609221+0000","last_undegraded":"2024-04-26T21:52:53.609221+0000","last_fullsized":"2024-04-26T21:52:53.609221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:24.450981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093241+0000","last_change":"2024-04-26T21:50:57.143064+0000","last_active":"2024-04-26T21:52:52.093241+0000","last_peered":"2024-04-26T21:52:52.093241+0000","last_clean":"2024-04-26T21:52:52.093241+0000","last_became_active":"2024-04-26T21:50:57.142715+0000","last_became_peered":"2024-04-26T21:50:57.142715+0000","last_unstale":"2024-04-26T21:52:52.093241+0000","last_undegraded":"2024-04-26T21:52:52.093241+0000","last_fullsized":"2024-04-26T21:52:52.093241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:04:35.901855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422759+0000","last_change":"2024-04-26T21:50:56.369898+0000","last_active":"2024-04-26T21:52:53.422759+0000","last_peered":"2024-04-26T21:52:53.422759+0000","last_clean":"2024-04-26T21:52:53.422759+0000","last_became_active":"2024-04-26T21:50:56.369349+0000","last_became_peered":"2024-04-26T21:50:56.369349+0000","last_unstale":"2024-04-26T21:52:53.422759+0000","last_undegraded":"2024-04-26T21:52:53.422759+0000","last_fullsized":"2024-04-26T21:52:53.422759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:39:41.378193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422210+0000","last_change":"2024-04-26T21:50:57.141417+0000","last_active":"2024-04-26T21:52:53.422210+0000","last_peered":"2024-04-26T21:52:53.422210+0000","last_clean":"2024-04-26T21:52:53.422210+0000","last_became_active":"2024-04-26T21:50:57.141091+0000","last_became_peered":"2024-04-26T21:50:57.141091+0000","last_unstale":"2024-04-26T21:52:53.422210+0000","last_undegraded":"2024-04-26T21:52:53.422210+0000","last_fullsized":"2024-04-26T21:52:53.422210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:45.357827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587434+0000","last_change":"2024-04-26T21:50:56.364439+0000","last_active":"2024-04-26T21:52:52.587434+0000","last_peered":"2024-04-26T21:52:52.587434+0000","last_clean":"2024-04-26T21:52:52.587434+0000","last_became_active":"2024-04-26T21:50:56.364090+0000","last_became_peered":"2024-04-26T21:50:56.364090+0000","last_unstale":"2024-04-26T21:52:52.587434+0000","last_undegraded":"2024-04-26T21:52:52.587434+0000","last_fullsized":"2024-04-26T21:52:52.587434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:10:43.562189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093083+0000","last_change":"2024-04-26T21:50:57.145976+0000","last_active":"2024-04-26T21:52:52.093083+0000","last_peered":"2024-04-26T21:52:52.093083+0000","last_clean":"2024-04-26T21:52:52.093083+0000","last_became_active":"2024-04-26T21:50:57.145620+0000","last_became_peered":"2024-04-26T21:50:57.145620+0000","last_unstale":"2024-04-26T21:52:52.093083+0000","last_undegraded":"2024-04-26T21:52:52.093083+0000","last_fullsized":"2024-04-26T21:52:52.093083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:54:26.351362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'27","reported_seq":283,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.640672+0000","last_change":"2024-04-26T21:50:56.365786+0000","last_active":"2024-04-26T21:52:53.640672+0000","last_peered":"2024-04-26T21:52:53.640672+0000","last_clean":"2024-04-26T21:52:53.640672+0000","last_became_active":"2024-04-26T21:50:56.365410+0000","last_became_peered":"2024-04-26T21:50:56.365410+0000","last_unstale":"2024-04-26T21:52:53.640672+0000","last_undegraded":"2024-04-26T21:52:53.640672+0000","last_fullsized":"2024-04-26T21:52:53.640672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:01.321661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49785,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":233,"num_read_kb":164,"num_write":28,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422593+0000","last_change":"2024-04-26T21:50:57.137333+0000","last_active":"2024-04-26T21:52:53.422593+0000","last_peered":"2024-04-26T21:52:53.422593+0000","last_clean":"2024-04-26T21:52:53.422593+0000","last_became_active":"2024-04-26T21:50:57.136983+0000","last_became_peered":"2024-04-26T21:50:57.136983+0000","last_unstale":"2024-04-26T21:52:53.422593+0000","last_undegraded":"2024-04-26T21:52:53.422593+0000","last_fullsized":"2024-04-26T21:52:53.422593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:18.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554581+0000","last_change":"2024-04-26T21:50:56.358340+0000","last_active":"2024-04-26T21:52:51.554581+0000","last_peered":"2024-04-26T21:52:51.554581+0000","last_clean":"2024-04-26T21:52:51.554581+0000","last_became_active":"2024-04-26T21:50:56.357972+0000","last_became_peered":"2024-04-26T21:50:56.357972+0000","last_unstale":"2024-04-26T21:52:51.554581+0000","last_undegraded":"2024-04-26T21:52:51.554581+0000","last_fullsized":"2024-04-26T21:52:51.554581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:52.256528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529185+0000","last_change":"2024-04-26T21:50:57.137027+0000","last_active":"2024-04-26T21:52:52.529185+0000","last_peered":"2024-04-26T21:52:52.529185+0000","last_clean":"2024-04-26T21:52:52.529185+0000","last_became_active":"2024-04-26T21:50:57.136400+0000","last_became_peered":"2024-04-26T21:50:57.136400+0000","last_unstale":"2024-04-26T21:52:52.529185+0000","last_undegraded":"2024-04-26T21:52:52.529185+0000","last_fullsized":"2024-04-26T21:52:52.529185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:45.474708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587015+0000","last_change":"2024-04-26T21:50:56.369784+0000","last_active":"2024-04-26T21:52:52.587015+0000","last_peered":"2024-04-26T21:52:52.587015+0000","last_clean":"2024-04-26T21:52:52.587015+0000","last_became_active":"2024-04-26T21:50:56.369093+0000","last_became_peered":"2024-04-26T21:50:56.369093+0000","last_unstale":"2024-04-26T21:52:52.587015+0000","last_undegraded":"2024-04-26T21:52:52.587015+0000","last_fullsized":"2024-04-26T21:52:52.587015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:06:27.595146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422543+0000","last_change":"2024-04-26T21:50:57.141404+0000","last_active":"2024-04-26T21:52:53.422543+0000","last_peered":"2024-04-26T21:52:53.422543+0000","last_clean":"2024-04-26T21:52:53.422543+0000","last_became_active":"2024-04-26T21:50:57.140973+0000","last_became_peered":"2024-04-26T21:50:57.140973+0000","last_unstale":"2024-04-26T21:52:53.422543+0000","last_undegraded":"2024-04-26T21:52:53.422543+0000","last_fullsized":"2024-04-26T21:52:53.422543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:09.405982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.532387+0000","last_change":"2024-04-26T21:50:56.370727+0000","last_active":"2024-04-26T21:52:53.532387+0000","last_peered":"2024-04-26T21:52:53.532387+0000","last_clean":"2024-04-26T21:52:53.532387+0000","last_became_active":"2024-04-26T21:50:56.370373+0000","last_became_peered":"2024-04-26T21:50:56.370373+0000","last_unstale":"2024-04-26T21:52:53.532387+0000","last_undegraded":"2024-04-26T21:52:53.532387+0000","last_fullsized":"2024-04-26T21:52:53.532387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:35:12.689699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422251+0000","last_change":"2024-04-26T21:50:57.145771+0000","last_active":"2024-04-26T21:52:53.422251+0000","last_peered":"2024-04-26T21:52:53.422251+0000","last_clean":"2024-04-26T21:52:53.422251+0000","last_became_active":"2024-04-26T21:50:57.145424+0000","last_became_peered":"2024-04-26T21:50:57.145424+0000","last_unstale":"2024-04-26T21:52:53.422251+0000","last_undegraded":"2024-04-26T21:52:53.422251+0000","last_fullsized":"2024-04-26T21:52:53.422251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:10.485045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422292+0000","last_change":"2024-04-26T21:51:50.560040+0000","last_active":"2024-04-26T21:52:53.422292+0000","last_peered":"2024-04-26T21:52:53.422292+0000","last_clean":"2024-04-26T21:52:53.422292+0000","last_became_active":"2024-04-26T21:51:50.546278+0000","last_became_peered":"2024-04-26T21:51:50.546278+0000","last_unstale":"2024-04-26T21:52:53.422292+0000","last_undegraded":"2024-04-26T21:52:53.422292+0000","last_fullsized":"2024-04-26T21:52:53.422292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:59.851719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.586948+0000","last_change":"2024-04-26T21:51:50.938948+0000","last_active":"2024-04-26T21:52:52.586948+0000","last_peered":"2024-04-26T21:52:52.586948+0000","last_clean":"2024-04-26T21:52:52.586948+0000","last_became_active":"2024-04-26T21:51:50.938501+0000","last_became_peered":"2024-04-26T21:51:50.938501+0000","last_unstale":"2024-04-26T21:52:52.586948+0000","last_undegraded":"2024-04-26T21:52:52.586948+0000","last_fullsized":"2024-04-26T21:52:52.586948+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:12:44.936013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422714+0000","last_change":"2024-04-26T21:50:56.358270+0000","last_active":"2024-04-26T21:52:53.422714+0000","last_peered":"2024-04-26T21:52:53.422714+0000","last_clean":"2024-04-26T21:52:53.422714+0000","last_became_active":"2024-04-26T21:50:56.357909+0000","last_became_peered":"2024-04-26T21:50:56.357909+0000","last_unstale":"2024-04-26T21:52:53.422714+0000","last_undegraded":"2024-04-26T21:52:53.422714+0000","last_fullsized":"2024-04-26T21:52:53.422714+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:45.627352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621736+0000","last_change":"2024-04-26T21:50:57.142181+0000","last_active":"2024-04-26T21:52:51.621736+0000","last_peered":"2024-04-26T21:52:51.621736+0000","last_clean":"2024-04-26T21:52:51.621736+0000","last_became_active":"2024-04-26T21:50:57.141837+0000","last_became_peered":"2024-04-26T21:50:57.141837+0000","last_unstale":"2024-04-26T21:52:51.621736+0000","last_undegraded":"2024-04-26T21:52:51.621736+0000","last_fullsized":"2024-04-26T21:52:51.621736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:42.150522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556519+0000","last_change":"2024-04-26T21:51:50.547619+0000","last_active":"2024-04-26T21:52:51.556519+0000","last_peered":"2024-04-26T21:52:51.556519+0000","last_clean":"2024-04-26T21:52:51.556519+0000","last_became_active":"2024-04-26T21:51:50.547180+0000","last_became_peered":"2024-04-26T21:51:50.547180+0000","last_unstale":"2024-04-26T21:52:51.556519+0000","last_undegraded":"2024-04-26T21:52:51.556519+0000","last_fullsized":"2024-04-26T21:52:51.556519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:17.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554589+0000","last_change":"2024-04-26T21:50:57.146682+0000","last_active":"2024-04-26T21:52:51.554589+0000","last_peered":"2024-04-26T21:52:51.554589+0000","last_clean":"2024-04-26T21:52:51.554589+0000","last_became_active":"2024-04-26T21:50:57.146329+0000","last_became_peered":"2024-04-26T21:50:57.146329+0000","last_unstale":"2024-04-26T21:52:51.554589+0000","last_undegraded":"2024-04-26T21:52:51.554589+0000","last_fullsized":"2024-04-26T21:52:51.554589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:41.035526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422540+0000","last_change":"2024-04-26T21:50:56.370533+0000","last_active":"2024-04-26T21:52:53.422540+0000","last_peered":"2024-04-26T21:52:53.422540+0000","last_clean":"2024-04-26T21:52:53.422540+0000","last_became_active":"2024-04-26T21:50:56.370029+0000","last_became_peered":"2024-04-26T21:50:56.370029+0000","last_unstale":"2024-04-26T21:52:53.422540+0000","last_undegraded":"2024-04-26T21:52:53.422540+0000","last_fullsized":"2024-04-26T21:52:53.422540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:23.005405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554152+0000","last_change":"2024-04-26T21:50:57.142420+0000","last_active":"2024-04-26T21:52:51.554152+0000","last_peered":"2024-04-26T21:52:51.554152+0000","last_clean":"2024-04-26T21:52:51.554152+0000","last_became_active":"2024-04-26T21:50:57.142023+0000","last_became_peered":"2024-04-26T21:50:57.142023+0000","last_unstale":"2024-04-26T21:52:51.554152+0000","last_undegraded":"2024-04-26T21:52:51.554152+0000","last_fullsized":"2024-04-26T21:52:51.554152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:31.567136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587065+0000","last_change":"2024-04-26T21:50:57.139924+0000","last_active":"2024-04-26T21:52:52.587065+0000","last_peered":"2024-04-26T21:52:52.587065+0000","last_clean":"2024-04-26T21:52:52.587065+0000","last_became_active":"2024-04-26T21:50:57.139561+0000","last_became_peered":"2024-04-26T21:50:57.139561+0000","last_unstale":"2024-04-26T21:52:52.587065+0000","last_undegraded":"2024-04-26T21:52:52.587065+0000","last_fullsized":"2024-04-26T21:52:52.587065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:44:52.519668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:54.533579+0000","last_change":"2024-04-26T21:50:56.365256+0000","last_active":"2024-04-26T21:52:54.533579+0000","last_peered":"2024-04-26T21:52:54.533579+0000","last_clean":"2024-04-26T21:52:54.533579+0000","last_became_active":"2024-04-26T21:50:56.364516+0000","last_became_peered":"2024-04-26T21:50:56.364516+0000","last_unstale":"2024-04-26T21:52:54.533579+0000","last_undegraded":"2024-04-26T21:52:54.533579+0000","last_fullsized":"2024-04-26T21:52:54.533579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:13.550129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400745+0000","last_change":"2024-04-26T21:50:57.138177+0000","last_active":"2024-04-26T21:52:52.400745+0000","last_peered":"2024-04-26T21:52:52.400745+0000","last_clean":"2024-04-26T21:52:52.400745+0000","last_became_active":"2024-04-26T21:50:57.137725+0000","last_became_peered":"2024-04-26T21:50:57.137725+0000","last_unstale":"2024-04-26T21:52:52.400745+0000","last_undegraded":"2024-04-26T21:52:52.400745+0000","last_fullsized":"2024-04-26T21:52:52.400745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:14.113935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.118868+0000","last_change":"2024-04-26T21:50:56.358280+0000","last_active":"2024-04-26T21:52:52.118868+0000","last_peered":"2024-04-26T21:52:52.118868+0000","last_clean":"2024-04-26T21:52:52.118868+0000","last_became_active":"2024-04-26T21:50:56.357854+0000","last_became_peered":"2024-04-26T21:50:56.357854+0000","last_unstale":"2024-04-26T21:52:52.118868+0000","last_undegraded":"2024-04-26T21:52:52.118868+0000","last_fullsized":"2024-04-26T21:52:52.118868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:16:22.499124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422416+0000","last_change":"2024-04-26T21:49:42.292617+0000","last_active":"2024-04-26T21:52:53.422416+0000","last_peered":"2024-04-26T21:52:53.422416+0000","last_clean":"2024-04-26T21:52:53.422416+0000","last_became_active":"2024-04-26T21:49:42.292160+0000","last_became_peered":"2024-04-26T21:49:42.292160+0000","last_unstale":"2024-04-26T21:52:53.422416+0000","last_undegraded":"2024-04-26T21:52:53.422416+0000","last_fullsized":"2024-04-26T21:52:53.422416+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:46:13.787665+0000","last_clean_scrub_stamp":"2024-04-26T21:46:13.787665+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:24.663652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529051+0000","last_change":"2024-04-26T21:50:57.145641+0000","last_active":"2024-04-26T21:52:52.529051+0000","last_peered":"2024-04-26T21:52:52.529051+0000","last_clean":"2024-04-26T21:52:52.529051+0000","last_became_active":"2024-04-26T21:50:57.145304+0000","last_became_peered":"2024-04-26T21:50:57.145304+0000","last_unstale":"2024-04-26T21:52:52.529051+0000","last_undegraded":"2024-04-26T21:52:52.529051+0000","last_fullsized":"2024-04-26T21:52:52.529051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:11.389728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422632+0000","last_change":"2024-04-26T21:51:50.549862+0000","last_active":"2024-04-26T21:52:53.422632+0000","last_peered":"2024-04-26T21:52:53.422632+0000","last_clean":"2024-04-26T21:52:53.422632+0000","last_became_active":"2024-04-26T21:51:50.546054+0000","last_became_peered":"2024-04-26T21:51:50.546054+0000","last_unstale":"2024-04-26T21:52:53.422632+0000","last_undegraded":"2024-04-26T21:52:53.422632+0000","last_fullsized":"2024-04-26T21:52:53.422632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:45:33.817688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422621+0000","last_change":"2024-04-26T21:50:56.364074+0000","last_active":"2024-04-26T21:52:53.422621+0000","last_peered":"2024-04-26T21:52:53.422621+0000","last_clean":"2024-04-26T21:52:53.422621+0000","last_became_active":"2024-04-26T21:50:56.363688+0000","last_became_peered":"2024-04-26T21:50:56.363688+0000","last_unstale":"2024-04-26T21:52:53.422621+0000","last_undegraded":"2024-04-26T21:52:53.422621+0000","last_fullsized":"2024-04-26T21:52:53.422621+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:35.526125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529330+0000","last_change":"2024-04-26T21:50:57.136811+0000","last_active":"2024-04-26T21:52:52.529330+0000","last_peered":"2024-04-26T21:52:52.529330+0000","last_clean":"2024-04-26T21:52:52.529330+0000","last_became_active":"2024-04-26T21:50:57.136252+0000","last_became_peered":"2024-04-26T21:50:57.136252+0000","last_unstale":"2024-04-26T21:52:52.529330+0000","last_undegraded":"2024-04-26T21:52:52.529330+0000","last_fullsized":"2024-04-26T21:52:52.529330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:53:01.061283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.556684+0000","last_change":"2024-04-26T21:50:56.364317+0000","last_active":"2024-04-26T21:52:51.556684+0000","last_peered":"2024-04-26T21:52:51.556684+0000","last_clean":"2024-04-26T21:52:51.556684+0000","last_became_active":"2024-04-26T21:50:56.363882+0000","last_became_peered":"2024-04-26T21:50:56.363882+0000","last_unstale":"2024-04-26T21:52:51.556684+0000","last_undegraded":"2024-04-26T21:52:51.556684+0000","last_fullsized":"2024-04-26T21:52:51.556684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:12.994293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.401010+0000","last_change":"2024-04-26T21:50:57.144856+0000","last_active":"2024-04-26T21:52:52.401010+0000","last_peered":"2024-04-26T21:52:52.401010+0000","last_clean":"2024-04-26T21:52:52.401010+0000","last_became_active":"2024-04-26T21:50:57.143717+0000","last_became_peered":"2024-04-26T21:50:57.143717+0000","last_unstale":"2024-04-26T21:52:52.401010+0000","last_undegraded":"2024-04-26T21:52:52.401010+0000","last_fullsized":"2024-04-26T21:52:52.401010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:53.049243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.119199+0000","last_change":"2024-04-26T21:50:56.370284+0000","last_active":"2024-04-26T21:52:52.119199+0000","last_peered":"2024-04-26T21:52:52.119199+0000","last_clean":"2024-04-26T21:52:52.119199+0000","last_became_active":"2024-04-26T21:50:56.369738+0000","last_became_peered":"2024-04-26T21:50:56.369738+0000","last_unstale":"2024-04-26T21:52:52.119199+0000","last_undegraded":"2024-04-26T21:52:52.119199+0000","last_fullsized":"2024-04-26T21:52:52.119199+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:23.390553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422256+0000","last_change":"2024-04-26T21:50:57.142687+0000","last_active":"2024-04-26T21:52:53.422256+0000","last_peered":"2024-04-26T21:52:53.422256+0000","last_clean":"2024-04-26T21:52:53.422256+0000","last_became_active":"2024-04-26T21:50:57.142365+0000","last_became_peered":"2024-04-26T21:50:57.142365+0000","last_unstale":"2024-04-26T21:52:53.422256+0000","last_undegraded":"2024-04-26T21:52:53.422256+0000","last_fullsized":"2024-04-26T21:52:53.422256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:14.261741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.092789+0000","last_change":"2024-04-26T21:50:56.364163+0000","last_active":"2024-04-26T21:52:52.092789+0000","last_peered":"2024-04-26T21:52:52.092789+0000","last_clean":"2024-04-26T21:52:52.092789+0000","last_became_active":"2024-04-26T21:50:56.363729+0000","last_became_peered":"2024-04-26T21:50:56.363729+0000","last_unstale":"2024-04-26T21:52:52.092789+0000","last_undegraded":"2024-04-26T21:52:52.092789+0000","last_fullsized":"2024-04-26T21:52:52.092789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:34:54.620884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422328+0000","last_change":"2024-04-26T21:50:57.142692+0000","last_active":"2024-04-26T21:52:53.422328+0000","last_peered":"2024-04-26T21:52:53.422328+0000","last_clean":"2024-04-26T21:52:53.422328+0000","last_became_active":"2024-04-26T21:50:57.142137+0000","last_became_peered":"2024-04-26T21:50:57.142137+0000","last_unstale":"2024-04-26T21:52:53.422328+0000","last_undegraded":"2024-04-26T21:52:53.422328+0000","last_fullsized":"2024-04-26T21:52:53.422328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:47:15.996485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400523+0000","last_change":"2024-04-26T21:50:56.366374+0000","last_active":"2024-04-26T21:52:52.400523+0000","last_peered":"2024-04-26T21:52:52.400523+0000","last_clean":"2024-04-26T21:52:52.400523+0000","last_became_active":"2024-04-26T21:50:56.366014+0000","last_became_peered":"2024-04-26T21:50:56.366014+0000","last_unstale":"2024-04-26T21:52:52.400523+0000","last_undegraded":"2024-04-26T21:52:52.400523+0000","last_fullsized":"2024-04-26T21:52:52.400523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:41.867792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621428+0000","last_change":"2024-04-26T21:50:57.142833+0000","last_active":"2024-04-26T21:52:51.621428+0000","last_peered":"2024-04-26T21:52:51.621428+0000","last_clean":"2024-04-26T21:52:51.621428+0000","last_became_active":"2024-04-26T21:50:57.142061+0000","last_became_peered":"2024-04-26T21:50:57.142061+0000","last_unstale":"2024-04-26T21:52:51.621428+0000","last_undegraded":"2024-04-26T21:52:51.621428+0000","last_fullsized":"2024-04-26T21:52:51.621428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:33.023159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554453+0000","last_change":"2024-04-26T21:50:56.363172+0000","last_active":"2024-04-26T21:52:51.554453+0000","last_peered":"2024-04-26T21:52:51.554453+0000","last_clean":"2024-04-26T21:52:51.554453+0000","last_became_active":"2024-04-26T21:50:56.362782+0000","last_became_peered":"2024-04-26T21:50:56.362782+0000","last_unstale":"2024-04-26T21:52:51.554453+0000","last_undegraded":"2024-04-26T21:52:51.554453+0000","last_fullsized":"2024-04-26T21:52:51.554453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:21:37.183497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400740+0000","last_change":"2024-04-26T21:50:57.142088+0000","last_active":"2024-04-26T21:52:52.400740+0000","last_peered":"2024-04-26T21:52:52.400740+0000","last_clean":"2024-04-26T21:52:52.400740+0000","last_became_active":"2024-04-26T21:50:57.141685+0000","last_became_peered":"2024-04-26T21:50:57.141685+0000","last_unstale":"2024-04-26T21:52:52.400740+0000","last_undegraded":"2024-04-26T21:52:52.400740+0000","last_fullsized":"2024-04-26T21:52:52.400740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:24.033787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422419+0000","last_change":"2024-04-26T21:50:56.371717+0000","last_active":"2024-04-26T21:52:53.422419+0000","last_peered":"2024-04-26T21:52:53.422419+0000","last_clean":"2024-04-26T21:52:53.422419+0000","last_became_active":"2024-04-26T21:50:56.370930+0000","last_became_peered":"2024-04-26T21:50:56.370930+0000","last_unstale":"2024-04-26T21:52:53.422419+0000","last_undegraded":"2024-04-26T21:52:53.422419+0000","last_fullsized":"2024-04-26T21:52:53.422419+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:36.977392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422495+0000","last_change":"2024-04-26T21:50:57.139671+0000","last_active":"2024-04-26T21:52:53.422495+0000","last_peered":"2024-04-26T21:52:53.422495+0000","last_clean":"2024-04-26T21:52:53.422495+0000","last_became_active":"2024-04-26T21:50:57.139245+0000","last_became_peered":"2024-04-26T21:50:57.139245+0000","last_unstale":"2024-04-26T21:52:53.422495+0000","last_undegraded":"2024-04-26T21:52:53.422495+0000","last_fullsized":"2024-04-26T21:52:53.422495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:26.903905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554682+0000","last_change":"2024-04-26T21:50:56.369836+0000","last_active":"2024-04-26T21:52:51.554682+0000","last_peered":"2024-04-26T21:52:51.554682+0000","last_clean":"2024-04-26T21:52:51.554682+0000","last_became_active":"2024-04-26T21:50:56.369522+0000","last_became_peered":"2024-04-26T21:50:56.369522+0000","last_unstale":"2024-04-26T21:52:51.554682+0000","last_undegraded":"2024-04-26T21:52:51.554682+0000","last_fullsized":"2024-04-26T21:52:51.554682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:40.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422787+0000","last_change":"2024-04-26T21:50:57.138551+0000","last_active":"2024-04-26T21:52:53.422787+0000","last_peered":"2024-04-26T21:52:53.422787+0000","last_clean":"2024-04-26T21:52:53.422787+0000","last_became_active":"2024-04-26T21:50:57.138187+0000","last_became_peered":"2024-04-26T21:50:57.138187+0000","last_unstale":"2024-04-26T21:52:53.422787+0000","last_undegraded":"2024-04-26T21:52:53.422787+0000","last_fullsized":"2024-04-26T21:52:53.422787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:46.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.587331+0000","last_change":"2024-04-26T21:50:56.362009+0000","last_active":"2024-04-26T21:52:52.587331+0000","last_peered":"2024-04-26T21:52:52.587331+0000","last_clean":"2024-04-26T21:52:52.587331+0000","last_became_active":"2024-04-26T21:50:56.361646+0000","last_became_peered":"2024-04-26T21:50:56.361646+0000","last_unstale":"2024-04-26T21:52:52.587331+0000","last_undegraded":"2024-04-26T21:52:52.587331+0000","last_fullsized":"2024-04-26T21:52:52.587331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:25.316195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422529+0000","last_change":"2024-04-26T21:50:57.139983+0000","last_active":"2024-04-26T21:52:53.422529+0000","last_peered":"2024-04-26T21:52:53.422529+0000","last_clean":"2024-04-26T21:52:53.422529+0000","last_became_active":"2024-04-26T21:50:57.138426+0000","last_became_peered":"2024-04-26T21:50:57.138426+0000","last_unstale":"2024-04-26T21:52:53.422529+0000","last_undegraded":"2024-04-26T21:52:53.422529+0000","last_fullsized":"2024-04-26T21:52:53.422529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:53.079650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529527+0000","last_change":"2024-04-26T21:50:56.369527+0000","last_active":"2024-04-26T21:52:52.529527+0000","last_peered":"2024-04-26T21:52:52.529527+0000","last_clean":"2024-04-26T21:52:52.529527+0000","last_became_active":"2024-04-26T21:50:56.369168+0000","last_became_peered":"2024-04-26T21:50:56.369168+0000","last_unstale":"2024-04-26T21:52:52.529527+0000","last_undegraded":"2024-04-26T21:52:52.529527+0000","last_fullsized":"2024-04-26T21:52:52.529527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:55.342499+0000","last_clean_scrub_stamp":"2024-04-26T21:50:55.342499+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:38.724692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422889+0000","last_change":"2024-04-26T21:50:57.137957+0000","last_active":"2024-04-26T21:52:53.422889+0000","last_peered":"2024-04-26T21:52:53.422889+0000","last_clean":"2024-04-26T21:52:53.422889+0000","last_became_active":"2024-04-26T21:50:57.137595+0000","last_became_peered":"2024-04-26T21:50:57.137595+0000","last_unstale":"2024-04-26T21:52:53.422889+0000","last_undegraded":"2024-04-26T21:52:53.422889+0000","last_fullsized":"2024-04-26T21:52:53.422889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:09.355937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621530+0000","last_change":"2024-04-26T21:50:57.143098+0000","last_active":"2024-04-26T21:52:51.621530+0000","last_peered":"2024-04-26T21:52:51.621530+0000","last_clean":"2024-04-26T21:52:51.621530+0000","last_became_active":"2024-04-26T21:50:57.142524+0000","last_became_peered":"2024-04-26T21:50:57.142524+0000","last_unstale":"2024-04-26T21:52:51.621530+0000","last_undegraded":"2024-04-26T21:52:51.621530+0000","last_fullsized":"2024-04-26T21:52:51.621530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:33.333382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554379+0000","last_change":"2024-04-26T21:50:57.143123+0000","last_active":"2024-04-26T21:52:51.554379+0000","last_peered":"2024-04-26T21:52:51.554379+0000","last_clean":"2024-04-26T21:52:51.554379+0000","last_became_active":"2024-04-26T21:50:57.142805+0000","last_became_peered":"2024-04-26T21:50:57.142805+0000","last_unstale":"2024-04-26T21:52:51.554379+0000","last_undegraded":"2024-04-26T21:52:51.554379+0000","last_fullsized":"2024-04-26T21:52:51.554379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:19.330202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554320+0000","last_change":"2024-04-26T21:50:57.140268+0000","last_active":"2024-04-26T21:52:51.554320+0000","last_peered":"2024-04-26T21:52:51.554320+0000","last_clean":"2024-04-26T21:52:51.554320+0000","last_became_active":"2024-04-26T21:50:57.139927+0000","last_became_peered":"2024-04-26T21:50:57.139927+0000","last_unstale":"2024-04-26T21:52:51.554320+0000","last_undegraded":"2024-04-26T21:52:51.554320+0000","last_fullsized":"2024-04-26T21:52:51.554320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:32:01.509634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.421994+0000","last_change":"2024-04-26T21:50:57.143564+0000","last_active":"2024-04-26T21:52:53.421994+0000","last_peered":"2024-04-26T21:52:53.421994+0000","last_clean":"2024-04-26T21:52:53.421994+0000","last_became_active":"2024-04-26T21:50:57.142740+0000","last_became_peered":"2024-04-26T21:50:57.142740+0000","last_unstale":"2024-04-26T21:52:53.421994+0000","last_undegraded":"2024-04-26T21:52:53.421994+0000","last_fullsized":"2024-04-26T21:52:53.421994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:58.498980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529451+0000","last_change":"2024-04-26T21:50:57.146883+0000","last_active":"2024-04-26T21:52:52.529451+0000","last_peered":"2024-04-26T21:52:52.529451+0000","last_clean":"2024-04-26T21:52:52.529451+0000","last_became_active":"2024-04-26T21:50:57.146540+0000","last_became_peered":"2024-04-26T21:50:57.146540+0000","last_unstale":"2024-04-26T21:52:52.529451+0000","last_undegraded":"2024-04-26T21:52:52.529451+0000","last_fullsized":"2024-04-26T21:52:52.529451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:40:56.787618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621730+0000","last_change":"2024-04-26T21:50:57.136938+0000","last_active":"2024-04-26T21:52:51.621730+0000","last_peered":"2024-04-26T21:52:51.621730+0000","last_clean":"2024-04-26T21:52:51.621730+0000","last_became_active":"2024-04-26T21:50:57.136474+0000","last_became_peered":"2024-04-26T21:50:57.136474+0000","last_unstale":"2024-04-26T21:52:51.621730+0000","last_undegraded":"2024-04-26T21:52:51.621730+0000","last_fullsized":"2024-04-26T21:52:51.621730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:53:42.534342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.554864+0000","last_change":"2024-04-26T21:50:57.150958+0000","last_active":"2024-04-26T21:52:51.554864+0000","last_peered":"2024-04-26T21:52:51.554864+0000","last_clean":"2024-04-26T21:52:51.554864+0000","last_became_active":"2024-04-26T21:50:57.150593+0000","last_became_peered":"2024-04-26T21:50:57.150593+0000","last_unstale":"2024-04-26T21:52:51.554864+0000","last_undegraded":"2024-04-26T21:52:51.554864+0000","last_fullsized":"2024-04-26T21:52:51.554864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:26.583983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400991+0000","last_change":"2024-04-26T21:50:57.147766+0000","last_active":"2024-04-26T21:52:52.400991+0000","last_peered":"2024-04-26T21:52:52.400991+0000","last_clean":"2024-04-26T21:52:52.400991+0000","last_became_active":"2024-04-26T21:50:57.147442+0000","last_became_peered":"2024-04-26T21:50:57.147442+0000","last_unstale":"2024-04-26T21:52:52.400991+0000","last_undegraded":"2024-04-26T21:52:52.400991+0000","last_fullsized":"2024-04-26T21:52:52.400991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.489117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:53.422382+0000","last_change":"2024-04-26T21:51:50.595554+0000","last_active":"2024-04-26T21:52:53.422382+0000","last_peered":"2024-04-26T21:52:53.422382+0000","last_clean":"2024-04-26T21:52:53.422382+0000","last_became_active":"2024-04-26T21:51:50.595248+0000","last_became_peered":"2024-04-26T21:51:50.595248+0000","last_unstale":"2024-04-26T21:52:53.422382+0000","last_undegraded":"2024-04-26T21:52:53.422382+0000","last_fullsized":"2024-04-26T21:52:53.422382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:36.853146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,3],"acting":[7,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.400448+0000","last_change":"2024-04-26T21:50:57.142804+0000","last_active":"2024-04-26T21:52:52.400448+0000","last_peered":"2024-04-26T21:52:52.400448+0000","last_clean":"2024-04-26T21:52:52.400448+0000","last_became_active":"2024-04-26T21:50:57.141902+0000","last_became_peered":"2024-04-26T21:50:57.141902+0000","last_unstale":"2024-04-26T21:52:52.400448+0000","last_undegraded":"2024-04-26T21:52:52.400448+0000","last_fullsized":"2024-04-26T21:52:52.400448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:01:37.590730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.093034+0000","last_change":"2024-04-26T21:50:57.138637+0000","last_active":"2024-04-26T21:52:52.093034+0000","last_peered":"2024-04-26T21:52:52.093034+0000","last_clean":"2024-04-26T21:52:52.093034+0000","last_became_active":"2024-04-26T21:50:57.138295+0000","last_became_peered":"2024-04-26T21:50:57.138295+0000","last_unstale":"2024-04-26T21:52:52.093034+0000","last_undegraded":"2024-04-26T21:52:52.093034+0000","last_fullsized":"2024-04-26T21:52:52.093034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:19.323645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:52.529384+0000","last_change":"2024-04-26T21:50:57.137377+0000","last_active":"2024-04-26T21:52:52.529384+0000","last_peered":"2024-04-26T21:52:52.529384+0000","last_clean":"2024-04-26T21:52:52.529384+0000","last_became_active":"2024-04-26T21:50:57.136893+0000","last_became_peered":"2024-04-26T21:50:57.136893+0000","last_unstale":"2024-04-26T21:52:52.529384+0000","last_undegraded":"2024-04-26T21:52:52.529384+0000","last_fullsized":"2024-04-26T21:52:52.529384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:46:44.257438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T21:52:51.621602+0000","last_change":"2024-04-26T21:50:57.141439+0000","last_active":"2024-04-26T21:52:51.621602+0000","last_peered":"2024-04-26T21:52:51.621602+0000","last_clean":"2024-04-26T21:52:51.621602+0000","last_became_active":"2024-04-26T21:50:57.141060+0000","last_became_peered":"2024-04-26T21:50:57.141060+0000","last_unstale":"2024-04-26T21:52:51.621602+0000","last_undegraded":"2024-04-26T21:52:51.621602+0000","last_fullsized":"2024-04-26T21:52:51.621602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T21:50:56.344113+0000","last_clean_scrub_stamp":"2024-04-26T21:50:56.344113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:27.174348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":250381,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54094,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":659,"num_read_kb":410,"num_write":160,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":161666,"data_compressed":5562,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":474342,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1101824,"data_stored":1114761,"data_compressed":49838,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":473648,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":473568,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1065806,"data_compressed":47984,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495680,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":473037,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":522307,"data_compressed":13369,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":474308,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":523545,"data_compressed":13369,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":464842,"data_compressed":8628,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":473977,"data_compressed":11515,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1064169,"data_compressed":47984,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34835,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36469,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":49785,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":50715,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":51754,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:52:58.515 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:52:58.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 -- 172.21.15.55:0/411910801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124102a60 msgr2=0x7f2124102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 --2- 172.21.15.55:0/411910801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124102a60 0x7f2124102ee0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2114009900 tx=0x7f211402a410 comp rx=0 tx=0).stop 2024-04-26T21:52:58.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 -- 172.21.15.55:0/411910801 shutdown_connections 2024-04-26T21:52:58.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 --2- 172.21.15.55:0/411910801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2124103420 0x7f2124109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 --2- 172.21.15.55:0/411910801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124102a60 0x7f2124102ee0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 --2- 172.21.15.55:0/411910801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124101870 0x7f2124101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 -- 172.21.15.55:0/411910801 >> 172.21.15.55:0/411910801 conn(0x7f21240fcde0 msgr2=0x7f21240ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:58.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 -- 172.21.15.55:0/411910801 shutdown_connections 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 -- 172.21.15.55:0/411910801 wait complete. 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 Processor -- start 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.653+0000 7f212bd4e640 1 -- start start 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212bd4e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124101870 0x7f2124198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212bd4e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2124102a60 0x7f2124199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212bd4e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124103420 0x7f21241a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212bd4e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f212410d3a0 con 0x7f2124103420 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212bd4e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f212410d220 con 0x7f2124102a60 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212bd4e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f212410d520 con 0x7f2124101870 2024-04-26T21:52:58.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124103420 0x7f21241a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124103420 0x7f21241a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45652/0 (socket says 172.21.15.55:45652) 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 -- 172.21.15.55:0/4185802098 learned_addr learned my addr 172.21.15.55:0/4185802098 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f21292e9640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2124102a60 0x7f2124199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f2129aea640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124101870 0x7f2124198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 -- 172.21.15.55:0/4185802098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124101870 msgr2=0x7f2124198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124101870 0x7f2124198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 -- 172.21.15.55:0/4185802098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2124102a60 msgr2=0x7f2124199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2124102a60 0x7f2124199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 -- 172.21.15.55:0/4185802098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21241a0860 con 0x7f2124103420 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212a2eb640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124103420 0x7f21241a01c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f211c00efd0 tx=0x7f211c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f211affd640 1 -- 172.21.15.55:0/4185802098 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c003b70 con 0x7f2124103420 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21241a0b50 con 0x7f2124103420 2024-04-26T21:52:58.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f211affd640 1 -- 172.21.15.55:0/4185802098 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f211c003d10 con 0x7f2124103420 2024-04-26T21:52:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.654+0000 7f211affd640 1 -- 172.21.15.55:0/4185802098 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0108a0 con 0x7f2124103420 2024-04-26T21:52:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.655+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21241a1050 con 0x7f2124103420 2024-04-26T21:52:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.656+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21241a1220 con 0x7f2124103420 2024-04-26T21:52:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.657+0000 7f211affd640 1 -- 172.21.15.55:0/4185802098 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f211c004020 con 0x7f2124103420 2024-04-26T21:52:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.657+0000 7f211affd640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f21100a3c40 0x7f21100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.657+0000 7f2129aea640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f21100a3c40 0x7f21100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.657+0000 7f211affd640 1 -- 172.21.15.55:0/4185802098 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f211c0d0ae0 con 0x7f2124103420 2024-04-26T21:52:58.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.658+0000 7f2129aea640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f21100a3c40 0x7f21100a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f212419a120 tx=0x7f2120005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:58.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.662+0000 7f211affd640 1 -- 172.21.15.55:0/4185802098 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f211c09d320 con 0x7f2124103420 2024-04-26T21:52:58.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.851+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21241a1490 con 0x7f2124103420 2024-04-26T21:52:58.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.852+0000 7f211affd640 1 -- 172.21.15.55:0/4185802098 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22760 (secure 0 0 0) 0x7f211c09cc20 con 0x7f2124103420 2024-04-26T21:52:58.853 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:52:58.853 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:52:51.546766+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:52:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.854+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f21100a3c40 msgr2=0x7f21100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.854+0000 7f212bd4e640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f21100a3c40 0x7f21100a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f212419a120 tx=0x7f2120005f90 comp rx=0 tx=0).stop 2024-04-26T21:52:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.854+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124103420 msgr2=0x7f21241a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.854+0000 7f212bd4e640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124103420 0x7f21241a01c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f211c00efd0 tx=0x7f211c00c6a0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.854+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 shutdown_connections 2024-04-26T21:52:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.854+0000 7f212bd4e640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f21100a3c40 0x7f21100a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.854+0000 7f212bd4e640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2124103420 0x7f21241a01c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.855+0000 7f212bd4e640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2124102a60 0x7f2124199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.855+0000 7f212bd4e640 1 --2- 172.21.15.55:0/4185802098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124101870 0x7f2124198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.855+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 >> 172.21.15.55:0/4185802098 conn(0x7f21240fcde0 msgr2=0x7f2124069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.855+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 shutdown_connections 2024-04-26T21:52:58.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.855+0000 7f212bd4e640 1 -- 172.21.15.55:0/4185802098 wait complete. 2024-04-26T21:52:58.866 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:52:58.867 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:52:58.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 -- 172.21.15.55:0/758884508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0101890 msgr2=0x7fcce0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 --2- 172.21.15.55:0/758884508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0101890 0x7fcce0101cb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009900 tx=0x7fccd002a410 comp rx=0 tx=0).stop 2024-04-26T21:52:58.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 -- 172.21.15.55:0/758884508 shutdown_connections 2024-04-26T21:52:58.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 --2- 172.21.15.55:0/758884508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcce0103440 0x7fcce0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 --2- 172.21.15.55:0/758884508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce0102a80 0x7fcce0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 --2- 172.21.15.55:0/758884508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0101890 0x7fcce0101cb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 -- 172.21.15.55:0/758884508 >> 172.21.15.55:0/758884508 conn(0x7fcce00fce20 msgr2=0x7fcce00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:58.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 -- 172.21.15.55:0/758884508 shutdown_connections 2024-04-26T21:52:58.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.972+0000 7fcce4938640 1 -- 172.21.15.55:0/758884508 wait complete. 2024-04-26T21:52:58.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fcce4938640 1 Processor -- start 2024-04-26T21:52:58.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fcce4938640 1 -- start start 2024-04-26T21:52:58.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fcce4938640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce0101890 0x7fcce019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fcce4938640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcce0102a80 0x7fcce019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fcce4938640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0103440 0x7fcce01a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fcce4938640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcce010d3b0 con 0x7fcce0103440 2024-04-26T21:52:58.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fcce4938640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcce010d230 con 0x7fcce0102a80 2024-04-26T21:52:58.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fcce4938640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcce010d530 con 0x7fcce0101890 2024-04-26T21:52:58.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0103440 0x7fcce01a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0103440 0x7fcce01a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45662/0 (socket says 172.21.15.55:45662) 2024-04-26T21:52:58.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 -- 172.21.15.55:0/806834038 learned_addr learned my addr 172.21.15.55:0/806834038 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:58.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccde59c640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce0101890 0x7fcce019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccddd9b640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcce0102a80 0x7fcce019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 -- 172.21.15.55:0/806834038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce0101890 msgr2=0x7fcce019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce0101890 0x7fcce019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 -- 172.21.15.55:0/806834038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcce0102a80 msgr2=0x7fcce019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcce0102a80 0x7fcce019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 -- 172.21.15.55:0/806834038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce01a2a40 con 0x7fcce0103440 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.973+0000 7fccded9d640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0103440 0x7fcce01a23d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fccd800cad0 tx=0x7fccd800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.974+0000 7fcccf7fe640 1 -- 172.21.15.55:0/806834038 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd800ae20 con 0x7fcce0103440 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.974+0000 7fcccf7fe640 1 -- 172.21.15.55:0/806834038 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd8005020 con 0x7fcce0103440 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.974+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce01a2d30 con 0x7fcce0103440 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.974+0000 7fcccf7fe640 1 -- 172.21.15.55:0/806834038 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd800f6b0 con 0x7fcce0103440 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.974+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce01a3220 con 0x7fcce0103440 2024-04-26T21:52:58.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.975+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce01a34b0 con 0x7fcce0103440 2024-04-26T21:52:58.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.976+0000 7fcccf7fe640 1 -- 172.21.15.55:0/806834038 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fccd80051d0 con 0x7fcce0103440 2024-04-26T21:52:58.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.976+0000 7fcccf7fe640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccc80a3cc0 0x7fccc80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:58.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.977+0000 7fccde59c640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccc80a3cc0 0x7fccc80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:58.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.977+0000 7fcccf7fe640 1 -- 172.21.15.55:0/806834038 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fccd80d0610 con 0x7fcce0103440 2024-04-26T21:52:58.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.977+0000 7fccde59c640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccc80a3cc0 0x7fccc80a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009230 tx=0x7fccd00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:58.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:58.981+0000 7fcccf7fe640 1 -- 172.21.15.55:0/806834038 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fccd8014030 con 0x7fcce0103440 2024-04-26T21:52:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:58 smithi165 ceph-mon[33296]: pgmap v353: 97 pgs: 97 active+clean; 630 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T21:52:59.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:58 smithi165 ceph-mon[33296]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:59.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:58 smithi165 ceph-mon[33296]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:59.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.165+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcce01a3810 con 0x7fcce0103440 2024-04-26T21:52:59.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.165+0000 7fcccf7fe640 1 -- 172.21.15.55:0/806834038 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fccd809ce50 con 0x7fcce0103440 2024-04-26T21:52:59.166 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T21:52:59.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.167+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccc80a3cc0 msgr2=0x7fccc80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.167+0000 7fcce4938640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccc80a3cc0 0x7fccc80a6180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009230 tx=0x7fccd00314a0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.167+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0103440 msgr2=0x7fcce01a23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0103440 0x7fcce01a23d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fccd800cad0 tx=0x7fccd800cf90 comp rx=0 tx=0).stop 2024-04-26T21:52:59.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 shutdown_connections 2024-04-26T21:52:59.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccc80a3cc0 0x7fccc80a6180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcce0103440 0x7fcce01a23d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcce0102a80 0x7fcce019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 --2- 172.21.15.55:0/806834038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce0101890 0x7fcce019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 >> 172.21.15.55:0/806834038 conn(0x7fcce00fce20 msgr2=0x7fcce006af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 shutdown_connections 2024-04-26T21:52:59.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.168+0000 7fcce4938640 1 -- 172.21.15.55:0/806834038 wait complete. 2024-04-26T21:52:59.178 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:58 smithi055 ceph-mon[29221]: pgmap v353: 97 pgs: 97 active+clean; 630 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T21:52:59.178 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:58 smithi055 ceph-mon[29221]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:59.178 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:58 smithi055 ceph-mon[29221]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:59.179 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:52:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:58 smithi064 ceph-mon[32765]: pgmap v353: 97 pgs: 97 active+clean; 630 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-26T21:52:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:58 smithi064 ceph-mon[32765]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:58 smithi064 ceph-mon[32765]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:52:59.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 -- 172.21.15.55:0/1210098404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0108670 msgr2=0x7fa2a010aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/1210098404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0108670 0x7fa2a010aac0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa298009ca0 tx=0x7fa29802a8c0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 -- 172.21.15.55:0/1210098404 shutdown_connections 2024-04-26T21:52:59.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/1210098404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0108670 0x7fa2a010aac0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/1210098404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2a0105d40 0x7fa2a0108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/1210098404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0103410 0x7fa2a0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 -- 172.21.15.55:0/1210098404 >> 172.21.15.55:0/1210098404 conn(0x7fa2a00fce20 msgr2=0x7fa2a00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 -- 172.21.15.55:0/1210098404 shutdown_connections 2024-04-26T21:52:59.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 -- 172.21.15.55:0/1210098404 wait complete. 2024-04-26T21:52:59.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 Processor -- start 2024-04-26T21:52:59.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 -- start start 2024-04-26T21:52:59.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.284+0000 7fa2a66bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0103410 0x7fa2a0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a66bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0105d40 0x7fa2a0199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a66bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2a0108670 0x7fa2a01a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a66bd640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a010e510 con 0x7fa2a0103410 2024-04-26T21:52:59.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a66bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a010e390 con 0x7fa2a0105d40 2024-04-26T21:52:59.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a66bd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a010e690 con 0x7fa2a0108670 2024-04-26T21:52:59.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0103410 0x7fa2a0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0103410 0x7fa2a0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45686/0 (socket says 172.21.15.55:45686) 2024-04-26T21:52:59.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 -- 172.21.15.55:0/328303511 learned_addr learned my addr 172.21.15.55:0/328303511 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:59.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29f7fe640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0105d40 0x7fa2a0199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a4c5a640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2a0108670 0x7fa2a01a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 -- 172.21.15.55:0/328303511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2a0108670 msgr2=0x7fa2a01a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2a0108670 0x7fa2a01a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 -- 172.21.15.55:0/328303511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0105d40 msgr2=0x7fa2a0199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0105d40 0x7fa2a0199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 -- 172.21.15.55:0/328303511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01a0720 con 0x7fa2a0103410 2024-04-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a4c5a640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2a0108670 0x7fa2a01a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29ffff640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0103410 0x7fa2a0198b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa29000c920 tx=0x7fa29000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29d7fa640 1 -- 172.21.15.55:0/328303511 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290007c70 con 0x7fa2a0103410 2024-04-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29d7fa640 1 -- 172.21.15.55:0/328303511 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa290007e10 con 0x7fa2a0103410 2024-04-26T21:52:59.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a01a0a10 con 0x7fa2a0103410 2024-04-26T21:52:59.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa29d7fa640 1 -- 172.21.15.55:0/328303511 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290018820 con 0x7fa2a0103410 2024-04-26T21:52:59.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.285+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01a0f60 con 0x7fa2a0103410 2024-04-26T21:52:59.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.286+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a00fe910 con 0x7fa2a0103410 2024-04-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.288+0000 7fa29d7fa640 1 -- 172.21.15.55:0/328303511 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa29000f450 con 0x7fa2a0103410 2024-04-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.288+0000 7fa29d7fa640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2880a3c40 0x7fa2880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.288+0000 7fa29f7fe640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2880a3c40 0x7fa2880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.288+0000 7fa29d7fa640 1 -- 172.21.15.55:0/328303511 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa2900d09b0 con 0x7fa2a0103410 2024-04-26T21:52:59.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.289+0000 7fa29f7fe640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2880a3c40 0x7fa2880a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2a019a010 tx=0x7fa29400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.292+0000 7fa29d7fa640 1 -- 172.21.15.55:0/328303511 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa290014030 con 0x7fa2a0103410 2024-04-26T21:52:59.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.478+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa2a0063530 con 0x7fa2a0103410 2024-04-26T21:52:59.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.479+0000 7fa29d7fa640 1 -- 172.21.15.55:0/328303511 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fa29009d280 con 0x7fa2a0103410 2024-04-26T21:52:59.479 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T21:52:59.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2880a3c40 msgr2=0x7fa2880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2880a3c40 0x7fa2880a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2a019a010 tx=0x7fa29400b410 comp rx=0 tx=0).stop 2024-04-26T21:52:59.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0103410 msgr2=0x7fa2a0198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0103410 0x7fa2a0198b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa29000c920 tx=0x7fa29000cde0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 shutdown_connections 2024-04-26T21:52:59.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2880a3c40 0x7fa2880a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2a0108670 0x7fa2a01a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2a0105d40 0x7fa2a0199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 --2- 172.21.15.55:0/328303511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2a0103410 0x7fa2a0198b00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 >> 172.21.15.55:0/328303511 conn(0x7fa2a00fce20 msgr2=0x7fa2a0103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 shutdown_connections 2024-04-26T21:52:59.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.481+0000 7fa2a66bd640 1 -- 172.21.15.55:0/328303511 wait complete. 2024-04-26T21:52:59.492 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:52:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- 172.21.15.55:0/1372364954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c102a80 msgr2=0x7ff45c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 --2- 172.21.15.55:0/1372364954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c102a80 0x7ff45c102f00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009900 tx=0x7ff44c02a410 comp rx=0 tx=0).stop 2024-04-26T21:52:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- 172.21.15.55:0/1372364954 shutdown_connections 2024-04-26T21:52:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 --2- 172.21.15.55:0/1372364954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff45c103440 0x7ff45c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 --2- 172.21.15.55:0/1372364954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c102a80 0x7ff45c102f00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 --2- 172.21.15.55:0/1372364954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c101890 0x7ff45c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- 172.21.15.55:0/1372364954 >> 172.21.15.55:0/1372364954 conn(0x7ff45c0fce20 msgr2=0x7ff45c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- 172.21.15.55:0/1372364954 shutdown_connections 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- 172.21.15.55:0/1372364954 wait complete. 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 Processor -- start 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- start start 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff45c101890 0x7ff45c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c102a80 0x7ff45c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c103440 0x7ff45c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10e540 con 0x7ff45c103440 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10e3c0 con 0x7ff45c102a80 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.597+0000 7ff464272640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10e6c0 con 0x7ff45c101890 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c103440 0x7ff45c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c103440 0x7ff45c1a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45708/0 (socket says 172.21.15.55:45708) 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 -- 172.21.15.55:0/887926012 learned_addr learned my addr 172.21.15.55:0/887926012 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46200e640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff45c101890 0x7ff45c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46180d640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c102a80 0x7ff45c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 -- 172.21.15.55:0/887926012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff45c101890 msgr2=0x7ff45c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff45c101890 0x7ff45c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 -- 172.21.15.55:0/887926012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c102a80 msgr2=0x7ff45c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c102a80 0x7ff45c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 -- 172.21.15.55:0/887926012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c1a07e0 con 0x7ff45c103440 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff46280f640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c103440 0x7ff45c1a0170 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff45400eaa0 tx=0x7ff45400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff4537fe640 1 -- 172.21.15.55:0/887926012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45400cdb0 con 0x7ff45c103440 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c1a0ad0 con 0x7ff45c103440 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff4537fe640 1 -- 172.21.15.55:0/887926012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff454005020 con 0x7ff45c103440 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff4537fe640 1 -- 172.21.15.55:0/887926012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4540116d0 con 0x7ff45c103440 2024-04-26T21:52:59.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.598+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a1040 con 0x7ff45c103440 2024-04-26T21:52:59.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.599+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c108c80 con 0x7ff45c103440 2024-04-26T21:52:59.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.600+0000 7ff4537fe640 1 -- 172.21.15.55:0/887926012 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff4540054a0 con 0x7ff45c103440 2024-04-26T21:52:59.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.601+0000 7ff4537fe640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4480a3c40 0x7ff4480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.601+0000 7ff46200e640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4480a3c40 0x7ff4480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.601+0000 7ff4537fe640 1 -- 172.21.15.55:0/887926012 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff4540d0800 con 0x7ff45c103440 2024-04-26T21:52:59.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.602+0000 7ff46200e640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4480a3c40 0x7ff4480a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff458005db0 tx=0x7ff458005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.606+0000 7ff4537fe640 1 -- 172.21.15.55:0/887926012 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff454016030 con 0x7ff45c103440 2024-04-26T21:52:59.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.792+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff45c02dca0 con 0x7ff45c103440 2024-04-26T21:52:59.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.793+0000 7ff4537fe640 1 -- 172.21.15.55:0/887926012 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7ff45409d040 con 0x7ff45c103440 2024-04-26T21:52:59.793 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T21:52:59.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4480a3c40 msgr2=0x7ff4480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4480a3c40 0x7ff4480a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff458005db0 tx=0x7ff458005d40 comp rx=0 tx=0).stop 2024-04-26T21:52:59.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c103440 msgr2=0x7ff45c1a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c103440 0x7ff45c1a0170 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff45400eaa0 tx=0x7ff45400ef60 comp rx=0 tx=0).stop 2024-04-26T21:52:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 shutdown_connections 2024-04-26T21:52:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff4480a3c40 0x7ff4480a6100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff45c103440 0x7ff45c1a0170 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c102a80 0x7ff45c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 --2- 172.21.15.55:0/887926012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff45c101890 0x7ff45c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 >> 172.21.15.55:0/887926012 conn(0x7ff45c0fce20 msgr2=0x7ff45c105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 shutdown_connections 2024-04-26T21:52:59.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.795+0000 7ff464272640 1 -- 172.21.15.55:0/887926012 wait complete. 2024-04-26T21:52:59.807 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:52:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 -- 172.21.15.55:0/3087942789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 msgr2=0x7f3048103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3087942789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 0x7f3048103ff0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3038009900 tx=0x7f303802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 -- 172.21.15.55:0/3087942789 shutdown_connections 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3087942789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3048104530 0x7f304810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3087942789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 0x7f3048103ff0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3087942789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3048102980 0x7f3048102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 -- 172.21.15.55:0/3087942789 >> 172.21.15.55:0/3087942789 conn(0x7f30480fdf10 msgr2=0x7f3048100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 -- 172.21.15.55:0/3087942789 shutdown_connections 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.908+0000 7f304f1b4640 1 -- 172.21.15.55:0/3087942789 wait complete. 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304f1b4640 1 Processor -- start 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304f1b4640 1 -- start start 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304f1b4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3048102980 0x7f3048198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304f1b4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 0x7f30481990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304f1b4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3048104530 0x7f30481a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304f1b4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f304810e4b0 con 0x7f3048103b70 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304f1b4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f304810e330 con 0x7f3048102980 2024-04-26T21:52:59.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304f1b4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f304810e630 con 0x7f3048104530 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 0x7f30481990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 0x7f30481990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45728/0 (socket says 172.21.15.55:45728) 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 -- 172.21.15.55:0/3726654095 learned_addr learned my addr 172.21.15.55:0/3726654095 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304cf50640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3048102980 0x7f3048198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f304d751640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3048104530 0x7f30481a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 -- 172.21.15.55:0/3726654095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3048104530 msgr2=0x7f30481a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3048104530 0x7f30481a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 -- 172.21.15.55:0/3726654095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3048102980 msgr2=0x7f3048198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3048102980 0x7f3048198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 -- 172.21.15.55:0/3726654095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30481a0780 con 0x7f3048103b70 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.909+0000 7f3047fff640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 0x7f30481990a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3038031500 tx=0x7f30380043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.910+0000 7f3045ffb640 1 -- 172.21.15.55:0/3726654095 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303803f030 con 0x7f3048103b70 2024-04-26T21:52:59.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.910+0000 7f3045ffb640 1 -- 172.21.15.55:0/3726654095 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3038003800 con 0x7f3048103b70 2024-04-26T21:52:59.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.910+0000 7f3045ffb640 1 -- 172.21.15.55:0/3726654095 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3038035070 con 0x7f3048103b70 2024-04-26T21:52:59.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.910+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30481a0a10 con 0x7f3048103b70 2024-04-26T21:52:59.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.910+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30481a0f00 con 0x7f3048103b70 2024-04-26T21:52:59.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.911+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3048109d20 con 0x7f3048103b70 2024-04-26T21:52:59.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.912+0000 7f3045ffb640 1 -- 172.21.15.55:0/3726654095 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f30380039b0 con 0x7f3048103b70 2024-04-26T21:52:59.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.913+0000 7f3045ffb640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30340a3c40 0x7f30340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:52:59.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.913+0000 7f304cf50640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30340a3c40 0x7f30340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:52:59.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.913+0000 7f3045ffb640 1 -- 172.21.15.55:0/3726654095 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f30380f1010 con 0x7f3048103b70 2024-04-26T21:52:59.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.913+0000 7f304cf50640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30340a3c40 0x7f30340a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f30481039d0 tx=0x7f3030005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:52:59.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:52:59.917+0000 7f3045ffb640 1 -- 172.21.15.55:0/3726654095 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f30380bd850 con 0x7f3048103b70 2024-04-26T21:53:00.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:59 smithi165 ceph-mon[33296]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:00.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4185802098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:00.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/806834038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/328303511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:52:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/887926012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.144+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f304802dcf0 con 0x7f3048103b70 2024-04-26T21:53:00.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.145+0000 7f3045ffb640 1 -- 172.21.15.55:0/3726654095 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11863 (secure 0 0 0) 0x7f30380bd150 con 0x7f3048103b70 2024-04-26T21:53:00.145 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:53:00.145 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:52:54.328649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24502,24509]},"id":1}]} 2024-04-26T21:53:00.146 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 21 2024-04-26T21:53:00.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30340a3c40 msgr2=0x7f30340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30340a3c40 0x7f30340a6100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f30481039d0 tx=0x7f3030005f90 comp rx=0 tx=0).stop 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 msgr2=0x7f30481990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 0x7f30481990a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3038031500 tx=0x7f30380043c0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 shutdown_connections 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30340a3c40 0x7f30340a6100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3048104530 0x7f30481a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3048103b70 0x7f30481990a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 --2- 172.21.15.55:0/3726654095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3048102980 0x7f3048198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 >> 172.21.15.55:0/3726654095 conn(0x7f30480fdf10 msgr2=0x7f3048106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 shutdown_connections 2024-04-26T21:53:00.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.147+0000 7f304f1b4640 1 -- 172.21.15.55:0/3726654095 wait complete. 2024-04-26T21:53:00.157 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:59 smithi055 ceph-mon[29221]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:00.157 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4185802098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:00.158 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/806834038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.158 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/328303511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.158 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:52:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/887926012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.159 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:53:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:59 smithi064 ceph-mon[32765]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T21:53:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4185802098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/806834038' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/328303511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:52:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/887926012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:53:00.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 -- 172.21.15.55:0/776316343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc41086b0 msgr2=0x7f5cc410aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 --2- 172.21.15.55:0/776316343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc41086b0 0x7f5cc410aaa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009ca0 tx=0x7f5cbc02a8c0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 -- 172.21.15.55:0/776316343 shutdown_connections 2024-04-26T21:53:00.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 --2- 172.21.15.55:0/776316343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc41086b0 0x7f5cc410aaa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 --2- 172.21.15.55:0/776316343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4105d80 0x7f5cc4108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 --2- 172.21.15.55:0/776316343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc4103450 0x7f5cc4105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 -- 172.21.15.55:0/776316343 >> 172.21.15.55:0/776316343 conn(0x7f5cc40fcde0 msgr2=0x7f5cc40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 -- 172.21.15.55:0/776316343 shutdown_connections 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.260+0000 7f5cca26d640 1 -- 172.21.15.55:0/776316343 wait complete. 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cca26d640 1 Processor -- start 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cca26d640 1 -- start start 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cca26d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc4103450 0x7f5cc4198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cca26d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4105d80 0x7f5cc4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cca26d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc41086b0 0x7f5cc41a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cca26d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410e4c0 con 0x7f5cc4103450 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cca26d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410e340 con 0x7f5cc4105d80 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cca26d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc410e640 con 0x7f5cc41086b0 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc4103450 0x7f5cc4198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.261+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc4103450 0x7f5cc4198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45736/0 (socket says 172.21.15.55:45736) 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc3fff640 1 -- 172.21.15.55:0/3650203204 learned_addr learned my addr 172.21.15.55:0/3650203204 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc37fe640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4105d80 0x7f5cc4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc880a640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc41086b0 0x7f5cc41a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc3fff640 1 -- 172.21.15.55:0/3650203204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc41086b0 msgr2=0x7f5cc41a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc3fff640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc41086b0 0x7f5cc41a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc3fff640 1 -- 172.21.15.55:0/3650203204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4105d80 msgr2=0x7f5cc4199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc3fff640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4105d80 0x7f5cc4199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc3fff640 1 -- 172.21.15.55:0/3650203204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc41a07a0 con 0x7f5cc4103450 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc3fff640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc4103450 0x7f5cc4198bd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400c970 tx=0x7f5cb400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc17fa640 1 -- 172.21.15.55:0/3650203204 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4007bb0 con 0x7f5cc4103450 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc41a0a90 con 0x7f5cc4103450 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc17fa640 1 -- 172.21.15.55:0/3650203204 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb4007d50 con 0x7f5cc4103450 2024-04-26T21:53:00.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cc17fa640 1 -- 172.21.15.55:0/3650203204 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400f450 con 0x7f5cc4103450 2024-04-26T21:53:00.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.262+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41a0f90 con 0x7f5cc4103450 2024-04-26T21:53:00.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.263+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc40fe8d0 con 0x7f5cc4103450 2024-04-26T21:53:00.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.265+0000 7f5cc17fa640 1 -- 172.21.15.55:0/3650203204 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5cb4004f20 con 0x7f5cc4103450 2024-04-26T21:53:00.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.265+0000 7f5cc17fa640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5cac0a3c40 0x7f5cac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.265+0000 7f5cc37fe640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5cac0a3c40 0x7f5cac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.265+0000 7f5cc17fa640 1 -- 172.21.15.55:0/3650203204 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5cb40d0340 con 0x7f5cc4103450 2024-04-26T21:53:00.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.266+0000 7f5cc37fe640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5cac0a3c40 0x7f5cac0a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5cc419a090 tx=0x7f5cb800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.270+0000 7f5cc17fa640 1 -- 172.21.15.55:0/3650203204 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5cb4014030 con 0x7f5cc4103450 2024-04-26T21:53:00.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.459+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cc40635d0 con 0x7f5cc4103450 2024-04-26T21:53:00.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.460+0000 7f5cc17fa640 1 -- 172.21.15.55:0/3650203204 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22760 (secure 0 0 0) 0x7f5cb409cb80 con 0x7f5cc4103450 2024-04-26T21:53:00.461 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:53:00.461 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:52:51.546766+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:53:00.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5cac0a3c40 msgr2=0x7f5cac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5cac0a3c40 0x7f5cac0a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5cc419a090 tx=0x7f5cb800a2d0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc4103450 msgr2=0x7f5cc4198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc4103450 0x7f5cc4198bd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400c970 tx=0x7f5cb400ce30 comp rx=0 tx=0).stop 2024-04-26T21:53:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 shutdown_connections 2024-04-26T21:53:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5cac0a3c40 0x7f5cac0a6100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc41086b0 0x7f5cc41a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4105d80 0x7f5cc4199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 --2- 172.21.15.55:0/3650203204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5cc4103450 0x7f5cc4198bd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 >> 172.21.15.55:0/3650203204 conn(0x7f5cc40fcde0 msgr2=0x7f5cc4103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 shutdown_connections 2024-04-26T21:53:00.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.463+0000 7f5cca26d640 1 -- 172.21.15.55:0/3650203204 wait complete. 2024-04-26T21:53:00.475 DEBUG:tasks.fwd_scrub:Yielding 2024-04-26T21:53:00.476 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T21:53:00.488 INFO:tasks.workunit:Pulling workunits from ref bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:53:00.488 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T21:53:00.488 INFO:tasks.workunit:timeout=3h 2024-04-26T21:53:00.488 INFO:tasks.workunit:cleanup=True 2024-04-26T21:53:00.489 DEBUG:teuthology.orchestra.run.smithi055:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T21:53:00.489 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-26T21:53:00.489 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:53:00.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:53:00.489 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-04-26 21:51:02.736621515 +0000 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-04-26 21:51:02.736621515 +0000 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-04-26 21:52:53.631222522 +0000 2024-04-26T21:53:00.505 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-04-26 21:51:02.736621515 +0000 2024-04-26T21:53:00.505 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T21:53:00.505 DEBUG:teuthology.orchestra.run.smithi055:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T21:53:00.579 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 -- 172.21.15.55:0/1771972195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60105d40 msgr2=0x7f1c60108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 --2- 172.21.15.55:0/1771972195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60105d40 0x7f1c60108130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009900 tx=0x7f1c5402a410 comp rx=0 tx=0).stop 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 -- 172.21.15.55:0/1771972195 shutdown_connections 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 --2- 172.21.15.55:0/1771972195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60108670 0x7f1c6010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 --2- 172.21.15.55:0/1771972195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60105d40 0x7f1c60108130 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 --2- 172.21.15.55:0/1771972195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c60103410 0x7f1c60105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 -- 172.21.15.55:0/1771972195 >> 172.21.15.55:0/1771972195 conn(0x7f1c600fce20 msgr2=0x7f1c600ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 -- 172.21.15.55:0/1771972195 shutdown_connections 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.597+0000 7f1c6863f640 1 -- 172.21.15.55:0/1771972195 wait complete. 2024-04-26T21:53:00.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c6863f640 1 Processor -- start 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c6863f640 1 -- start start 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c6863f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c60103410 0x7f1c60198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c6863f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60105d40 0x7f1c601991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c6863f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60108670 0x7f1c601a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c6863f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010e520 con 0x7f1c60108670 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c6863f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010e3a0 con 0x7f1c60105d40 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c6863f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6010e6a0 con 0x7f1c60103410 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c66bdc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60108670 0x7f1c601a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c66bdc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60108670 0x7f1c601a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45762/0 (socket says 172.21.15.55:45762) 2024-04-26T21:53:00.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c66bdc640 1 -- 172.21.15.55:0/3747661201 learned_addr learned my addr 172.21.15.55:0/3747661201 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c65bda640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60105d40 0x7f1c601991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c663db640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c60103410 0x7f1c60198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c66bdc640 1 -- 172.21.15.55:0/3747661201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c60103410 msgr2=0x7f1c60198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c66bdc640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c60103410 0x7f1c60198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c66bdc640 1 -- 172.21.15.55:0/3747661201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60105d40 msgr2=0x7f1c601991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c66bdc640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60105d40 0x7f1c601991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c66bdc640 1 -- 172.21.15.55:0/3747661201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c601a08e0 con 0x7f1c60108670 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.598+0000 7f1c663db640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c60103410 0x7f1c60198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.599+0000 7f1c66bdc640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60108670 0x7f1c601a0240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e970 tx=0x7f1c5800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.599+0000 7f1c5f7fe640 1 -- 172.21.15.55:0/3747661201 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800ccc0 con 0x7f1c60108670 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.599+0000 7f1c6863f640 1 -- 172.21.15.55:0/3747661201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c601a0bd0 con 0x7f1c60108670 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.599+0000 7f1c5f7fe640 1 -- 172.21.15.55:0/3747661201 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5800ce60 con 0x7f1c60108670 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.599+0000 7f1c5f7fe640 1 -- 172.21.15.55:0/3747661201 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c58010430 con 0x7f1c60108670 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.599+0000 7f1c6863f640 1 -- 172.21.15.55:0/3747661201 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c601a1170 con 0x7f1c60108670 2024-04-26T21:53:00.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.601+0000 7f1c5f7fe640 1 -- 172.21.15.55:0/3747661201 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1c58004ff0 con 0x7f1c60108670 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c6b430740 1 Processor -- start 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c6b430740 1 -- start start 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc2533ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533f2a0 0x55dc25343da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc25344410 0x55dc253468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c6b430740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dc252a3b00 con 0x55dc2533f2a0 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c6b430740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dc2523f920 con 0x55dc2533e8b0 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c6b430740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dc2529fac0 con 0x55dc25344410 2024-04-26T21:53:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c5f7fe640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1c480a3c40 0x7f1c480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c663db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533f2a0 0x55dc25343da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c663db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533f2a0 0x55dc25343da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45768/0 (socket says 172.21.15.55:45768) 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c663db640 1 -- 172.21.15.55:0/1682527552 learned_addr learned my addr 172.21.15.55:0/1682527552 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c66bdc640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1c480a3c40 0x7f1c480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.602+0000 7f1c65bda640 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc2533ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c65bda640 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc25344410 0x55dc253468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c663db640 1 -- 172.21.15.55:0/1682527552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc25344410 msgr2=0x55dc253468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c663db640 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc25344410 0x55dc253468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c663db640 1 -- 172.21.15.55:0/1682527552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 msgr2=0x55dc2533ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c663db640 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc2533ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c5f7fe640 1 -- 172.21.15.55:0/3747661201 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1c580d0a70 con 0x7f1c60108670 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c663db640 1 -- 172.21.15.55:0/1682527552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2507d7e0 con 0x55dc2533f2a0 2024-04-26T21:53:00.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c65bda640 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc2533ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c663db640 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533f2a0 0x55dc25343da0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c450 tx=0x7f1c5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0a96188fe245a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c66bdc640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1c480a3c40 0x7f1c480a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c580671d0 tx=0x7f1c5806f6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c5cbfe640 1 -- 172.21.15.55:0/1682527552 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5001a070 con 0x55dc2533f2a0 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c5cbfe640 1 -- 172.21.15.55:0/1682527552 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c500078e0 con 0x55dc2533f2a0 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.603+0000 7f1c5cbfe640 1 -- 172.21.15.55:0/1682527552 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50015830 con 0x55dc2533f2a0 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 -- 172.21.15.55:0/1682527552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533f2a0 msgr2=0x55dc25343da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533f2a0 0x55dc25343da0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c450 tx=0x7f1c5000c910 comp rx=0 tx=0).stop 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 -- 172.21.15.55:0/1682527552 shutdown_connections 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc25344410 0x55dc253468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533f2a0 0x55dc25343da0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1682527552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc2533ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 -- 172.21.15.55:0/1682527552 >> 172.21.15.55:0/1682527552 conn(0x55dc25339f20 msgr2=0x55dc2533a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 -- 172.21.15.55:0/1682527552 shutdown_connections 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 -- 172.21.15.55:0/1682527552 wait complete. 2024-04-26T21:53:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 -- start start 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.604+0000 7f1c6b430740 1 Processor -- start 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533e8b0 0x55dc25348c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533f2a0 0x55dc253491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25350200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c6b430740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dc253473c0 con 0x55dc25344410 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c6b430740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dc2523d5d0 con 0x55dc2533f2a0 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c6b430740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dc25347540 con 0x55dc2533e8b0 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c6b430740 10 client.0 _ll_register_callbacks cb 0x55dc2522b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25350200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25350200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45778/0 (socket says 172.21.15.55:45778) 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 -- 172.21.15.55:0/1641030901 learned_addr learned my addr 172.21.15.55:0/1641030901 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533e8b0 0x55dc25348c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c663db640 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533f2a0 0x55dc253491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 -- 172.21.15.55:0/1641030901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533e8b0 msgr2=0x55dc25348c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533e8b0 0x55dc25348c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 -- 172.21.15.55:0/1641030901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533f2a0 msgr2=0x55dc253491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533f2a0 0x55dc253491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 -- 172.21.15.55:0/1641030901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc250ddf70 con 0x55dc25344410 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c65bda640 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25350200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c54000c00 tx=0x7f1c5402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0ef9e4f036443 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c45ffb640 1 -- 172.21.15.55:0/1641030901 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54046040 con 0x55dc25344410 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c45ffb640 1 -- 172.21.15.55:0/1641030901 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1c54004450 con 0x55dc25344410 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.605+0000 7f1c6b430740 1 -- 172.21.15.55:0/1641030901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c500078e0 con 0x55dc25344410 2024-04-26T21:53:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c45ffb640 1 -- 172.21.15.55:0/1641030901 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1c54035040 con 0x55dc25344410 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c6b430740 10 client.15366 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c6b430740 1 -- 172.21.15.55:0/1641030901 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dc24f75170 con 0x55dc25344410 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c45ffb640 1 -- 172.21.15.55:0/1641030901 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c500078e0 con 0x55dc25344410 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c45ffb640 1 -- 172.21.15.55:0/1641030901 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1c5403d600 con 0x55dc25344410 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c6b430740 10 client.15366 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c6b430740 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c6b430740 20 client.15366 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dc253546a0 0x55dc25356b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c6b430740 4 client.15366 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c6b430740 1 -- 172.21.15.55:0/1641030901 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc252b1710 con 0x55dc253546a0 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.606+0000 7f1c663db640 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dc253546a0 0x55dc25356b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.607+0000 7f1c45ffb640 10 client.15366 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.607+0000 7f1c663db640 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dc253546a0 0x55dc25356b10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca321f5e18b7162 server_cookie=9aedaa9c6703d14 in_seq=0 out_seq=0 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.607+0000 7f1c45ffb640 1 -- 172.21.15.55:0/1641030901 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc252b1710 con 0x55dc253546a0 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.607+0000 7f1c45ffb640 10 client.15366 handle_command_reply: tid=0 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.607+0000 7f1c6b430740 1 client.15366 shutdown 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 -- 172.21.15.55:0/1641030901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 msgr2=0x55dc25350200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25350200 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1c54000c00 tx=0x7f1c5402a410 comp rx=0 tx=0).stop 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 -- 172.21.15.55:0/1641030901 shutdown_connections 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dc253546a0 0x55dc25356b10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25350200 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533f2a0 0x55dc253491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 --2- 172.21.15.55:0/1641030901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533e8b0 0x55dc25348c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 -- 172.21.15.55:0/1641030901 >> 172.21.15.55:0/1641030901 conn(0x55dc25339f20 msgr2=0x55dc253390e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c45ffb640 0 client.15366 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 -- 172.21.15.55:0/1641030901 shutdown_connections 2024-04-26T21:53:00.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 1 -- 172.21.15.55:0/1641030901 wait complete. 2024-04-26T21:53:00.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.608+0000 7f1c6b430740 20 client.15366 trim_cache size 0 max 16384 2024-04-26T21:53:00.616 INFO:tasks.workunit.client.0.smithi055.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c6b430740 1 Processor -- start 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c6b430740 1 -- start start 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc25338850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc253501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25338d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c6b430740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601ac860 con 0x55dc25344410 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c6b430740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601aa130 con 0x55dc2533e8b0 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c6b430740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601a7f80 con 0x55dc2533f2a0 2024-04-26T21:53:00.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25338d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25338d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45786/0 (socket says 172.21.15.55:45786) 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 -- 172.21.15.55:0/2370040616 learned_addr learned my addr 172.21.15.55:0/2370040616 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc25338850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c663db640 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc253501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 -- 172.21.15.55:0/2370040616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 msgr2=0x55dc253501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc253501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 -- 172.21.15.55:0/2370040616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 msgr2=0x55dc25338850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc25338850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c65bda640 1 -- 172.21.15.55:0/2370040616 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc25356e80 con 0x55dc25344410 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.626+0000 7f1c663db640 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc253501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.627+0000 7f1c65bda640 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25338d90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1c540096c0 tx=0x7f1c54032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4c37769a6d5dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.627+0000 7f1c47fff640 1 -- 172.21.15.55:0/2370040616 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5405a040 con 0x55dc25344410 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.627+0000 7f1c47fff640 1 -- 172.21.15.55:0/2370040616 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5405b040 con 0x55dc25344410 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.627+0000 7f1c47fff640 1 -- 172.21.15.55:0/2370040616 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54003af0 con 0x55dc25344410 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- 172.21.15.55:0/2370040616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 msgr2=0x55dc25338d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25338d90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1c540096c0 tx=0x7f1c54032d90 comp rx=0 tx=0).stop 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- 172.21.15.55:0/2370040616 shutdown_connections 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc25344410 0x55dc25338d90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc253501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 --2- 172.21.15.55:0/2370040616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc2533e8b0 0x55dc25338850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- 172.21.15.55:0/2370040616 >> 172.21.15.55:0/2370040616 conn(0x55dc25339f20 msgr2=0x55dc25348c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- 172.21.15.55:0/2370040616 shutdown_connections 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- 172.21.15.55:0/2370040616 wait complete. 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- start start 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 Processor -- start 2024-04-26T21:53:00.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533e8b0 0x55dc2534dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc2534e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc25344410 0x55dc2534f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601a85f0 con 0x55dc2533e8b0 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601a9a60 con 0x55dc25344410 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1c601a8cc0 con 0x55dc2533f2a0 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c6b430740 10 client.0 _ll_register_callbacks cb 0x7f1c540315a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c65bda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533e8b0 0x55dc2534dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c65bda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533e8b0 0x55dc2534dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45790/0 (socket says 172.21.15.55:45790) 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.628+0000 7f1c65bda640 1 -- 172.21.15.55:0/3772665737 learned_addr learned my addr 172.21.15.55:0/3772665737 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c663db640 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc2534e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c65bda640 1 -- 172.21.15.55:0/3772665737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 msgr2=0x55dc2534e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c65bda640 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc2534e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c65bda640 1 -- 172.21.15.55:0/3772665737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc25344410 msgr2=0x55dc2534f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c65bda640 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc25344410 0x55dc2534f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c65bda640 1 -- 172.21.15.55:0/3772665737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2534f620 con 0x55dc2533e8b0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c65bda640 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533e8b0 0x55dc2534dae0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1c540316b0 tx=0x7f1c5403dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7f4db17f9014d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c45ffb640 1 -- 172.21.15.55:0/3772665737 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5405a030 con 0x55dc2533e8b0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c45ffb640 1 -- 172.21.15.55:0/3772665737 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1c5403d660 con 0x55dc2533e8b0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c6b430740 1 -- 172.21.15.55:0/3772665737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c5405b040 con 0x55dc2533e8b0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c45ffb640 1 -- 172.21.15.55:0/3772665737 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1c54031b00 con 0x55dc2533e8b0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c6b430740 10 client.15372 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c6b430740 1 -- 172.21.15.55:0/3772665737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dc253baec0 con 0x55dc2533e8b0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.629+0000 7f1c45ffb640 1 -- 172.21.15.55:0/3772665737 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c5405b040 con 0x55dc2533e8b0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c45ffb640 1 -- 172.21.15.55:0/3772665737 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1c5405e070 con 0x55dc2533e8b0 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c6b430740 10 client.15372 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c6b430740 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:00.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c6b430740 20 client.15372 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:00.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c6b430740 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dc253bb840 0x55dc253bdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c6b430740 4 client.15372 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:53:00.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c6b430740 1 -- 172.21.15.55:0/3772665737 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dc252ab580 con 0x55dc253bb840 2024-04-26T21:53:00.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c663db640 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dc253bb840 0x55dc253bdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c663db640 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dc253bb840 0x55dc253bdc60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d43d8bd6d876803 server_cookie=c6f89b233b431db3 in_seq=0 out_seq=0 2024-04-26T21:53:00.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.630+0000 7f1c45ffb640 10 client.15372 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:00.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.631+0000 7f1c45ffb640 1 -- 172.21.15.55:0/3772665737 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dc252ab580 con 0x55dc253bb840 2024-04-26T21:53:00.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.631+0000 7f1c45ffb640 10 client.15372 handle_command_reply: tid=0 2024-04-26T21:53:00.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.631+0000 7f1c6b430740 1 client.15372 shutdown 2024-04-26T21:53:00.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 -- 172.21.15.55:0/3772665737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533e8b0 msgr2=0x55dc2534dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533e8b0 0x55dc2534dae0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1c540316b0 tx=0x7f1c5403dac0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 -- 172.21.15.55:0/3772665737 shutdown_connections 2024-04-26T21:53:00.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dc25344410 0x55dc2534f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc2533f2a0 0x55dc2534e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dc253bb840 0x55dc253bdc60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 --2- 172.21.15.55:0/3772665737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dc2533e8b0 0x55dc2534dae0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 -- 172.21.15.55:0/3772665737 >> 172.21.15.55:0/3772665737 conn(0x55dc25339f20 msgr2=0x55dc25338850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c45ffb640 0 client.15372 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 -- 172.21.15.55:0/3772665737 shutdown_connections 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 1 -- 172.21.15.55:0/3772665737 wait complete. 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6b430740 20 client.15372 trim_cache size 0 max 16384 2024-04-26T21:53:00.638 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:53:00.639 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:53:00.639 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "f9a6f706-c944-4c4f-a561-13d28f9f08d1", 2024-04-26T21:53:00.639 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:53:00.639 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:53:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 -- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1c480a3c40 msgr2=0x7f1c480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1c480a3c40 0x7f1c480a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c580671d0 tx=0x7f1c5806f6f0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 -- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60108670 msgr2=0x7f1c601a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60108670 0x7f1c601a0240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e970 tx=0x7f1c5800ed70 comp rx=0 tx=0).stop 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 -- 172.21.15.55:0/3747661201 shutdown_connections 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1c480a3c40 0x7f1c480a6100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c60108670 0x7f1c601a0240 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c60105d40 0x7f1c601991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 --2- 172.21.15.55:0/3747661201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c60103410 0x7f1c60198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 -- 172.21.15.55:0/3747661201 >> 172.21.15.55:0/3747661201 conn(0x7f1c600fce20 msgr2=0x7f1c601052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 -- 172.21.15.55:0/3747661201 shutdown_connections 2024-04-26T21:53:00.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.632+0000 7f1c6863f640 1 -- 172.21.15.55:0/3747661201 wait complete. 2024-04-26T21:53:00.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:53:00.644 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:53:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 -- 172.21.15.55:0/3028891217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c1034b0 msgr2=0x7f518c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 --2- 172.21.15.55:0/3028891217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c1034b0 0x7f518c1058a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f517c009900 tx=0x7f517c02a410 comp rx=0 tx=0).stop 2024-04-26T21:53:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 -- 172.21.15.55:0/3028891217 shutdown_connections 2024-04-26T21:53:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 --2- 172.21.15.55:0/3028891217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c108710 0x7f518c10ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 --2- 172.21.15.55:0/3028891217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c105de0 0x7f518c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 --2- 172.21.15.55:0/3028891217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c1034b0 0x7f518c1058a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 -- 172.21.15.55:0/3028891217 >> 172.21.15.55:0/3028891217 conn(0x7f518c0fcde0 msgr2=0x7f518c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 -- 172.21.15.55:0/3028891217 shutdown_connections 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 -- 172.21.15.55:0/3028891217 wait complete. 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 Processor -- start 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.792+0000 7f519376a640 1 -- start start 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f519376a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1034b0 0x7f518c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f519376a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c105de0 0x7f518c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f519376a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108710 0x7f518c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f519376a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10e4d0 con 0x7f518c105de0 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f519376a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10e350 con 0x7f518c108710 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f519376a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f518c10e650 con 0x7f518c1034b0 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c105de0 0x7f518c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5191506640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1034b0 0x7f518c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c105de0 0x7f518c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53956/0 (socket says 172.21.15.55:53956) 2024-04-26T21:53:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 -- 172.21.15.55:0/1916779515 learned_addr learned my addr 172.21.15.55:0/1916779515 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5191d07640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108710 0x7f518c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 -- 172.21.15.55:0/1916779515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1034b0 msgr2=0x7f518c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1034b0 0x7f518c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 -- 172.21.15.55:0/1916779515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108710 msgr2=0x7f518c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108710 0x7f518c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 -- 172.21.15.55:0/1916779515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1a0870 con 0x7f518c105de0 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f5190d05640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c105de0 0x7f518c1991a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f518800d770 tx=0x7f518800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.793+0000 7f51827fc640 1 -- 172.21.15.55:0/1916779515 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188017070 con 0x7f518c105de0 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.794+0000 7f519376a640 1 -- 172.21.15.55:0/1916779515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1a0b60 con 0x7f518c105de0 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.794+0000 7f51827fc640 1 -- 172.21.15.55:0/1916779515 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5188009c30 con 0x7f518c105de0 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.794+0000 7f51827fc640 1 -- 172.21.15.55:0/1916779515 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518800f620 con 0x7f518c105de0 2024-04-26T21:53:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.794+0000 7f519376a640 1 -- 172.21.15.55:0/1916779515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1a1100 con 0x7f518c105de0 2024-04-26T21:53:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.796+0000 7f5196482740 1 Processor -- start 2024-04-26T21:53:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.796+0000 7f51827fc640 1 -- 172.21.15.55:0/1916779515 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f518801a070 con 0x7f518c105de0 2024-04-26T21:53:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.796+0000 7f5196482740 1 -- start start 2024-04-26T21:53:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.796+0000 7f5196482740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c53300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.796+0000 7f5196482740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c540b0 0x556a48c54530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.796+0000 7f5196482740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c54a70 0x556a48c5b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5196482740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556a48bb7d80 con 0x556a48c540b0 2024-04-26T21:53:00.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5196482740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556a48b718b0 con 0x556a48c52ee0 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5196482740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556a48bb3d40 con 0x556a48c54a70 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c540b0 0x556a48c54530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c540b0 0x556a48c54530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53960/0 (socket says 172.21.15.55:53960) 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 -- 172.21.15.55:0/2609137425 learned_addr learned my addr 172.21.15.55:0/2609137425 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191d07640 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c53300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 -- 172.21.15.55:0/2609137425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c54a70 msgr2=0x556a48c5b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c54a70 0x556a48c5b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 -- 172.21.15.55:0/2609137425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 msgr2=0x556a48c53300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c53300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 -- 172.21.15.55:0/2609137425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a48b42a40 con 0x556a48c540b0 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f51827fc640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51780a3c40 0x7f51780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191d07640 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c53300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5190d05640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51780a3c40 0x7f51780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f5191506640 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c540b0 0x556a48c54530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c00 tx=0x7f517c031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8fd38538d2e44e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f51827fc640 1 -- 172.21.15.55:0/1916779515 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f51880d00f0 con 0x7f518c105de0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.797+0000 7f51777fe640 1 -- 172.21.15.55:0/2609137425 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c035030 con 0x556a48c540b0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f51777fe640 1 -- 172.21.15.55:0/2609137425 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517c040070 con 0x556a48c540b0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f51777fe640 1 -- 172.21.15.55:0/2609137425 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c003ab0 con 0x556a48c540b0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 -- 172.21.15.55:0/2609137425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c540b0 msgr2=0x556a48c54530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5190d05640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51780a3c40 0x7f51780a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f518800d740 tx=0x7f518806bb80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c540b0 0x556a48c54530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c00 tx=0x7f517c031b90 comp rx=0 tx=0).stop 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 -- 172.21.15.55:0/2609137425 shutdown_connections 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c54a70 0x556a48c5b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c540b0 0x556a48c54530 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 --2- 172.21.15.55:0/2609137425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c53300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 -- 172.21.15.55:0/2609137425 >> 172.21.15.55:0/2609137425 conn(0x556a48c4e3d0 msgr2=0x556a48c4e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 -- 172.21.15.55:0/2609137425 shutdown_connections 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.798+0000 7f5196482740 1 -- 172.21.15.55:0/2609137425 wait complete. 2024-04-26T21:53:00.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 1 -- start start 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 1 Processor -- start 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c52ee0 0x556a48c5d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c5d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556a48c5ba20 con 0x556a48c54a70 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556a48b3ed10 con 0x556a48c540b0 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556a48c5bba0 con 0x556a48c52ee0 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5196482740 10 client.0 _ll_register_callbacks cb 0x556a48b6b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5191d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5191d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53964/0 (socket says 172.21.15.55:53964) 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5191d07640 1 -- 172.21.15.55:0/2961395175 learned_addr learned my addr 172.21.15.55:0/2961395175 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5191506640 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c5d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.799+0000 7f5191d07640 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c52ee0 0x556a48c5d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5191d07640 1 -- 172.21.15.55:0/2961395175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c52ee0 msgr2=0x556a48c5d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5191d07640 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c52ee0 0x556a48c5d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5191d07640 1 -- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 msgr2=0x556a48c5d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5191d07640 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c5d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5191d07640 1 -- 172.21.15.55:0/2961395175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a489937d0 con 0x556a48c54a70 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5191506640 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5191d07640 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c648b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5184013eb0 tx=0x7f5184009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d128009bede36820 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5174ff9640 1 -- 172.21.15.55:0/2961395175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004b20 con 0x556a48c54a70 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5174ff9640 1 -- 172.21.15.55:0/2961395175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5184004cc0 con 0x556a48c54a70 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5196482740 1 -- 172.21.15.55:0/2961395175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517c040070 con 0x556a48c54a70 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5174ff9640 1 -- 172.21.15.55:0/2961395175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5184027050 con 0x556a48c54a70 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5196482740 10 client.15384 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:00.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5196482740 1 -- 172.21.15.55:0/2961395175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556a488c93f0 con 0x556a48c54a70 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.800+0000 7f5174ff9640 1 -- 172.21.15.55:0/2961395175 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517c040070 con 0x556a48c54a70 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5174ff9640 1 -- 172.21.15.55:0/2961395175 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f518400f040 con 0x556a48c54a70 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5196482740 10 client.15384 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5196482740 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5196482740 20 client.15384 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5196482740 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556a48c68ce0 0x556a48c6b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5196482740 4 client.15384 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5196482740 1 -- 172.21.15.55:0/2961395175 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a48bc5990 con 0x556a48c68ce0 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5191506640 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556a48c68ce0 0x556a48c6b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5191506640 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556a48c68ce0 0x556a48c6b1a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a5eec2b5268f51 server_cookie=f0ea5d5d11895d23 in_seq=0 out_seq=0 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.801+0000 7f5174ff9640 10 client.15384 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5174ff9640 1 -- 172.21.15.55:0/2961395175 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a48bc5990 con 0x556a48c68ce0 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5174ff9640 10 client.15384 handle_command_reply: tid=0 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 client.15384 shutdown 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 -- 172.21.15.55:0/2961395175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 msgr2=0x556a48c648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c648b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5184013eb0 tx=0x7f5184009a90 comp rx=0 tx=0).stop 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 -- 172.21.15.55:0/2961395175 shutdown_connections 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556a48c68ce0 0x556a48c6b1a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c648b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 --2- 172.21.15.55:0/2961395175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c52ee0 0x556a48c5d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5174ff9640 0 client.15384 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 -- 172.21.15.55:0/2961395175 >> 172.21.15.55:0/2961395175 conn(0x556a48c4e3d0 msgr2=0x556a48c4d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 -- 172.21.15.55:0/2961395175 shutdown_connections 2024-04-26T21:53:00.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 1 -- 172.21.15.55:0/2961395175 wait complete. 2024-04-26T21:53:00.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.802+0000 7f5196482740 20 client.15384 trim_cache size 0 max 16384 2024-04-26T21:53:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5196482740 1 Processor -- start 2024-04-26T21:53:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5196482740 1 -- start start 2024-04-26T21:53:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5196482740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c5b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5196482740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c540b0 0x556a48c64870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5196482740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5196482740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1ac850 con 0x556a48c54a70 2024-04-26T21:53:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5196482740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1aa120 con 0x556a48c52ee0 2024-04-26T21:53:00.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5196482740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1a7f70 con 0x556a48c540b0 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53966/0 (socket says 172.21.15.55:53966) 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191506640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c540b0 0x556a48c64870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 -- 172.21.15.55:0/2283751276 learned_addr learned my addr 172.21.15.55:0/2283751276 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c5b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 -- 172.21.15.55:0/2283751276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c540b0 msgr2=0x556a48c64870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c540b0 0x556a48c64870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 -- 172.21.15.55:0/2283751276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 msgr2=0x556a48c5b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c5b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191d07640 1 -- 172.21.15.55:0/2283751276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a488890f0 con 0x556a48c54a70 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.821+0000 7f5191506640 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c540b0 0x556a48c64870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5191d07640 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c512e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5184009ca0 tx=0x7f51840135d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927cf8b6cb67ebb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5176ffd640 1 -- 172.21.15.55:0/2283751276 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004a40 con 0x556a48c54a70 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5176ffd640 1 -- 172.21.15.55:0/2283751276 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5184004be0 con 0x556a48c54a70 2024-04-26T21:53:00.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5176ffd640 1 -- 172.21.15.55:0/2283751276 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400cb30 con 0x556a48c54a70 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5196482740 1 -- 172.21.15.55:0/2283751276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 msgr2=0x556a48c512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5196482740 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c512e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5184009ca0 tx=0x7f51840135d0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5196482740 1 -- 172.21.15.55:0/2283751276 shutdown_connections 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5196482740 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c54a70 0x556a48c512e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5196482740 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c540b0 0x556a48c64870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5196482740 1 --2- 172.21.15.55:0/2283751276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c52ee0 0x556a48c5b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.822+0000 7f5196482740 1 -- 172.21.15.55:0/2283751276 >> 172.21.15.55:0/2283751276 conn(0x556a48c4e3d0 msgr2=0x556a48c57b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 -- 172.21.15.55:0/2283751276 shutdown_connections 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 -- 172.21.15.55:0/2283751276 wait complete. 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 -- start start 2024-04-26T21:53:00.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 Processor -- start 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c52ee0 0x556a48c61720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c61c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c54a70 0x556a48ccd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1a85e0 con 0x556a48c52ee0 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1a9a50 con 0x556a48c540b0 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1a8cb0 con 0x556a48c54a70 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5196482740 10 client.0 _ll_register_callbacks cb 0x7f51840099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5191d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c52ee0 0x556a48c61720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5191d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c52ee0 0x556a48c61720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53980/0 (socket says 172.21.15.55:53980) 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.823+0000 7f5191d07640 1 -- 172.21.15.55:0/818660329 learned_addr learned my addr 172.21.15.55:0/818660329 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5191506640 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c61c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5191d07640 1 -- 172.21.15.55:0/818660329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c54a70 msgr2=0x556a48ccd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5191d07640 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c54a70 0x556a48ccd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5191d07640 1 -- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 msgr2=0x556a48c61c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5191d07640 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c61c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5191d07640 1 -- 172.21.15.55:0/818660329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a48b784b0 con 0x556a48c52ee0 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5191506640 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c61c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5191d07640 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c52ee0 0x556a48c61720 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5184013bd0 tx=0x7f5184005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e35152fa56cb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5174ff9640 1 -- 172.21.15.55:0/818660329 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400ccc0 con 0x556a48c52ee0 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5174ff9640 1 -- 172.21.15.55:0/818660329 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5184002030 con 0x556a48c52ee0 2024-04-26T21:53:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5196482740 1 -- 172.21.15.55:0/818660329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5184004be0 con 0x556a48c52ee0 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5174ff9640 1 -- 172.21.15.55:0/818660329 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f518400c370 con 0x556a48c52ee0 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5196482740 10 client.15390 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5196482740 1 -- 172.21.15.55:0/818660329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556a48cce8e0 con 0x556a48c52ee0 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.824+0000 7f5174ff9640 1 -- 172.21.15.55:0/818660329 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5184004be0 con 0x556a48c52ee0 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5174ff9640 1 -- 172.21.15.55:0/818660329 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f518403d080 con 0x556a48c52ee0 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5196482740 10 client.15390 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5196482740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5196482740 20 client.15390 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5196482740 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556a48ccf260 0x556a48cd1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5196482740 4 client.15390 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5196482740 1 -- 172.21.15.55:0/818660329 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a48bbf800 con 0x556a48ccf260 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5191506640 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556a48ccf260 0x556a48cd1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5174ff9640 10 client.15390 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.825+0000 7f5191506640 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556a48ccf260 0x556a48cd1680 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92601a3f7d6309 server_cookie=29f96a3cc3765957 in_seq=0 out_seq=0 2024-04-26T21:53:00.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5174ff9640 1 -- 172.21.15.55:0/818660329 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556a48bbf800 con 0x556a48ccf260 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5174ff9640 10 client.15390 handle_command_reply: tid=0 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 client.15390 shutdown 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 -- 172.21.15.55:0/818660329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c52ee0 msgr2=0x556a48c61720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c52ee0 0x556a48c61720 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5184013bd0 tx=0x7f5184005f00 comp rx=0 tx=0).stop 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 -- 172.21.15.55:0/818660329 shutdown_connections 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556a48c54a70 0x556a48ccd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a48c540b0 0x556a48c61c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556a48ccf260 0x556a48cd1680 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 --2- 172.21.15.55:0/818660329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556a48c52ee0 0x556a48c61720 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5174ff9640 0 client.15390 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 -- 172.21.15.55:0/818660329 >> 172.21.15.55:0/818660329 conn(0x556a48c4e3d0 msgr2=0x556a48c4ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 -- 172.21.15.55:0/818660329 shutdown_connections 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 1 -- 172.21.15.55:0/818660329 wait complete. 2024-04-26T21:53:00.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f5196482740 20 client.15390 trim_cache size 0 max 16384 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout: "f9a6f706-c944-4c4f-a561-13d28f9f08d1": { 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "f9a6f706-c944-4c4f-a561-13d28f9f08d1", 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:53:00.833 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:53:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f519376a640 1 -- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51780a3c40 msgr2=0x7f51780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f519376a640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51780a3c40 0x7f51780a6100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f518800d740 tx=0x7f518806bb80 comp rx=0 tx=0).stop 2024-04-26T21:53:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f519376a640 1 -- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c105de0 msgr2=0x7f518c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.826+0000 7f519376a640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c105de0 0x7f518c1991a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f518800d770 tx=0x7f518800dc30 comp rx=0 tx=0).stop 2024-04-26T21:53:00.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.827+0000 7f519376a640 1 -- 172.21.15.55:0/1916779515 shutdown_connections 2024-04-26T21:53:00.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.827+0000 7f519376a640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51780a3c40 0x7f51780a6100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.827+0000 7f519376a640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f518c108710 0x7f518c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.827+0000 7f519376a640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518c105de0 0x7f518c1991a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.827+0000 7f519376a640 1 --2- 172.21.15.55:0/1916779515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1034b0 0x7f518c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:00.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.827+0000 7f519376a640 1 -- 172.21.15.55:0/1916779515 >> 172.21.15.55:0/1916779515 conn(0x7f518c0fcde0 msgr2=0x7f518c103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:00.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.827+0000 7f519376a640 1 -- 172.21.15.55:0/1916779515 shutdown_connections 2024-04-26T21:53:00.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:00.827+0000 7f519376a640 1 -- 172.21.15.55:0/1916779515 wait complete. 2024-04-26T21:53:00.838 INFO:tasks.cephfs.filesystem:scrub status for tag:f9a6f706-c944-4c4f-a561-13d28f9f08d1 - {'path': '/', 'tag': 'f9a6f706-c944-4c4f-a561-13d28f9f08d1', 'options': 'recursive,force'} 2024-04-26T21:53:01.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:00 smithi165 ceph-mon[33296]: pgmap v354: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T21:53:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:00 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3726654095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:01.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:00 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3650203204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:00 smithi055 ceph-mon[29221]: pgmap v354: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T21:53:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:00 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3726654095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:00 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3650203204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:00 smithi064 ceph-mon[32765]: pgmap v354: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-26T21:53:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:00 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3726654095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:00 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3650203204' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:53:02.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:01 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:53:02.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:01 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:02.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:01 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:53:02.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:01 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:02.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:01 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:53:02.158 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:01 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:53:02.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:01 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:02.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:01 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:53:02.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:01 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:02.159 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:01 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:53:02.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:01 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:53:02.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:01 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:02.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:01 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:53:02.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:01 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:01 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:53:03.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:02 smithi055 ceph-mon[29221]: pgmap v355: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T21:53:03.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:02 smithi064 ceph-mon[32765]: pgmap v355: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T21:53:03.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:02 smithi165 ceph-mon[33296]: pgmap v355: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-26T21:53:05.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:04 smithi055 ceph-mon[29221]: pgmap v356: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.0 KiB/s wr, 6 op/s 2024-04-26T21:53:05.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:04 smithi064 ceph-mon[32765]: pgmap v356: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.0 KiB/s wr, 6 op/s 2024-04-26T21:53:05.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:04 smithi165 ceph-mon[33296]: pgmap v356: 97 pgs: 97 active+clean; 634 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.0 KiB/s wr, 6 op/s 2024-04-26T21:53:07.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:06 smithi055 ceph-mon[29221]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T21:53:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:06 smithi064 ceph-mon[32765]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T21:53:07.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:06 smithi165 ceph-mon[33296]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T21:53:09.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:08 smithi055 ceph-mon[29221]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-26T21:53:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:08 smithi064 ceph-mon[32765]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-26T21:53:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:08 smithi165 ceph-mon[33296]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-26T21:53:11.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:10 smithi055 ceph-mon[29221]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T21:53:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:10 smithi064 ceph-mon[32765]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T21:53:11.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:10 smithi165 ceph-mon[33296]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-26T21:53:13.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:12 smithi055 ceph-mon[29221]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-26T21:53:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:12 smithi064 ceph-mon[32765]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-26T21:53:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:12 smithi165 ceph-mon[33296]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-04-26T21:53:15.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:14 smithi055 ceph-mon[29221]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T21:53:15.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:14 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:15.213 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:14 smithi165 ceph-mon[33296]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T21:53:15.213 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:14 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:15.249 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:14 smithi064 ceph-mon[32765]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-26T21:53:15.250 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:14 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:17.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:17.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T21:53:17.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:17.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:17.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T21:53:17.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:17.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:17.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:17.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:53:19.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:19 smithi055 ceph-mon[29221]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:19 smithi064 ceph-mon[32765]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:19.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:19 smithi165 ceph-mon[33296]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:21.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:21 smithi055 ceph-mon[29221]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:21 smithi064 ceph-mon[32765]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:21.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:21 smithi165 ceph-mon[33296]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:23.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:23 smithi055 ceph-mon[29221]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:23 smithi064 ceph-mon[32765]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:23.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:23 smithi165 ceph-mon[33296]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:25.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:25 smithi055 ceph-mon[29221]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:25.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:25 smithi064 ceph-mon[32765]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:25.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:25 smithi165 ceph-mon[33296]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:27.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:27 smithi055 ceph-mon[29221]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:27 smithi064 ceph-mon[32765]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:27.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:27 smithi165 ceph-mon[33296]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:29.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:29 smithi055 ceph-mon[29221]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:29.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:29 smithi064 ceph-mon[32765]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:29.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:29 smithi165 ceph-mon[33296]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:30.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:53:30.840 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:53:30.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 -- 172.21.15.55:0/3833239217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4102a60 msgr2=0x7fc2e4102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:30.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 --2- 172.21.15.55:0/3833239217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4102a60 0x7fc2e4102ee0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009900 tx=0x7fc2d802a410 comp rx=0 tx=0).stop 2024-04-26T21:53:30.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 -- 172.21.15.55:0/3833239217 shutdown_connections 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 --2- 172.21.15.55:0/3833239217 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2e4103420 0x7fc2e4109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 --2- 172.21.15.55:0/3833239217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4102a60 0x7fc2e4102ee0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 --2- 172.21.15.55:0/3833239217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e4101870 0x7fc2e4101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 -- 172.21.15.55:0/3833239217 >> 172.21.15.55:0/3833239217 conn(0x7fc2e40fcde0 msgr2=0x7fc2e40ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 -- 172.21.15.55:0/3833239217 shutdown_connections 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 -- 172.21.15.55:0/3833239217 wait complete. 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.961+0000 7fc2eae46640 1 Processor -- start 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2eae46640 1 -- start start 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2eae46640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e4101870 0x7fc2e4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2eae46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2e4102a60 0x7fc2e4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2eae46640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4103420 0x7fc2e41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2eae46640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e410e4c0 con 0x7fc2e4103420 2024-04-26T21:53:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2eae46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e410e340 con 0x7fc2e4102a60 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2eae46640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e410e640 con 0x7fc2e4101870 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4103420 0x7fc2e41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4103420 0x7fc2e41a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37030/0 (socket says 172.21.15.55:37030) 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 -- 172.21.15.55:0/596661281 learned_addr learned my addr 172.21.15.55:0/596661281 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2e4102a60 0x7fc2e4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e4101870 0x7fc2e4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 -- 172.21.15.55:0/596661281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e4101870 msgr2=0x7fc2e4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e4101870 0x7fc2e4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 -- 172.21.15.55:0/596661281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2e4102a60 msgr2=0x7fc2e4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2e4102a60 0x7fc2e4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 -- 172.21.15.55:0/596661281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e41a0860 con 0x7fc2e4103420 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.962+0000 7fc2e93e3640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4103420 0x7fc2e41a01c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00cad0 tx=0x7fc2dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.963+0000 7fc2e1ffb640 1 -- 172.21.15.55:0/596661281 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc00ae20 con 0x7fc2e4103420 2024-04-26T21:53:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.963+0000 7fc2e1ffb640 1 -- 172.21.15.55:0/596661281 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2dc005020 con 0x7fc2e4103420 2024-04-26T21:53:30.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.963+0000 7fc2eae46640 1 -- 172.21.15.55:0/596661281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e41a0b50 con 0x7fc2e4103420 2024-04-26T21:53:30.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.963+0000 7fc2e1ffb640 1 -- 172.21.15.55:0/596661281 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc00f6b0 con 0x7fc2e4103420 2024-04-26T21:53:30.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.963+0000 7fc2eae46640 1 -- 172.21.15.55:0/596661281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e41a1050 con 0x7fc2e4103420 2024-04-26T21:53:30.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2e1ffb640 1 -- 172.21.15.55:0/596661281 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc2dc0051d0 con 0x7fc2e4103420 2024-04-26T21:53:30.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2edc35740 1 Processor -- start 2024-04-26T21:53:30.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2edc35740 1 -- start start 2024-04-26T21:53:30.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000b2f0 0x55e17000b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17000c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000ce80 0x55e170013710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2edc35740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e16ff6bd80 con 0x55e17000c4c0 2024-04-26T21:53:30.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2edc35740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e16ff258b0 con 0x55e17000ce80 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.965+0000 7fc2edc35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e16ff67d40 con 0x55e17000b2f0 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e1ffb640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2cc0a3c40 0x7fc2cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e8be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17000c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000b2f0 0x55e17000b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000b2f0 0x55e17000b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36258/0 (socket says 172.21.15.55:36258) 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e93e3640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2cc0a3c40 0x7fc2cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e8be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17000c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37046/0 (socket says 172.21.15.55:37046) 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e3fff640 1 -- 172.21.15.55:0/1457357905 learned_addr learned my addr 172.21.15.55:0/1457357905 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000ce80 0x55e170013710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e1ffb640 1 -- 172.21.15.55:0/596661281 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2dc0d0650 con 0x7fc2e4103420 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e8be2640 1 -- 172.21.15.55:0/1457357905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000b2f0 msgr2=0x55e17000b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000b2f0 0x55e17000b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e8be2640 1 -- 172.21.15.55:0/1457357905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000ce80 msgr2=0x55e170013710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000ce80 0x55e170013710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e8be2640 1 -- 172.21.15.55:0/1457357905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e16fef6a40 con 0x55e17000c4c0 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000b2f0 0x55e17000b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e93e3640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2cc0a3c40 0x7fc2cc0a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc098c10 tx=0x7fc2dc087040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17000c940 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc2e419a120 tx=0x7fc2d400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff944918c2860e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2caffd640 1 -- 172.21.15.55:0/1457357905 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4004030 con 0x55e17000c4c0 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2caffd640 1 -- 172.21.15.55:0/1457357905 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d40054d0 con 0x55e17000c4c0 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.966+0000 7fc2caffd640 1 -- 172.21.15.55:0/1457357905 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4004bc0 con 0x55e17000c4c0 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 -- 172.21.15.55:0/1457357905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 msgr2=0x55e17000c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17000c940 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc2e419a120 tx=0x7fc2d400f3c0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 -- 172.21.15.55:0/1457357905 shutdown_connections 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000ce80 0x55e170013710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17000c940 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 --2- 172.21.15.55:0/1457357905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000b2f0 0x55e17000b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 -- 172.21.15.55:0/1457357905 >> 172.21.15.55:0/1457357905 conn(0x55e1700023d0 msgr2=0x55e1700027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:30.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 -- 172.21.15.55:0/1457357905 shutdown_connections 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 -- 172.21.15.55:0/1457357905 wait complete. 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.967+0000 7fc2edc35740 1 -- start start 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2edc35740 1 Processor -- start 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e170015780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 0x55e170015cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e17001cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2edc35740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e170013e90 con 0x55e17000ce80 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2edc35740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e16fef2d10 con 0x55e17000b2f0 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2edc35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e170014010 con 0x55e17000c4c0 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2edc35740 10 client.0 _ll_register_callbacks cb 0x55e16ff1f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e17001cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e17001cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37052/0 (socket says 172.21.15.55:37052) 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e3fff640 1 -- 172.21.15.55:0/3876101478 learned_addr learned my addr 172.21.15.55:0/3876101478 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 0x55e170015cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e3fff640 1 -- 172.21.15.55:0/3876101478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 msgr2=0x55e170015cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 0x55e170015cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e3fff640 1 -- 172.21.15.55:0/3876101478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 msgr2=0x55e170015780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e170015780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.968+0000 7fc2e3fff640 1 -- 172.21.15.55:0/3876101478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e16fd477d0 con 0x55e17000ce80 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e17001cd00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8000c00 tx=0x7fc2d802a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae69633f5842eef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2c3fff640 1 -- 172.21.15.55:0/3876101478 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8035040 con 0x55e17000ce80 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2c3fff640 1 -- 172.21.15.55:0/3876101478 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2d80035f0 con 0x55e17000ce80 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2edc35740 1 -- 172.21.15.55:0/3876101478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2d40054d0 con 0x55e17000ce80 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2c3fff640 1 -- 172.21.15.55:0/3876101478 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc2d80037f0 con 0x55e17000ce80 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2edc35740 10 client.15402 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2edc35740 1 -- 172.21.15.55:0/3876101478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e16fc7d3f0 con 0x55e17000ce80 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2c3fff640 1 -- 172.21.15.55:0/3876101478 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d40054d0 con 0x55e17000ce80 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2c3fff640 1 -- 172.21.15.55:0/3876101478 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc2d8047070 con 0x55e17000ce80 2024-04-26T21:53:30.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2edc35740 10 client.15402 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2edc35740 10 client.15402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2edc35740 20 client.15402 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.969+0000 7fc2edc35740 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1700211a0 0x55e170023640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.970+0000 7fc2edc35740 4 client.15402 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.970+0000 7fc2edc35740 1 -- 172.21.15.55:0/3876101478 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e16ff79990 con 0x55e1700211a0 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.970+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1700211a0 0x55e170023640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.970+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1700211a0 0x55e170023640 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64bfd58e49bd14a2 server_cookie=c97a2b0dc15bb98e in_seq=0 out_seq=0 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.970+0000 7fc2c3fff640 10 client.15402 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2c3fff640 1 -- 172.21.15.55:0/3876101478 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e16ff79990 con 0x55e1700211a0 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2c3fff640 10 client.15402 handle_command_reply: tid=0 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 client.15402 shutdown 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 -- 172.21.15.55:0/3876101478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 msgr2=0x55e17001cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e17001cd00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8000c00 tx=0x7fc2d802a9b0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 -- 172.21.15.55:0/3876101478 shutdown_connections 2024-04-26T21:53:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1700211a0 0x55e170023640 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e17001cd00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 0x55e170015cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 --2- 172.21.15.55:0/3876101478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e170015780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2c3fff640 0 client.15402 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:30.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 -- 172.21.15.55:0/3876101478 >> 172.21.15.55:0/3876101478 conn(0x55e1700023d0 msgr2=0x55e170001550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:30.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 -- 172.21.15.55:0/3876101478 shutdown_connections 2024-04-26T21:53:30.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 1 -- 172.21.15.55:0/3876101478 wait complete. 2024-04-26T21:53:30.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.971+0000 7fc2edc35740 20 client.15402 trim_cache size 0 max 16384 2024-04-26T21:53:30.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2edc35740 1 Processor -- start 2024-04-26T21:53:30.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2edc35740 1 -- start start 2024-04-26T21:53:30.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e1700059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 0x55e17001cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e170027f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2edc35740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e41aa5e0 con 0x55e17000ce80 2024-04-26T21:53:30.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2edc35740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e41a7eb0 con 0x55e17000b2f0 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2edc35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e41a5d00 con 0x55e17000c4c0 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e170027f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e170027f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37064/0 (socket says 172.21.15.55:37064) 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2e3fff640 1 -- 172.21.15.55:0/1885530440 learned_addr learned my addr 172.21.15.55:0/1885530440 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 0x55e17001cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.989+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e1700059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2e3fff640 1 -- 172.21.15.55:0/1885530440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 msgr2=0x55e17001cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 0x55e17001cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2e3fff640 1 -- 172.21.15.55:0/1885530440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 msgr2=0x55e1700059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e1700059f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2e3fff640 1 -- 172.21.15.55:0/1885530440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e16fc3d0f0 con 0x55e17000ce80 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e170027f00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc2d802ac00 tx=0x7fc2d8040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca26c78e478b8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2ca7fc640 1 -- 172.21.15.55:0/1885530440 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8058070 con 0x55e17000ce80 2024-04-26T21:53:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2ca7fc640 1 -- 172.21.15.55:0/1885530440 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d8031760 con 0x55e17000ce80 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2ca7fc640 1 -- 172.21.15.55:0/1885530440 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8035030 con 0x55e17000ce80 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2edc35740 1 -- 172.21.15.55:0/1885530440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 msgr2=0x55e170027f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.990+0000 7fc2edc35740 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e170027f00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc2d802ac00 tx=0x7fc2d8040040 comp rx=0 tx=0).stop 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 -- 172.21.15.55:0/1885530440 shutdown_connections 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000ce80 0x55e170027f00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000c4c0 0x55e17001cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 --2- 172.21.15.55:0/1885530440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e1700059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 -- 172.21.15.55:0/1885530440 >> 172.21.15.55:0/1885530440 conn(0x55e1700023d0 msgr2=0x55e1700052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 -- 172.21.15.55:0/1885530440 shutdown_connections 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 -- 172.21.15.55:0/1885530440 wait complete. 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 -- start start 2024-04-26T21:53:30.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 Processor -- start 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e17001c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17001ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000ce80 0x55e17001ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e41a6370 con 0x55e17000c4c0 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e41a77e0 con 0x55e17000b2f0 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e41a6a40 con 0x55e17000ce80 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2edc35740 10 client.0 _ll_register_callbacks cb 0x7fc2d80316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.991+0000 7fc2e8be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17001ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e8be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17001ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37070/0 (socket says 172.21.15.55:37070) 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e8be2640 1 -- 172.21.15.55:0/195659026 learned_addr learned my addr 172.21.15.55:0/195659026 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e17001c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000ce80 0x55e17001ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e8be2640 1 -- 172.21.15.55:0/195659026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000ce80 msgr2=0x55e17001ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000ce80 0x55e17001ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e8be2640 1 -- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 msgr2=0x55e17001c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e17001c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e8be2640 1 -- 172.21.15.55:0/195659026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e16ff2c4b0 con 0x55e17000c4c0 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e17001c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2e8be2640 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17001ca60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400a520 tx=0x7fc2d4005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e04392ed67cfb9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2c3fff640 1 -- 172.21.15.55:0/195659026 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d402d070 con 0x55e17000c4c0 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2c3fff640 1 -- 172.21.15.55:0/195659026 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2d4029570 con 0x55e17000c4c0 2024-04-26T21:53:30.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2edc35740 1 -- 172.21.15.55:0/195659026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e16ff73800 con 0x55e17000c4c0 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2c3fff640 1 -- 172.21.15.55:0/195659026 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc2d4039050 con 0x55e17000c4c0 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2edc35740 10 client.15408 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.992+0000 7fc2edc35740 1 -- 172.21.15.55:0/195659026 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e170088020 con 0x55e17000c4c0 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2c3fff640 1 -- 172.21.15.55:0/195659026 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e16ff73800 con 0x55e17000c4c0 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2c3fff640 1 -- 172.21.15.55:0/195659026 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc2d4023040 con 0x55e17000c4c0 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2edc35740 10 client.15408 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2edc35740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2edc35740 20 client.15408 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2edc35740 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1700886a0 0x55e17008aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2edc35740 4 client.15408 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2edc35740 1 -- 172.21.15.55:0/195659026 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e16feee4a0 con 0x55e1700886a0 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1700886a0 0x55e17008aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2e3fff640 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1700886a0 0x55e17008aac0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db5b0a4e7e9a97af server_cookie=aff52db35344d211 in_seq=0 out_seq=0 2024-04-26T21:53:30.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.993+0000 7fc2c3fff640 10 client.15408 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2c3fff640 1 -- 172.21.15.55:0/195659026 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e16feee4a0 con 0x55e1700886a0 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2c3fff640 10 client.15408 handle_command_reply: tid=0 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 client.15408 shutdown 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 -- 172.21.15.55:0/195659026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 msgr2=0x55e17001ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17001ca60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400a520 tx=0x7fc2d4005aa0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 -- 172.21.15.55:0/195659026 shutdown_connections 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e17000ce80 0x55e17001ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1700886a0 0x55e17008aac0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e17000c4c0 0x55e17001ca60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 --2- 172.21.15.55:0/195659026 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e17000b2f0 0x55e17001c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2c3fff640 0 client.15408 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 -- 172.21.15.55:0/195659026 >> 172.21.15.55:0/195659026 conn(0x55e1700023d0 msgr2=0x55e170001550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 -- 172.21.15.55:0/195659026 shutdown_connections 2024-04-26T21:53:31.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 1 -- 172.21.15.55:0/195659026 wait complete. 2024-04-26T21:53:31.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2edc35740 20 client.15408 trim_cache size 0 max 16384 2024-04-26T21:53:31.001 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:53:31.001 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:53:31.001 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:53:31.001 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:53:31.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2eae46640 1 -- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2cc0a3c40 msgr2=0x7fc2cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2eae46640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2cc0a3c40 0x7fc2cc0a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc098c10 tx=0x7fc2dc087040 comp rx=0 tx=0).stop 2024-04-26T21:53:31.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2eae46640 1 -- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4103420 msgr2=0x7fc2e41a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.994+0000 7fc2eae46640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4103420 0x7fc2e41a01c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00cad0 tx=0x7fc2dc00cf90 comp rx=0 tx=0).stop 2024-04-26T21:53:31.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.995+0000 7fc2eae46640 1 -- 172.21.15.55:0/596661281 shutdown_connections 2024-04-26T21:53:31.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.995+0000 7fc2eae46640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2cc0a3c40 0x7fc2cc0a6100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.995+0000 7fc2eae46640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2e4103420 0x7fc2e41a01c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.995+0000 7fc2eae46640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2e4102a60 0x7fc2e4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.995+0000 7fc2eae46640 1 --2- 172.21.15.55:0/596661281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e4101870 0x7fc2e4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.995+0000 7fc2eae46640 1 -- 172.21.15.55:0/596661281 >> 172.21.15.55:0/596661281 conn(0x7fc2e40fcde0 msgr2=0x7fc2e4105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.995+0000 7fc2eae46640 1 -- 172.21.15.55:0/596661281 shutdown_connections 2024-04-26T21:53:31.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:30.995+0000 7fc2eae46640 1 -- 172.21.15.55:0/596661281 wait complete. 2024-04-26T21:53:31.006 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:53:31.006 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 -- 172.21.15.55:0/141923910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 msgr2=0x7f8918100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 --2- 172.21.15.55:0/141923910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 0x7f8918100e80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8900009900 tx=0x7f890002a410 comp rx=0 tx=0).stop 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 -- 172.21.15.55:0/141923910 shutdown_connections 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 --2- 172.21.15.55:0/141923910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89181013c0 0x7f891810af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 --2- 172.21.15.55:0/141923910 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 0x7f8918100e80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 --2- 172.21.15.55:0/141923910 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8918103f90 0x7f89181043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 -- 172.21.15.55:0/141923910 >> 172.21.15.55:0/141923910 conn(0x7f89180fc5a0 msgr2=0x7f89180fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 -- 172.21.15.55:0/141923910 shutdown_connections 2024-04-26T21:53:31.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.108+0000 7f891cd97640 1 -- 172.21.15.55:0/141923910 wait complete. 2024-04-26T21:53:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891cd97640 1 Processor -- start 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891cd97640 1 -- start start 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891cd97640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 0x7f8918198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891cd97640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89181013c0 0x7f8918199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891cd97640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8918103f90 0x7f89181a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891cd97640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f891810e4a0 con 0x7f8918100a00 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891cd97640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f891810e320 con 0x7f89181013c0 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891cd97640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f891810e620 con 0x7f8918103f90 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 0x7f8918198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 0x7f8918198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37090/0 (socket says 172.21.15.55:37090) 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f891659c640 1 -- 172.21.15.55:0/2644665325 learned_addr learned my addr 172.21.15.55:0/2644665325 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f8916d9d640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8918103f90 0x7f89181a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.109+0000 7f890ffff640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89181013c0 0x7f8918199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f891659c640 1 -- 172.21.15.55:0/2644665325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8918103f90 msgr2=0x7f89181a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f891659c640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8918103f90 0x7f89181a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f891659c640 1 -- 172.21.15.55:0/2644665325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89181013c0 msgr2=0x7f8918199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f891659c640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89181013c0 0x7f8918199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f891659c640 1 -- 172.21.15.55:0/2644665325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89181a0710 con 0x7f8918100a00 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f891659c640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 0x7f8918198af0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f890800c970 tx=0x7f890800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f890f7fe640 1 -- 172.21.15.55:0/2644665325 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908007c10 con 0x7f8918100a00 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f890f7fe640 1 -- 172.21.15.55:0/2644665325 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8908007db0 con 0x7f8918100a00 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89181a0a00 con 0x7f8918100a00 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f890f7fe640 1 -- 172.21.15.55:0/2644665325 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890800f870 con 0x7f8918100a00 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.110+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89181a0f50 con 0x7f8918100a00 2024-04-26T21:53:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.111+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8918109e50 con 0x7f8918100a00 2024-04-26T21:53:31.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.112+0000 7f890f7fe640 1 -- 172.21.15.55:0/2644665325 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f890800fa10 con 0x7f8918100a00 2024-04-26T21:53:31.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.113+0000 7f890f7fe640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89040a3c40 0x7f89040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.113+0000 7f890ffff640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89040a3c40 0x7f89040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.113+0000 7f890f7fe640 1 -- 172.21.15.55:0/2644665325 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f89080d0c70 con 0x7f8918100a00 2024-04-26T21:53:31.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.113+0000 7f890ffff640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89040a3c40 0x7f89040a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f890002a930 tx=0x7f8900000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.116+0000 7f890f7fe640 1 -- 172.21.15.55:0/2644665325 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f890809d570 con 0x7f8918100a00 2024-04-26T21:53:31.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.343+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f891802dad0 con 0x7f8918100a00 2024-04-26T21:53:31.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.344+0000 7f890f7fe640 1 -- 172.21.15.55:0/2644665325 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11863 (secure 0 0 0) 0x7f890809ce70 con 0x7f8918100a00 2024-04-26T21:53:31.345 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:53:31.345 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:52:54.328649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24502,24509]},"id":1}]} 2024-04-26T21:53:31.345 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 21 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89040a3c40 msgr2=0x7f89040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89040a3c40 0x7f89040a6100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f890002a930 tx=0x7f8900000bc0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 msgr2=0x7f8918198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 0x7f8918198af0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f890800c970 tx=0x7f890800ce30 comp rx=0 tx=0).stop 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 shutdown_connections 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f89040a3c40 0x7f89040a6100 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8918103f90 0x7f89181a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89181013c0 0x7f8918199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 --2- 172.21.15.55:0/2644665325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8918100a00 0x7f8918198af0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 >> 172.21.15.55:0/2644665325 conn(0x7f89180fc5a0 msgr2=0x7f8918107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 shutdown_connections 2024-04-26T21:53:31.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.346+0000 7f891cd97640 1 -- 172.21.15.55:0/2644665325 wait complete. 2024-04-26T21:53:31.358 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:53:31.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:31 smithi055 ceph-mon[29221]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:31.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 -- 172.21.15.55:0/4196268489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08101870 msgr2=0x7f2c08101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 --2- 172.21.15.55:0/4196268489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08101870 0x7f2c08101c90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009900 tx=0x7f2bf802a410 comp rx=0 tx=0).stop 2024-04-26T21:53:31.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 -- 172.21.15.55:0/4196268489 shutdown_connections 2024-04-26T21:53:31.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 --2- 172.21.15.55:0/4196268489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c08103420 0x7f2c08109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 --2- 172.21.15.55:0/4196268489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08102a60 0x7f2c08102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 --2- 172.21.15.55:0/4196268489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08101870 0x7f2c08101c90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 -- 172.21.15.55:0/4196268489 >> 172.21.15.55:0/4196268489 conn(0x7f2c080fcde0 msgr2=0x7f2c080ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 -- 172.21.15.55:0/4196268489 shutdown_connections 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 -- 172.21.15.55:0/4196268489 wait complete. 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 Processor -- start 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.462+0000 7f2c0d319640 1 -- start start 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c0d319640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c08101870 0x7f2c08198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c0d319640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08102a60 0x7f2c08199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c0d319640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08103420 0x7f2c081a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c0d319640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0810e520 con 0x7f2c08102a60 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c0d319640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0810e3a0 con 0x7f2c08103420 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c0d319640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0810e6a0 con 0x7f2c08101870 2024-04-26T21:53:31.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08102a60 0x7f2c08199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08102a60 0x7f2c08199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37112/0 (socket says 172.21.15.55:37112) 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 -- 172.21.15.55:0/3345833407 learned_addr learned my addr 172.21.15.55:0/3345833407 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c077fe640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08103420 0x7f2c081a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c08101870 0x7f2c08198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 -- 172.21.15.55:0/3345833407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c08101870 msgr2=0x7f2c08198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c08101870 0x7f2c08198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 -- 172.21.15.55:0/3345833407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08103420 msgr2=0x7f2c081a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08103420 0x7f2c081a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 -- 172.21.15.55:0/3345833407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c081a0800 con 0x7f2c08102a60 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c08101870 0x7f2c08198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c077fe640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08103420 0x7f2c081a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c067fc640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08102a60 0x7f2c08199120 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00e9f0 tx=0x7f2bfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2beffff640 1 -- 172.21.15.55:0/3345833407 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc00cd80 con 0x7f2c08102a60 2024-04-26T21:53:31.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2beffff640 1 -- 172.21.15.55:0/3345833407 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bfc0108e0 con 0x7f2c08102a60 2024-04-26T21:53:31.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c0d319640 1 -- 172.21.15.55:0/3345833407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c081a0af0 con 0x7f2c08102a60 2024-04-26T21:53:31.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2beffff640 1 -- 172.21.15.55:0/3345833407 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc018980 con 0x7f2c08102a60 2024-04-26T21:53:31.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.463+0000 7f2c0d319640 1 -- 172.21.15.55:0/3345833407 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c081a1040 con 0x7f2c08102a60 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2beffff640 1 -- 172.21.15.55:0/3345833407 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2bfc010430 con 0x7f2c08102a60 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c1003c740 1 Processor -- start 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c1003c740 1 -- start start 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b260310c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603119f0 0x55b260311e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b260318c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c1003c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b260274d80 con 0x55b2603123b0 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c1003c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b26022ea20 con 0x55b2603119f0 2024-04-26T21:53:31.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c1003c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b260270d40 con 0x55b260310820 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b260318c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b260318c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37126/0 (socket says 172.21.15.55:37126) 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c067fc640 1 -- 172.21.15.55:0/1649663696 learned_addr learned my addr 172.21.15.55:0/1649663696 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603119f0 0x55b260311e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2beffff640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2bf00a3c40 0x7f2bf00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.466+0000 7f2c077fe640 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b260310c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2c077fe640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2bf00a3c40 0x7f2bf00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2c067fc640 1 -- 172.21.15.55:0/1649663696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 msgr2=0x55b260310c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2c067fc640 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b260310c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2c067fc640 1 -- 172.21.15.55:0/1649663696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603119f0 msgr2=0x55b260311e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2c067fc640 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603119f0 0x55b260311e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2c067fc640 1 -- 172.21.15.55:0/1649663696 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b260239c30 con 0x55b2603123b0 2024-04-26T21:53:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2beffff640 1 -- 172.21.15.55:0/3345833407 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2bfc0d09e0 con 0x7f2c08102a60 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2c067fc640 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b260318c40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc09c820 tx=0x7f2bfc05c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60127804b8c2a997 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2c077fe640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2bf00a3c40 0x7f2bf00a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55b2603118f0 tx=0x7f2c000094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2bed7fa640 1 -- 172.21.15.55:0/1649663696 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0da040 con 0x55b2603123b0 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2bed7fa640 1 -- 172.21.15.55:0/1649663696 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bfc0d0e50 con 0x55b2603123b0 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.467+0000 7f2bed7fa640 1 -- 172.21.15.55:0/1649663696 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc09cb20 con 0x55b2603123b0 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- 172.21.15.55:0/1649663696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 msgr2=0x55b260318c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b260318c40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc09c820 tx=0x7f2bfc05c6d0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- 172.21.15.55:0/1649663696 shutdown_connections 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b260318c40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603119f0 0x55b260311e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 --2- 172.21.15.55:0/1649663696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b260310c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- 172.21.15.55:0/1649663696 >> 172.21.15.55:0/1649663696 conn(0x55b260251910 msgr2=0x55b26030c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- 172.21.15.55:0/1649663696 shutdown_connections 2024-04-26T21:53:31.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- 172.21.15.55:0/1649663696 wait complete. 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- start start 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 Processor -- start 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b26031a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26031aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603123b0 0x55b260321f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b260319360 con 0x55b2603119f0 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b2601fbdc0 con 0x55b2603123b0 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b2603194e0 con 0x55b260310820 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c1003c740 10 client.0 _ll_register_callbacks cb 0x55b260217600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26031aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26031aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37142/0 (socket says 172.21.15.55:37142) 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.468+0000 7f2c067fc640 1 -- 172.21.15.55:0/2663506985 learned_addr learned my addr 172.21.15.55:0/2663506985 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b26031a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c077fe640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603123b0 0x55b260321f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c067fc640 1 -- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 msgr2=0x55b26031a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c067fc640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b26031a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c067fc640 1 -- 172.21.15.55:0/2663506985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603123b0 msgr2=0x55b260321f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c067fc640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603123b0 0x55b260321f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c067fc640 1 -- 172.21.15.55:0/2663506985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25ff863f0 con 0x55b2603119f0 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b26031a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c077fe640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603123b0 0x55b260321f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c067fc640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26031aef0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc01f040 tx=0x7f2bfc014fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d6eacb6246b7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2beaff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2beaff5640 1 -- 172.21.15.55:0/2663506985 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0d7070 con 0x55b2603119f0 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2beaff5640 1 -- 172.21.15.55:0/2663506985 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2bfc0ce5a0 con 0x55b2603119f0 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c1003c740 1 -- 172.21.15.55:0/2663506985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bfc0d0e50 con 0x55b2603119f0 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2beaff5640 1 -- 172.21.15.55:0/2663506985 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2bfc0a0ed0 con 0x55b2603119f0 2024-04-26T21:53:31.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c1003c740 10 client.15426 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2c1003c740 1 -- 172.21.15.55:0/2663506985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b2600507d0 con 0x55b2603119f0 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2beaff5640 1 -- 172.21.15.55:0/2663506985 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bfc0d0e50 con 0x55b2603119f0 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.469+0000 7f2beaff5640 1 -- 172.21.15.55:0/2663506985 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2bfc0bbce0 con 0x55b2603119f0 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2c1003c740 10 client.15426 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2c1003c740 10 client.15426 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2c1003c740 20 client.15426 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2c1003c740 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55b260326420 0x55b260328810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2c1003c740 4 client.15426 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2c1003c740 1 -- 172.21.15.55:0/2663506985 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b260282990 con 0x55b260326420 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55b260326420 0x55b260328810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2beaff5640 10 client.15426 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.470+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55b260326420 0x55b260328810 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9083b96d2e29bb3 server_cookie=41c04d38693f2c in_seq=0 out_seq=0 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2beaff5640 1 -- 172.21.15.55:0/2663506985 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b260282990 con 0x55b260326420 2024-04-26T21:53:31.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2beaff5640 10 client.15426 handle_command_reply: tid=0 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 client.15426 shutdown 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 -- 172.21.15.55:0/2663506985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 msgr2=0x55b26031aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26031aef0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc01f040 tx=0x7f2bfc014fd0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 -- 172.21.15.55:0/2663506985 shutdown_connections 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55b260326420 0x55b260328810 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b2603123b0 0x55b260321f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26031aef0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 --2- 172.21.15.55:0/2663506985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b260310820 0x55b26031a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2beaff5640 0 client.15426 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 -- 172.21.15.55:0/2663506985 >> 172.21.15.55:0/2663506985 conn(0x55b260251910 msgr2=0x55b26030b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 -- 172.21.15.55:0/2663506985 shutdown_connections 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 1 -- 172.21.15.55:0/2663506985 wait complete. 2024-04-26T21:53:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.471+0000 7f2c1003c740 20 client.15426 trim_cache size 0 max 16384 2024-04-26T21:53:31.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c1003c740 1 Processor -- start 2024-04-26T21:53:31.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c1003c740 1 -- start start 2024-04-26T21:53:31.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b26030b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26030b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603123b0 0x55b26032cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c1003c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081ac870 con 0x55b2603119f0 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c1003c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081aa140 con 0x55b260310820 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c1003c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081a7f90 con 0x55b2603123b0 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26030b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26030b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37146/0 (socket says 172.21.15.55:37146) 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 -- 172.21.15.55:0/125490121 learned_addr learned my addr 172.21.15.55:0/125490121 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b26030b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 -- 172.21.15.55:0/125490121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603123b0 msgr2=0x55b26032cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603123b0 0x55b26032cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 -- 172.21.15.55:0/125490121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 msgr2=0x55b26030b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b26030b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 -- 172.21.15.55:0/125490121 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b260306660 con 0x55b2603119f0 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b26030b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2c067fc640 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26030b640 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc09cab0 tx=0x7f2bfc0b1490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8e4360d79b839a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.485+0000 7f2becff9640 1 -- 172.21.15.55:0/125490121 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0d7070 con 0x55b2603119f0 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2becff9640 1 -- 172.21.15.55:0/125490121 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bfc09de50 con 0x55b2603119f0 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2becff9640 1 -- 172.21.15.55:0/125490121 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc010a90 con 0x55b2603119f0 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 -- 172.21.15.55:0/125490121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 msgr2=0x55b26030b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26030b640 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc09cab0 tx=0x7f2bfc0b1490 comp rx=0 tx=0).stop 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 -- 172.21.15.55:0/125490121 shutdown_connections 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603123b0 0x55b26032cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603119f0 0x55b26030b640 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 --2- 172.21.15.55:0/125490121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b26030b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 -- 172.21.15.55:0/125490121 >> 172.21.15.55:0/125490121 conn(0x55b260251910 msgr2=0x55b26030a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 -- 172.21.15.55:0/125490121 shutdown_connections 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 -- 172.21.15.55:0/125490121 wait complete. 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 -- start start 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.486+0000 7f2c1003c740 1 Processor -- start 2024-04-26T21:53:31.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b260388050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603119f0 0x55b260388590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c1003c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b26038f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c1003c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081a8600 con 0x55b2603123b0 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c1003c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081a9a70 con 0x55b260310820 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c1003c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c081a8cd0 con 0x55b2603119f0 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c1003c740 10 client.0 _ll_register_callbacks cb 0x7f2bfc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b26038f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b26038f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37152/0 (socket says 172.21.15.55:37152) 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 -- 172.21.15.55:0/3815792339 learned_addr learned my addr 172.21.15.55:0/3815792339 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c067fc640 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603119f0 0x55b260388590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b260388050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 -- 172.21.15.55:0/3815792339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603119f0 msgr2=0x55b260388590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603119f0 0x55b260388590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 -- 172.21.15.55:0/3815792339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 msgr2=0x55b260388050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b260388050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 -- 172.21.15.55:0/3815792339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26038fbc0 con 0x55b2603123b0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b260388050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c077fe640 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b26038f5d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2c00010a10 tx=0x7f2c00009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc8080843efbea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2beaff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2beaff5640 1 -- 172.21.15.55:0/3815792339 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000fb90 con 0x55b2603123b0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2beaff5640 1 -- 172.21.15.55:0/3815792339 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2c0000fd30 con 0x55b2603123b0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.487+0000 7f2c1003c740 1 -- 172.21.15.55:0/3815792339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bfc09de50 con 0x55b2603123b0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2beaff5640 1 -- 172.21.15.55:0/3815792339 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2c00004b20 con 0x55b2603123b0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c1003c740 10 client.15432 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c1003c740 1 -- 172.21.15.55:0/3815792339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b260390f00 con 0x55b2603123b0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2beaff5640 1 -- 172.21.15.55:0/3815792339 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bfc09de50 con 0x55b2603123b0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2beaff5640 1 -- 172.21.15.55:0/3815792339 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2c00027420 con 0x55b2603123b0 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c1003c740 10 client.15432 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:31.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c1003c740 10 client.15432 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c1003c740 20 client.15432 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c1003c740 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55b2603918a0 0x55b260393cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c1003c740 4 client.15432 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c1003c740 1 -- 172.21.15.55:0/3815792339 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b26031a7a0 con 0x55b2603918a0 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.488+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55b2603918a0 0x55b260393cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2beaff5640 10 client.15432 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c06ffd640 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55b2603918a0 0x55b260393cc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=341626c0e43caf84 server_cookie=86d39589beb0ec16 in_seq=0 out_seq=0 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2beaff5640 1 -- 172.21.15.55:0/3815792339 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b26031a7a0 con 0x55b2603918a0 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2beaff5640 10 client.15432 handle_command_reply: tid=0 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 client.15432 shutdown 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 -- 172.21.15.55:0/3815792339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 msgr2=0x55b26038f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b26038f5d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2c00010a10 tx=0x7f2c00009a70 comp rx=0 tx=0).stop 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 -- 172.21.15.55:0/3815792339 shutdown_connections 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55b2603918a0 0x55b260393cc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b2603123b0 0x55b26038f5d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b2603119f0 0x55b260388590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 --2- 172.21.15.55:0/3815792339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b260310820 0x55b260388050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2beaff5640 0 client.15432 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:53:31.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 -- 172.21.15.55:0/3815792339 >> 172.21.15.55:0/3815792339 conn(0x55b260251910 msgr2=0x55b26030b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 -- 172.21.15.55:0/3815792339 shutdown_connections 2024-04-26T21:53:31.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 1 -- 172.21.15.55:0/3815792339 wait complete. 2024-04-26T21:53:31.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.489+0000 7f2c1003c740 20 client.15432 trim_cache size 0 max 16384 2024-04-26T21:53:31.496 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 -- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2bf00a3c40 msgr2=0x7f2bf00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2bf00a3c40 0x7f2bf00a6100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55b2603118f0 tx=0x7f2c000094a0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 -- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08102a60 msgr2=0x7f2c08199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08102a60 0x7f2c08199120 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00e9f0 tx=0x7f2bfc00eeb0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 -- 172.21.15.55:0/3345833407 shutdown_connections 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2bf00a3c40 0x7f2bf00a6100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c08103420 0x7f2c081a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c08102a60 0x7f2c08199120 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 --2- 172.21.15.55:0/3345833407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c08101870 0x7f2c08198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 -- 172.21.15.55:0/3345833407 >> 172.21.15.55:0/3345833407 conn(0x7f2c080fcde0 msgr2=0x7f2c08105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 -- 172.21.15.55:0/3345833407 shutdown_connections 2024-04-26T21:53:31.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.490+0000 7f2c0d319640 1 -- 172.21.15.55:0/3345833407 wait complete. 2024-04-26T21:53:31.501 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:53:31.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:31 smithi064 ceph-mon[32765]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:31.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:31 smithi165 ceph-mon[33296]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:31.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.604+0000 7ff004adc640 1 -- 172.21.15.55:0/3131667360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff000108710 msgr2=0x7ff00010ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.604+0000 7ff004adc640 1 --2- 172.21.15.55:0/3131667360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff000108710 0x7ff00010ab00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feff8009be0 tx=0x7feff802a6c0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.604+0000 7ff004adc640 1 -- 172.21.15.55:0/3131667360 shutdown_connections 2024-04-26T21:53:31.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.605+0000 7ff004adc640 1 --2- 172.21.15.55:0/3131667360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff000108710 0x7ff00010ab00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.605+0000 7ff004adc640 1 --2- 172.21.15.55:0/3131667360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff000105de0 0x7ff0001081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.605+0000 7ff004adc640 1 --2- 172.21.15.55:0/3131667360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0001034b0 0x7ff0001058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.605+0000 7ff004adc640 1 -- 172.21.15.55:0/3131667360 >> 172.21.15.55:0/3131667360 conn(0x7ff0000fcde0 msgr2=0x7ff0000ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.605+0000 7ff004adc640 1 -- 172.21.15.55:0/3131667360 shutdown_connections 2024-04-26T21:53:31.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.605+0000 7ff004adc640 1 -- 172.21.15.55:0/3131667360 wait complete. 2024-04-26T21:53:31.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.605+0000 7ff004adc640 1 Processor -- start 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.605+0000 7ff004adc640 1 -- start start 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7ff004adc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0001034b0 0x7ff000102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7ff004adc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff000105de0 0x7ff000101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7ff004adc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff000108710 0x7ff0001019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7ff004adc640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010e4d0 con 0x7ff0001034b0 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7ff004adc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010e350 con 0x7ff000105de0 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7ff004adc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff00010e650 con 0x7ff000108710 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0001034b0 0x7ff000102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0001034b0 0x7ff000102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37176/0 (socket says 172.21.15.55:37176) 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 -- 172.21.15.55:0/582465989 learned_addr learned my addr 172.21.15.55:0/582465989 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffed9d640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff000108710 0x7ff0001019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feff5d9b640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff000105de0 0x7ff000101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 -- 172.21.15.55:0/582465989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff000108710 msgr2=0x7ff0001019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff000108710 0x7ff0001019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 -- 172.21.15.55:0/582465989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff000105de0 msgr2=0x7ff000101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff000105de0 0x7ff000101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 -- 172.21.15.55:0/582465989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff000102230 con 0x7ff0001034b0 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feff5d9b640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff000105de0 0x7ff000101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.606+0000 7feffe59c640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0001034b0 0x7ff000102e20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9f0 tx=0x7feff000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.607+0000 7feff7fff640 1 -- 172.21.15.55:0/582465989 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff000ccc0 con 0x7ff0001034b0 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.607+0000 7ff004adc640 1 -- 172.21.15.55:0/582465989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0001a2cf0 con 0x7ff0001034b0 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.607+0000 7feff7fff640 1 -- 172.21.15.55:0/582465989 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff00054d0 con 0x7ff0001034b0 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.607+0000 7feff7fff640 1 -- 172.21.15.55:0/582465989 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0010430 con 0x7ff0001034b0 2024-04-26T21:53:31.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.607+0000 7ff004adc640 1 -- 172.21.15.55:0/582465989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0001a3190 con 0x7ff0001034b0 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.609+0000 7feff7fff640 1 -- 172.21.15.55:0/582465989 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feff00046c0 con 0x7ff0001034b0 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.609+0000 7ff0077f3740 1 Processor -- start 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.609+0000 7ff0077f3740 1 -- start start 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff7fff640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fefec0a3c40 0x7fefec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389caf820 0x561389cafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb13b0 0x561389cb7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7ff0077f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561389c13d80 con 0x561389cb09f0 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7ff0077f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561389bcda20 con 0x561389cb13b0 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7ff0077f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561389c0fd40 con 0x561389caf820 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feffed9d640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fefec0a3c40 0x7fefec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37186/0 (socket says 172.21.15.55:37186) 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 -- 172.21.15.55:0/3505155642 learned_addr learned my addr 172.21.15.55:0/3505155642 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff7fff640 1 -- 172.21.15.55:0/582465989 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7feff00d09b0 con 0x7ff0001034b0 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feffed9d640 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb13b0 0x561389cb7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feffe59c640 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389caf820 0x561389cafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 -- 172.21.15.55:0/3505155642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389caf820 msgr2=0x561389cafc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389caf820 0x561389cafc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 -- 172.21.15.55:0/3505155642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb13b0 msgr2=0x561389cb7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb13b0 0x561389cb7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 -- 172.21.15.55:0/3505155642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561389bd8c30 con 0x561389cb09f0 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feffed9d640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fefec0a3c40 0x7fefec0a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feff8006010 tx=0x7feff8030710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff5d9b640 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb0e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fefe800bce0 tx=0x7fefe800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3d6723b589b8a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.610+0000 7feff4d99640 1 -- 172.21.15.55:0/3505155642 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8015070 con 0x561389cb09f0 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7feff4d99640 1 -- 172.21.15.55:0/3505155642 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe80118f0 con 0x561389cb09f0 2024-04-26T21:53:31.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7feff4d99640 1 -- 172.21.15.55:0/3505155642 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8010820 con 0x561389cb09f0 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 -- 172.21.15.55:0/3505155642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 msgr2=0x561389cb0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb0e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fefe800bce0 tx=0x7fefe800f730 comp rx=0 tx=0).stop 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 -- 172.21.15.55:0/3505155642 shutdown_connections 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb13b0 0x561389cb7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb0e70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 --2- 172.21.15.55:0/3505155642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389caf820 0x561389cafc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 -- 172.21.15.55:0/3505155642 >> 172.21.15.55:0/3505155642 conn(0x561389bf0910 msgr2=0x561389cab120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 -- 172.21.15.55:0/3505155642 shutdown_connections 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.611+0000 7ff0077f3740 1 -- 172.21.15.55:0/3505155642 wait complete. 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 1 -- start start 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 1 Processor -- start 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 0x561389cc0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561389cb83c0 con 0x561389cb09f0 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561389b9adc0 con 0x561389caf820 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561389cb8540 con 0x561389cb13b0 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7ff0077f3740 10 client.0 _ll_register_callbacks cb 0x561389bb6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feff5d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feff5d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37196/0 (socket says 172.21.15.55:37196) 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feff5d9b640 1 -- 172.21.15.55:0/2517507895 learned_addr learned my addr 172.21.15.55:0/2517507895 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feffed9d640 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 0x561389cc0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feffe59c640 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feff5d9b640 1 -- 172.21.15.55:0/2517507895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 msgr2=0x561389cc0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feff5d9b640 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 0x561389cc0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feff5d9b640 1 -- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 msgr2=0x561389cb9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feff5d9b640 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feff5d9b640 1 -- 172.21.15.55:0/2517507895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613899253f0 con 0x561389cb09f0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.612+0000 7feffe59c640 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7feff5d9b640 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb9fa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fefe8000c00 tx=0x7fefe8011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcfb148c5c3b576 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7fefe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7fefe5ffb640 1 -- 172.21.15.55:0/2517507895 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8015050 con 0x561389cb09f0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7fefe5ffb640 1 -- 172.21.15.55:0/2517507895 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fefe8011cb0 con 0x561389cb09f0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 1 -- 172.21.15.55:0/2517507895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefe80118f0 con 0x561389cb09f0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7fefe5ffb640 1 -- 172.21.15.55:0/2517507895 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fefe8004030 con 0x561389cb09f0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 10 client.15444 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 1 -- 172.21.15.55:0/2517507895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5613899ef7d0 con 0x561389cb09f0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7fefe5ffb640 1 -- 172.21.15.55:0/2517507895 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe80118f0 con 0x561389cb09f0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7fefe5ffb640 1 -- 172.21.15.55:0/2517507895 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fefe80105a0 con 0x561389cb09f0 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 10 client.15444 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 10 client.15444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 20 client.15444 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x561389cc5570 0x561389cc79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 4 client.15444 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.613+0000 7ff0077f3740 1 -- 172.21.15.55:0/2517507895 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561389c21990 con 0x561389cc5570 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.614+0000 7feffe59c640 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x561389cc5570 0x561389cc79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.614+0000 7feffe59c640 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x561389cc5570 0x561389cc79e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e912163120993e5f server_cookie=d8fa2f8bbea9d186 in_seq=0 out_seq=0 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.614+0000 7fefe5ffb640 10 client.15444 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7fefe5ffb640 1 -- 172.21.15.55:0/2517507895 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561389c21990 con 0x561389cc5570 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7fefe5ffb640 10 client.15444 handle_command_reply: tid=0 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 client.15444 shutdown 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 -- 172.21.15.55:0/2517507895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 msgr2=0x561389cb9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb9fa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fefe8000c00 tx=0x7fefe8011670 comp rx=0 tx=0).stop 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 -- 172.21.15.55:0/2517507895 shutdown_connections 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x561389cc5570 0x561389cc79e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 0x561389cc0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7fefe5ffb640 0 client.15444 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:53:31.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb09f0 0x561389cb9fa0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 --2- 172.21.15.55:0/2517507895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 -- 172.21.15.55:0/2517507895 >> 172.21.15.55:0/2517507895 conn(0x561389bf0910 msgr2=0x561389caa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 -- 172.21.15.55:0/2517507895 shutdown_connections 2024-04-26T21:53:31.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 1 -- 172.21.15.55:0/2517507895 wait complete. 2024-04-26T21:53:31.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.615+0000 7ff0077f3740 20 client.15444 trim_cache size 0 max 16384 2024-04-26T21:53:31.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.628+0000 7ff0077f3740 1 Processor -- start 2024-04-26T21:53:31.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.628+0000 7ff0077f3740 1 -- start start 2024-04-26T21:53:31.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.628+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.628+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb09f0 0x561389cc0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.628+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb13b0 0x561389cb4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.628+0000 7ff0077f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001ae9b0 con 0x561389cb13b0 2024-04-26T21:53:31.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7ff0077f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001ac280 con 0x561389caf820 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7ff0077f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001aa0d0 con 0x561389cb09f0 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb13b0 0x561389cb4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb13b0 0x561389cb4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37208/0 (socket says 172.21.15.55:37208) 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 -- 172.21.15.55:0/3380056922 learned_addr learned my addr 172.21.15.55:0/3380056922 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffe59c640 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 -- 172.21.15.55:0/3380056922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb09f0 msgr2=0x561389cc0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feff5d9b640 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb09f0 0x561389cc0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb09f0 0x561389cc0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 -- 172.21.15.55:0/3380056922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 msgr2=0x561389cb9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 -- 172.21.15.55:0/3380056922 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561389cb8690 con 0x561389cb13b0 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feff5d9b640 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb09f0 0x561389cc0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffe59c640 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:31.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7feffed9d640 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb13b0 0x561389cb4250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feff8033000 tx=0x7feff8044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d1b7d9c9b70f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7fefe7fff640 1 -- 172.21.15.55:0/3380056922 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8051070 con 0x561389cb13b0 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7fefe7fff640 1 -- 172.21.15.55:0/3380056922 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff8003cf0 con 0x561389cb13b0 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.629+0000 7fefe7fff640 1 -- 172.21.15.55:0/3380056922 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8036040 con 0x561389cb13b0 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- 172.21.15.55:0/3380056922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb13b0 msgr2=0x561389cb4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb13b0 0x561389cb4250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feff8033000 tx=0x7feff8044000 comp rx=0 tx=0).stop 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- 172.21.15.55:0/3380056922 shutdown_connections 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389cb13b0 0x561389cb4250 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb09f0 0x561389cc0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 --2- 172.21.15.55:0/3380056922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389caf820 0x561389cb9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- 172.21.15.55:0/3380056922 >> 172.21.15.55:0/3380056922 conn(0x561389bf0910 msgr2=0x561389cb9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- 172.21.15.55:0/3380056922 shutdown_connections 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- 172.21.15.55:0/3380056922 wait complete. 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- start start 2024-04-26T21:53:31.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 Processor -- start 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389caf820 0x561389cbde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb09f0 0x561389cbe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 0x561389d29b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001aa740 con 0x561389caf820 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001abbb0 con 0x561389cb09f0 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.630+0000 7ff0077f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001aae10 con 0x561389cb13b0 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7ff0077f3740 10 client.0 _ll_register_callbacks cb 0x7fefe800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389caf820 0x561389cbde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389caf820 0x561389cbde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37220/0 (socket says 172.21.15.55:37220) 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 -- 172.21.15.55:0/1255242931 learned_addr learned my addr 172.21.15.55:0/1255242931 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feff5d9b640 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb09f0 0x561389cbe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffed9d640 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 0x561389d29b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 -- 172.21.15.55:0/1255242931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 msgr2=0x561389d29b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 0x561389d29b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 -- 172.21.15.55:0/1255242931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb09f0 msgr2=0x561389cbe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb09f0 0x561389cbe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 -- 172.21.15.55:0/1255242931 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561389d2a050 con 0x561389caf820 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7feffe59c640 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389caf820 0x561389cbde80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feff0099460 tx=0x7feff001c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb3a806eeedee02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7fefe5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7fefe5ffb640 1 -- 172.21.15.55:0/1255242931 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff009ca90 con 0x561389caf820 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7fefe5ffb640 1 -- 172.21.15.55:0/1255242931 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7feff0098a20 con 0x561389caf820 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7ff0077f3740 1 -- 172.21.15.55:0/1255242931 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff8003cf0 con 0x561389caf820 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7fefe5ffb640 1 -- 172.21.15.55:0/1255242931 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7feff00a0e40 con 0x561389caf820 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7ff0077f3740 10 client.15450 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:31.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7ff0077f3740 1 -- 172.21.15.55:0/1255242931 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561389d2b340 con 0x561389caf820 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.631+0000 7fefe5ffb640 1 -- 172.21.15.55:0/1255242931 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff8003cf0 con 0x561389caf820 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7fefe5ffb640 1 -- 172.21.15.55:0/1255242931 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7feff00b9030 con 0x561389caf820 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7ff0077f3740 10 client.15450 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7ff0077f3740 10 client.15450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7ff0077f3740 20 client.15450 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7ff0077f3740 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x561389d2bce0 0x561389d2e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7ff0077f3740 4 client.15450 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7ff0077f3740 1 -- 172.21.15.55:0/1255242931 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561389c1b800 con 0x561389d2bce0 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7feff5d9b640 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x561389d2bce0 0x561389d2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7feff5d9b640 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x561389d2bce0 0x561389d2e100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be124242c9324086 server_cookie=88a1bc42c218ca6a in_seq=0 out_seq=0 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7fefe5ffb640 10 client.15450 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:53:31.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7fefe5ffb640 1 -- 172.21.15.55:0/1255242931 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561389c1b800 con 0x561389d2bce0 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7fefe5ffb640 10 client.15450 handle_command_reply: tid=0 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.632+0000 7ff0077f3740 1 client.15450 shutdown 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 -- 172.21.15.55:0/1255242931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389caf820 msgr2=0x561389cbde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389caf820 0x561389cbde80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7feff0099460 tx=0x7feff001c590 comp rx=0 tx=0).stop 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 -- 172.21.15.55:0/1255242931 shutdown_connections 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x561389d2bce0 0x561389d2e100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561389cb13b0 0x561389d29b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7fefe5ffb640 0 client.15450 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561389cb09f0 0x561389cbe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 --2- 172.21.15.55:0/1255242931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561389caf820 0x561389cbde80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 -- 172.21.15.55:0/1255242931 >> 172.21.15.55:0/1255242931 conn(0x561389bf0910 msgr2=0x561389cb9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 -- 172.21.15.55:0/1255242931 shutdown_connections 2024-04-26T21:53:31.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 1 -- 172.21.15.55:0/1255242931 wait complete. 2024-04-26T21:53:31.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff0077f3740 20 client.15450 trim_cache size 0 max 16384 2024-04-26T21:53:31.642 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 -- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fefec0a3c40 msgr2=0x7fefec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fefec0a3c40 0x7fefec0a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feff8006010 tx=0x7feff8030710 comp rx=0 tx=0).stop 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 -- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0001034b0 msgr2=0x7ff000102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0001034b0 0x7ff000102e20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feff000e9f0 tx=0x7feff000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 -- 172.21.15.55:0/582465989 shutdown_connections 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fefec0a3c40 0x7fefec0a6100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff000108710 0x7ff0001019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff000105de0 0x7ff000101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 --2- 172.21.15.55:0/582465989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0001034b0 0x7ff000102e20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 -- 172.21.15.55:0/582465989 >> 172.21.15.55:0/582465989 conn(0x7ff0000fcde0 msgr2=0x7ff000103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 -- 172.21.15.55:0/582465989 shutdown_connections 2024-04-26T21:53:31.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.633+0000 7ff004adc640 1 -- 172.21.15.55:0/582465989 wait complete. 2024-04-26T21:53:31.644 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:53:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 -- 172.21.15.55:0/2071838262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558103410 msgr2=0x7f1558105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 --2- 172.21.15.55:0/2071838262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558103410 0x7f1558105800 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f154c009900 tx=0x7f154c02a410 comp rx=0 tx=0).stop 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 -- 172.21.15.55:0/2071838262 shutdown_connections 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 --2- 172.21.15.55:0/2071838262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1558108670 0x7f155810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 --2- 172.21.15.55:0/2071838262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1558105d40 0x7f1558108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 --2- 172.21.15.55:0/2071838262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558103410 0x7f1558105800 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 -- 172.21.15.55:0/2071838262 >> 172.21.15.55:0/2071838262 conn(0x7f15580fce20 msgr2=0x7f15580ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 -- 172.21.15.55:0/2071838262 shutdown_connections 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.748+0000 7f155ee47640 1 -- 172.21.15.55:0/2071838262 wait complete. 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155ee47640 1 Processor -- start 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155ee47640 1 -- start start 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155ee47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1558103410 0x7f1558198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155ee47640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1558105d40 0x7f1558199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155ee47640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558108670 0x7f15581a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155ee47640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f155810e4c0 con 0x7f1558108670 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155ee47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f155810e340 con 0x7f1558103410 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155ee47640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f155810e640 con 0x7f1558105d40 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558108670 0x7f15581a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558108670 0x7f15581a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37226/0 (socket says 172.21.15.55:37226) 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 -- 172.21.15.55:0/344516159 learned_addr learned my addr 172.21.15.55:0/344516159 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f1557fff640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1558105d40 0x7f1558199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155cbe3640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1558103410 0x7f1558198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 -- 172.21.15.55:0/344516159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1558105d40 msgr2=0x7f1558199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1558105d40 0x7f1558199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 -- 172.21.15.55:0/344516159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1558103410 msgr2=0x7f1558198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1558103410 0x7f1558198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 -- 172.21.15.55:0/344516159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15581a0830 con 0x7f1558108670 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155cbe3640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1558103410 0x7f1558198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.749+0000 7f155d3e4640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558108670 0x7f15581a0190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f155000eb10 tx=0x7f155000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.750+0000 7f1555ffb640 1 -- 172.21.15.55:0/344516159 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550004b20 con 0x7f1558108670 2024-04-26T21:53:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.750+0000 7f155ee47640 1 -- 172.21.15.55:0/344516159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15581a0b20 con 0x7f1558108670 2024-04-26T21:53:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.750+0000 7f1555ffb640 1 -- 172.21.15.55:0/344516159 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1550004cc0 con 0x7f1558108670 2024-04-26T21:53:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.750+0000 7f1555ffb640 1 -- 172.21.15.55:0/344516159 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550010670 con 0x7f1558108670 2024-04-26T21:53:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.750+0000 7f155ee47640 1 -- 172.21.15.55:0/344516159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15581a1020 con 0x7f1558108670 2024-04-26T21:53:31.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f1555ffb640 1 -- 172.21.15.55:0/344516159 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1550005020 con 0x7f1558108670 2024-04-26T21:53:31.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f15617fd740 1 Processor -- start 2024-04-26T21:53:31.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f15617fd740 1 -- start start 2024-04-26T21:53:31.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f15617fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 0x56476a210c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f15617fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2119f0 0x56476a211e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f15617fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2123b0 0x56476a218c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f15617fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56476a174d80 con 0x56476a2119f0 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f15617fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56476a12ea20 con 0x56476a2123b0 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.752+0000 7f15617fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56476a170d40 con 0x56476a210820 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2119f0 0x56476a211e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f1555ffb640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15480a3c40 0x7f15480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2119f0 0x56476a211e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37236/0 (socket says 172.21.15.55:37236) 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 -- 172.21.15.55:0/1086201740 learned_addr learned my addr 172.21.15.55:0/1086201740 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155d3e4640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15480a3c40 0x7f15480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f1557fff640 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 0x56476a210c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f1555ffb640 1 -- 172.21.15.55:0/344516159 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f15500d0d40 con 0x7f1558108670 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 -- 172.21.15.55:0/1086201740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 msgr2=0x56476a210c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f1557fff640 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2123b0 0x56476a218c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 0x56476a210c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 -- 172.21.15.55:0/1086201740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2123b0 msgr2=0x56476a218c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2123b0 0x56476a218c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 -- 172.21.15.55:0/1086201740 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56476a139c30 con 0x56476a2119f0 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f1557fff640 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2123b0 0x56476a218c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155cbe3640 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2119f0 0x56476a211e70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f154c005ab0 tx=0x7f154c032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9968a55e64e191b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f155d3e4640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15480a3c40 0x7f15480a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15500997c0 tx=0x7f15500bd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f1542ffd640 1 -- 172.21.15.55:0/1086201740 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c03f030 con 0x56476a2119f0 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f1542ffd640 1 -- 172.21.15.55:0/1086201740 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154c032d90 con 0x56476a2119f0 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.753+0000 7f1542ffd640 1 -- 172.21.15.55:0/1086201740 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c035070 con 0x56476a2119f0 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 -- 172.21.15.55:0/1086201740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2119f0 msgr2=0x56476a211e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2119f0 0x56476a211e70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f154c005ab0 tx=0x7f154c032ba0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 -- 172.21.15.55:0/1086201740 shutdown_connections 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2123b0 0x56476a218c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2119f0 0x56476a211e70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 --2- 172.21.15.55:0/1086201740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 0x56476a210c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 -- 172.21.15.55:0/1086201740 >> 172.21.15.55:0/1086201740 conn(0x56476a151910 msgr2=0x56476a20c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 -- 172.21.15.55:0/1086201740 shutdown_connections 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 -- 172.21.15.55:0/1086201740 wait complete. 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 -- start start 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.754+0000 7f15617fd740 1 Processor -- start 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f15617fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 0x56476a21aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f15617fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a21af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f15617fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2123b0 0x56476a221f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f15617fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56476a219360 con 0x56476a2123b0 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f15617fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56476a0fbdc0 con 0x56476a2119f0 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f15617fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56476a2194e0 con 0x56476a210820 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f15617fd740 10 client.0 _ll_register_callbacks cb 0x56476a117600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2123b0 0x56476a221f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2123b0 0x56476a221f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37242/0 (socket says 172.21.15.55:37242) 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 -- 172.21.15.55:0/3123922451 learned_addr learned my addr 172.21.15.55:0/3123922451 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f155cbe3640 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a21af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 0x56476a21aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 -- 172.21.15.55:0/3123922451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 msgr2=0x56476a21aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 0x56476a21aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 -- 172.21.15.55:0/3123922451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 msgr2=0x56476a21af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a21af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.755+0000 7f1557fff640 1 -- 172.21.15.55:0/3123922451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564769e863f0 con 0x56476a2123b0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f1557fff640 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2123b0 0x56476a221f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1544012da0 tx=0x7f15440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8849ab6e980770c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f1537fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f1537fff640 1 -- 172.21.15.55:0/3123922451 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544020400 con 0x56476a2123b0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 1 -- 172.21.15.55:0/3123922451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154c032d90 con 0x56476a2123b0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f1537fff640 1 -- 172.21.15.55:0/3123922451 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f154401f3a0 con 0x56476a2123b0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f1537fff640 1 -- 172.21.15.55:0/3123922451 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f15440205a0 con 0x56476a2123b0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 10 client.15462 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 1 -- 172.21.15.55:0/3123922451 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564769f507d0 con 0x56476a2123b0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f1537fff640 1 -- 172.21.15.55:0/3123922451 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154c032d90 con 0x56476a2123b0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f1537fff640 1 -- 172.21.15.55:0/3123922451 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f154401f840 con 0x56476a2123b0 2024-04-26T21:53:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 10 client.15462 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 20 client.15462 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56476a226780 0x56476a228c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 4 client.15462 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.756+0000 7f15617fd740 1 -- 172.21.15.55:0/3123922451 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56476a182990 con 0x56476a226780 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.757+0000 7f155cbe3640 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56476a226780 0x56476a228c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.757+0000 7f1537fff640 10 client.15462 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.757+0000 7f155cbe3640 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56476a226780 0x56476a228c20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2873598b382aaff8 server_cookie=7a0d7c142e6620de in_seq=0 out_seq=0 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f1537fff640 1 -- 172.21.15.55:0/3123922451 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56476a182990 con 0x56476a226780 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f1537fff640 10 client.15462 handle_command_reply: tid=0 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 client.15462 shutdown 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 -- 172.21.15.55:0/3123922451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2123b0 msgr2=0x56476a221f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2123b0 0x56476a221f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1544012da0 tx=0x7f15440046c0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 -- 172.21.15.55:0/3123922451 shutdown_connections 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56476a226780 0x56476a228c20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a2123b0 0x56476a221f90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a21af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f1537fff640 0 client.15462 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 --2- 172.21.15.55:0/3123922451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a210820 0x56476a21aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 -- 172.21.15.55:0/3123922451 >> 172.21.15.55:0/3123922451 conn(0x56476a151910 msgr2=0x56476a20b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 -- 172.21.15.55:0/3123922451 shutdown_connections 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 1 -- 172.21.15.55:0/3123922451 wait complete. 2024-04-26T21:53:31.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.758+0000 7f15617fd740 20 client.15462 trim_cache size 0 max 16384 2024-04-26T21:53:31.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.771+0000 7f15617fd740 1 Processor -- start 2024-04-26T21:53:31.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.771+0000 7f15617fd740 1 -- start start 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.771+0000 7f15617fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a20bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.771+0000 7f15617fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a20a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.771+0000 7f15617fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 0x56476a20af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.771+0000 7f15617fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15581ac7a0 con 0x56476a210820 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.771+0000 7f15617fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15581aa070 con 0x56476a2119f0 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.771+0000 7f15617fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f15581a7ec0 con 0x56476a2123b0 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a20bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a20bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37258/0 (socket says 172.21.15.55:37258) 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 -- 172.21.15.55:0/3544166737 learned_addr learned my addr 172.21.15.55:0/3544166737 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 -- 172.21.15.55:0/3544166737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 msgr2=0x56476a20af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 --2- 172.21.15.55:0/3544166737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 0x56476a20af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 -- 172.21.15.55:0/3544166737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 msgr2=0x56476a20a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 --2- 172.21.15.55:0/3544166737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a20a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 -- 172.21.15.55:0/3544166737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56476a2251b0 con 0x56476a210820 2024-04-26T21:53:31.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f1557fff640 1 --2- 172.21.15.55:0/3544166737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a20bba0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f15440058c0 tx=0x7f1544010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10106ef70d88ca87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f15427fc640 1 -- 172.21.15.55:0/3544166737 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544020400 con 0x56476a210820 2024-04-26T21:53:31.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f15427fc640 1 -- 172.21.15.55:0/3544166737 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154402f070 con 0x56476a210820 2024-04-26T21:53:31.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.772+0000 7f15427fc640 1 -- 172.21.15.55:0/3544166737 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544005190 con 0x56476a210820 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- 172.21.15.55:0/3544166737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 msgr2=0x56476a20bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 --2- 172.21.15.55:0/3544166737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a20bba0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f15440058c0 tx=0x7f1544010f50 comp rx=0 tx=0).stop 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- 172.21.15.55:0/3544166737 shutdown_connections 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 --2- 172.21.15.55:0/3544166737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 0x56476a20af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 --2- 172.21.15.55:0/3544166737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a20a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 --2- 172.21.15.55:0/3544166737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a20bba0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- 172.21.15.55:0/3544166737 >> 172.21.15.55:0/3544166737 conn(0x56476a151910 msgr2=0x56476a215250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- 172.21.15.55:0/3544166737 shutdown_connections 2024-04-26T21:53:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- 172.21.15.55:0/3544166737 wait complete. 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- start start 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 Processor -- start 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a288350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a288890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 0x56476a28f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f15581a8530 con 0x56476a210820 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15581a99a0 con 0x56476a2119f0 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.773+0000 7f15617fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f15581a8c00 con 0x56476a2123b0 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f15617fd740 10 client.0 _ll_register_callbacks cb 0x7f1544009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a288350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a288350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37266/0 (socket says 172.21.15.55:37266) 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 -- 172.21.15.55:0/3677767508 learned_addr learned my addr 172.21.15.55:0/3677767508 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f155cbe3640 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a288890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 0x56476a28f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 -- 172.21.15.55:0/3677767508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 msgr2=0x56476a28f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 0x56476a28f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 -- 172.21.15.55:0/3677767508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 msgr2=0x56476a288890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a288890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 -- 172.21.15.55:0/3677767508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56476a28fec0 con 0x56476a210820 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1557fff640 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a288350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f15440092e0 tx=0x7f1544005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa6e8d4dd681fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1537fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1537fff640 1 -- 172.21.15.55:0/3677767508 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544020400 con 0x56476a210820 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1537fff640 1 -- 172.21.15.55:0/3677767508 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f154401f5b0 con 0x56476a210820 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f15617fd740 1 -- 172.21.15.55:0/3677767508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154402f070 con 0x56476a210820 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f1537fff640 1 -- 172.21.15.55:0/3677767508 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f154401f7b0 con 0x56476a210820 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f15617fd740 10 client.15468 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.774+0000 7f15617fd740 1 -- 172.21.15.55:0/3677767508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56476a2911c0 con 0x56476a210820 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f1537fff640 1 -- 172.21.15.55:0/3677767508 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154402f070 con 0x56476a210820 2024-04-26T21:53:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f1537fff640 1 -- 172.21.15.55:0/3677767508 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1544003710 con 0x56476a210820 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f15617fd740 10 client.15468 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f15617fd740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f15617fd740 20 client.15468 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f15617fd740 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56476a291a80 0x56476a293ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f15617fd740 4 client.15468 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f15617fd740 1 -- 172.21.15.55:0/3677767508 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56476a17c800 con 0x56476a291a80 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f155cbe3640 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56476a291a80 0x56476a293ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f155cbe3640 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56476a291a80 0x56476a293ea0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c682937c333a71f4 server_cookie=f577e4a1ceaacac1 in_seq=0 out_seq=0 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f1537fff640 10 client.15468 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.775+0000 7f1537fff640 1 -- 172.21.15.55:0/3677767508 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56476a17c800 con 0x56476a291a80 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f1537fff640 10 client.15468 handle_command_reply: tid=0 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 client.15468 shutdown 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 -- 172.21.15.55:0/3677767508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 msgr2=0x56476a288350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a288350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f15440092e0 tx=0x7f1544005f90 comp rx=0 tx=0).stop 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 -- 172.21.15.55:0/3677767508 shutdown_connections 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56476a2123b0 0x56476a28f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56476a2119f0 0x56476a288890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56476a291a80 0x56476a293ea0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 --2- 172.21.15.55:0/3677767508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56476a210820 0x56476a288350 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f1537fff640 0 client.15468 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 -- 172.21.15.55:0/3677767508 >> 172.21.15.55:0/3677767508 conn(0x56476a151910 msgr2=0x56476a20f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 -- 172.21.15.55:0/3677767508 shutdown_connections 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 1 -- 172.21.15.55:0/3677767508 wait complete. 2024-04-26T21:53:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f15617fd740 20 client.15468 trim_cache size 0 max 16384 2024-04-26T21:53:31.786 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:53:31.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 -- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15480a3c40 msgr2=0x7f15480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15480a3c40 0x7f15480a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15500997c0 tx=0x7f15500bd040 comp rx=0 tx=0).stop 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 -- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558108670 msgr2=0x7f15581a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558108670 0x7f15581a0190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f155000eb10 tx=0x7f155000c2d0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 -- 172.21.15.55:0/344516159 shutdown_connections 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f15480a3c40 0x7f15480a6100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1558108670 0x7f15581a0190 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1558105d40 0x7f1558199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 --2- 172.21.15.55:0/344516159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1558103410 0x7f1558198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.776+0000 7f155ee47640 1 -- 172.21.15.55:0/344516159 >> 172.21.15.55:0/344516159 conn(0x7f15580fce20 msgr2=0x7f1558105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.777+0000 7f155ee47640 1 -- 172.21.15.55:0/344516159 shutdown_connections 2024-04-26T21:53:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:31.777+0000 7f155ee47640 1 -- 172.21.15.55:0/344516159 wait complete. 2024-04-26T21:53:32.422 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:32 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2644665325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:32.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:32 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2644665325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:32.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:32 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2644665325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:53:32.789 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:53:32.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:53:32.789 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:53:32.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 -- 172.21.15.55:0/3323846851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 msgr2=0x7fcc28101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3323846851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 0x7fcc28101c90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009900 tx=0x7fcc1802a410 comp rx=0 tx=0).stop 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 -- 172.21.15.55:0/3323846851 shutdown_connections 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3323846851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc28103420 0x7fcc28109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3323846851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc28102a60 0x7fcc28102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3323846851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 0x7fcc28101c90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 -- 172.21.15.55:0/3323846851 >> 172.21.15.55:0/3323846851 conn(0x7fcc280fcde0 msgr2=0x7fcc280ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 -- 172.21.15.55:0/3323846851 shutdown_connections 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 -- 172.21.15.55:0/3323846851 wait complete. 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 Processor -- start 2024-04-26T21:53:32.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.908+0000 7fcc2e322640 1 -- start start 2024-04-26T21:53:32.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2e322640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 0x7fcc28105800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2e322640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc28102a60 0x7fcc28103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2e322640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc28103420 0x7fcc28104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2e322640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810d390 con 0x7fcc28101870 2024-04-26T21:53:32.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2e322640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810d210 con 0x7fcc28103420 2024-04-26T21:53:32.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2e322640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810d510 con 0x7fcc28102a60 2024-04-26T21:53:32.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 0x7fcc28105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 0x7fcc28105800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37286/0 (socket says 172.21.15.55:37286) 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc27fff640 1 -- 172.21.15.55:0/3844224096 learned_addr learned my addr 172.21.15.55:0/3844224096 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2c8bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc28103420 0x7fcc28104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc277fe640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc28102a60 0x7fcc28103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2c8bf640 1 -- 172.21.15.55:0/3844224096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc28102a60 msgr2=0x7fcc28103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2c8bf640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc28102a60 0x7fcc28103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2c8bf640 1 -- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 msgr2=0x7fcc28105800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2c8bf640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 0x7fcc28105800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2c8bf640 1 -- 172.21.15.55:0/3844224096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc28104c10 con 0x7fcc28103420 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc27fff640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 0x7fcc28105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.909+0000 7fcc2c8bf640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc28103420 0x7fcc28104380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009fd0 tx=0x7fcc2000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.910+0000 7fcc257fa640 1 -- 172.21.15.55:0/3844224096 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2000ce20 con 0x7fcc28103420 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.910+0000 7fcc2e322640 1 -- 172.21.15.55:0/3844224096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc281a2d90 con 0x7fcc28103420 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.910+0000 7fcc257fa640 1 -- 172.21.15.55:0/3844224096 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc20005020 con 0x7fcc28103420 2024-04-26T21:53:32.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.910+0000 7fcc257fa640 1 -- 172.21.15.55:0/3844224096 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20010690 con 0x7fcc28103420 2024-04-26T21:53:32.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.910+0000 7fcc2e322640 1 -- 172.21.15.55:0/3844224096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc281a3220 con 0x7fcc28103420 2024-04-26T21:53:32.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc31036740 1 Processor -- start 2024-04-26T21:53:32.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc257fa640 1 -- 172.21.15.55:0/3844224096 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcc200054d0 con 0x7fcc28103420 2024-04-26T21:53:32.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc31036740 1 -- start start 2024-04-26T21:53:32.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc31036740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc31036740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc31036740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc31036740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f022b00 con 0x55cd1f0bd8b0 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc31036740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cd1efbe920 con 0x55cd1f0c3410 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.912+0000 7fcc31036740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f01eac0 con 0x55cd1f0be2a0 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37298/0 (socket says 172.21.15.55:37298) 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 -- 172.21.15.55:0/211488795 learned_addr learned my addr 172.21.15.55:0/211488795 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc27fff640 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc257fa640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcc100a3c40 0x7fcc100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 -- 172.21.15.55:0/211488795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0be2a0 msgr2=0x55cd1f0c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 -- 172.21.15.55:0/211488795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 msgr2=0x55cd1f0c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 -- 172.21.15.55:0/211488795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1edfc7e0 con 0x55cd1f0bd8b0 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc2c8bf640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcc100a3c40 0x7fcc100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc277fe640 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bdcd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00be10 tx=0x7fcc1c004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795124c7974b90b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc257fa640 1 -- 172.21.15.55:0/3844224096 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fcc200d0d30 con 0x7fcc28103420 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc0e7fc640 1 -- 172.21.15.55:0/211488795 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c003e80 con 0x55cd1f0bd8b0 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc0e7fc640 1 -- 172.21.15.55:0/211488795 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c0054d0 con 0x55cd1f0bd8b0 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.913+0000 7fcc0e7fc640 1 -- 172.21.15.55:0/211488795 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c012450 con 0x55cd1f0bd8b0 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 -- 172.21.15.55:0/211488795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0bd8b0 msgr2=0x55cd1f0bdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bdcd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00be10 tx=0x7fcc1c004160 comp rx=0 tx=0).stop 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc2c8bf640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcc100a3c40 0x7fcc100a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcc2009cb10 tx=0x7fcc200bd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 -- 172.21.15.55:0/211488795 shutdown_connections 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 --2- 172.21.15.55:0/211488795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bdcd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 -- 172.21.15.55:0/211488795 >> 172.21.15.55:0/211488795 conn(0x55cd1f0b8f20 msgr2=0x55cd1f0b9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 -- 172.21.15.55:0/211488795 shutdown_connections 2024-04-26T21:53:32.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 -- 172.21.15.55:0/211488795 wait complete. 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.914+0000 7fcc31036740 1 -- start start 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc31036740 1 Processor -- start 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc31036740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc31036740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc31036740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc31036740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f0c6360 con 0x55cd1f0be2a0 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc31036740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cd1efbc5d0 con 0x55cd1f0c3410 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc31036740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cd1f0c64e0 con 0x55cd1f0bd8b0 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc31036740 10 client.0 _ll_register_callbacks cb 0x55cd1efaa080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37308/0 (socket says 172.21.15.55:37308) 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc27fff640 1 -- 172.21.15.55:0/662687098 learned_addr learned my addr 172.21.15.55:0/662687098 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:32.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc277fe640 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc277fe640 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc277fe640 1 -- 172.21.15.55:0/662687098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 msgr2=0x55cd1f0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc277fe640 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc277fe640 1 -- 172.21.15.55:0/662687098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 msgr2=0x55cd1f0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc277fe640 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc277fe640 1 -- 172.21.15.55:0/662687098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1ee5cf70 con 0x55cd1f0c3410 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.915+0000 7fcc27fff640 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc277fe640 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cf250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c005fd0 tx=0x7fcc1c005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c9d7d3f5c9e11dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc077fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc077fe640 1 -- 172.21.15.55:0/662687098 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c005230 con 0x55cd1f0c3410 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 1 -- 172.21.15.55:0/662687098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc1c0054d0 con 0x55cd1f0c3410 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc077fe640 1 -- 172.21.15.55:0/662687098 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fcc1c012450 con 0x55cd1f0c3410 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc077fe640 1 -- 172.21.15.55:0/662687098 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcc1c012650 con 0x55cd1f0c3410 2024-04-26T21:53:32.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 10 client.25151 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 1 -- 172.21.15.55:0/662687098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cd1ecf4170 con 0x55cd1f0c3410 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc077fe640 1 -- 172.21.15.55:0/662687098 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c0054d0 con 0x55cd1f0c3410 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc077fe640 1 -- 172.21.15.55:0/662687098 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fcc1c02f450 con 0x55cd1f0c3410 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 10 client.25151 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 10 client.25151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 20 client.25151 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cd1f0d3420 0x55cd1f0d5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 4 client.25151 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.916+0000 7fcc31036740 1 -- 172.21.15.55:0/662687098 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd1f030710 con 0x55cd1f0d3420 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.917+0000 7fcc27fff640 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cd1f0d3420 0x55cd1f0d5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.917+0000 7fcc27fff640 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cd1f0d3420 0x55cd1f0d5830 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21daf74355d36864 server_cookie=e02c62d20369bec9 in_seq=0 out_seq=0 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.917+0000 7fcc077fe640 10 client.25151 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc077fe640 1 -- 172.21.15.55:0/662687098 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd1f030710 con 0x55cd1f0d3420 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc077fe640 10 client.25151 handle_command_reply: tid=0 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 client.25151 shutdown 2024-04-26T21:53:32.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 -- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 msgr2=0x55cd1f0cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cf250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c005fd0 tx=0x7fcc1c005990 comp rx=0 tx=0).stop 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 -- 172.21.15.55:0/662687098 shutdown_connections 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cf250 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cd1f0d3420 0x55cd1f0d5830 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc077fe640 0 client.25151 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 --2- 172.21.15.55:0/662687098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 -- 172.21.15.55:0/662687098 >> 172.21.15.55:0/662687098 conn(0x55cd1f0b8f20 msgr2=0x55cd1f0b80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 -- 172.21.15.55:0/662687098 shutdown_connections 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 1 -- 172.21.15.55:0/662687098 wait complete. 2024-04-26T21:53:32.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.918+0000 7fcc31036740 20 client.25151 trim_cache size 0 max 16384 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc31036740 1 Processor -- start 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc31036740 1 -- start start 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc31036740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc31036740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc31036740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0c3410 0x55cd1f0bc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc31036740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281aea60 con 0x55cd1f0be2a0 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc31036740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281ac330 con 0x55cd1f0bd8b0 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc31036740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281aa180 con 0x55cd1f0c3410 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37314/0 (socket says 172.21.15.55:37314) 2024-04-26T21:53:32.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc27fff640 1 -- 172.21.15.55:0/2954749331 learned_addr learned my addr 172.21.15.55:0/2954749331 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.936+0000 7fcc277fe640 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc277fe640 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0c3410 0x55cd1f0bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc27fff640 1 -- 172.21.15.55:0/2954749331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0c3410 msgr2=0x55cd1f0bc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc27fff640 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0c3410 0x55cd1f0bc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc27fff640 1 -- 172.21.15.55:0/2954749331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0bd8b0 msgr2=0x55cd1f0bbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc27fff640 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc27fff640 1 -- 172.21.15.55:0/2954749331 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1f0bd410 con 0x55cd1f0be2a0 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc277fe640 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0c3410 0x55cd1f0bc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc27fff640 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cf210 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcc18005ab0 tx=0x7fcc180314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca2cb51351cfd47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc0dffb640 1 -- 172.21.15.55:0/2954749331 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18035040 con 0x55cd1f0be2a0 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc0dffb640 1 -- 172.21.15.55:0/2954749331 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1804a070 con 0x55cd1f0be2a0 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc0dffb640 1 -- 172.21.15.55:0/2954749331 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc180452c0 con 0x55cd1f0be2a0 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc31036740 1 -- 172.21.15.55:0/2954749331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 msgr2=0x55cd1f0cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.937+0000 7fcc31036740 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cf210 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcc18005ab0 tx=0x7fcc180314a0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 -- 172.21.15.55:0/2954749331 shutdown_connections 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0c3410 0x55cd1f0bc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cf210 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 --2- 172.21.15.55:0/2954749331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0bbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 -- 172.21.15.55:0/2954749331 >> 172.21.15.55:0/2954749331 conn(0x55cd1f0b8f20 msgr2=0x55cd1f0bf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 -- 172.21.15.55:0/2954749331 shutdown_connections 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 -- 172.21.15.55:0/2954749331 wait complete. 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 -- start start 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 Processor -- start 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281aa7f0 con 0x55cd1f0be2a0 2024-04-26T21:53:32.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281abc60 con 0x55cd1f0c3410 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcc281aaec0 con 0x55cd1f0bd8b0 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.938+0000 7fcc31036740 10 client.0 _ll_register_callbacks cb 0x7fcc1c003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc27fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37316/0 (socket says 172.21.15.55:37316) 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc27fff640 1 -- 172.21.15.55:0/710423280 learned_addr learned my addr 172.21.15.55:0/710423280 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc277fe640 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc277fe640 1 -- 172.21.15.55:0/710423280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 msgr2=0x55cd1f0cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc277fe640 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc277fe640 1 -- 172.21.15.55:0/710423280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 msgr2=0x55cd1f0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc277fe640 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc277fe640 1 -- 172.21.15.55:0/710423280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1f0ce030 con 0x55cd1f0c3410 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc27fff640 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:53:32.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc277fe640 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cda40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c003e70 tx=0x7fcc1c014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7655abb20e4531 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc077fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc077fe640 1 -- 172.21.15.55:0/710423280 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c012450 con 0x55cd1f0c3410 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc31036740 1 -- 172.21.15.55:0/710423280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc1804a070 con 0x55cd1f0c3410 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc077fe640 1 -- 172.21.15.55:0/710423280 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fcc1c004160 con 0x55cd1f0c3410 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc077fe640 1 -- 172.21.15.55:0/710423280 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcc1c005230 con 0x55cd1f0c3410 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc31036740 10 client.25157 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.939+0000 7fcc31036740 1 -- 172.21.15.55:0/710423280 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cd1f0cf2e0 con 0x55cd1f0c3410 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc077fe640 1 -- 172.21.15.55:0/710423280 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1804a070 con 0x55cd1f0c3410 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc077fe640 1 -- 172.21.15.55:0/710423280 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fcc1c0127f0 con 0x55cd1f0c3410 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc31036740 10 client.25157 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc31036740 10 client.25157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc31036740 20 client.25157 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc31036740 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cd1f139d00 0x55cd1f13c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:32.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc31036740 4 client.25157 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:53:32.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc31036740 1 -- 172.21.15.55:0/710423280 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cd1f02a580 con 0x55cd1f139d00 2024-04-26T21:53:32.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc27fff640 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cd1f139d00 0x55cd1f13c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:32.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc27fff640 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cd1f139d00 0x55cd1f13c120 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee2dcd204938b358 server_cookie=79780af80a51d887 in_seq=0 out_seq=0 2024-04-26T21:53:32.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.940+0000 7fcc077fe640 10 client.25157 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.941+0000 7fcc077fe640 1 -- 172.21.15.55:0/710423280 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cd1f02a580 con 0x55cd1f139d00 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.941+0000 7fcc077fe640 10 client.25157 handle_command_reply: tid=0 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.941+0000 7fcc31036740 1 client.25157 shutdown 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 -- 172.21.15.55:0/710423280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 msgr2=0x55cd1f0cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cda40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c003e70 tx=0x7fcc1c014040 comp rx=0 tx=0).stop 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 -- 172.21.15.55:0/710423280 shutdown_connections 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cd1f0c3410 0x55cd1f0cda40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cd1f139d00 0x55cd1f13c120 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cd1f0be2a0 0x55cd1f0cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 --2- 172.21.15.55:0/710423280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cd1f0bd8b0 0x55cd1f0cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 -- 172.21.15.55:0/710423280 >> 172.21.15.55:0/710423280 conn(0x55cd1f0b8f20 msgr2=0x55cd1f0b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc077fe640 0 client.25157 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 -- 172.21.15.55:0/710423280 shutdown_connections 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 1 -- 172.21.15.55:0/710423280 wait complete. 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc31036740 20 client.25157 trim_cache size 0 max 16384 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "28d1ff5a-ddc4-46ea-899d-b208fbab7d9c", 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:53:32.948 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:53:32.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 -- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcc100a3c40 msgr2=0x7fcc100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcc100a3c40 0x7fcc100a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcc2009cb10 tx=0x7fcc200bd040 comp rx=0 tx=0).stop 2024-04-26T21:53:32.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 -- 172.21.15.55:0/3844224096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc28103420 msgr2=0x7fcc28104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:32.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc28103420 0x7fcc28104380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009fd0 tx=0x7fcc2000ef10 comp rx=0 tx=0).stop 2024-04-26T21:53:32.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 -- 172.21.15.55:0/3844224096 shutdown_connections 2024-04-26T21:53:32.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fcc100a3c40 0x7fcc100a6100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc28103420 0x7fcc28104380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc28102a60 0x7fcc28103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 --2- 172.21.15.55:0/3844224096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc28101870 0x7fcc28105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:32.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.942+0000 7fcc2e322640 1 -- 172.21.15.55:0/3844224096 >> 172.21.15.55:0/3844224096 conn(0x7fcc280fcde0 msgr2=0x7fcc28069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:32.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.943+0000 7fcc2e322640 1 -- 172.21.15.55:0/3844224096 shutdown_connections 2024-04-26T21:53:32.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:32.943+0000 7fcc2e322640 1 -- 172.21.15.55:0/3844224096 wait complete. 2024-04-26T21:53:32.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:53:32.954 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:53:33.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 -- 172.21.15.55:0/679385464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0103470 msgr2=0x7f80f0105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 --2- 172.21.15.55:0/679385464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0103470 0x7f80f0105860 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009900 tx=0x7f80ec02a410 comp rx=0 tx=0).stop 2024-04-26T21:53:33.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 -- 172.21.15.55:0/679385464 shutdown_connections 2024-04-26T21:53:33.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 --2- 172.21.15.55:0/679385464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f01086d0 0x7f80f010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 --2- 172.21.15.55:0/679385464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80f0105da0 0x7f80f0108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 --2- 172.21.15.55:0/679385464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0103470 0x7f80f0105860 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 -- 172.21.15.55:0/679385464 >> 172.21.15.55:0/679385464 conn(0x7f80f00fce20 msgr2=0x7f80f00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 -- 172.21.15.55:0/679385464 shutdown_connections 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.058+0000 7f80f7719640 1 -- 172.21.15.55:0/679385464 wait complete. 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f7719640 1 Processor -- start 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f7719640 1 -- start start 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f7719640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f0103470 0x7f80f0198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f7719640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0105da0 0x7f80f0199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f7719640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80f01086d0 0x7f80f01a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f7719640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80f010e4b0 con 0x7f80f0105da0 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f7719640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f80f010e330 con 0x7f80f0103470 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f7719640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80f010e630 con 0x7f80f01086d0 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0105da0 0x7f80f0199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0105da0 0x7f80f0199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37322/0 (socket says 172.21.15.55:37322) 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f54b5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f0103470 0x7f80f0198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 -- 172.21.15.55:0/1283063411 learned_addr learned my addr 172.21.15.55:0/1283063411 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80f01086d0 0x7f80f01a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 -- 172.21.15.55:0/1283063411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80f01086d0 msgr2=0x7f80f01a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80f01086d0 0x7f80f01a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 -- 172.21.15.55:0/1283063411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f0103470 msgr2=0x7f80f0198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f0103470 0x7f80f0198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 -- 172.21.15.55:0/1283063411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f01a0850 con 0x7f80f0105da0 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f54b5640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f0103470 0x7f80f0198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.059+0000 7f80f4cb4640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0105da0 0x7f80f0199180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f80e000c970 tx=0x7f80e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.060+0000 7f80e67fc640 1 -- 172.21.15.55:0/1283063411 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0007bb0 con 0x7f80f0105da0 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.060+0000 7f80e67fc640 1 -- 172.21.15.55:0/1283063411 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e0007d50 con 0x7f80f0105da0 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.060+0000 7f80f7719640 1 -- 172.21.15.55:0/1283063411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f01a0b40 con 0x7f80f0105da0 2024-04-26T21:53:33.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.060+0000 7f80e67fc640 1 -- 172.21.15.55:0/1283063411 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e000f450 con 0x7f80f0105da0 2024-04-26T21:53:33.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.060+0000 7f80f7719640 1 -- 172.21.15.55:0/1283063411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f01a10e0 con 0x7f80f0105da0 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.062+0000 7f80e67fc640 1 -- 172.21.15.55:0/1283063411 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f80e0004f20 con 0x7f80f0105da0 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.062+0000 7f80fa438740 1 Processor -- start 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.062+0000 7f80fa438740 1 -- start start 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.062+0000 7f80fa438740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ada710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80fa438740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adb4c0 0x559fb1adb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80e67fc640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80dc0a3cc0 0x7f80dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80fa438740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1ae2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80fa438740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559fb1a3ad80 con 0x559fb1adbe80 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80fa438740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559fb19f48b0 con 0x559fb1ada2f0 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80fa438740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559fb1a36d40 con 0x559fb1adb4c0 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1ae2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1ae2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37328/0 (socket says 172.21.15.55:37328) 2024-04-26T21:53:33.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f4cb4640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80dc0a3cc0 0x7f80dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 -- 172.21.15.55:0/1586146917 learned_addr learned my addr 172.21.15.55:0/1586146917 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80e67fc640 1 -- 172.21.15.55:0/1283063411 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f80e00d0660 con 0x7f80f0105da0 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ada710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f54b5640 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adb4c0 0x559fb1adb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 -- 172.21.15.55:0/1586146917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adb4c0 msgr2=0x559fb1adb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adb4c0 0x559fb1adb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 -- 172.21.15.55:0/1586146917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 msgr2=0x559fb1ada710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ada710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 -- 172.21.15.55:0/1586146917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb19c5a40 con 0x559fb1adbe80 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1ae2710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009fd0 tx=0x7f80e8009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9eea32ddc7f87cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80f4cb4640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80dc0a3cc0 0x7f80dc0a6180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f80e00cdfa0 tx=0x7f80e0087040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80db7fe640 1 -- 172.21.15.55:0/1586146917 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8003970 con 0x559fb1adbe80 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80db7fe640 1 -- 172.21.15.55:0/1586146917 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e8010890 con 0x559fb1adbe80 2024-04-26T21:53:33.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.063+0000 7f80db7fe640 1 -- 172.21.15.55:0/1586146917 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80043f0 con 0x559fb1adbe80 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 -- 172.21.15.55:0/1586146917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 msgr2=0x559fb1ae2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1ae2710 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009fd0 tx=0x7f80e8009aa0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 -- 172.21.15.55:0/1586146917 shutdown_connections 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1ae2710 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adb4c0 0x559fb1adb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 --2- 172.21.15.55:0/1586146917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ada710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 -- 172.21.15.55:0/1586146917 >> 172.21.15.55:0/1586146917 conn(0x559fb1ad13d0 msgr2=0x559fb1ad17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 -- 172.21.15.55:0/1586146917 shutdown_connections 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 -- 172.21.15.55:0/1586146917 wait complete. 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.064+0000 7f80fa438740 1 -- start start 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80fa438740 1 Processor -- start 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80fa438740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ae47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80fa438740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80fa438740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80fa438740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559fb1ae2e90 con 0x559fb1adb4c0 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80fa438740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559fb19c1d10 con 0x559fb1ada2f0 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80fa438740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559fb1ae3010 con 0x559fb1adbe80 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80fa438740 10 client.0 _ll_register_callbacks cb 0x559fb19ee210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f54b5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f54b5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37342/0 (socket says 172.21.15.55:37342) 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f54b5640 1 -- 172.21.15.55:0/73840793 learned_addr learned my addr 172.21.15.55:0/73840793 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f5cb6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ae47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f54b5640 1 -- 172.21.15.55:0/73840793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 msgr2=0x559fb1aebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f54b5640 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f54b5640 1 -- 172.21.15.55:0/73840793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 msgr2=0x559fb1ae47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.065+0000 7f80f54b5640 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ae47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80f54b5640 1 -- 172.21.15.55:0/73840793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb18167d0 con 0x559fb1adb4c0 2024-04-26T21:53:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80f54b5640 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae4d10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f80ec032fd0 tx=0x7f80ec031910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3a4650f0d693a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80d8ff9640 1 -- 172.21.15.55:0/73840793 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec035040 con 0x559fb1adb4c0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80d8ff9640 1 -- 172.21.15.55:0/73840793 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f80ec004240 con 0x559fb1adb4c0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 1 -- 172.21.15.55:0/73840793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e8010890 con 0x559fb1adb4c0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80d8ff9640 1 -- 172.21.15.55:0/73840793 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f80ec004440 con 0x559fb1adb4c0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 10 client.15498 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 1 -- 172.21.15.55:0/73840793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559fb174c3f0 con 0x559fb1adb4c0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80d8ff9640 1 -- 172.21.15.55:0/73840793 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e8010890 con 0x559fb1adb4c0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80d8ff9640 1 -- 172.21.15.55:0/73840793 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f80ec0035f0 con 0x559fb1adb4c0 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 10 client.15498 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 20 client.15498 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559fb1af0260 0x559fb1af2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 4 client.15498 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.066+0000 7f80fa438740 1 -- 172.21.15.55:0/73840793 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fb1a48990 con 0x559fb1af0260 2024-04-26T21:53:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.067+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559fb1af0260 0x559fb1af2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.067+0000 7f80d8ff9640 10 client.15498 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.067+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559fb1af0260 0x559fb1af2700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a14d7f1b488a4f4 server_cookie=c2fb72338822b5e1 in_seq=0 out_seq=0 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80d8ff9640 1 -- 172.21.15.55:0/73840793 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fb1a48990 con 0x559fb1af0260 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80d8ff9640 10 client.15498 handle_command_reply: tid=0 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 client.15498 shutdown 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 -- 172.21.15.55:0/73840793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 msgr2=0x559fb1ae4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae4d10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f80ec032fd0 tx=0x7f80ec031910 comp rx=0 tx=0).stop 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 -- 172.21.15.55:0/73840793 shutdown_connections 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559fb1af0260 0x559fb1af2700 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae4d10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 --2- 172.21.15.55:0/73840793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ae47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80d8ff9640 0 client.15498 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 -- 172.21.15.55:0/73840793 >> 172.21.15.55:0/73840793 conn(0x559fb1ad13d0 msgr2=0x559fb1ad0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 -- 172.21.15.55:0/73840793 shutdown_connections 2024-04-26T21:53:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 1 -- 172.21.15.55:0/73840793 wait complete. 2024-04-26T21:53:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.068+0000 7f80fa438740 20 client.15498 trim_cache size 0 max 16384 2024-04-26T21:53:33.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.085+0000 7f80fa438740 1 Processor -- start 2024-04-26T21:53:33.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.085+0000 7f80fa438740 1 -- start start 2024-04-26T21:53:33.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80fa438740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1ada2f0 0x559fb1acc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80fa438740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1adb4c0 0x559fb1aebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80fa438740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1acfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80fa438740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01aa670 con 0x559fb1adbe80 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80fa438740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01a7f40 con 0x559fb1adb4c0 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80fa438740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01a5d90 con 0x559fb1ada2f0 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1acfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1acfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37356/0 (socket says 172.21.15.55:37356) 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 -- 172.21.15.55:0/792335054 learned_addr learned my addr 172.21.15.55:0/792335054 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f54b5640 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1adb4c0 0x559fb1aebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1ada2f0 0x559fb1acc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 -- 172.21.15.55:0/792335054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1ada2f0 msgr2=0x559fb1acc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1ada2f0 0x559fb1acc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 -- 172.21.15.55:0/792335054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1adb4c0 msgr2=0x559fb1aebd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1adb4c0 0x559fb1aebd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 -- 172.21.15.55:0/792335054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb170c0f0 con 0x559fb1adbe80 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f54b5640 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1adb4c0 0x559fb1aebd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:33.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1acfcc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009270 tx=0x7f80e8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb10981370ac78a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80daffd640 1 -- 172.21.15.55:0/792335054 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8010dc0 con 0x559fb1adbe80 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80daffd640 1 -- 172.21.15.55:0/792335054 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e8003e20 con 0x559fb1adbe80 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.086+0000 7f80daffd640 1 -- 172.21.15.55:0/792335054 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8005720 con 0x559fb1adbe80 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 -- 172.21.15.55:0/792335054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 msgr2=0x559fb1acfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1acfcc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009270 tx=0x7f80e8009ea0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 -- 172.21.15.55:0/792335054 shutdown_connections 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adbe80 0x559fb1acfcc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1adb4c0 0x559fb1aebd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 --2- 172.21.15.55:0/792335054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1ada2f0 0x559fb1acc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 -- 172.21.15.55:0/792335054 >> 172.21.15.55:0/792335054 conn(0x559fb1ad13d0 msgr2=0x559fb1ad17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 -- 172.21.15.55:0/792335054 shutdown_connections 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 -- 172.21.15.55:0/792335054 wait complete. 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 -- start start 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.087+0000 7f80fa438740 1 Processor -- start 2024-04-26T21:53:33.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ae90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01a6400 con 0x559fb1adb4c0 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01a7870 con 0x559fb1ada2f0 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80f01a6ad0 con 0x559fb1adbe80 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 10 client.0 _ll_register_callbacks cb 0x7f80ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37360/0 (socket says 172.21.15.55:37360) 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 -- 172.21.15.55:0/2594237197 learned_addr learned my addr 172.21.15.55:0/2594237197 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ae90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 -- 172.21.15.55:0/2594237197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 msgr2=0x559fb1aea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 -- 172.21.15.55:0/2594237197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 msgr2=0x559fb1ae90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ae90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 -- 172.21.15.55:0/2594237197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb19fb4b0 con 0x559fb1adb4c0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80f54b5640 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae9620 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009850 tx=0x7f80ec0049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0681a83e3ac4a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80d8ff9640 1 -- 172.21.15.55:0/2594237197 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec035030 con 0x559fb1adb4c0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80d8ff9640 1 -- 172.21.15.55:0/2594237197 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f80ec031d90 con 0x559fb1adb4c0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 1 -- 172.21.15.55:0/2594237197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e8003e20 con 0x559fb1adb4c0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80d8ff9640 1 -- 172.21.15.55:0/2594237197 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f80ec003c80 con 0x559fb1adb4c0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 10 client.15504 fetch_fsmap learned FSMap version 21 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.088+0000 7f80fa438740 1 -- 172.21.15.55:0/2594237197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559fb1aebdd0 con 0x559fb1adb4c0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80d8ff9640 1 -- 172.21.15.55:0/2594237197 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e8003e20 con 0x559fb1adb4c0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80d8ff9640 1 -- 172.21.15.55:0/2594237197 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f80ec0323c0 con 0x559fb1adb4c0 2024-04-26T21:53:33.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80fa438740 10 client.15504 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80fa438740 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80fa438740 20 client.15504 populate_metadata read hostname 'smithi055' 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80fa438740 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559fb1b56da0 0x559fb1b591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80fa438740 4 client.15504 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80fa438740 1 -- 172.21.15.55:0/2594237197 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559fb1a42800 con 0x559fb1b56da0 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559fb1b56da0 0x559fb1b591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80d8ff9640 10 client.15504 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.089+0000 7f80f5cb6640 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559fb1b56da0 0x559fb1b591c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=931a47c0f854aed server_cookie=a199a854d79098e0 in_seq=0 out_seq=0 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80d8ff9640 1 -- 172.21.15.55:0/2594237197 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559fb1a42800 con 0x559fb1b56da0 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80d8ff9640 10 client.15504 handle_command_reply: tid=0 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 client.15504 shutdown 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 -- 172.21.15.55:0/2594237197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 msgr2=0x559fb1ae9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae9620 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009850 tx=0x7f80ec0049f0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 -- 172.21.15.55:0/2594237197 shutdown_connections 2024-04-26T21:53:33.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559fb1adbe80 0x559fb1aea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559fb1b56da0 0x559fb1b591c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559fb1adb4c0 0x559fb1ae9620 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80d8ff9640 0 client.15504 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 --2- 172.21.15.55:0/2594237197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559fb1ada2f0 0x559fb1ae90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 -- 172.21.15.55:0/2594237197 >> 172.21.15.55:0/2594237197 conn(0x559fb1ad13d0 msgr2=0x559fb1ae21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 -- 172.21.15.55:0/2594237197 shutdown_connections 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 1 -- 172.21.15.55:0/2594237197 wait complete. 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80fa438740 20 client.15504 trim_cache size 0 max 16384 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:53:33.097 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:53:33.098 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:53:33.098 INFO:teuthology.orchestra.run.smithi055.stdout: "28d1ff5a-ddc4-46ea-899d-b208fbab7d9c": { 2024-04-26T21:53:33.098 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:53:33.098 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "28d1ff5a-ddc4-46ea-899d-b208fbab7d9c", 2024-04-26T21:53:33.098 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:53:33.098 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:53:33.098 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:53:33.098 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 -- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80dc0a3cc0 msgr2=0x7f80dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80dc0a3cc0 0x7f80dc0a6180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f80e00cdfa0 tx=0x7f80e0087040 comp rx=0 tx=0).stop 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 -- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0105da0 msgr2=0x7f80f0199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0105da0 0x7f80f0199180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f80e000c970 tx=0x7f80e000ce30 comp rx=0 tx=0).stop 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 -- 172.21.15.55:0/1283063411 shutdown_connections 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f80dc0a3cc0 0x7f80dc0a6180 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80f01086d0 0x7f80f01a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80f0105da0 0x7f80f0199180 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.090+0000 7f80f7719640 1 --2- 172.21.15.55:0/1283063411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f80f0103470 0x7f80f0198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:53:33.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.091+0000 7f80f7719640 1 -- 172.21.15.55:0/1283063411 >> 172.21.15.55:0/1283063411 conn(0x7f80f00fce20 msgr2=0x7f80f0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:53:33.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.091+0000 7f80f7719640 1 -- 172.21.15.55:0/1283063411 shutdown_connections 2024-04-26T21:53:33.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:53:33.091+0000 7f80f7719640 1 -- 172.21.15.55:0/1283063411 wait complete. 2024-04-26T21:53:33.102 INFO:tasks.cephfs.filesystem:scrub status for tag:28d1ff5a-ddc4-46ea-899d-b208fbab7d9c - {'path': '/', 'tag': '28d1ff5a-ddc4-46ea-899d-b208fbab7d9c', 'options': 'recursive,force'} 2024-04-26T21:53:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:33 smithi055 ceph-mon[29221]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:33 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:53:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:33 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:33 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:53:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:33 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:33 smithi064 ceph-mon[32765]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:33 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:53:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:33 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:33 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:53:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:33 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:33.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:33 smithi165 ceph-mon[33296]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:33.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:33 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:53:33.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:33 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:33.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:33 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:53:33.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:33 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:53:35.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:35 smithi055 ceph-mon[29221]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:35 smithi064 ceph-mon[32765]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:35.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:35 smithi165 ceph-mon[33296]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:36.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:36 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:53:36.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:36 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:53:36.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:36 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:53:37.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:37 smithi055 ceph-mon[29221]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:53:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:37 smithi064 ceph-mon[32765]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:53:37.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:37 smithi165 ceph-mon[33296]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:53:39.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:39 smithi055 ceph-mon[29221]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:53:39.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:39 smithi064 ceph-mon[32765]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:53:39.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:39 smithi165 ceph-mon[33296]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:53:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:41 smithi064 ceph-mon[32765]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:41.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:41 smithi165 ceph-mon[33296]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:41.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:41 smithi055 ceph-mon[29221]: pgmap v374: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:43 smithi064 ceph-mon[32765]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:43 smithi165 ceph-mon[33296]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:43.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:43 smithi055 ceph-mon[29221]: pgmap v375: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:45.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:45 smithi064 ceph-mon[32765]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-26T21:53:45.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:45 smithi165 ceph-mon[33296]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-26T21:53:45.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:45 smithi055 ceph-mon[29221]: pgmap v376: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-26T21:53:47.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:47 smithi064 ceph-mon[32765]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T21:53:47.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:47 smithi165 ceph-mon[33296]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T21:53:47.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:47 smithi055 ceph-mon[29221]: pgmap v377: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-04-26T21:53:49.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:49 smithi064 ceph-mon[32765]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:49.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:49 smithi165 ceph-mon[33296]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:49.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:49 smithi055 ceph-mon[29221]: pgmap v378: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:51.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:51 smithi064 ceph-mon[32765]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:51.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:51 smithi165 ceph-mon[33296]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:51.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:51 smithi055 ceph-mon[29221]: pgmap v379: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T21:53:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:53 smithi064 ceph-mon[32765]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:53.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:53 smithi165 ceph-mon[33296]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:53.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:53 smithi055 ceph-mon[29221]: pgmap v380: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:55 smithi064 ceph-mon[32765]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:55 smithi165 ceph-mon[33296]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:55.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:55 smithi055 ceph-mon[29221]: pgmap v381: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:53:56.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:56 smithi165 ceph-mon[33296]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:56.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:56 smithi055 ceph-mon[29221]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:56.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:56 smithi064 ceph-mon[32765]: pgmap v382: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:53:58.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:53:58 smithi064 ceph-mon[32765]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:58.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:53:58 smithi165 ceph-mon[33296]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:53:58.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:53:58 smithi055 ceph-mon[29221]: pgmap v383: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:00.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:00 smithi064 ceph-mon[32765]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:00.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:00 smithi165 ceph-mon[33296]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:00.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:00 smithi055 ceph-mon[29221]: pgmap v384: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:02.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:02 smithi064 ceph-mon[32765]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:02.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:02 smithi165 ceph-mon[33296]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:02.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:02 smithi055 ceph-mon[29221]: pgmap v385: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:03.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:54:03.103 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:54:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 -- 172.21.15.55:0/1579615714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 msgr2=0x7fc5dc1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 --2- 172.21.15.55:0/1579615714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 0x7fc5dc1029c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009ca0 tx=0x7fc5d402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 -- 172.21.15.55:0/1579615714 shutdown_connections 2024-04-26T21:54:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 --2- 172.21.15.55:0/1579615714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 0x7fc5dc1029c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 --2- 172.21.15.55:0/1579615714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5dc101ba0 0x7fc5dc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 --2- 172.21.15.55:0/1579615714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5dc101240 0x7fc5dc101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 -- 172.21.15.55:0/1579615714 >> 172.21.15.55:0/1579615714 conn(0x7fc5dc0fcde0 msgr2=0x7fc5dc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 -- 172.21.15.55:0/1579615714 shutdown_connections 2024-04-26T21:54:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.266+0000 7fc5e232b640 1 -- 172.21.15.55:0/1579615714 wait complete. 2024-04-26T21:54:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e232b640 1 Processor -- start 2024-04-26T21:54:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e232b640 1 -- start start 2024-04-26T21:54:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e232b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5dc101240 0x7fc5dc19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e232b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5dc101ba0 0x7fc5dc19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e232b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 0x7fc5dc1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e232b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1106e0 con 0x7fc5dc102540 2024-04-26T21:54:03.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e232b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc110560 con 0x7fc5dc101240 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e232b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc110860 con 0x7fc5dc101ba0 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e08c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 0x7fc5dc1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e08c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 0x7fc5dc1a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49268/0 (socket says 172.21.15.55:49268) 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.267+0000 7fc5e08c8640 1 -- 172.21.15.55:0/2169166404 learned_addr learned my addr 172.21.15.55:0/2169166404 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5dc101ba0 0x7fc5dc19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5dc101240 0x7fc5dc19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5db7fe640 1 -- 172.21.15.55:0/2169166404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5dc101240 msgr2=0x7fc5dc19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5dc101240 0x7fc5dc19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5db7fe640 1 -- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 msgr2=0x7fc5dc1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 0x7fc5dc1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5db7fe640 1 -- 172.21.15.55:0/2169166404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc1a2980 con 0x7fc5dc101ba0 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5e08c8640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 0x7fc5dc1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5dc101ba0 0x7fc5dc19b2a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000e9f0 tx=0x7fc5d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.268+0000 7fc5d97fa640 1 -- 172.21.15.55:0/2169166404 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d000ccc0 con 0x7fc5dc101ba0 2024-04-26T21:54:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.269+0000 7fc5e232b640 1 -- 172.21.15.55:0/2169166404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc1a2c70 con 0x7fc5dc101ba0 2024-04-26T21:54:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.269+0000 7fc5d97fa640 1 -- 172.21.15.55:0/2169166404 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d00044d0 con 0x7fc5dc101ba0 2024-04-26T21:54:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.269+0000 7fc5d97fa640 1 -- 172.21.15.55:0/2169166404 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d001c070 con 0x7fc5dc101ba0 2024-04-26T21:54:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.269+0000 7fc5e232b640 1 -- 172.21.15.55:0/2169166404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc1a3160 con 0x7fc5dc101ba0 2024-04-26T21:54:03.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e5049740 1 Processor -- start 2024-04-26T21:54:03.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e5049740 1 -- start start 2024-04-26T21:54:03.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5d97fa640 1 -- 172.21.15.55:0/2169166404 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc5d0004a00 con 0x7fc5dc101ba0 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 0x55c05baf4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf50b0 0x55c05baf5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bafc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e5049740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c05ba58d80 con 0x55c05baf5a70 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e5049740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c05ba128b0 con 0x55c05baf50b0 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e5049740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c05ba54d40 con 0x55c05baf3ee0 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bafc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bafc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49282/0 (socket says 172.21.15.55:49282) 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5db7fe640 1 -- 172.21.15.55:0/1580188104 learned_addr learned my addr 172.21.15.55:0/1580188104 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf50b0 0x55c05baf5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.272+0000 7fc5e08c8640 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 0x55c05baf4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5db7fe640 1 -- 172.21.15.55:0/1580188104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 msgr2=0x55c05baf4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 0x55c05baf4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5db7fe640 1 -- 172.21.15.55:0/1580188104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf50b0 msgr2=0x55c05baf5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf50b0 0x55c05baf5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5db7fe640 1 -- 172.21.15.55:0/1580188104 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c05b9e3a40 con 0x55c05baf5a70 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5d97fa640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5c40a3c40 0x7fc5c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5e08c8640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5c40a3c40 0x7fc5c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bafc300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00cde10 tx=0x7fc5d00d6800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb57ce855647834 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5d97fa640 1 -- 172.21.15.55:0/2169166404 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc5d00d0560 con 0x7fc5dc101ba0 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5c27fc640 1 -- 172.21.15.55:0/1580188104 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0098940 con 0x55c05baf5a70 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5c27fc640 1 -- 172.21.15.55:0/1580188104 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d0005020 con 0x55c05baf5a70 2024-04-26T21:54:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.273+0000 7fc5c27fc640 1 -- 172.21.15.55:0/1580188104 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d009c000 con 0x55c05baf5a70 2024-04-26T21:54:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.274+0000 7fc5e5049740 1 -- 172.21.15.55:0/1580188104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 msgr2=0x55c05bafc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.274+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bafc300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00cde10 tx=0x7fc5d00d6800 comp rx=0 tx=0).stop 2024-04-26T21:54:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.274+0000 7fc5e08c8640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5c40a3c40 0x7fc5c40a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55c05baf4fb0 tx=0x7fc5d4031af0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.274+0000 7fc5e5049740 1 -- 172.21.15.55:0/1580188104 shutdown_connections 2024-04-26T21:54:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.274+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bafc300 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.274+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf50b0 0x55c05baf5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.274+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1580188104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 0x55c05baf4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.274+0000 7fc5e5049740 1 -- 172.21.15.55:0/1580188104 >> 172.21.15.55:0/1580188104 conn(0x55c05baef3d0 msgr2=0x55c05baef7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 -- 172.21.15.55:0/1580188104 shutdown_connections 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 -- 172.21.15.55:0/1580188104 wait complete. 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 -- start start 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 Processor -- start 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bafe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05bafe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf5a70 0x55c05bb05860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c05bafca80 con 0x55c05baf50b0 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c05b9dfd10 con 0x55c05baf3ee0 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c05bafcc00 con 0x55c05baf5a70 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.275+0000 7fc5e5049740 10 client.0 _ll_register_callbacks cb 0x55c05ba0c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05bafe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bafe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bafe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:57640/0 (socket says 172.21.15.55:57640) 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 -- 172.21.15.55:0/3642289789 learned_addr learned my addr 172.21.15.55:0/3642289789 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5e08c8640 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf5a70 0x55c05bb05860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 -- 172.21.15.55:0/3642289789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf5a70 msgr2=0x55c05bb05860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf5a70 0x55c05bb05860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 -- 172.21.15.55:0/3642289789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 msgr2=0x55c05bafe820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05bafe820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 -- 172.21.15.55:0/3642289789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c05b8347d0 con 0x55c05baf3ee0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05bafe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bafe2e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00be50 tx=0x7fc5cc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ded35bb31108d171 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.276+0000 7fc5bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/3642289789 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc004220 con 0x55c05baf3ee0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/3642289789 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc5cc0043c0 con 0x55c05baf3ee0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 1 -- 172.21.15.55:0/3642289789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d0005020 con 0x55c05baf3ee0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/3642289789 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc5cc0035f0 con 0x55c05baf3ee0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 10 client.25187 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 1 -- 172.21.15.55:0/3642289789 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c05b76a3f0 con 0x55c05baf3ee0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/3642289789 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc003c40 con 0x55c05baf3ee0 2024-04-26T21:54:03.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/3642289789 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc5cc02cad0 con 0x55c05baf3ee0 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 10 client.25187 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 10 client.25187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 20 client.25187 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c05bb09da0 0x55c05bb0c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 4 client.25187 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.277+0000 7fc5e5049740 1 -- 172.21.15.55:0/3642289789 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc5d0005020 con 0x55c05bb09da0 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.278+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c05bb09da0 0x55c05bb0c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.278+0000 7fc5bb7fe640 10 client.25187 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.278+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c05bb09da0 0x55c05bb0c220 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c78ff7b45debe5 server_cookie=39b9cb2d42adc22a in_seq=0 out_seq=0 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/3642289789 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fc5d0005020 con 0x55c05bb09da0 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5bb7fe640 10 client.25187 handle_command_reply: tid=0 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 client.25187 shutdown 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 -- 172.21.15.55:0/3642289789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 msgr2=0x55c05bafe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bafe2e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00be50 tx=0x7fc5cc011810 comp rx=0 tx=0).stop 2024-04-26T21:54:03.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 -- 172.21.15.55:0/3642289789 shutdown_connections 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf5a70 0x55c05bb05860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c05bb09da0 0x55c05bb0c220 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05bafe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5bb7fe640 0 client.25187 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 --2- 172.21.15.55:0/3642289789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bafe2e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 -- 172.21.15.55:0/3642289789 >> 172.21.15.55:0/3642289789 conn(0x55c05baef3d0 msgr2=0x55c05baee550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 -- 172.21.15.55:0/3642289789 shutdown_connections 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 1 -- 172.21.15.55:0/3642289789 wait complete. 2024-04-26T21:54:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.279+0000 7fc5e5049740 20 client.25187 trim_cache size 0 max 16384 2024-04-26T21:54:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5e5049740 1 Processor -- start 2024-04-26T21:54:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5e5049740 1 -- start start 2024-04-26T21:54:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 0x55c05baee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05baee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf5a70 0x55c05bb05820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5e5049740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1a6180 con 0x55c05baf50b0 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5e5049740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1a3a50 con 0x55c05baf5a70 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5e5049740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc073010 con 0x55c05baf3ee0 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05baee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05baee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49312/0 (socket says 172.21.15.55:49312) 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.297+0000 7fc5db7fe640 1 -- 172.21.15.55:0/2744932118 learned_addr learned my addr 172.21.15.55:0/2744932118 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 0x55c05baee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5db7fe640 1 -- 172.21.15.55:0/2744932118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 msgr2=0x55c05baee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 0x55c05baee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5e08c8640 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf5a70 0x55c05bb05820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5db7fe640 1 -- 172.21.15.55:0/2744932118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf5a70 msgr2=0x55c05bb05820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf5a70 0x55c05bb05820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5db7fe640 1 -- 172.21.15.55:0/2744932118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c05b72a0f0 con 0x55c05baf50b0 2024-04-26T21:54:03.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05baee910 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00d6d00 tx=0x7fc5d00d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6d6205cecb3fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5c1ffb640 1 -- 172.21.15.55:0/2744932118 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d009c1e0 con 0x55c05baf50b0 2024-04-26T21:54:03.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5c1ffb640 1 -- 172.21.15.55:0/2744932118 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d0005020 con 0x55c05baf50b0 2024-04-26T21:54:03.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.298+0000 7fc5c1ffb640 1 -- 172.21.15.55:0/2744932118 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d009c470 con 0x55c05baf50b0 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 -- 172.21.15.55:0/2744932118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 msgr2=0x55c05baee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05baee910 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00d6d00 tx=0x7fc5d00d0040 comp rx=0 tx=0).stop 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 -- 172.21.15.55:0/2744932118 shutdown_connections 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf5a70 0x55c05bb05820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf50b0 0x55c05baee910 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 --2- 172.21.15.55:0/2744932118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf3ee0 0x55c05baee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 -- 172.21.15.55:0/2744932118 >> 172.21.15.55:0/2744932118 conn(0x55c05baef3d0 msgr2=0x55c05baedcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 -- 172.21.15.55:0/2744932118 shutdown_connections 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.299+0000 7fc5e5049740 1 -- 172.21.15.55:0/2744932118 wait complete. 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e5049740 1 -- start start 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e5049740 1 Processor -- start 2024-04-26T21:54:03.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bb031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf50b0 0x55c05bb036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e5049740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bb046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e5049740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc073680 con 0x55c05baf5a70 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e5049740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1a33f0 con 0x55c05baf3ee0 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e5049740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc073d50 con 0x55c05baf50b0 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e08c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bb046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e08c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bb046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49326/0 (socket says 172.21.15.55:49326) 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5e08c8640 1 -- 172.21.15.55:0/1962100674 learned_addr learned my addr 172.21.15.55:0/1962100674 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf50b0 0x55c05bb036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.300+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bb031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5e5049740 10 client.0 _ll_register_callbacks cb 0x7fc5cc00ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5dbfff640 1 -- 172.21.15.55:0/1962100674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf50b0 msgr2=0x55c05bb036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf50b0 0x55c05bb036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5dbfff640 1 -- 172.21.15.55:0/1962100674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 msgr2=0x55c05bb046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bb046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5dbfff640 1 -- 172.21.15.55:0/1962100674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c05ba194b0 con 0x55c05baf3ee0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5e08c8640 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bb046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5dbfff640 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bb031a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0105f0 tx=0x7fc5cc0052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e23db6affa569914 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/1962100674 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc03b040 con 0x55c05baf3ee0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5e5049740 1 -- 172.21.15.55:0/1962100674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d0005020 con 0x55c05baf3ee0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/1962100674 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc5cc0056e0 con 0x55c05baf3ee0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/1962100674 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc5cc011c50 con 0x55c05baf3ee0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5e5049740 10 client.25190 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.301+0000 7fc5e5049740 1 -- 172.21.15.55:0/1962100674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c05bb70360 con 0x55c05baf3ee0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/1962100674 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc046070 con 0x55c05baf3ee0 2024-04-26T21:54:03.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/1962100674 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc5cc0109e0 con 0x55c05baf3ee0 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5e5049740 10 client.25190 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5e5049740 10 client.25190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5e5049740 20 client.25190 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c05bb09da0 0x55c05bb72cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5e5049740 4 client.25190 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5e5049740 1 -- 172.21.15.55:0/1962100674 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc5d0005020 con 0x55c05bb09da0 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.302+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c05bb09da0 0x55c05bb72cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5bb7fe640 10 client.25190 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5db7fe640 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c05bb09da0 0x55c05bb72cb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3924ff237341a8ad server_cookie=5b47b734157c36e8 in_seq=0 out_seq=0 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5bb7fe640 1 -- 172.21.15.55:0/1962100674 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc5d0005020 con 0x55c05bb09da0 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5bb7fe640 10 client.25190 handle_command_reply: tid=0 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 client.25190 shutdown 2024-04-26T21:54:03.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 -- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 msgr2=0x55c05bb031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bb031a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0105f0 tx=0x7fc5cc0052f0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 -- 172.21.15.55:0/1962100674 shutdown_connections 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c05bb09da0 0x55c05bb72cb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5bb7fe640 0 client.25190 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c05baf5a70 0x55c05bb046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c05baf50b0 0x55c05bb036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 --2- 172.21.15.55:0/1962100674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c05baf3ee0 0x55c05bb031a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 -- 172.21.15.55:0/1962100674 >> 172.21.15.55:0/1962100674 conn(0x55c05baef3d0 msgr2=0x55c05baee550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 -- 172.21.15.55:0/1962100674 shutdown_connections 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 1 -- 172.21.15.55:0/1962100674 wait complete. 2024-04-26T21:54:03.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.303+0000 7fc5e5049740 20 client.25190 trim_cache size 0 max 16384 2024-04-26T21:54:03.310 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:54:03.310 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:54:03.310 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:54:03.310 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:54:03.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 -- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5c40a3c40 msgr2=0x7fc5c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5c40a3c40 0x7fc5c40a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55c05baf4fb0 tx=0x7fc5d4031af0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 -- 172.21.15.55:0/2169166404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5dc101ba0 msgr2=0x7fc5dc19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5dc101ba0 0x7fc5dc19b2a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000e9f0 tx=0x7fc5d000eeb0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 -- 172.21.15.55:0/2169166404 shutdown_connections 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc5c40a3c40 0x7fc5c40a6100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5dc102540 0x7fc5dc1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5dc101ba0 0x7fc5dc19b2a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 --2- 172.21.15.55:0/2169166404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5dc101240 0x7fc5dc19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 -- 172.21.15.55:0/2169166404 >> 172.21.15.55:0/2169166404 conn(0x7fc5dc0fcde0 msgr2=0x7fc5dc0fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 -- 172.21.15.55:0/2169166404 shutdown_connections 2024-04-26T21:54:03.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.304+0000 7fc5e232b640 1 -- 172.21.15.55:0/2169166404 wait complete. 2024-04-26T21:54:03.316 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:54:03.316 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:54:03.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.421+0000 7f30cf951640 1 -- 172.21.15.55:0/22382211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c81033f0 msgr2=0x7f30c81057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.421+0000 7f30cf951640 1 --2- 172.21.15.55:0/22382211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c81033f0 0x7f30c81057e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009900 tx=0x7f30c402a410 comp rx=0 tx=0).stop 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 -- 172.21.15.55:0/22382211 shutdown_connections 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 --2- 172.21.15.55:0/22382211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c8108650 0x7f30c810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 --2- 172.21.15.55:0/22382211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c8105d20 0x7f30c8108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 --2- 172.21.15.55:0/22382211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c81033f0 0x7f30c81057e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 -- 172.21.15.55:0/22382211 >> 172.21.15.55:0/22382211 conn(0x7f30c80fcde0 msgr2=0x7f30c80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 -- 172.21.15.55:0/22382211 shutdown_connections 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 -- 172.21.15.55:0/22382211 wait complete. 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 Processor -- start 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.422+0000 7f30cf951640 1 -- start start 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cf951640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c8105d20 0x7f30c8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cf951640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c8108650 0x7f30c8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cf951640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c819e3a0 0x7f30c81a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cf951640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30c810e4c0 con 0x7f30c819e3a0 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cf951640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30c810e340 con 0x7f30c8108650 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cf951640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30c810e640 con 0x7f30c8105d20 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c819e3a0 0x7f30c81a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c819e3a0 0x7f30c81a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49346/0 (socket says 172.21.15.55:49346) 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 -- 172.21.15.55:0/377136155 learned_addr learned my addr 172.21.15.55:0/377136155 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cceec640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c8108650 0x7f30c8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cd6ed640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c8105d20 0x7f30c8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 -- 172.21.15.55:0/377136155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c8105d20 msgr2=0x7f30c8198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c8105d20 0x7f30c8198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 -- 172.21.15.55:0/377136155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c8108650 msgr2=0x7f30c8199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c8108650 0x7f30c8199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 -- 172.21.15.55:0/377136155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c81a0e00 con 0x7f30c819e3a0 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30cdeee640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c819e3a0 0x7f30c81a0790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f30c000eb10 tx=0x7f30c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.423+0000 7f30be7fc640 1 -- 172.21.15.55:0/377136155 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0004b20 con 0x7f30c819e3a0 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.424+0000 7f30be7fc640 1 -- 172.21.15.55:0/377136155 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c0004cc0 con 0x7f30c819e3a0 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.424+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c8101240 con 0x7f30c819e3a0 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.424+0000 7f30be7fc640 1 -- 172.21.15.55:0/377136155 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0010670 con 0x7f30c819e3a0 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.424+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c8101790 con 0x7f30c819e3a0 2024-04-26T21:54:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.425+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c80fe8d0 con 0x7f30c819e3a0 2024-04-26T21:54:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.427+0000 7f30be7fc640 1 -- 172.21.15.55:0/377136155 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f30c0005000 con 0x7f30c819e3a0 2024-04-26T21:54:03.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.428+0000 7f30be7fc640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30b40a3d40 0x7f30b40a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.428+0000 7f30be7fc640 1 -- 172.21.15.55:0/377136155 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f30c00d0db0 con 0x7f30c819e3a0 2024-04-26T21:54:03.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.428+0000 7f30cd6ed640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30b40a3d40 0x7f30b40a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.429+0000 7f30cd6ed640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30b40a3d40 0x7f30b40a6200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009230 tx=0x7f30c40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.432+0000 7f30be7fc640 1 -- 172.21.15.55:0/377136155 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f30c0054060 con 0x7f30c819e3a0 2024-04-26T21:54:03.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.662+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30c802dad0 con 0x7f30c819e3a0 2024-04-26T21:54:03.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.663+0000 7f30be7fc640 1 -- 172.21.15.55:0/377136155 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11863 (secure 0 0 0) 0x7f30c009d590 con 0x7f30c819e3a0 2024-04-26T21:54:03.664 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:54:03.664 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:52:54.328649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24502,24509]},"id":1}]} 2024-04-26T21:54:03.664 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 21 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30b40a3d40 msgr2=0x7f30b40a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30b40a3d40 0x7f30b40a6200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009230 tx=0x7f30c40314a0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c819e3a0 msgr2=0x7f30c81a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c819e3a0 0x7f30c81a0790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f30c000eb10 tx=0x7f30c000c2d0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 shutdown_connections 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30b40a3d40 0x7f30b40a6200 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30c819e3a0 0x7f30c81a0790 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30c8108650 0x7f30c8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 --2- 172.21.15.55:0/377136155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30c8105d20 0x7f30c8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 >> 172.21.15.55:0/377136155 conn(0x7f30c80fcde0 msgr2=0x7f30c8105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 shutdown_connections 2024-04-26T21:54:03.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.666+0000 7f30cf951640 1 -- 172.21.15.55:0/377136155 wait complete. 2024-04-26T21:54:03.678 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:54:03.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 -- 172.21.15.55:0/2151580996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c1045d0 msgr2=0x7fb71c10ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 --2- 172.21.15.55:0/2151580996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c1045d0 0x7fb71c10ae60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb714009cd0 tx=0x7fb71402a8e0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 -- 172.21.15.55:0/2151580996 shutdown_connections 2024-04-26T21:54:03.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 --2- 172.21.15.55:0/2151580996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c1045d0 0x7fb71c10ae60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 --2- 172.21.15.55:0/2151580996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb71c103c10 0x7fb71c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 --2- 172.21.15.55:0/2151580996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb71c100c40 0x7fb71c101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 -- 172.21.15.55:0/2151580996 >> 172.21.15.55:0/2151580996 conn(0x7fb71c0fc570 msgr2=0x7fb71c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 -- 172.21.15.55:0/2151580996 shutdown_connections 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.785+0000 7fb722abd640 1 -- 172.21.15.55:0/2151580996 wait complete. 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb722abd640 1 Processor -- start 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb722abd640 1 -- start start 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb722abd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb71c100c40 0x7fb71c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb722abd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c103c10 0x7fb71c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb722abd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb71c1045d0 0x7fb71c1a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb722abd640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c10e4d0 con 0x7fb71c103c10 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb722abd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c10e350 con 0x7fb71c100c40 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb722abd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c10e650 con 0x7fb71c1045d0 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c103c10 0x7fb71c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c103c10 0x7fb71c1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49366/0 (socket says 172.21.15.55:49366) 2024-04-26T21:54:03.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb71bfff640 1 -- 172.21.15.55:0/2577151873 learned_addr learned my addr 172.21.15.55:0/2577151873 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.786+0000 7fb720859640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb71c100c40 0x7fb71c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb72105a640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb71c1045d0 0x7fb71c1a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb720859640 1 -- 172.21.15.55:0/2577151873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb71c1045d0 msgr2=0x7fb71c1a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb720859640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb71c1045d0 0x7fb71c1a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb720859640 1 -- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c103c10 msgr2=0x7fb71c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb720859640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c103c10 0x7fb71c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb720859640 1 -- 172.21.15.55:0/2577151873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c1a07d0 con 0x7fb71c100c40 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb71bfff640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c103c10 0x7fb71c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:03.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb720859640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb71c100c40 0x7fb71c198bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb71000e9c0 tx=0x7fb71000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb719ffb640 1 -- 172.21.15.55:0/2577151873 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71000ccc0 con 0x7fb71c100c40 2024-04-26T21:54:03.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb722abd640 1 -- 172.21.15.55:0/2577151873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c1a0ac0 con 0x7fb71c100c40 2024-04-26T21:54:03.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb719ffb640 1 -- 172.21.15.55:0/2577151873 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7100035f0 con 0x7fb71c100c40 2024-04-26T21:54:03.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.787+0000 7fb719ffb640 1 -- 172.21.15.55:0/2577151873 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71000ce60 con 0x7fb71c100c40 2024-04-26T21:54:03.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.788+0000 7fb722abd640 1 -- 172.21.15.55:0/2577151873 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c1a1060 con 0x7fb71c100c40 2024-04-26T21:54:03.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb719ffb640 1 -- 172.21.15.55:0/2577151873 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb710004000 con 0x7fb71c100c40 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb7257d2740 1 Processor -- start 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb7257d2740 1 -- start start 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9757c30 0x563bb9758050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9759280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb9760050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb7257d2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563bb96b7d80 con 0x563bb97597c0 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb7257d2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563bb9671a20 con 0x563bb9757c30 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.791+0000 7fb7257d2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563bb96b3d40 con 0x563bb9758e00 2024-04-26T21:54:03.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb719ffb640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb70c0a3c40 0x7fb70c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb720859640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9759280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb720859640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9759280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:55386/0 (socket says 172.21.15.55:55386) 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb720859640 1 -- 172.21.15.55:0/2352229922 learned_addr learned my addr 172.21.15.55:0/2352229922 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb719ffb640 1 -- 172.21.15.55:0/2577151873 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb7100d0ab0 con 0x7fb71c100c40 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb72105a640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb70c0a3c40 0x7fb70c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb720859640 1 -- 172.21.15.55:0/2352229922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9757c30 msgr2=0x563bb9758050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb720859640 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9757c30 0x563bb9758050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb720859640 1 -- 172.21.15.55:0/2352229922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 msgr2=0x563bb9760050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb71bfff640 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb9760050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb720859640 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb9760050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb720859640 1 -- 172.21.15.55:0/2352229922 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb967cc30 con 0x563bb9758e00 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.792+0000 7fb71bfff640 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb9760050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.793+0000 7fb720859640 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9759280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb71009c8f0 tx=0x7fb7100676e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b8c37b53710c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.793+0000 7fb72105a640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb70c0a3c40 0x7fb70c0a6100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb714009430 tx=0x7fb714031850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.793+0000 7fb706ffd640 1 -- 172.21.15.55:0/2352229922 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710067750 con 0x563bb9758e00 2024-04-26T21:54:03.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.793+0000 7fb706ffd640 1 -- 172.21.15.55:0/2352229922 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb710087040 con 0x563bb9758e00 2024-04-26T21:54:03.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.793+0000 7fb706ffd640 1 -- 172.21.15.55:0/2352229922 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100a5860 con 0x563bb9758e00 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 -- 172.21.15.55:0/2352229922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 msgr2=0x563bb9759280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9759280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb71009c8f0 tx=0x7fb7100676e0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 -- 172.21.15.55:0/2352229922 shutdown_connections 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb9760050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9759280 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 --2- 172.21.15.55:0/2352229922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9757c30 0x563bb9758050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 -- 172.21.15.55:0/2352229922 >> 172.21.15.55:0/2352229922 conn(0x563bb9694910 msgr2=0x563bb974f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 -- 172.21.15.55:0/2352229922 shutdown_connections 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 -- 172.21.15.55:0/2352229922 wait complete. 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 -- start start 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.794+0000 7fb7257d2740 1 Processor -- start 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9761f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 0x563bb9762450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb97597c0 0x563bb9769490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb7257d2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563bb97607d0 con 0x563bb9757c30 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb7257d2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563bb963edc0 con 0x563bb9758e00 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb7257d2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563bb9760950 con 0x563bb97597c0 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb7257d2740 10 client.0 _ll_register_callbacks cb 0x563bb965a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb720859640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9761f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb720859640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9761f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49382/0 (socket says 172.21.15.55:49382) 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb720859640 1 -- 172.21.15.55:0/3914415831 learned_addr learned my addr 172.21.15.55:0/3914415831 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb72105a640 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb97597c0 0x563bb9769490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb71bfff640 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 0x563bb9762450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb720859640 1 -- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb97597c0 msgr2=0x563bb9769490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb720859640 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb97597c0 0x563bb9769490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb720859640 1 -- 172.21.15.55:0/3914415831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 msgr2=0x563bb9762450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb720859640 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 0x563bb9762450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb720859640 1 -- 172.21.15.55:0/3914415831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb93c93f0 con 0x563bb9757c30 2024-04-26T21:54:03.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.795+0000 7fb72105a640 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb97597c0 0x563bb9769490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb720859640 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9761f10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb710003fd0 tx=0x7fb710098870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dc529e3df760ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb6fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb6fbfff640 1 -- 172.21.15.55:0/3914415831 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100a5860 con 0x563bb9757c30 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb6fbfff640 1 -- 172.21.15.55:0/3914415831 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb7100037a0 con 0x563bb9757c30 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb7257d2740 1 -- 172.21.15.55:0/3914415831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb710087040 con 0x563bb9757c30 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb6fbfff640 1 -- 172.21.15.55:0/3914415831 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb7100bd0e0 con 0x563bb9757c30 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb7257d2740 10 client.15537 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb7257d2740 1 -- 172.21.15.55:0/3914415831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563bb94937d0 con 0x563bb9757c30 2024-04-26T21:54:03.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.796+0000 7fb6fbfff640 1 -- 172.21.15.55:0/3914415831 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb710087040 con 0x563bb9757c30 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.797+0000 7fb6fbfff640 1 -- 172.21.15.55:0/3914415831 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb7100042f0 con 0x563bb9757c30 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.797+0000 7fb7257d2740 10 client.15537 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.797+0000 7fb7257d2740 10 client.15537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.797+0000 7fb7257d2740 20 client.15537 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.798+0000 7fb7257d2740 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563bb976d8f0 0x563bb976fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.798+0000 7fb7257d2740 4 client.15537 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.798+0000 7fb7257d2740 1 -- 172.21.15.55:0/3914415831 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bb96c5990 con 0x563bb976d8f0 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.798+0000 7fb71bfff640 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563bb976d8f0 0x563bb976fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.798+0000 7fb6fbfff640 10 client.15537 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:54:03.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.798+0000 7fb71bfff640 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563bb976d8f0 0x563bb976fce0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a42c9ff64ac3d4b3 server_cookie=1f7fa93f6e82ccd1 in_seq=0 out_seq=0 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb6fbfff640 1 -- 172.21.15.55:0/3914415831 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bb96c5990 con 0x563bb976d8f0 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb6fbfff640 10 client.15537 handle_command_reply: tid=0 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 client.15537 shutdown 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 -- 172.21.15.55:0/3914415831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 msgr2=0x563bb9761f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9761f10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb710003fd0 tx=0x7fb710098870 comp rx=0 tx=0).stop 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 -- 172.21.15.55:0/3914415831 shutdown_connections 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563bb976d8f0 0x563bb976fce0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb6fbfff640 0 client.15537 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb97597c0 0x563bb9769490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 0x563bb9762450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 --2- 172.21.15.55:0/3914415831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9761f10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.799+0000 7fb7257d2740 1 -- 172.21.15.55:0/3914415831 >> 172.21.15.55:0/3914415831 conn(0x563bb9694910 msgr2=0x563bb974e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.800+0000 7fb7257d2740 1 -- 172.21.15.55:0/3914415831 shutdown_connections 2024-04-26T21:54:03.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.800+0000 7fb7257d2740 1 -- 172.21.15.55:0/3914415831 wait complete. 2024-04-26T21:54:03.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.800+0000 7fb7257d2740 20 client.15537 trim_cache size 0 max 16384 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.826+0000 7fb7257d2740 1 Processor -- start 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.826+0000 7fb7257d2740 1 -- start start 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.826+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9762230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.826+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9769450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.826+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb97597c0 0x563bb974d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.826+0000 7fb7257d2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1aa560 con 0x563bb9757c30 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.826+0000 7fb720859640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9762230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb720859640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9762230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49390/0 (socket says 172.21.15.55:49390) 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb720859640 1 -- 172.21.15.55:0/2406963828 learned_addr learned my addr 172.21.15.55:0/2406963828 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.826+0000 7fb7257d2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1a7e30 con 0x563bb97597c0 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb7257d2740 1 -- 172.21.15.55:0/2406963828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1a5c80 con 0x563bb9758e00 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb71bfff640 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9769450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb72105a640 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb97597c0 0x563bb974d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb720859640 1 -- 172.21.15.55:0/2406963828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 msgr2=0x563bb9769450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb720859640 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9769450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb720859640 1 -- 172.21.15.55:0/2406963828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb97597c0 msgr2=0x563bb974d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb720859640 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb97597c0 0x563bb974d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb720859640 1 -- 172.21.15.55:0/2406963828 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb974a8b0 con 0x563bb9757c30 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb720859640 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9762230 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7100d0e20 tx=0x7fb71009b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6567e6c5fb21b77f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb7067fc640 1 -- 172.21.15.55:0/2406963828 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100a5860 con 0x563bb9757c30 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb7067fc640 1 -- 172.21.15.55:0/2406963828 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb710087070 con 0x563bb9757c30 2024-04-26T21:54:03.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.827+0000 7fb7067fc640 1 -- 172.21.15.55:0/2406963828 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71009cbf0 con 0x563bb9757c30 2024-04-26T21:54:03.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.828+0000 7fb7257d2740 1 -- 172.21.15.55:0/2406963828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 msgr2=0x563bb9762230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.828+0000 7fb7257d2740 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9762230 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7100d0e20 tx=0x7fb71009b100 comp rx=0 tx=0).stop 2024-04-26T21:54:03.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.828+0000 7fb7257d2740 1 -- 172.21.15.55:0/2406963828 shutdown_connections 2024-04-26T21:54:03.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.828+0000 7fb7257d2740 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb97597c0 0x563bb974d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.828+0000 7fb7257d2740 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9758e00 0x563bb9769450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.828+0000 7fb7257d2740 1 --2- 172.21.15.55:0/2406963828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb9757c30 0x563bb9762230 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.828+0000 7fb7257d2740 1 -- 172.21.15.55:0/2406963828 >> 172.21.15.55:0/2406963828 conn(0x563bb9694910 msgr2=0x563bb9761b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.829+0000 7fb7257d2740 1 -- 172.21.15.55:0/2406963828 shutdown_connections 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.829+0000 7fb7257d2740 1 -- 172.21.15.55:0/2406963828 wait complete. 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.829+0000 7fb7257d2740 1 -- start start 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.829+0000 7fb7257d2740 1 Processor -- start 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.829+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9757c30 0x563bb97663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.829+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 0x563bb9766910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.829+0000 7fb7257d2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb97d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb97d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb97d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49392/0 (socket says 172.21.15.55:49392) 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 -- 172.21.15.55:0/1439466546 learned_addr learned my addr 172.21.15.55:0/1439466546 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1a62f0 con 0x563bb97597c0 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1a7760 con 0x563bb9758e00 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c1a69c0 con 0x563bb9757c30 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb720859640 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9757c30 0x563bb97663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb71bfff640 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 0x563bb9766910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb7257d2740 10 client.0 _ll_register_callbacks cb 0x7fb71009c9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 -- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9757c30 msgr2=0x563bb97663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9757c30 0x563bb97663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 -- 172.21.15.55:0/1439466546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 msgr2=0x563bb9766910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 0x563bb9766910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 -- 172.21.15.55:0/1439466546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb97d2160 con 0x563bb97597c0 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb720859640 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9757c30 0x563bb97663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb72105a640 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb97d1b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb714009cd0 tx=0x7fb71402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44efe0d02d62b96f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.830+0000 7fb6fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb710087070 con 0x563bb97597c0 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb6fbfff640 1 -- 172.21.15.55:0/1439466546 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7140046a0 con 0x563bb97597c0 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb6fbfff640 1 -- 172.21.15.55:0/1439466546 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb7140305a0 con 0x563bb97597c0 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb6fbfff640 1 -- 172.21.15.55:0/1439466546 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb714004840 con 0x563bb97597c0 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 10 client.15543 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=21}) -- 0x563bb97d3360 con 0x563bb97597c0 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb6fbfff640 1 -- 172.21.15.55:0/1439466546 <== mon.0 v2:172.21.15.55:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb714058030 con 0x563bb97597c0 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 10 client.15543 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 10 client.15543 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 20 client.15543 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563bb97d3bf0 0x563bb97d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 4 client.15543 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.831+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb710087070 con 0x563bb97d3bf0 2024-04-26T21:54:03.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.832+0000 7fb720859640 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563bb97d3bf0 0x563bb97d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.832+0000 7fb6fbfff640 10 client.15543 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.832+0000 7fb720859640 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563bb97d3bf0 0x563bb97d6010 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e09a324b5452a333 server_cookie=4d7a693a893cb07b in_seq=0 out_seq=0 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.832+0000 7fb6fbfff640 1 -- 172.21.15.55:0/1439466546 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb710087070 con 0x563bb97d3bf0 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.832+0000 7fb6fbfff640 10 client.15543 handle_command_reply: tid=0 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.832+0000 7fb7257d2740 1 client.15543 shutdown 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 msgr2=0x563bb97d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb97d1b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb714009cd0 tx=0x7fb71402a8e0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 shutdown_connections 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563bb97d3bf0 0x563bb97d6010 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563bb97597c0 0x563bb97d1b70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb6fbfff640 0 client.15543 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:54:03.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb9758e00 0x563bb9766910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 --2- 172.21.15.55:0/1439466546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bb9757c30 0x563bb97663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 >> 172.21.15.55:0/1439466546 conn(0x563bb9694910 msgr2=0x563bb9752010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 shutdown_connections 2024-04-26T21:54:03.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 1 -- 172.21.15.55:0/1439466546 wait complete. 2024-04-26T21:54:03.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.833+0000 7fb7257d2740 20 client.15543 trim_cache size 0 max 16384 2024-04-26T21:54:03.839 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:54:03.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 -- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb70c0a3c40 msgr2=0x7fb70c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb70c0a3c40 0x7fb70c0a6100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb714009430 tx=0x7fb714031850 comp rx=0 tx=0).stop 2024-04-26T21:54:03.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 -- 172.21.15.55:0/2577151873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb71c100c40 msgr2=0x7fb71c198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb71c100c40 0x7fb71c198bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb71000e9c0 tx=0x7fb71000ee80 comp rx=0 tx=0).stop 2024-04-26T21:54:03.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 -- 172.21.15.55:0/2577151873 shutdown_connections 2024-04-26T21:54:03.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb70c0a3c40 0x7fb70c0a6100 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb71c1045d0 0x7fb71c1a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb71c103c10 0x7fb71c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 --2- 172.21.15.55:0/2577151873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb71c100c40 0x7fb71c198bb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 -- 172.21.15.55:0/2577151873 >> 172.21.15.55:0/2577151873 conn(0x7fb71c0fc570 msgr2=0x7fb71c106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 -- 172.21.15.55:0/2577151873 shutdown_connections 2024-04-26T21:54:03.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.834+0000 7fb722abd640 1 -- 172.21.15.55:0/2577151873 wait complete. 2024-04-26T21:54:03.849 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 -- 172.21.15.55:0/3436579342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb108103450 msgr2=0x7fb108105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3436579342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb108103450 0x7fb108105840 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009900 tx=0x7fb0f802a410 comp rx=0 tx=0).stop 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 -- 172.21.15.55:0/3436579342 shutdown_connections 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3436579342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1081086b0 0x7fb10810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3436579342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108105d80 0x7fb108108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3436579342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb108103450 0x7fb108105840 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 -- 172.21.15.55:0/3436579342 >> 172.21.15.55:0/3436579342 conn(0x7fb1080fcde0 msgr2=0x7fb1080ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 -- 172.21.15.55:0/3436579342 shutdown_connections 2024-04-26T21:54:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.958+0000 7fb10dc59640 1 -- 172.21.15.55:0/3436579342 wait complete. 2024-04-26T21:54:03.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.959+0000 7fb10dc59640 1 Processor -- start 2024-04-26T21:54:03.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.959+0000 7fb10dc59640 1 -- start start 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.959+0000 7fb10dc59640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb108103450 0x7fb108196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.959+0000 7fb10dc59640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108105d80 0x7fb108196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.959+0000 7fb10dc59640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1081086b0 0x7fb10819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb10dc59640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb10810c2e0 con 0x7fb1081086b0 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb10dc59640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb10810c160 con 0x7fb108105d80 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb10dc59640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb10810c460 con 0x7fb108103450 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1081086b0 0x7fb10819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1081086b0 0x7fb10819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49408/0 (socket says 172.21.15.55:49408) 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 -- 172.21.15.55:0/3493151559 learned_addr learned my addr 172.21.15.55:0/3493151559 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb1077fe640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb108103450 0x7fb108196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb106ffd640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108105d80 0x7fb108196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 -- 172.21.15.55:0/3493151559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb108103450 msgr2=0x7fb108196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb108103450 0x7fb108196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 -- 172.21.15.55:0/3493151559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108105d80 msgr2=0x7fb108196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108105d80 0x7fb108196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 -- 172.21.15.55:0/3493151559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10819e620 con 0x7fb1081086b0 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb107fff640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1081086b0 0x7fb10819dfb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb10000edb0 tx=0x7fb100012490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.960+0000 7fb104ff9640 1 -- 172.21.15.55:0/3493151559 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10001c030 con 0x7fb1081086b0 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.961+0000 7fb104ff9640 1 -- 172.21.15.55:0/3493151559 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb100012db0 con 0x7fb1081086b0 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.961+0000 7fb10dc59640 1 -- 172.21.15.55:0/3493151559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10819e910 con 0x7fb1081086b0 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.961+0000 7fb104ff9640 1 -- 172.21.15.55:0/3493151559 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100018400 con 0x7fb1081086b0 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.961+0000 7fb10dc59640 1 -- 172.21.15.55:0/3493151559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb10819eeb0 con 0x7fb1081086b0 2024-04-26T21:54:03.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.963+0000 7fb104ff9640 1 -- 172.21.15.55:0/3493151559 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb10001f070 con 0x7fb1081086b0 2024-04-26T21:54:03.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb104ff9640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0f00a3cc0 0x7fb0f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb110972740 1 Processor -- start 2024-04-26T21:54:03.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb1077fe640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0f00a3cc0 0x7fb0f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb104ff9640 1 -- 172.21.15.55:0/3493151559 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb1000d4180 con 0x7fb1081086b0 2024-04-26T21:54:03.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb110972740 1 -- start start 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb110972740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb110972740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb110972740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb110972740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa709d80 con 0x55dcaa7a73b0 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb110972740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa6c3a20 con 0x55dcaa7a69f0 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.964+0000 7fb110972740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa705d40 con 0x55dcaa7a5820 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49414/0 (socket says 172.21.15.55:49414) 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 -- 172.21.15.55:0/490470170 learned_addr learned my addr 172.21.15.55:0/490470170 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb106ffd640 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 -- 172.21.15.55:0/490470170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 msgr2=0x55dcaa7a5c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb1077fe640 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 -- 172.21.15.55:0/490470170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a69f0 msgr2=0x55dcaa7a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7a6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 -- 172.21.15.55:0/490470170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcaa6cec30 con 0x55dcaa7a73b0 2024-04-26T21:54:03.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb106ffd640 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:03.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.965+0000 7fb107fff640 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7adc40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb1000a8a00 tx=0x7fb1000a0b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43e93085f574428 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.966+0000 7fb0edffb640 1 -- 172.21.15.55:0/490470170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000b9070 con 0x55dcaa7a73b0 2024-04-26T21:54:03.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.966+0000 7fb0edffb640 1 -- 172.21.15.55:0/490470170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1000d1d40 con 0x55dcaa7a73b0 2024-04-26T21:54:03.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.966+0000 7fb0edffb640 1 -- 172.21.15.55:0/490470170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000a43d0 con 0x55dcaa7a73b0 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 -- 172.21.15.55:0/490470170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 msgr2=0x55dcaa7adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb1077fe640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0f00a3cc0 0x7fb0f00a6180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8000c00 tx=0x7fb0f8005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7adc40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb1000a8a00 tx=0x7fb1000a0b30 comp rx=0 tx=0).stop 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 -- 172.21.15.55:0/490470170 shutdown_connections 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7adc40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 --2- 172.21.15.55:0/490470170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 -- 172.21.15.55:0/490470170 >> 172.21.15.55:0/490470170 conn(0x55dcaa6e6910 msgr2=0x55dcaa7a1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 -- 172.21.15.55:0/490470170 shutdown_connections 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 -- 172.21.15.55:0/490470170 wait complete. 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.967+0000 7fb110972740 1 -- start start 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb110972740 1 Processor -- start 2024-04-26T21:54:03.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb110972740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 0x55dcaa7afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb110972740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7affd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb110972740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb110972740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa7ae3c0 con 0x55dcaa7a69f0 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb110972740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa690dc0 con 0x55dcaa7a5820 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb110972740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dcaa7ae540 con 0x55dcaa7a73b0 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb110972740 10 client.0 _ll_register_callbacks cb 0x55dcaa6ac600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49430/0 (socket says 172.21.15.55:49430) 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb106ffd640 1 -- 172.21.15.55:0/684386022 learned_addr learned my addr 172.21.15.55:0/684386022 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.968+0000 7fb107fff640 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb106ffd640 1 -- 172.21.15.55:0/684386022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a73b0 msgr2=0x55dcaa7b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb106ffd640 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb106ffd640 1 -- 172.21.15.55:0/684386022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 msgr2=0x55dcaa7afa90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:54:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb106ffd640 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 0x55dcaa7afa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb106ffd640 1 -- 172.21.15.55:0/684386022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcaa41b3f0 con 0x55dcaa7a69f0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb106ffd640 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7affd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00bad0 tx=0x7fb0fc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f016970d75d75acf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb0e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/684386022 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc003da0 con 0x55dcaa7a69f0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/684386022 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb0fc005020 con 0x55dcaa7a69f0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb110972740 1 -- 172.21.15.55:0/684386022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1000d1d40 con 0x55dcaa7a69f0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.969+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/684386022 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb0fc005260 con 0x55dcaa7a69f0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb110972740 10 client.15555 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb110972740 1 -- 172.21.15.55:0/684386022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dcaa4e57d0 con 0x55dcaa7a69f0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/684386022 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1000d1d40 con 0x55dcaa7a69f0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/684386022 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb0fc01b590 con 0x55dcaa7a69f0 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb110972740 10 client.15555 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb110972740 10 client.15555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb110972740 20 client.15555 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:03.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb110972740 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55dcaa7bb8c0 0x55dcaa7bdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:03.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb110972740 4 client.15555 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:03.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.970+0000 7fb110972740 1 -- 172.21.15.55:0/684386022 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcaa717990 con 0x55dcaa7bb8c0 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.971+0000 7fb1077fe640 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55dcaa7bb8c0 0x55dcaa7bdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.971+0000 7fb1077fe640 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55dcaa7bb8c0 0x55dcaa7bdd60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f8a18a6688907b1 server_cookie=78defc440b8d7be2 in_seq=0 out_seq=0 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.971+0000 7fb0e6ffd640 10 client.15555 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/684386022 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcaa717990 con 0x55dcaa7bb8c0 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb0e6ffd640 10 client.15555 handle_command_reply: tid=0 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 client.15555 shutdown 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 -- 172.21.15.55:0/684386022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a69f0 msgr2=0x55dcaa7affd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7affd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00bad0 tx=0x7fb0fc00bf90 comp rx=0 tx=0).stop 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 -- 172.21.15.55:0/684386022 shutdown_connections 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55dcaa7bb8c0 0x55dcaa7bdd60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a69f0 0x55dcaa7affd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb0e6ffd640 0 client.15555 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:54:03.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 --2- 172.21.15.55:0/684386022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 0x55dcaa7afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:03.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 -- 172.21.15.55:0/684386022 >> 172.21.15.55:0/684386022 conn(0x55dcaa6e6910 msgr2=0x55dcaa7a0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:03.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 -- 172.21.15.55:0/684386022 shutdown_connections 2024-04-26T21:54:03.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.972+0000 7fb110972740 1 -- 172.21.15.55:0/684386022 wait complete. 2024-04-26T21:54:03.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.973+0000 7fb110972740 20 client.15555 trim_cache size 0 max 16384 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb110972740 1 Processor -- start 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb110972740 1 -- start start 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb110972740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb110972740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb110972740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb110972740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081aa670 con 0x55dcaa7a73b0 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb110972740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081a7f40 con 0x55dcaa7a5820 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb110972740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081a5d90 con 0x55dcaa7aa9c0 2024-04-26T21:54:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49432/0 (socket says 172.21.15.55:49432) 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb106ffd640 1 -- 172.21.15.55:0/4087695991 learned_addr learned my addr 172.21.15.55:0/4087695991 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb107fff640 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb1077fe640 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb106ffd640 1 -- 172.21.15.55:0/4087695991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7aa9c0 msgr2=0x55dcaa7a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb106ffd640 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb106ffd640 1 -- 172.21.15.55:0/4087695991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 msgr2=0x55dcaa7a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb106ffd640 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb106ffd640 1 -- 172.21.15.55:0/4087695991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcaa7ba2c0 con 0x55dcaa7a73b0 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:03.999+0000 7fb1077fe640 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.000+0000 7fb106ffd640 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b6fd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc01bb40 tx=0x7fb0fc004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebc6228a8b27dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.000+0000 7fb0ed7fa640 1 -- 172.21.15.55:0/4087695991 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc02b040 con 0x55dcaa7a73b0 2024-04-26T21:54:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.000+0000 7fb0ed7fa640 1 -- 172.21.15.55:0/4087695991 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0fc021030 con 0x55dcaa7a73b0 2024-04-26T21:54:04.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.000+0000 7fb0ed7fa640 1 -- 172.21.15.55:0/4087695991 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc01b6d0 con 0x55dcaa7a73b0 2024-04-26T21:54:04.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 -- 172.21.15.55:0/4087695991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 msgr2=0x55dcaa7b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b6fd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc01bb40 tx=0x7fb0fc004400 comp rx=0 tx=0).stop 2024-04-26T21:54:04.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 -- 172.21.15.55:0/4087695991 shutdown_connections 2024-04-26T21:54:04.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b6fd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 --2- 172.21.15.55:0/4087695991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7a5820 0x55dcaa7a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 -- 172.21.15.55:0/4087695991 >> 172.21.15.55:0/4087695991 conn(0x55dcaa6e6910 msgr2=0x55dcaa7aa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 -- 172.21.15.55:0/4087695991 shutdown_connections 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 -- 172.21.15.55:0/4087695991 wait complete. 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.001+0000 7fb110972740 1 -- start start 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb110972740 1 Processor -- start 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb110972740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb110972740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb110972740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb106ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49440/0 (socket says 172.21.15.55:49440) 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb106ffd640 1 -- 172.21.15.55:0/628094643 learned_addr learned my addr 172.21.15.55:0/628094643 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081a6400 con 0x55dcaa7a73b0 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081a7870 con 0x55dcaa7aa9c0 2024-04-26T21:54:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb1081a6ad0 con 0x55dcaa7a5820 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb1077fe640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.002+0000 7fb110972740 10 client.0 _ll_register_callbacks cb 0x7fb0fc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb106ffd640 1 -- 172.21.15.55:0/628094643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 msgr2=0x55dcaa7b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb107fff640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb106ffd640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb106ffd640 1 -- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7aa9c0 msgr2=0x55dcaa7b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb106ffd640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb106ffd640 1 -- 172.21.15.55:0/628094643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcaa7b6300 con 0x55dcaa7a73b0 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb107fff640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb1077fe640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb106ffd640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b4d50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc01ba70 tx=0x7fb0fc01bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66901c0d643c19f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb0e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:04.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/628094643 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc03c060 con 0x55dcaa7a73b0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/628094643 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb0fc01b470 con 0x55dcaa7a73b0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0fc021030 con 0x55dcaa7a73b0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/628094643 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb0fc02b040 con 0x55dcaa7a73b0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb110972740 10 client.15561 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.003+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dcaa821f00 con 0x55dcaa7a73b0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/628094643 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0fc021030 con 0x55dcaa7a73b0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/628094643 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb0fc042420 con 0x55dcaa7a73b0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb110972740 10 client.15561 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb110972740 10 client.15561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb110972740 20 client.15561 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb110972740 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55dcaa8228d0 0x55dcaa824cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb110972740 4 client.15561 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcaa711800 con 0x55dcaa8228d0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.004+0000 7fb1077fe640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55dcaa8228d0 0x55dcaa824cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb0e6ffd640 10 client.15561 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb1077fe640 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55dcaa8228d0 0x55dcaa824cf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a847ba15d8f78484 server_cookie=e83d64f60c6cd177 in_seq=0 out_seq=0 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb0e6ffd640 1 -- 172.21.15.55:0/628094643 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcaa711800 con 0x55dcaa8228d0 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb0e6ffd640 10 client.15561 handle_command_reply: tid=0 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 client.15561 shutdown 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 msgr2=0x55dcaa7b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b4d50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc01ba70 tx=0x7fb0fc01bc00 comp rx=0 tx=0).stop 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 shutdown_connections 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55dcaa8228d0 0x55dcaa824cf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb0e6ffd640 0 client.15561 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dcaa7aa9c0 0x55dcaa7b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcaa7a73b0 0x55dcaa7b4d50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 --2- 172.21.15.55:0/628094643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dcaa7a5820 0x55dcaa7b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 >> 172.21.15.55:0/628094643 conn(0x55dcaa6e6910 msgr2=0x55dcaa79f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.005+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 shutdown_connections 2024-04-26T21:54:04.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb110972740 1 -- 172.21.15.55:0/628094643 wait complete. 2024-04-26T21:54:04.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb110972740 20 client.15561 trim_cache size 0 max 16384 2024-04-26T21:54:04.011 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 -- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0f00a3cc0 msgr2=0x7fb0f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0f00a3cc0 0x7fb0f00a6180 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8000c00 tx=0x7fb0f8005d70 comp rx=0 tx=0).stop 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 -- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1081086b0 msgr2=0x7fb10819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1081086b0 0x7fb10819dfb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb10000edb0 tx=0x7fb100012490 comp rx=0 tx=0).stop 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 -- 172.21.15.55:0/3493151559 shutdown_connections 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0f00a3cc0 0x7fb0f00a6180 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1081086b0 0x7fb10819dfb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb108105d80 0x7fb108196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 --2- 172.21.15.55:0/3493151559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb108103450 0x7fb108196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.006+0000 7fb10dc59640 1 -- 172.21.15.55:0/3493151559 >> 172.21.15.55:0/3493151559 conn(0x7fb1080fcde0 msgr2=0x7fb1081052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.007+0000 7fb10dc59640 1 -- 172.21.15.55:0/3493151559 shutdown_connections 2024-04-26T21:54:04.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.007+0000 7fb10dc59640 1 -- 172.21.15.55:0/3493151559 wait complete. 2024-04-26T21:54:04.021 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:54:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 -- 172.21.15.55:0/260405449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c103470 msgr2=0x7f331c105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 --2- 172.21.15.55:0/260405449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c103470 0x7f331c105860 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f330c009900 tx=0x7f330c02a410 comp rx=0 tx=0).stop 2024-04-26T21:54:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 -- 172.21.15.55:0/260405449 shutdown_connections 2024-04-26T21:54:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 --2- 172.21.15.55:0/260405449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f331c1086d0 0x7f331c10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 --2- 172.21.15.55:0/260405449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c105da0 0x7f331c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 --2- 172.21.15.55:0/260405449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c103470 0x7f331c105860 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 -- 172.21.15.55:0/260405449 >> 172.21.15.55:0/260405449 conn(0x7f331c0fcde0 msgr2=0x7f331c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 -- 172.21.15.55:0/260405449 shutdown_connections 2024-04-26T21:54:04.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.126+0000 7f3321389640 1 -- 172.21.15.55:0/260405449 wait complete. 2024-04-26T21:54:04.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f3321389640 1 Processor -- start 2024-04-26T21:54:04.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f3321389640 1 -- start start 2024-04-26T21:54:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f3321389640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c103470 0x7f331c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f3321389640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c105da0 0x7f331c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f3321389640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f331c1086d0 0x7f331c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f3321389640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f331c10e4c0 con 0x7f331c105da0 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f3321389640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f331c10e340 con 0x7f331c103470 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f3321389640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f331c10e640 con 0x7f331c1086d0 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f331a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c105da0 0x7f331c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f331a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c105da0 0x7f331c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49444/0 (socket says 172.21.15.55:49444) 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.127+0000 7f331a7fc640 1 -- 172.21.15.55:0/2443918620 learned_addr learned my addr 172.21.15.55:0/2443918620 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331b7fe640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f331c1086d0 0x7f331c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331affd640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c103470 0x7f331c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331a7fc640 1 -- 172.21.15.55:0/2443918620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f331c1086d0 msgr2=0x7f331c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331a7fc640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f331c1086d0 0x7f331c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331a7fc640 1 -- 172.21.15.55:0/2443918620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c103470 msgr2=0x7f331c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331a7fc640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c103470 0x7f331c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331a7fc640 1 -- 172.21.15.55:0/2443918620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c1a0840 con 0x7f331c105da0 2024-04-26T21:54:04.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331b7fe640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f331c1086d0 0x7f331c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f331a7fc640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c105da0 0x7f331c1991c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3310005fd0 tx=0x7f331000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f3303fff640 1 -- 172.21.15.55:0/2443918620 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310017070 con 0x7f331c105da0 2024-04-26T21:54:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f3303fff640 1 -- 172.21.15.55:0/2443918620 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3310009c30 con 0x7f331c105da0 2024-04-26T21:54:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f3321389640 1 -- 172.21.15.55:0/2443918620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331c1012c0 con 0x7f331c105da0 2024-04-26T21:54:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f3303fff640 1 -- 172.21.15.55:0/2443918620 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310003d30 con 0x7f331c105da0 2024-04-26T21:54:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.128+0000 7f3321389640 1 -- 172.21.15.55:0/2443918620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331c101860 con 0x7f331c105da0 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3303fff640 1 -- 172.21.15.55:0/2443918620 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3310003ed0 con 0x7f331c105da0 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3324180740 1 Processor -- start 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3324180740 1 -- start start 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3324180740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3bb0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3324180740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bb1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3324180740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bb8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3324180740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cde3b14d80 con 0x55cde3bb19f0 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3324180740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cde3acea20 con 0x55cde3bb23b0 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f3324180740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cde3b10d40 con 0x55cde3bb0820 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f331affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bb1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f331affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bb1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49458/0 (socket says 172.21.15.55:49458) 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.131+0000 7f331affd640 1 -- 172.21.15.55:0/888903496 learned_addr learned my addr 172.21.15.55:0/888903496 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331b7fe640 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3bb0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f3303fff640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33040a3cc0 0x7f33040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331a7fc640 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bb8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331affd640 1 -- 172.21.15.55:0/888903496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 msgr2=0x55cde3bb0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331affd640 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3bb0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331affd640 1 -- 172.21.15.55:0/888903496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 msgr2=0x55cde3bb8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331affd640 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bb8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331affd640 1 -- 172.21.15.55:0/888903496 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cde3ad9c30 con 0x55cde3bb19f0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f3303fff640 1 -- 172.21.15.55:0/2443918620 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f33100d0390 con 0x7f331c105da0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331b7fe640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33040a3cc0 0x7f33040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f331affd640 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bb1e70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f330c02a930 tx=0x7f330c032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beea580c1aceaa23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.132+0000 7f33017fa640 1 -- 172.21.15.55:0/888903496 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c048070 con 0x55cde3bb19f0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f33017fa640 1 -- 172.21.15.55:0/888903496 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f330c004040 con 0x55cde3bb19f0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f331b7fe640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33040a3cc0 0x7f33040a6180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f33140092f0 tx=0x7f33140096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f33017fa640 1 -- 172.21.15.55:0/888903496 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c035050 con 0x55cde3bb19f0 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 -- 172.21.15.55:0/888903496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 msgr2=0x55cde3bb1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bb1e70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f330c02a930 tx=0x7f330c032bd0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 -- 172.21.15.55:0/888903496 shutdown_connections 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bb8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bb1e70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 --2- 172.21.15.55:0/888903496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3bb0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 -- 172.21.15.55:0/888903496 >> 172.21.15.55:0/888903496 conn(0x55cde3af1910 msgr2=0x55cde3bac120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 -- 172.21.15.55:0/888903496 shutdown_connections 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.133+0000 7f3324180740 1 -- 172.21.15.55:0/888903496 wait complete. 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 1 -- start start 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 1 Processor -- start 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3bbab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bbb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cde3bb9400 con 0x55cde3bb19f0 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cde3a9bdc0 con 0x55cde3bb23b0 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cde3bb9580 con 0x55cde3bb0820 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f3324180740 10 client.0 _ll_register_callbacks cb 0x55cde3ab7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bbb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bbb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49468/0 (socket says 172.21.15.55:49468) 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331a7fc640 1 -- 172.21.15.55:0/1770395705 learned_addr learned my addr 172.21.15.55:0/1770395705 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331affd640 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3bbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331b7fe640 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331a7fc640 1 -- 172.21.15.55:0/1770395705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 msgr2=0x55cde3bbab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331a7fc640 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3bbab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331a7fc640 1 -- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 msgr2=0x55cde3bc2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.134+0000 7f331a7fc640 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f331a7fc640 1 -- 172.21.15.55:0/1770395705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cde38263f0 con 0x55cde3bb19f0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f331b7fe640 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f331a7fc640 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bbb040 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f33100988a0 tx=0x7f3310009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a511b659c798752 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f32feff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f32feff5640 1 -- 172.21.15.55:0/1770395705 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331009c4d0 con 0x55cde3bb19f0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f32feff5640 1 -- 172.21.15.55:0/1770395705 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3310098ab0 con 0x55cde3bb19f0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f3324180740 1 -- 172.21.15.55:0/1770395705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f330c004040 con 0x55cde3bb19f0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f32feff5640 1 -- 172.21.15.55:0/1770395705 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f33100a0880 con 0x55cde3bb19f0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f3324180740 10 client.15573 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f3324180740 1 -- 172.21.15.55:0/1770395705 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cde38f07d0 con 0x55cde3bb19f0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.135+0000 7f32feff5640 1 -- 172.21.15.55:0/1770395705 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f330c004040 con 0x55cde3bb19f0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f32feff5640 1 -- 172.21.15.55:0/1770395705 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f33100041c0 con 0x55cde3bb19f0 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f3324180740 10 client.15573 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f3324180740 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f3324180740 20 client.15573 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:04.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f3324180740 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cde3bc69b0 0x55cde3bc8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f3324180740 4 client.15573 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f3324180740 1 -- 172.21.15.55:0/1770395705 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cde3b22990 con 0x55cde3bc69b0 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f331affd640 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cde3bc69b0 0x55cde3bc8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f331affd640 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cde3bc69b0 0x55cde3bc8da0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be43566ff4b51cba server_cookie=3bd13065563cae7f in_seq=0 out_seq=0 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.136+0000 7f32feff5640 10 client.15573 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f32feff5640 1 -- 172.21.15.55:0/1770395705 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cde3b22990 con 0x55cde3bc69b0 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f32feff5640 10 client.15573 handle_command_reply: tid=0 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 client.15573 shutdown 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 -- 172.21.15.55:0/1770395705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 msgr2=0x55cde3bbb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bbb040 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f33100988a0 tx=0x7f3310009de0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 -- 172.21.15.55:0/1770395705 shutdown_connections 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cde3bc69b0 0x55cde3bc8da0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f32feff5640 0 client.15573 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:04.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bb19f0 0x55cde3bbb040 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 --2- 172.21.15.55:0/1770395705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3bbab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 -- 172.21.15.55:0/1770395705 >> 172.21.15.55:0/1770395705 conn(0x55cde3af1910 msgr2=0x55cde3bab280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.137+0000 7f3324180740 1 -- 172.21.15.55:0/1770395705 shutdown_connections 2024-04-26T21:54:04.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.138+0000 7f3324180740 1 -- 172.21.15.55:0/1770395705 wait complete. 2024-04-26T21:54:04.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.138+0000 7f3324180740 20 client.15573 trim_cache size 0 max 16384 2024-04-26T21:54:04.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f3324180740 1 Processor -- start 2024-04-26T21:54:04.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f3324180740 1 -- start start 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f3324180740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3baa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f3324180740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f3324180740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3baaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f3324180740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1aa5a0 con 0x55cde3bc69b0 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f3324180740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1a7e70 con 0x55cde3bb23b0 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f3324180740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1a5cc0 con 0x55cde3bb0820 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3baaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3baaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49482/0 (socket says 172.21.15.55:49482) 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f331b7fe640 1 -- 172.21.15.55:0/1669817692 learned_addr learned my addr 172.21.15.55:0/1669817692 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f331affd640 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3baa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.151+0000 7f331a7fc640 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f331b7fe640 1 -- 172.21.15.55:0/1669817692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 msgr2=0x55cde3baa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f331b7fe640 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3baa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f331b7fe640 1 -- 172.21.15.55:0/1669817692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 msgr2=0x55cde3bc2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f331b7fe640 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f331b7fe640 1 -- 172.21.15.55:0/1669817692 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cde3ba78b0 con 0x55cde3bc69b0 2024-04-26T21:54:04.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f331affd640 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3baa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:04.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f331b7fe640 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3baaf30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3314004d10 tx=0x7f3314017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1db95660709f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f3300ff9640 1 -- 172.21.15.55:0/1669817692 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314012d30 con 0x55cde3bc69b0 2024-04-26T21:54:04.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f3300ff9640 1 -- 172.21.15.55:0/1669817692 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f331401f8c0 con 0x55cde3bc69b0 2024-04-26T21:54:04.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.152+0000 7f3300ff9640 1 -- 172.21.15.55:0/1669817692 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331400f940 con 0x55cde3bc69b0 2024-04-26T21:54:04.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 -- 172.21.15.55:0/1669817692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 msgr2=0x55cde3baaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3baaf30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3314004d10 tx=0x7f3314017000 comp rx=0 tx=0).stop 2024-04-26T21:54:04.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 -- 172.21.15.55:0/1669817692 shutdown_connections 2024-04-26T21:54:04.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3baaf30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb23b0 0x55cde3bc2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 --2- 172.21.15.55:0/1669817692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb0820 0x55cde3baa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 -- 172.21.15.55:0/1669817692 >> 172.21.15.55:0/1669817692 conn(0x55cde3af1910 msgr2=0x55cde3bb52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 -- 172.21.15.55:0/1669817692 shutdown_connections 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 -- 172.21.15.55:0/1669817692 wait complete. 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 -- start start 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.153+0000 7f3324180740 1 Processor -- start 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f3324180740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb0820 0x55cde3bbf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f3324180740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb23b0 0x55cde3bbf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f3324180740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3bc06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f3324180740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1a6330 con 0x55cde3bc69b0 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f3324180740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1a77a0 con 0x55cde3bb0820 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f3324180740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f331c1a6a00 con 0x55cde3bb23b0 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3bc06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3bc06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49484/0 (socket says 172.21.15.55:49484) 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331b7fe640 1 -- 172.21.15.55:0/3333326344 learned_addr learned my addr 172.21.15.55:0/3333326344 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f3324180740 10 client.0 _ll_register_callbacks cb 0x7f331009be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331a7fc640 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb23b0 0x55cde3bbf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331affd640 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb0820 0x55cde3bbf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331b7fe640 1 -- 172.21.15.55:0/3333326344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb23b0 msgr2=0x55cde3bbf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331b7fe640 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb23b0 0x55cde3bbf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331b7fe640 1 -- 172.21.15.55:0/3333326344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb0820 msgr2=0x55cde3bbf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331b7fe640 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb0820 0x55cde3bbf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331b7fe640 1 -- 172.21.15.55:0/3333326344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cde3bc0c90 con 0x55cde3bc69b0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.154+0000 7f331a7fc640 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb23b0 0x55cde3bbf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f331b7fe640 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3bc06a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3314009580 tx=0x7f331400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe89cb4a96dffbe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f32feff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f32feff5640 1 -- 172.21.15.55:0/3333326344 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314034020 con 0x55cde3bc69b0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f32feff5640 1 -- 172.21.15.55:0/3333326344 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3314012d40 con 0x55cde3bc69b0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f3324180740 1 -- 172.21.15.55:0/3333326344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f331401f8c0 con 0x55cde3bc69b0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f32feff5640 1 -- 172.21.15.55:0/3333326344 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f331403f050 con 0x55cde3bc69b0 2024-04-26T21:54:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f3324180740 10 client.15579 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f3324180740 1 -- 172.21.15.55:0/3333326344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cde3bc1f70 con 0x55cde3bc69b0 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f32feff5640 1 -- 172.21.15.55:0/3333326344 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f331401f8c0 con 0x55cde3bc69b0 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f32feff5640 1 -- 172.21.15.55:0/3333326344 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f3314028430 con 0x55cde3bc69b0 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.155+0000 7f3324180740 10 client.15579 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f3324180740 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f3324180740 20 client.15579 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f3324180740 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cde3c2d180 0x55cde3c2f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f3324180740 4 client.15579 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f3324180740 1 -- 172.21.15.55:0/3333326344 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cde3b1c800 con 0x55cde3c2d180 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f331affd640 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cde3c2d180 0x55cde3c2f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f331affd640 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cde3c2d180 0x55cde3c2f5a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e3f925cc6464690 server_cookie=b74f1f03f9c0cc60 in_seq=0 out_seq=0 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f32feff5640 10 client.15579 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f32feff5640 1 -- 172.21.15.55:0/3333326344 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cde3b1c800 con 0x55cde3c2d180 2024-04-26T21:54:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f32feff5640 10 client.15579 handle_command_reply: tid=0 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.156+0000 7f3324180740 1 client.15579 shutdown 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 -- 172.21.15.55:0/3333326344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 msgr2=0x55cde3bc06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3bc06a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3314009580 tx=0x7f331400f520 comp rx=0 tx=0).stop 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 -- 172.21.15.55:0/3333326344 shutdown_connections 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cde3c2d180 0x55cde3c2f5a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cde3bc69b0 0x55cde3bc06a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cde3bb23b0 0x55cde3bbf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 --2- 172.21.15.55:0/3333326344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cde3bb0820 0x55cde3bbf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 -- 172.21.15.55:0/3333326344 >> 172.21.15.55:0/3333326344 conn(0x55cde3af1910 msgr2=0x55cde3baf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f32feff5640 0 client.15579 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 -- 172.21.15.55:0/3333326344 shutdown_connections 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 1 -- 172.21.15.55:0/3333326344 wait complete. 2024-04-26T21:54:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3324180740 20 client.15579 trim_cache size 0 max 16384 2024-04-26T21:54:04.163 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:54:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3321389640 1 -- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33040a3cc0 msgr2=0x7f33040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3321389640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33040a3cc0 0x7f33040a6180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f33140092f0 tx=0x7f33140096d0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3321389640 1 -- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c105da0 msgr2=0x7f331c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3321389640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c105da0 0x7f331c1991c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3310005fd0 tx=0x7f331000dde0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3321389640 1 -- 172.21.15.55:0/2443918620 shutdown_connections 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.157+0000 7f3321389640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f33040a3cc0 0x7f33040a6180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.158+0000 7f3321389640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f331c1086d0 0x7f331c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.158+0000 7f3321389640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f331c105da0 0x7f331c1991c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.158+0000 7f3321389640 1 --2- 172.21.15.55:0/2443918620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f331c103470 0x7f331c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.158+0000 7f3321389640 1 -- 172.21.15.55:0/2443918620 >> 172.21.15.55:0/2443918620 conn(0x7f331c0fcde0 msgr2=0x7f331c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.158+0000 7f3321389640 1 -- 172.21.15.55:0/2443918620 shutdown_connections 2024-04-26T21:54:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:04.158+0000 7f3321389640 1 -- 172.21.15.55:0/2443918620 wait complete. 2024-04-26T21:54:04.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:04 smithi064 ceph-mon[32765]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:54:04.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:04 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/377136155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:04.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:04 smithi165 ceph-mon[33296]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:54:04.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:04 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/377136155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:04.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:04 smithi055 ceph-mon[29221]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:54:04.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:04 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/377136155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:05.170 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:54:05.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:54:05.171 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:54:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 -- 172.21.15.55:0/1168046366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868103420 msgr2=0x7f3868109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 --2- 172.21.15.55:0/1168046366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868103420 0x7f3868109cb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3860009ca0 tx=0x7f386002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 -- 172.21.15.55:0/1168046366 shutdown_connections 2024-04-26T21:54:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 --2- 172.21.15.55:0/1168046366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868103420 0x7f3868109cb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 --2- 172.21.15.55:0/1168046366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868102a60 0x7f3868102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 --2- 172.21.15.55:0/1168046366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3868101870 0x7f3868101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 -- 172.21.15.55:0/1168046366 >> 172.21.15.55:0/1168046366 conn(0x7f38680fcde0 msgr2=0x7f38680ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 -- 172.21.15.55:0/1168046366 shutdown_connections 2024-04-26T21:54:05.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.283+0000 7f386ebe7640 1 -- 172.21.15.55:0/1168046366 wait complete. 2024-04-26T21:54:05.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.284+0000 7f386ebe7640 1 Processor -- start 2024-04-26T21:54:05.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.284+0000 7f386ebe7640 1 -- start start 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.284+0000 7f386ebe7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3868101870 0x7f3868198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.284+0000 7f386ebe7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868102a60 0x7f3868199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868102a60 0x7f3868199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868102a60 0x7f3868199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49496/0 (socket says 172.21.15.55:49496) 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386ebe7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868103420 0x7f38681a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 -- 172.21.15.55:0/352433173 learned_addr learned my addr 172.21.15.55:0/352433173 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f386810e4c0 con 0x7f3868102a60 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f386810e340 con 0x7f3868101870 2024-04-26T21:54:05.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f386810e640 con 0x7f3868103420 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386c983640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3868101870 0x7f3868198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386d184640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868103420 0x7f38681a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 -- 172.21.15.55:0/352433173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868103420 msgr2=0x7f38681a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868103420 0x7f38681a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 -- 172.21.15.55:0/352433173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3868101870 msgr2=0x7f3868198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3868101870 0x7f3868198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 -- 172.21.15.55:0/352433173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38681a07f0 con 0x7f3868102a60 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386c983640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3868101870 0x7f3868198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386d184640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868103420 0x7f38681a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3867fff640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868102a60 0x7f3868199110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f385400d770 tx=0x7f385400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3865ffb640 1 -- 172.21.15.55:0/352433173 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854017070 con 0x7f3868102a60 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f3865ffb640 1 -- 172.21.15.55:0/352433173 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38540054d0 con 0x7f3868102a60 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.285+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38681a0ae0 con 0x7f3868102a60 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.286+0000 7f3865ffb640 1 -- 172.21.15.55:0/352433173 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854004c30 con 0x7f3868102a60 2024-04-26T21:54:05.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.286+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38681a0fe0 con 0x7f3868102a60 2024-04-26T21:54:05.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.289+0000 7f3865ffb640 1 -- 172.21.15.55:0/352433173 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3854004df0 con 0x7f3868102a60 2024-04-26T21:54:05.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.289+0000 7f38719d9740 1 Processor -- start 2024-04-26T21:54:05.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.289+0000 7f38719d9740 1 -- start start 2024-04-26T21:54:05.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.289+0000 7f38719d9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce25830 0x557e9ce25c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.289+0000 7f38719d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f38719d9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f38719d9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557e9cd8ab00 con 0x557e9ce29380 2024-04-26T21:54:05.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f38719d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557e9cd26920 con 0x557e9ce26220 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f38719d9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557e9cd86ac0 con 0x557e9ce25830 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce361f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49498/0 (socket says 172.21.15.55:49498) 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3867fff640 1 -- 172.21.15.55:0/1791424413 learned_addr learned my addr 172.21.15.55:0/1791424413 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f386d184640 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce25830 0x557e9ce25c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f386c983640 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3865ffb640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38580a3c40 0x7f38580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3865ffb640 1 -- 172.21.15.55:0/352433173 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f38540d3170 con 0x7f3868102a60 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3867fff640 1 -- 172.21.15.55:0/1791424413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce25830 msgr2=0x557e9ce25c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3867fff640 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce25830 0x557e9ce25c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3867fff640 1 -- 172.21.15.55:0/1791424413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 msgr2=0x557e9ce28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3867fff640 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce28d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f3867fff640 1 -- 172.21.15.55:0/1791424413 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9cb647e0 con 0x557e9ce29380 2024-04-26T21:54:05.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f386d184640 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce25830 0x557e9ce25c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:05.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.290+0000 7f386d184640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38580a3c40 0x7f38580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.291+0000 7f3867fff640 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce361f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3854097ba0 tx=0x7f3854097b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec98d1651756fb05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.291+0000 7f3852ffd640 1 -- 172.21.15.55:0/1791424413 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38540d4060 con 0x557e9ce29380 2024-04-26T21:54:05.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.291+0000 7f3852ffd640 1 -- 172.21.15.55:0/1791424413 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3854005680 con 0x557e9ce29380 2024-04-26T21:54:05.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.291+0000 7f3852ffd640 1 -- 172.21.15.55:0/1791424413 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38540a43b0 con 0x557e9ce29380 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 -- 172.21.15.55:0/1791424413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 msgr2=0x557e9ce361f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f386d184640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38580a3c40 0x7f38580a6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f386002f500 tx=0x7f38600095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce361f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3854097ba0 tx=0x7f3854097b10 comp rx=0 tx=0).stop 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 -- 172.21.15.55:0/1791424413 shutdown_connections 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce361f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 --2- 172.21.15.55:0/1791424413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce25830 0x557e9ce25c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 -- 172.21.15.55:0/1791424413 >> 172.21.15.55:0/1791424413 conn(0x557e9ce20f20 msgr2=0x557e9ce21320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 -- 172.21.15.55:0/1791424413 shutdown_connections 2024-04-26T21:54:05.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.292+0000 7f38719d9740 1 -- 172.21.15.55:0/1791424413 wait complete. 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.293+0000 7f38719d9740 1 -- start start 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.293+0000 7f38719d9740 1 Processor -- start 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.293+0000 7f38719d9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce25830 0x557e9ce2fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.293+0000 7f38719d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce30160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f38719d9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce29380 0x557e9ce35190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386c983640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce25830 0x557e9ce2fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386c983640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce25830 0x557e9ce2fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49512/0 (socket says 172.21.15.55:49512) 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386c983640 1 -- 172.21.15.55:0/942387642 learned_addr learned my addr 172.21.15.55:0/942387642 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x557e9ce29d60 con 0x557e9ce25830 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557e9cd245d0 con 0x557e9ce26220 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557e9ce29ee0 con 0x557e9ce29380 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f3867fff640 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce30160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386d184640 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce29380 0x557e9ce35190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386c983640 1 -- 172.21.15.55:0/942387642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce29380 msgr2=0x557e9ce35190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386c983640 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce29380 0x557e9ce35190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386c983640 1 -- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 msgr2=0x557e9ce30160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386c983640 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce30160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f386c983640 1 -- 172.21.15.55:0/942387642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9cbc4f70 con 0x557e9ce25830 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.294+0000 7f3867fff640 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce30160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f386c983640 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce25830 0x557e9ce2fc20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f385c012700 tx=0x7f385c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f87c464d6bf38a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f3847fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f38719d9740 10 client.0 _ll_register_callbacks cb 0x557e9cd12080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f3847fff640 1 -- 172.21.15.55:0/942387642 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c0043e0 con 0x557e9ce25830 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f3847fff640 1 -- 172.21.15.55:0/942387642 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f385c011940 con 0x557e9ce25830 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3854005680 con 0x557e9ce25830 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f3847fff640 1 -- 172.21.15.55:0/942387642 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f385c011b40 con 0x557e9ce25830 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f38719d9740 10 client.15591 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:05.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.295+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=21}) -- 0x557e9ca5c170 con 0x557e9ce25830 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.296+0000 7f3847fff640 1 -- 172.21.15.55:0/942387642 <== mon.0 v2:172.21.15.55:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f385c011ce0 con 0x557e9ce25830 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.296+0000 7f38719d9740 10 client.15591 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.296+0000 7f38719d9740 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.296+0000 7f38719d9740 20 client.15591 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.296+0000 7f38719d9740 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x557e9ce4c610 0x557e9ce4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.296+0000 7f38719d9740 4 client.15591 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.296+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3854005680 con 0x557e9ce4c610 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.296+0000 7f3867fff640 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x557e9ce4c610 0x557e9ce4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.297+0000 7f3847fff640 10 client.15591 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:05.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.297+0000 7f3867fff640 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x557e9ce4c610 0x557e9ce4eaa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d024b04af0d4bcac server_cookie=4250fb629c0f283 in_seq=0 out_seq=0 2024-04-26T21:54:05.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.297+0000 7f3847fff640 1 -- 172.21.15.55:0/942387642 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3854005680 con 0x557e9ce4c610 2024-04-26T21:54:05.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.297+0000 7f3847fff640 10 client.15591 handle_command_reply: tid=0 2024-04-26T21:54:05.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.297+0000 7f38719d9740 1 client.15591 shutdown 2024-04-26T21:54:05.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce25830 msgr2=0x557e9ce2fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce25830 0x557e9ce2fc20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f385c012700 tx=0x7f385c012bc0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 shutdown_connections 2024-04-26T21:54:05.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce29380 0x557e9ce35190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce26220 0x557e9ce30160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x557e9ce4c610 0x557e9ce4eaa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 --2- 172.21.15.55:0/942387642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce25830 0x557e9ce2fc20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f3847fff640 0 client.15591 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:05.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 >> 172.21.15.55:0/942387642 conn(0x557e9ce20f20 msgr2=0x557e9ce200e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 shutdown_connections 2024-04-26T21:54:05.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 1 -- 172.21.15.55:0/942387642 wait complete. 2024-04-26T21:54:05.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.298+0000 7f38719d9740 20 client.15591 trim_cache size 0 max 16384 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f38719d9740 1 Processor -- start 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f38719d9740 1 -- start start 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f38719d9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce26220 0x557e9ce2e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f38719d9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce2f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f38719d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce2fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f38719d9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f38681aa6c0 con 0x557e9ce29380 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f38719d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f38681a7f90 con 0x557e9ce2f7b0 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f38719d9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f38681a5de0 con 0x557e9ce26220 2024-04-26T21:54:05.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f3867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce2f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f3867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce2f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49522/0 (socket says 172.21.15.55:49522) 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f3867fff640 1 -- 172.21.15.55:0/3337982190 learned_addr learned my addr 172.21.15.55:0/3337982190 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.316+0000 7f386c983640 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce26220 0x557e9ce2e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f386d184640 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce2fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f3867fff640 1 -- 172.21.15.55:0/3337982190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce26220 msgr2=0x557e9ce2e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f3867fff640 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce26220 0x557e9ce2e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f3867fff640 1 -- 172.21.15.55:0/3337982190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 msgr2=0x557e9ce2fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f3867fff640 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce2fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f3867fff640 1 -- 172.21.15.55:0/3337982190 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9ce26600 con 0x557e9ce29380 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f386d184640 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce2fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:54:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f3867fff640 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce2f270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3854097ba0 tx=0x7f3854009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa410a887f33a0c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f38527fc640 1 -- 172.21.15.55:0/3337982190 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38540a43b0 con 0x557e9ce29380 2024-04-26T21:54:05.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f38527fc640 1 -- 172.21.15.55:0/3337982190 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38540b9070 con 0x557e9ce29380 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.317+0000 7f38527fc640 1 -- 172.21.15.55:0/3337982190 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3854097db0 con 0x557e9ce29380 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 -- 172.21.15.55:0/3337982190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 msgr2=0x557e9ce2f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce2f270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3854097ba0 tx=0x7f3854009f90 comp rx=0 tx=0).stop 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 -- 172.21.15.55:0/3337982190 shutdown_connections 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce2fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce29380 0x557e9ce2f270 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 --2- 172.21.15.55:0/3337982190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ce26220 0x557e9ce2e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 -- 172.21.15.55:0/3337982190 >> 172.21.15.55:0/3337982190 conn(0x557e9ce20f20 msgr2=0x557e9ce247a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 -- 172.21.15.55:0/3337982190 shutdown_connections 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.318+0000 7f38719d9740 1 -- 172.21.15.55:0/3337982190 wait complete. 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 1 -- start start 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 1 Processor -- start 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce26220 0x557e9ce5ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce5f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ceb0740 0x557e9ceb2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f38681a6450 con 0x557e9ce26220 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f38681a78c0 con 0x557e9ce2f7b0 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f38681a6b20 con 0x557e9ceb0740 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f38719d9740 10 client.0 _ll_register_callbacks cb 0x7f385c009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f386c983640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce26220 0x557e9ce5ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f386c983640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce26220 0x557e9ce5ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49528/0 (socket says 172.21.15.55:49528) 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f386c983640 1 -- 172.21.15.55:0/1910007730 learned_addr learned my addr 172.21.15.55:0/1910007730 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.319+0000 7f3867fff640 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce5f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f386d184640 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ceb0740 0x557e9ceb2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f386c983640 1 -- 172.21.15.55:0/1910007730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ceb0740 msgr2=0x557e9ceb2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f386c983640 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ceb0740 0x557e9ceb2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f386c983640 1 -- 172.21.15.55:0/1910007730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 msgr2=0x557e9ce5f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f386c983640 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce5f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f386c983640 1 -- 172.21.15.55:0/1910007730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9ceb30f0 con 0x557e9ce26220 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f386c983640 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce26220 0x557e9ce5ef90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x557e9ce2f240 tx=0x7f385c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9910bd3bcfe675a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f3847fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f3847fff640 1 -- 172.21.15.55:0/1910007730 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c037060 con 0x557e9ce26220 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f38719d9740 1 -- 172.21.15.55:0/1910007730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38540b9070 con 0x557e9ce26220 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f3847fff640 1 -- 172.21.15.55:0/1910007730 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f385c011ce0 con 0x557e9ce26220 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f3847fff640 1 -- 172.21.15.55:0/1910007730 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f385c02d070 con 0x557e9ce26220 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f38719d9740 10 client.15597 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.320+0000 7f38719d9740 1 -- 172.21.15.55:0/1910007730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557e9ceb44a0 con 0x557e9ce26220 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.321+0000 7f3847fff640 1 -- 172.21.15.55:0/1910007730 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38540b9070 con 0x557e9ce26220 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.321+0000 7f3847fff640 1 -- 172.21.15.55:0/1910007730 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f385c03d730 con 0x557e9ce26220 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.321+0000 7f38719d9740 10 client.15597 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.321+0000 7f38719d9740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.321+0000 7f38719d9740 20 client.15597 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:05.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.321+0000 7f38719d9740 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x557e9ceb4df0 0x557e9ceb7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.321+0000 7f38719d9740 4 client.15597 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:54:05.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.321+0000 7f38719d9740 1 -- 172.21.15.55:0/1910007730 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557e9cd98710 con 0x557e9ceb4df0 2024-04-26T21:54:05.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.322+0000 7f3867fff640 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x557e9ceb4df0 0x557e9ceb7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.322+0000 7f3867fff640 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x557e9ceb4df0 0x557e9ceb7210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2a40a5234b6cd3 server_cookie=58902e0de13fdb7d in_seq=0 out_seq=0 2024-04-26T21:54:05.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.322+0000 7f3847fff640 10 client.15597 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:05.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f3847fff640 1 -- 172.21.15.55:0/1910007730 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557e9cd98710 con 0x557e9ceb4df0 2024-04-26T21:54:05.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f3847fff640 10 client.15597 handle_command_reply: tid=0 2024-04-26T21:54:05.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f38719d9740 1 client.15597 shutdown 2024-04-26T21:54:05.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f38719d9740 1 -- 172.21.15.55:0/1910007730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce26220 msgr2=0x557e9ce5ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f38719d9740 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce26220 0x557e9ce5ef90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x557e9ce2f240 tx=0x7f385c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f38719d9740 1 -- 172.21.15.55:0/1910007730 shutdown_connections 2024-04-26T21:54:05.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f38719d9740 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557e9ceb0740 0x557e9ceb2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f38719d9740 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557e9ce2f7b0 0x557e9ce5f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f38719d9740 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x557e9ceb4df0 0x557e9ceb7210 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.323+0000 7f3847fff640 0 client.15597 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.324+0000 7f38719d9740 1 --2- 172.21.15.55:0/1910007730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x557e9ce26220 0x557e9ce5ef90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.324+0000 7f38719d9740 1 -- 172.21.15.55:0/1910007730 >> 172.21.15.55:0/1910007730 conn(0x557e9ce20f20 msgr2=0x557e9ce247a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.324+0000 7f38719d9740 1 -- 172.21.15.55:0/1910007730 shutdown_connections 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.324+0000 7f38719d9740 1 -- 172.21.15.55:0/1910007730 wait complete. 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.324+0000 7f38719d9740 20 client.15597 trim_cache size 0 max 16384 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "04dd0363-337e-43d0-8295-f40c52358dd3", 2024-04-26T21:54:05.328 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:54:05.329 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38580a3c40 msgr2=0x7f38580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38580a3c40 0x7f38580a6100 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f386002f500 tx=0x7f38600095c0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868102a60 msgr2=0x7f3868199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868102a60 0x7f3868199110 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f385400d770 tx=0x7f385400dc30 comp rx=0 tx=0).stop 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 shutdown_connections 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38580a3c40 0x7f38580a6100 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3868103420 0x7f38681a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3868102a60 0x7f3868199110 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.325+0000 7f386ebe7640 1 --2- 172.21.15.55:0/352433173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3868101870 0x7f3868198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.326+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 >> 172.21.15.55:0/352433173 conn(0x7f38680fcde0 msgr2=0x7f3868105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.326+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 shutdown_connections 2024-04-26T21:54:05.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.326+0000 7f386ebe7640 1 -- 172.21.15.55:0/352433173 wait complete. 2024-04-26T21:54:05.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:54:05.341 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 -- 172.21.15.55:0/2323711847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 msgr2=0x7fc9cc10ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 --2- 172.21.15.55:0/2323711847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 0x7fc9cc10ae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009ca0 tx=0x7fc9c402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 -- 172.21.15.55:0/2323711847 shutdown_connections 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 --2- 172.21.15.55:0/2323711847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 0x7fc9cc10ae50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 --2- 172.21.15.55:0/2323711847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9cc103c00 0x7fc9cc104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 --2- 172.21.15.55:0/2323711847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9cc100c30 0x7fc9cc101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 -- 172.21.15.55:0/2323711847 >> 172.21.15.55:0/2323711847 conn(0x7fc9cc0fc560 msgr2=0x7fc9cc0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 -- 172.21.15.55:0/2323711847 shutdown_connections 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.453+0000 7fc9d1704640 1 -- 172.21.15.55:0/2323711847 wait complete. 2024-04-26T21:54:05.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9d1704640 1 Processor -- start 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9d1704640 1 -- start start 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9d1704640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9cc100c30 0x7fc9cc103570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9d1704640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9cc103c00 0x7fc9cc101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9d1704640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 0x7fc9cc1020f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9d1704640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc10e490 con 0x7fc9cc100c30 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9d1704640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc10e310 con 0x7fc9cc1045c0 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9d1704640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc10e610 con 0x7fc9cc103c00 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.454+0000 7fc9caffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9cc100c30 0x7fc9cc103570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9caffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9cc100c30 0x7fc9cc103570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49556/0 (socket says 172.21.15.55:49556) 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9caffd640 1 -- 172.21.15.55:0/611598046 learned_addr learned my addr 172.21.15.55:0/611598046 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9cc103c00 0x7fc9cc101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 0x7fc9cc1020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9caffd640 1 -- 172.21.15.55:0/611598046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9cc103c00 msgr2=0x7fc9cc101bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9caffd640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9cc103c00 0x7fc9cc101bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9caffd640 1 -- 172.21.15.55:0/611598046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 msgr2=0x7fc9cc1020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9caffd640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 0x7fc9cc1020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9caffd640 1 -- 172.21.15.55:0/611598046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc102980 con 0x7fc9cc100c30 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 0x7fc9cc1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.455+0000 7fc9caffd640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9cc100c30 0x7fc9cc103570 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00e9c0 tx=0x7fc9bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.456+0000 7fc9b3fff640 1 -- 172.21.15.55:0/611598046 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc00cd80 con 0x7fc9cc100c30 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.456+0000 7fc9b3fff640 1 -- 172.21.15.55:0/611598046 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9bc0035f0 con 0x7fc9cc100c30 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.456+0000 7fc9b3fff640 1 -- 172.21.15.55:0/611598046 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc010690 con 0x7fc9cc100c30 2024-04-26T21:54:05.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.456+0000 7fc9d1704640 1 -- 172.21.15.55:0/611598046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9cc1a2c90 con 0x7fc9cc100c30 2024-04-26T21:54:05.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.456+0000 7fc9d1704640 1 -- 172.21.15.55:0/611598046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9cc1a3180 con 0x7fc9cc100c30 2024-04-26T21:54:05.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.459+0000 7fc9d441b740 1 Processor -- start 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.459+0000 7fc9d441b740 1 -- start start 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9d2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d30b0 0x563f8e9d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 0x563f8e9da300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9d441b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563f8e936d80 con 0x563f8e9d1ee0 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9d441b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563f8e8f08b0 con 0x563f8e9d30b0 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9d441b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563f8e932d40 con 0x563f8e9d3a70 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9b3fff640 1 -- 172.21.15.55:0/611598046 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc9bc0108f0 con 0x7fc9cc100c30 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49566/0 (socket says 172.21.15.55:49566) 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/1138623376 learned_addr learned my addr 172.21.15.55:0/1138623376 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 0x563f8e9da300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9caffd640 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d30b0 0x563f8e9d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/1138623376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 msgr2=0x563f8e9da300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 0x563f8e9da300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/1138623376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d30b0 msgr2=0x563f8e9d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d30b0 0x563f8e9d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/1138623376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8e8c1a40 con 0x563f8e9d1ee0 2024-04-26T21:54:05.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.460+0000 7fc9caffd640 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d30b0 0x563f8e9d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:05.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.461+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9d2300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4005f70 tx=0x7fc9c4005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ecc68a8a0c05c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.463+0000 7fc9b3fff640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9b40a3d40 0x7fc9b40a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.463+0000 7fc9caffd640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9b40a3d40 0x7fc9b40a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.463+0000 7fc9b3fff640 1 -- 172.21.15.55:0/611598046 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc9bc0d0a90 con 0x7fc9cc100c30 2024-04-26T21:54:05.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.464+0000 7fc9caffd640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9b40a3d40 0x7fc9b40a6200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc099540 tx=0x7fc9bc0bd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.464+0000 7fc9b17fa640 1 -- 172.21.15.55:0/1138623376 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4031de0 con 0x563f8e9d1ee0 2024-04-26T21:54:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.465+0000 7fc9b17fa640 1 -- 172.21.15.55:0/1138623376 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c4036070 con 0x563f8e9d1ee0 2024-04-26T21:54:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.465+0000 7fc9b17fa640 1 -- 172.21.15.55:0/1138623376 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40043e0 con 0x563f8e9d1ee0 2024-04-26T21:54:05.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.465+0000 7fc9d441b740 1 -- 172.21.15.55:0/1138623376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 msgr2=0x563f8e9d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.465+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9d2300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4005f70 tx=0x7fc9c4005fa0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.466+0000 7fc9d441b740 1 -- 172.21.15.55:0/1138623376 shutdown_connections 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.466+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 0x563f8e9da300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.466+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d30b0 0x563f8e9d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.466+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1138623376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9d2300 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.466+0000 7fc9d441b740 1 -- 172.21.15.55:0/1138623376 >> 172.21.15.55:0/1138623376 conn(0x563f8e9cd3d0 msgr2=0x563f8e9cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.466+0000 7fc9d441b740 1 -- 172.21.15.55:0/1138623376 shutdown_connections 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.466+0000 7fc9d441b740 1 -- 172.21.15.55:0/1138623376 wait complete. 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.466+0000 7fc9d441b740 1 -- start start 2024-04-26T21:54:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9d441b740 1 Processor -- start 2024-04-26T21:54:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9dc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d3a70 0x563f8e9e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9d441b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563f8e9daa20 con 0x563f8e9d3a70 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9d441b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563f8e8bdd10 con 0x563f8e9d1ee0 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d3a70 0x563f8e9e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9ca7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d3a70 0x563f8e9e38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49578/0 (socket says 172.21.15.55:49578) 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/1223245173 learned_addr learned my addr 172.21.15.55:0/1223245173 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9ca7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9dc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:55662/0 (socket says 172.21.15.55:55662) 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.467+0000 7fc9d441b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563f8e9daba0 con 0x563f8e9d30b0 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9d441b740 10 client.0 _ll_register_callbacks cb 0x563f8e8ea210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/1223245173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 msgr2=0x563f8e9dc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9dc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/1223245173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 msgr2=0x563f8e9dc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9dc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/1223245173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8e7127d0 con 0x563f8e9d3a70 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9dc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9d441b740 1 -- 172.21.15.55:0/1223245173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c4036070 con 0x563f8e9d3a70 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d3a70 0x563f8e9e38b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009620 tx=0x7fc9c4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad333e2dc9a40080 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9aeff5640 10 client.15609 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.468+0000 7fc9aeff5640 1 -- 172.21.15.55:0/1223245173 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4049040 con 0x563f8e9d3a70 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.469+0000 7fc9aeff5640 1 -- 172.21.15.55:0/1223245173 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc9c4004d40 con 0x563f8e9d3a70 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.469+0000 7fc9aeff5640 1 -- 172.21.15.55:0/1223245173 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc9c4004180 con 0x563f8e9d3a70 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.469+0000 7fc9d441b740 10 client.15609 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.469+0000 7fc9d441b740 1 -- 172.21.15.55:0/1223245173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=21}) -- 0x563f8e6483f0 con 0x563f8e9d3a70 2024-04-26T21:54:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.469+0000 7fc9aeff5640 1 -- 172.21.15.55:0/1223245173 <== mon.0 v2:172.21.15.55:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc9c404f070 con 0x563f8e9d3a70 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.469+0000 7fc9d441b740 10 client.15609 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.469+0000 7fc9d441b740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.469+0000 7fc9d441b740 20 client.15609 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.470+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563f8e9e77c0 0x563f8e9e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.470+0000 7fc9d441b740 4 client.15609 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.470+0000 7fc9d441b740 1 -- 172.21.15.55:0/1223245173 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f8e944990 con 0x563f8e9e77c0 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.470+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563f8e9e77c0 0x563f8e9e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.470+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563f8e9e77c0 0x563f8e9e9bb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d946d56c112402 server_cookie=618503d34169c044 in_seq=0 out_seq=0 2024-04-26T21:54:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.470+0000 7fc9aeff5640 10 client.15609 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:05.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9aeff5640 1 -- 172.21.15.55:0/1223245173 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f8e944990 con 0x563f8e9e77c0 2024-04-26T21:54:05.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9aeff5640 10 client.15609 handle_command_reply: tid=0 2024-04-26T21:54:05.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 client.15609 shutdown 2024-04-26T21:54:05.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 -- 172.21.15.55:0/1223245173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d3a70 msgr2=0x563f8e9e38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d3a70 0x563f8e9e38b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009620 tx=0x7fc9c4005d80 comp rx=0 tx=0).stop 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 -- 172.21.15.55:0/1223245173 shutdown_connections 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563f8e9e77c0 0x563f8e9e9bb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d3a70 0x563f8e9e38b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9dc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 --2- 172.21.15.55:0/1223245173 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9dc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9d441b740 1 -- 172.21.15.55:0/1223245173 >> 172.21.15.55:0/1223245173 conn(0x563f8e9cd3d0 msgr2=0x563f8e9cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.471+0000 7fc9aeff5640 0 client.15609 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.472+0000 7fc9d441b740 1 -- 172.21.15.55:0/1223245173 shutdown_connections 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.472+0000 7fc9d441b740 1 -- 172.21.15.55:0/1223245173 wait complete. 2024-04-26T21:54:05.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.472+0000 7fc9d441b740 20 client.15609 trim_cache size 0 max 16384 2024-04-26T21:54:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.507+0000 7fc9d441b740 1 Processor -- start 2024-04-26T21:54:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.507+0000 7fc9d441b740 1 -- start start 2024-04-26T21:54:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.507+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.507+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9e3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.507+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d3a70 0x563f8e9ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.507+0000 7fc9d441b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1b2eb0 con 0x563f8e9d1ee0 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.507+0000 7fc9d441b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1b0780 con 0x563f8e9d3a70 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.507+0000 7fc9d441b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1ae5d0 con 0x563f8e9d30b0 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49588/0 (socket says 172.21.15.55:49588) 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/3295425819 learned_addr learned my addr 172.21.15.55:0/3295425819 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/3295425819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/3295425819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 msgr2=0x563f8e9e3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/3295425819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9e3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/3295425819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d3a70 msgr2=0x563f8e9ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/3295425819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d3a70 0x563f8e9ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 -- 172.21.15.55:0/3295425819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8e6080f0 con 0x563f8e9d1ee0 2024-04-26T21:54:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/3295425819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9cc3d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc9c402c010 tx=0x7fc9c404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17c0e2b291ecc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.508+0000 7fc9b0ff9640 1 -- 172.21.15.55:0/3295425819 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4031450 con 0x563f8e9d1ee0 2024-04-26T21:54:05.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.509+0000 7fc9b0ff9640 1 -- 172.21.15.55:0/3295425819 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c4036070 con 0x563f8e9d1ee0 2024-04-26T21:54:05.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.509+0000 7fc9b0ff9640 1 -- 172.21.15.55:0/3295425819 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c40305c0 con 0x563f8e9d1ee0 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.509+0000 7fc9d441b740 1 -- 172.21.15.55:0/3295425819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 msgr2=0x563f8e9cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.509+0000 7fc9d441b740 1 --2- 172.21.15.55:0/3295425819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9cc3d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc9c402c010 tx=0x7fc9c404f000 comp rx=0 tx=0).stop 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 -- 172.21.15.55:0/3295425819 shutdown_connections 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 --2- 172.21.15.55:0/3295425819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d3a70 0x563f8e9ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 --2- 172.21.15.55:0/3295425819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d30b0 0x563f8e9e3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 --2- 172.21.15.55:0/3295425819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d1ee0 0x563f8e9cc3d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 -- 172.21.15.55:0/3295425819 >> 172.21.15.55:0/3295425819 conn(0x563f8e9cd3d0 msgr2=0x563f8e9cbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 -- 172.21.15.55:0/3295425819 shutdown_connections 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 -- 172.21.15.55:0/3295425819 wait complete. 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 -- start start 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 Processor -- start 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.510+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d30b0 0x563f8e9e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9d441b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 0x563f8e9e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9d441b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1aec40 con 0x563f8e9d30b0 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9d441b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1b00b0 con 0x563f8e9d1ee0 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9d441b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc1af310 con 0x563f8e9d3a70 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9d441b740 10 client.0 _ll_register_callbacks cb 0x7fc9c4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9ca7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d30b0 0x563f8e9e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9ca7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d30b0 0x563f8e9e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49596/0 (socket says 172.21.15.55:49596) 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9ca7fc640 1 -- 172.21.15.55:0/957968831 learned_addr learned my addr 172.21.15.55:0/957968831 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 0x563f8e9e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9ca7fc640 1 -- 172.21.15.55:0/957968831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 msgr2=0x563f8e9e21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 0x563f8e9e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9ca7fc640 1 -- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 msgr2=0x563f8e9e0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9e0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9ca7fc640 1 -- 172.21.15.55:0/957968831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8e8f74b0 con 0x563f8e9d30b0 2024-04-26T21:54:05.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.511+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9e0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:05.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.512+0000 7fc9ca7fc640 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d30b0 0x563f8e9e11e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc1032e0 tx=0x7fc9c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2fa0f9f927da40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:05.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.512+0000 7fc9aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:05.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.512+0000 7fc9aeff5640 1 -- 172.21.15.55:0/957968831 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c001d790 con 0x563f8e9d30b0 2024-04-26T21:54:05.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.512+0000 7fc9aeff5640 1 -- 172.21.15.55:0/957968831 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc9c0004020 con 0x563f8e9d30b0 2024-04-26T21:54:05.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.513+0000 7fc9d441b740 1 -- 172.21.15.55:0/957968831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c4036070 con 0x563f8e9d30b0 2024-04-26T21:54:05.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.513+0000 7fc9aeff5640 1 -- 172.21.15.55:0/957968831 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc9c001d930 con 0x563f8e9d30b0 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.513+0000 7fc9d441b740 10 client.15615 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.513+0000 7fc9d441b740 1 -- 172.21.15.55:0/957968831 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f8e9e3a20 con 0x563f8e9d30b0 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.513+0000 7fc9aeff5640 1 -- 172.21.15.55:0/957968831 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c0004220 con 0x563f8e9d30b0 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.513+0000 7fc9aeff5640 1 -- 172.21.15.55:0/957968831 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fc9c002f420 con 0x563f8e9d30b0 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9d441b740 10 client.15615 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9d441b740 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9d441b740 20 client.15615 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9d441b740 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563f8ea4e1d0 0x563f8ea505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9d441b740 4 client.15615 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9d441b740 1 -- 172.21.15.55:0/957968831 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc9c4036070 con 0x563f8ea4e1d0 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563f8ea4e1d0 0x563f8ea505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9cb7fe640 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563f8ea4e1d0 0x563f8ea505f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c347177ed395dedc server_cookie=1fa1e4b4fcd51d6b in_seq=0 out_seq=0 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.514+0000 7fc9aeff5640 10 client.15615 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9aeff5640 1 -- 172.21.15.55:0/957968831 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fc9c4036070 con 0x563f8ea4e1d0 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9aeff5640 10 client.15615 handle_command_reply: tid=0 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 client.15615 shutdown 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 -- 172.21.15.55:0/957968831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d30b0 msgr2=0x563f8e9e11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d30b0 0x563f8e9e11e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc1032e0 tx=0x7fc9c0011810 comp rx=0 tx=0).stop 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 -- 172.21.15.55:0/957968831 shutdown_connections 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f8e9d3a70 0x563f8e9e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563f8ea4e1d0 0x563f8ea505f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f8e9d30b0 0x563f8e9e11e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 --2- 172.21.15.55:0/957968831 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f8e9d1ee0 0x563f8e9e0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 -- 172.21.15.55:0/957968831 >> 172.21.15.55:0/957968831 conn(0x563f8e9cd3d0 msgr2=0x563f8e9d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9aeff5640 0 client.15615 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 -- 172.21.15.55:0/957968831 shutdown_connections 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 1 -- 172.21.15.55:0/957968831 wait complete. 2024-04-26T21:54:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.515+0000 7fc9d441b740 20 client.15615 trim_cache size 0 max 16384 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout: "04dd0363-337e-43d0-8295-f40c52358dd3": { 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "04dd0363-337e-43d0-8295-f40c52358dd3", 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:54:05.519 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:54:05.520 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:54:05.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 -- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9b40a3d40 msgr2=0x7fc9b40a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9b40a3d40 0x7fc9b40a6200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc099540 tx=0x7fc9bc0bd040 comp rx=0 tx=0).stop 2024-04-26T21:54:05.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 -- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9cc100c30 msgr2=0x7fc9cc103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:05.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9cc100c30 0x7fc9cc103570 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00e9c0 tx=0x7fc9bc00ee80 comp rx=0 tx=0).stop 2024-04-26T21:54:05.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 -- 172.21.15.55:0/611598046 shutdown_connections 2024-04-26T21:54:05.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc9b40a3d40 0x7fc9b40a6200 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc9cc1045c0 0x7fc9cc1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9cc103c00 0x7fc9cc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 --2- 172.21.15.55:0/611598046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9cc100c30 0x7fc9cc103570 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:05.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 -- 172.21.15.55:0/611598046 >> 172.21.15.55:0/611598046 conn(0x7fc9cc0fc560 msgr2=0x7fc9cc106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:05.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 -- 172.21.15.55:0/611598046 shutdown_connections 2024-04-26T21:54:05.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:05.516+0000 7fc9d1704640 1 -- 172.21.15.55:0/611598046 wait complete. 2024-04-26T21:54:05.533 INFO:tasks.cephfs.filesystem:scrub status for tag:04dd0363-337e-43d0-8295-f40c52358dd3 - {'path': '/', 'tag': '04dd0363-337e-43d0-8295-f40c52358dd3', 'options': 'recursive,force'} 2024-04-26T21:54:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:06 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:54:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:06 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:06 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:54:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:06 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:06 smithi064 ceph-mon[32765]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:06 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:54:06.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:06 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:54:06.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:06 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:06.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:06 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:54:06.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:06 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:06.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:06 smithi165 ceph-mon[33296]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:06.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:06 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:54:06.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:06 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:54:06.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:06 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:06.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:06 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:54:06.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:06 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:06.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:06 smithi055 ceph-mon[29221]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:06.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:06 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:54:08.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:08 smithi064 ceph-mon[32765]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:08.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:08 smithi165 ceph-mon[33296]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:08.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:08 smithi055 ceph-mon[29221]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:10.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:10 smithi064 ceph-mon[32765]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:10.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:10 smithi165 ceph-mon[33296]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:10.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:10 smithi055 ceph-mon[29221]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:12.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:12 smithi064 ceph-mon[32765]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:54:12.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:12 smithi165 ceph-mon[33296]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:54:12.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:12 smithi055 ceph-mon[29221]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:54:14.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:14 smithi064 ceph-mon[32765]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T21:54:14.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:14 smithi165 ceph-mon[33296]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T21:54:14.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:14 smithi055 ceph-mon[29221]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-26T21:54:16.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:16 smithi165 ceph-mon[33296]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:16.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:16 smithi055 ceph-mon[29221]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:17.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:16 smithi064 ceph-mon[32765]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:17.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:17 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:17.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:17 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:17 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:18.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:18 smithi055 ceph-mon[29221]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:54:18.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:18.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:18 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:18 smithi064 ceph-mon[32765]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:54:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:18 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:18 smithi165 ceph-mon[33296]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-26T21:54:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:18 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:54:20.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:20 smithi055 ceph-mon[29221]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:21.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:20 smithi064 ceph-mon[32765]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:21.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:20 smithi165 ceph-mon[33296]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-26T21:54:22.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:22 smithi055 ceph-mon[29221]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:23.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:22 smithi064 ceph-mon[32765]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:23.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:22 smithi165 ceph-mon[33296]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-26T21:54:23.640 INFO:tasks.workunit.client.0.smithi055.stderr:Updating files: 93% (11451/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-26T21:54:24.418 INFO:tasks.workunit.client.0.smithi055.stderr:Note: switching to 'bc612ffb9c1d5bc27536bc917a705262a36d4387'. 2024-04-26T21:54:24.418 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:24.418 INFO:tasks.workunit.client.0.smithi055.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T21:54:24.418 INFO:tasks.workunit.client.0.smithi055.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr: git switch -c 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr:Or undo this operation with: 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr: git switch - 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:24.419 INFO:tasks.workunit.client.0.smithi055.stderr:HEAD is now at bc612ffb9c1 Merge PR #55144 into wip-rishabh-testing-20240426.111959 2024-04-26T21:54:24.423 DEBUG:teuthology.orchestra.run.smithi055:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T21:54:24.463 INFO:tasks.workunit.client.0.smithi055.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T21:54:24.465 INFO:tasks.workunit.client.0.smithi055.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T21:54:24.465 INFO:tasks.workunit.client.0.smithi055.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T21:54:24.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:24 smithi055 ceph-mon[29221]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:54:25.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:24 smithi064 ceph-mon[32765]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:54:25.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:24 smithi165 ceph-mon[33296]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T21:54:25.085 INFO:tasks.workunit.client.0.smithi055.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T21:54:25.154 INFO:tasks.workunit.client.0.smithi055.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T21:54:25.194 INFO:tasks.workunit.client.0.smithi055.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T21:54:25.195 INFO:tasks.workunit.client.0.smithi055.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T21:54:25.196 INFO:tasks.workunit.client.0.smithi055.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T21:54:25.238 INFO:tasks.workunit.client.0.smithi055.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T21:54:25.239 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T21:54:25.239 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T21:54:25.297 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-04-26T21:54:25.299 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-04-26T21:54:25.300 DEBUG:teuthology.orchestra.run.smithi055:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc612ffb9c1d5bc27536bc917a705262a36d4387 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-04-26T21:54:25.360 INFO:tasks.workunit.client.0.smithi055.stdout:getting iogen 2024-04-26T21:54:25.360 INFO:tasks.workunit.client.0.smithi055.stderr:+ echo 'getting iogen' 2024-04-26T21:54:25.361 INFO:tasks.workunit.client.0.smithi055.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-04-26T21:54:25.389 INFO:tasks.workunit.client.0.smithi055.stderr:--2024-04-26 21:54:25-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-04-26T21:54:25.389 INFO:tasks.workunit.client.0.smithi055.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-26T21:54:25.410 INFO:tasks.workunit.client.0.smithi055.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-26T21:54:25.431 INFO:tasks.workunit.client.0.smithi055.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-26T21:54:25.431 INFO:tasks.workunit.client.0.smithi055.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-04-26T21:54:25.432 INFO:tasks.workunit.client.0.smithi055.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-04-26T21:54:25.435 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:25.435 INFO:tasks.workunit.client.0.smithi055.stderr: 0K ....... 100% 1.26G=0s 2024-04-26T21:54:25.435 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:25.435 INFO:tasks.workunit.client.0.smithi055.stderr:2024-04-26 21:54:25 (1.26 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-04-26T21:54:25.435 INFO:tasks.workunit.client.0.smithi055.stderr: 2024-04-26T21:54:25.435 INFO:tasks.workunit.client.0.smithi055.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-04-26T21:54:25.450 INFO:tasks.workunit.client.0.smithi055.stdout:iogen_3.1p0 2024-04-26T21:54:25.451 INFO:tasks.workunit.client.0.smithi055.stdout:iogen_3.1p0/src 2024-04-26T21:54:25.453 INFO:tasks.workunit.client.0.smithi055.stdout:iogen_3.1p0/src/iogen.8 2024-04-26T21:54:25.454 INFO:tasks.workunit.client.0.smithi055.stdout:iogen_3.1p0/src/iogen.c 2024-04-26T21:54:25.456 INFO:tasks.workunit.client.0.smithi055.stdout:iogen_3.1p0/README 2024-04-26T21:54:25.458 INFO:tasks.workunit.client.0.smithi055.stdout:iogen_3.1p0/Makefile.bsd 2024-04-26T21:54:25.460 INFO:tasks.workunit.client.0.smithi055.stdout:iogen_3.1p0/Makefile 2024-04-26T21:54:25.462 INFO:tasks.workunit.client.0.smithi055.stderr:+ cd iogen_3.1p0 2024-04-26T21:54:25.462 INFO:tasks.workunit.client.0.smithi055.stdout:making iogen 2024-04-26T21:54:25.462 INFO:tasks.workunit.client.0.smithi055.stderr:+ echo 'making iogen' 2024-04-26T21:54:25.462 INFO:tasks.workunit.client.0.smithi055.stderr:+ make 2024-04-26T21:54:25.471 INFO:tasks.workunit.client.0.smithi055.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-04-26T21:54:25.714 INFO:tasks.workunit.client.0.smithi055.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-04-26T21:54:25.792 INFO:tasks.workunit.client.0.smithi055.stdout:running iogen 2024-04-26T21:54:25.794 INFO:tasks.workunit.client.0.smithi055.stderr:+ echo 'running iogen' 2024-04-26T21:54:25.794 INFO:tasks.workunit.client.0.smithi055.stderr:+ ./iogen -n 5 -s 2g 2024-04-26T21:54:25.794 INFO:tasks.workunit.client.0.smithi055.stderr:child spawned 81212 2024-04-26T21:54:25.794 INFO:tasks.workunit.client.0.smithi055.stderr:child spawned 81213 2024-04-26T21:54:25.794 INFO:tasks.workunit.client.0.smithi055.stderr:child spawned 81214 2024-04-26T21:54:25.794 INFO:tasks.workunit.client.0.smithi055.stderr:child spawned 81215 2024-04-26T21:54:25.795 INFO:tasks.workunit.client.0.smithi055.stderr:child spawned 81216 2024-04-26T21:54:25.795 INFO:tasks.workunit.client.0.smithi055.stdout:sleep for 10 min 2024-04-26T21:54:25.795 INFO:tasks.workunit.client.0.smithi055.stderr:+ echo 'sleep for 10 min' 2024-04-26T21:54:25.795 INFO:tasks.workunit.client.0.smithi055.stderr:+ sleep 600 2024-04-26T21:54:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:26 smithi064 ceph-mon[32765]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:27.079 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:26 smithi165 ceph-mon[33296]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:27.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:26 smithi055 ceph-mon[29221]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T21:54:29.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:28 smithi064 ceph-mon[32765]: pgmap v398: 97 pgs: 97 active+clean; 73 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-04-26T21:54:29.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:28 smithi165 ceph-mon[33296]: pgmap v398: 97 pgs: 97 active+clean; 73 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-04-26T21:54:29.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:28 smithi055 ceph-mon[29221]: pgmap v398: 97 pgs: 97 active+clean; 73 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6.0 MiB/s wr, 7 op/s 2024-04-26T21:54:31.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:30 smithi064 ceph-mon[32765]: pgmap v399: 97 pgs: 97 active+clean; 999 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 83 MiB/s wr, 96 op/s 2024-04-26T21:54:31.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:30 smithi165 ceph-mon[33296]: pgmap v399: 97 pgs: 97 active+clean; 999 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 83 MiB/s wr, 96 op/s 2024-04-26T21:54:31.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:30 smithi055 ceph-mon[29221]: pgmap v399: 97 pgs: 97 active+clean; 999 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 83 MiB/s wr, 96 op/s 2024-04-26T21:54:33.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:32 smithi064 ceph-mon[32765]: pgmap v400: 97 pgs: 97 active+clean; 2.4 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 207 MiB/s wr, 127 op/s 2024-04-26T21:54:33.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:32 smithi165 ceph-mon[33296]: pgmap v400: 97 pgs: 97 active+clean; 2.4 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 207 MiB/s wr, 127 op/s 2024-04-26T21:54:33.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:32 smithi055 ceph-mon[29221]: pgmap v400: 97 pgs: 97 active+clean; 2.4 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 207 MiB/s wr, 127 op/s 2024-04-26T21:54:35.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:34 smithi064 ceph-mon[32765]: pgmap v401: 97 pgs: 97 active+clean; 3.4 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 287 MiB/s wr, 287 op/s 2024-04-26T21:54:35.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:34 smithi165 ceph-mon[33296]: pgmap v401: 97 pgs: 97 active+clean; 3.4 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 287 MiB/s wr, 287 op/s 2024-04-26T21:54:35.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:34 smithi055 ceph-mon[29221]: pgmap v401: 97 pgs: 97 active+clean; 3.4 GiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 287 MiB/s wr, 287 op/s 2024-04-26T21:54:35.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:54:35.535 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:54:35.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.826+0000 7f702515f640 1 -- 172.21.15.55:0/3275478051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 msgr2=0x7f70201057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.826+0000 7f702515f640 1 --2- 172.21.15.55:0/3275478051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 0x7f70201057d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7010009900 tx=0x7f701002a410 comp rx=0 tx=0).stop 2024-04-26T21:54:35.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.827+0000 7f702515f640 1 -- 172.21.15.55:0/3275478051 shutdown_connections 2024-04-26T21:54:35.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.827+0000 7f702515f640 1 --2- 172.21.15.55:0/3275478051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7020108640 0x7f702010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.827+0000 7f702515f640 1 --2- 172.21.15.55:0/3275478051 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7020105d10 0x7f7020108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.827+0000 7f702515f640 1 --2- 172.21.15.55:0/3275478051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 0x7f70201057d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.827+0000 7f702515f640 1 -- 172.21.15.55:0/3275478051 >> 172.21.15.55:0/3275478051 conn(0x7f70200fcdd0 msgr2=0x7f70200ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:35.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.827+0000 7f702515f640 1 -- 172.21.15.55:0/3275478051 shutdown_connections 2024-04-26T21:54:35.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.827+0000 7f702515f640 1 -- 172.21.15.55:0/3275478051 wait complete. 2024-04-26T21:54:35.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.827+0000 7f702515f640 1 Processor -- start 2024-04-26T21:54:35.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f702515f640 1 -- start start 2024-04-26T21:54:35.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f702515f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 0x7f7020198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f702515f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7020105d10 0x7f70201992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f702515f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7020108640 0x7f70201a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f702515f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f702010e5d0 con 0x7f70201033e0 2024-04-26T21:54:35.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f702515f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f702010e450 con 0x7f7020105d10 2024-04-26T21:54:35.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f702515f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f702010e750 con 0x7f7020108640 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f701ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 0x7f7020198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f701ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 0x7f7020198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58512/0 (socket says 172.21.15.55:58512) 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.828+0000 7f701ed9d640 1 -- 172.21.15.55:0/3773034145 learned_addr learned my addr 172.21.15.55:0/3773034145 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f701f59e640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7020108640 0x7f70201a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f701ed9d640 1 -- 172.21.15.55:0/3773034145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7020108640 msgr2=0x7f70201a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f701ed9d640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7020108640 0x7f70201a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f701ed9d640 1 -- 172.21.15.55:0/3773034145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7020105d10 msgr2=0x7f70201992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f701ed9d640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7020105d10 0x7f70201992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f701ed9d640 1 -- 172.21.15.55:0/3773034145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70201a0950 con 0x7f70201033e0 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f701f59e640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7020108640 0x7f70201a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f701ed9d640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 0x7f7020198d90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7010031500 tx=0x7f7010003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f7007fff640 1 -- 172.21.15.55:0/3773034145 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701003f070 con 0x7f70201033e0 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f7007fff640 1 -- 172.21.15.55:0/3773034145 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701003b460 con 0x7f70201033e0 2024-04-26T21:54:35.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f702515f640 1 -- 172.21.15.55:0/3773034145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70201a0be0 con 0x7f70201033e0 2024-04-26T21:54:35.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f7007fff640 1 -- 172.21.15.55:0/3773034145 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010035030 con 0x7f70201033e0 2024-04-26T21:54:35.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.829+0000 7f702515f640 1 -- 172.21.15.55:0/3773034145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70201a1120 con 0x7f70201033e0 2024-04-26T21:54:35.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.833+0000 7f7007fff640 1 -- 172.21.15.55:0/3773034145 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7010042070 con 0x7f70201033e0 2024-04-26T21:54:35.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.834+0000 7f7007fff640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70080a3cc0 0x7f70080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.834+0000 7f7007fff640 1 -- 172.21.15.55:0/3773034145 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f70100f1040 con 0x7f70201033e0 2024-04-26T21:54:35.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.835+0000 7f701e59c640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70080a3cc0 0x7f70080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.834+0000 7f7027e75740 1 Processor -- start 2024-04-26T21:54:35.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.836+0000 7f701e59c640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70080a3cc0 0x7f70080a6180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f702006ba90 tx=0x7f7014006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.836+0000 7f7027e75740 1 -- start start 2024-04-26T21:54:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f7027e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494dee0 0x564d9494e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f7027e75740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 0x564d9494f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f7027e75740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d94956300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f701f59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d94956300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f701f59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d94956300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58528/0 (socket says 172.21.15.55:58528) 2024-04-26T21:54:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f701f59e640 1 -- 172.21.15.55:0/3711140308 learned_addr learned my addr 172.21.15.55:0/3711140308 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f7027e75740 1 -- 172.21.15.55:0/3711140308 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564d948b2d80 con 0x564d9494fa70 2024-04-26T21:54:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f7027e75740 1 -- 172.21.15.55:0/3711140308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564d9486c8b0 con 0x564d9494f0b0 2024-04-26T21:54:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.837+0000 7f7027e75740 1 -- 172.21.15.55:0/3711140308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564d948aed40 con 0x564d9494dee0 2024-04-26T21:54:35.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.838+0000 7f701f59e640 1 -- 172.21.15.55:0/3711140308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494dee0 msgr2=0x564d9494e300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:54:35.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.838+0000 7f701f59e640 1 --2- 172.21.15.55:0/3711140308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494dee0 0x564d9494e300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.838+0000 7f701f59e640 1 -- 172.21.15.55:0/3711140308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 msgr2=0x564d9494f530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:54:35.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.838+0000 7f701f59e640 1 --2- 172.21.15.55:0/3711140308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 0x564d9494f530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.838+0000 7f701f59e640 1 -- 172.21.15.55:0/3711140308 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9483da40 con 0x564d9494fa70 2024-04-26T21:54:35.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.839+0000 7f701f59e640 1 --2- 172.21.15.55:0/3711140308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d94956300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7018007f30 tx=0x7f701800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58e257458448aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:35.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.840+0000 7f70057fa640 1 -- 172.21.15.55:0/3711140308 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018014070 con 0x564d9494fa70 2024-04-26T21:54:35.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.840+0000 7f70057fa640 1 -- 172.21.15.55:0/3711140308 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701800fd00 con 0x564d9494fa70 2024-04-26T21:54:35.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.840+0000 7f70057fa640 1 -- 172.21.15.55:0/3711140308 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701800ed30 con 0x564d9494fa70 2024-04-26T21:54:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.845+0000 7f7027e75740 1 -- 172.21.15.55:0/3711140308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 msgr2=0x564d94956300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.845+0000 7f7027e75740 1 --2- 172.21.15.55:0/3711140308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d94956300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7018007f30 tx=0x7f701800c750 comp rx=0 tx=0).stop 2024-04-26T21:54:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.845+0000 7f7027e75740 1 -- 172.21.15.55:0/3711140308 shutdown_connections 2024-04-26T21:54:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.845+0000 7f7027e75740 1 --2- 172.21.15.55:0/3711140308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d94956300 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.845+0000 7f7027e75740 1 --2- 172.21.15.55:0/3711140308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 0x564d9494f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.845+0000 7f7027e75740 1 --2- 172.21.15.55:0/3711140308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494dee0 0x564d9494e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.845+0000 7f7027e75740 1 -- 172.21.15.55:0/3711140308 >> 172.21.15.55:0/3711140308 conn(0x564d949493d0 msgr2=0x564d949497d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:35.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.852+0000 7f7027e75740 1 -- 172.21.15.55:0/3711140308 shutdown_connections 2024-04-26T21:54:35.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.852+0000 7f7027e75740 1 -- 172.21.15.55:0/3711140308 wait complete. 2024-04-26T21:54:35.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.853+0000 7f7027e75740 1 -- start start 2024-04-26T21:54:35.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.853+0000 7f7027e75740 1 Processor -- start 2024-04-26T21:54:35.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.854+0000 7f7027e75740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94960c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.854+0000 7f7027e75740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 0x564d949611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.854+0000 7f7027e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494fa70 0x564d949681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.854+0000 7f7027e75740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564d94956a80 con 0x564d9494dee0 2024-04-26T21:54:35.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.854+0000 7f7027e75740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564d94839d10 con 0x564d9494f0b0 2024-04-26T21:54:35.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.854+0000 7f7027e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564d94956c00 con 0x564d9494fa70 2024-04-26T21:54:35.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.854+0000 7f7027e75740 10 client.0 _ll_register_callbacks cb 0x564d94866210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:35.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.855+0000 7f701e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 0x564d949611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.855+0000 7f701f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494fa70 0x564d949681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.855+0000 7f701f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494fa70 0x564d949681f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48322/0 (socket says 172.21.15.55:48322) 2024-04-26T21:54:35.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.856+0000 7f701f59e640 1 -- 172.21.15.55:0/3968716421 learned_addr learned my addr 172.21.15.55:0/3968716421 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:35.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.859+0000 7f701e59c640 1 -- 172.21.15.55:0/3968716421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494fa70 msgr2=0x564d949681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:35.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.859+0000 7f701e59c640 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494fa70 0x564d949681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.859+0000 7f701e59c640 1 -- 172.21.15.55:0/3968716421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 msgr2=0x564d94960c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:35.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.859+0000 7f701e59c640 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94960c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.859+0000 7f701e59c640 1 -- 172.21.15.55:0/3968716421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9468e7d0 con 0x564d9494f0b0 2024-04-26T21:54:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.872+0000 7f701e59c640 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 0x564d949611b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7014009fd0 tx=0x7f7014004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f73a1f08366928 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.872+0000 7f7002ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:54:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.873+0000 7f7002ff5640 1 -- 172.21.15.55:0/3968716421 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014023060 con 0x564d9494f0b0 2024-04-26T21:54:35.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.873+0000 7f7002ff5640 1 -- 172.21.15.55:0/3968716421 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f70140150d0 con 0x564d9494f0b0 2024-04-26T21:54:35.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.873+0000 7f7027e75740 1 -- 172.21.15.55:0/3968716421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f701800fd00 con 0x564d9494f0b0 2024-04-26T21:54:35.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.876+0000 7f7002ff5640 1 -- 172.21.15.55:0/3968716421 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f701401e410 con 0x564d9494f0b0 2024-04-26T21:54:35.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.878+0000 7f7027e75740 10 client.25265 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:35.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.878+0000 7f7027e75740 1 -- 172.21.15.55:0/3968716421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564d945c43f0 con 0x564d9494f0b0 2024-04-26T21:54:35.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.885+0000 7f7002ff5640 1 -- 172.21.15.55:0/3968716421 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701800fd00 con 0x564d9494f0b0 2024-04-26T21:54:35.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.886+0000 7f7002ff5640 1 -- 172.21.15.55:0/3968716421 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f7014012070 con 0x564d9494f0b0 2024-04-26T21:54:35.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.886+0000 7f7027e75740 10 client.25265 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:35.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.886+0000 7f7027e75740 10 client.25265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:35.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.886+0000 7f7027e75740 20 client.25265 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:35.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.886+0000 7f7027e75740 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564d9497d640 0x564d9497fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.886+0000 7f7027e75740 4 client.25265 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:35.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.886+0000 7f7027e75740 1 -- 172.21.15.55:0/3968716421 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d948c0990 con 0x564d9497d640 2024-04-26T21:54:35.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.894+0000 7f701ed9d640 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564d9497d640 0x564d9497fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.912+0000 7f701ed9d640 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564d9497d640 0x564d9497fad0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2fa8788efc90d61 server_cookie=c451898a78f6fb66 in_seq=0 out_seq=0 2024-04-26T21:54:35.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.912+0000 7f7002ff5640 10 client.25265 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:35.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.920+0000 7f7002ff5640 1 -- 172.21.15.55:0/3968716421 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d948c0990 con 0x564d9497d640 2024-04-26T21:54:35.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.920+0000 7f7002ff5640 10 client.25265 handle_command_reply: tid=0 2024-04-26T21:54:35.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.920+0000 7f7027e75740 1 client.25265 shutdown 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.921+0000 7f7027e75740 1 -- 172.21.15.55:0/3968716421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 msgr2=0x564d949611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.921+0000 7f7027e75740 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 0x564d949611b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7014009fd0 tx=0x7f7014004750 comp rx=0 tx=0).stop 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.922+0000 7f7027e75740 1 -- 172.21.15.55:0/3968716421 shutdown_connections 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.922+0000 7f7027e75740 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494fa70 0x564d949681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.922+0000 7f7027e75740 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494f0b0 0x564d949611b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.922+0000 7f7027e75740 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564d9497d640 0x564d9497fad0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.922+0000 7f7027e75740 1 --2- 172.21.15.55:0/3968716421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94960c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.922+0000 7f7027e75740 1 -- 172.21.15.55:0/3968716421 >> 172.21.15.55:0/3968716421 conn(0x564d949493d0 msgr2=0x564d94948550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.922+0000 7f7002ff5640 0 client.25265 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:35.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.925+0000 7f7027e75740 1 -- 172.21.15.55:0/3968716421 shutdown_connections 2024-04-26T21:54:35.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.929+0000 7f7027e75740 1 -- 172.21.15.55:0/3968716421 wait complete. 2024-04-26T21:54:35.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.930+0000 7f7027e75740 20 client.25265 trim_cache size 0 max 16384 2024-04-26T21:54:35.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.992+0000 7f7027e75740 1 Processor -- start 2024-04-26T21:54:35.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.992+0000 7f7027e75740 1 -- start start 2024-04-26T21:54:35.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.993+0000 7f7027e75740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494dee0 0x564d94984390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.994+0000 7f7027e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 0x564d949681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.994+0000 7f7027e75740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d949848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:35.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.994+0000 7f7027e75740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f70201a1f80 con 0x564d9494fa70 2024-04-26T21:54:35.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.994+0000 7f7027e75740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7020073060 con 0x564d9494dee0 2024-04-26T21:54:35.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.994+0000 7f7027e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7020070eb0 con 0x564d9494f0b0 2024-04-26T21:54:35.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.996+0000 7f701ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494dee0 0x564d94984390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.996+0000 7f701f59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d949848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:35.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.996+0000 7f701f59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d949848d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58552/0 (socket says 172.21.15.55:58552) 2024-04-26T21:54:35.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.996+0000 7f701f59e640 1 -- 172.21.15.55:0/1409488658 learned_addr learned my addr 172.21.15.55:0/1409488658 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:35.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.997+0000 7f701e59c640 1 --2- 172.21.15.55:0/1409488658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 0x564d949681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.998+0000 7f701f59e640 1 -- 172.21.15.55:0/1409488658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 msgr2=0x564d949681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:36.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.998+0000 7f701f59e640 1 --2- 172.21.15.55:0/1409488658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 0x564d949681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.998+0000 7f701f59e640 1 -- 172.21.15.55:0/1409488658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494dee0 msgr2=0x564d94984390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:36.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.998+0000 7f701f59e640 1 --2- 172.21.15.55:0/1409488658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494dee0 0x564d94984390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:35.998+0000 7f701f59e640 1 -- 172.21.15.55:0/1409488658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d945840f0 con 0x564d9494fa70 2024-04-26T21:54:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.000+0000 7f701f59e640 1 --2- 172.21.15.55:0/1409488658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d949848d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7018007920 tx=0x7f701800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcf621dd12a2624 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.002+0000 7f7004ff9640 1 -- 172.21.15.55:0/1409488658 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018014040 con 0x564d9494fa70 2024-04-26T21:54:36.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.002+0000 7f7004ff9640 1 -- 172.21.15.55:0/1409488658 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7018028070 con 0x564d9494fa70 2024-04-26T21:54:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.004+0000 7f7027e75740 1 -- 172.21.15.55:0/1409488658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 msgr2=0x564d949848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.004+0000 7f7027e75740 1 --2- 172.21.15.55:0/1409488658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d949848d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7018007920 tx=0x7f701800cf20 comp rx=0 tx=0).stop 2024-04-26T21:54:36.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.005+0000 7f7027e75740 1 -- 172.21.15.55:0/1409488658 shutdown_connections 2024-04-26T21:54:36.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.005+0000 7f7027e75740 1 --2- 172.21.15.55:0/1409488658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494fa70 0x564d949848d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.005+0000 7f7027e75740 1 --2- 172.21.15.55:0/1409488658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 0x564d949681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.005+0000 7f7027e75740 1 --2- 172.21.15.55:0/1409488658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494dee0 0x564d94984390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.005+0000 7f7027e75740 1 -- 172.21.15.55:0/1409488658 >> 172.21.15.55:0/1409488658 conn(0x564d949493d0 msgr2=0x564d94948550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.005+0000 7f7027e75740 1 -- 172.21.15.55:0/1409488658 shutdown_connections 2024-04-26T21:54:36.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.005+0000 7f7027e75740 1 -- 172.21.15.55:0/1409488658 wait complete. 2024-04-26T21:54:36.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.006+0000 7f7027e75740 1 -- start start 2024-04-26T21:54:36.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.007+0000 7f7027e75740 1 Processor -- start 2024-04-26T21:54:36.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.008+0000 7f7027e75740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94965c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.008+0000 7f7027e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 0x564d94966180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.009+0000 7f7027e75740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494fa70 0x564d94967120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.009+0000 7f7027e75740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7020071520 con 0x564d9494dee0 2024-04-26T21:54:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.009+0000 7f7027e75740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7020072990 con 0x564d9494fa70 2024-04-26T21:54:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.009+0000 7f7027e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7020071bf0 con 0x564d9494f0b0 2024-04-26T21:54:36.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.009+0000 7f7027e75740 10 client.0 _ll_register_callbacks cb 0x7f7014015420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:36.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.011+0000 7f701ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94965c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.011+0000 7f701ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94965c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58568/0 (socket says 172.21.15.55:58568) 2024-04-26T21:54:36.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.011+0000 7f701ed9d640 1 -- 172.21.15.55:0/53049437 learned_addr learned my addr 172.21.15.55:0/53049437 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:36.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.012+0000 7f701e59c640 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 0x564d94966180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.013+0000 7f701f59e640 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494fa70 0x564d94967120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.013+0000 7f701ed9d640 1 -- 172.21.15.55:0/53049437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 msgr2=0x564d94966180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:36.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.013+0000 7f701ed9d640 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 0x564d94966180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.013+0000 7f701ed9d640 1 -- 172.21.15.55:0/53049437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494fa70 msgr2=0x564d94967120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:36.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.013+0000 7f701ed9d640 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494fa70 0x564d94967120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.013+0000 7f701ed9d640 1 -- 172.21.15.55:0/53049437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d948734b0 con 0x564d9494dee0 2024-04-26T21:54:36.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.014+0000 7f701ed9d640 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94965c40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f70100ee9e0 tx=0x7f701003bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48173a6e399e7e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.014+0000 7f7002ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:36.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.014+0000 7f7002ff5640 1 -- 172.21.15.55:0/53049437 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010049040 con 0x564d9494dee0 2024-04-26T21:54:36.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.014+0000 7f7002ff5640 1 -- 172.21.15.55:0/53049437 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f70100b9760 con 0x564d9494dee0 2024-04-26T21:54:36.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.014+0000 7f7027e75740 1 -- 172.21.15.55:0/53049437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7018028070 con 0x564d9494dee0 2024-04-26T21:54:36.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7002ff5640 1 -- 172.21.15.55:0/53049437 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f70100bd180 con 0x564d9494dee0 2024-04-26T21:54:36.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7027e75740 10 client.15630 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:36.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7027e75740 1 -- 172.21.15.55:0/53049437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x564d949e3e60 con 0x564d9494dee0 2024-04-26T21:54:36.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7002ff5640 1 -- 172.21.15.55:0/53049437 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7018028070 con 0x564d9494dee0 2024-04-26T21:54:36.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7002ff5640 1 -- 172.21.15.55:0/53049437 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f70100c1120 con 0x564d9494dee0 2024-04-26T21:54:36.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7027e75740 10 client.15630 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:36.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7027e75740 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:36.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7027e75740 20 client.15630 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:36.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7027e75740 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564d949e4840 0x564d949e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7027e75740 4 client.15630 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:54:36.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.015+0000 7f7027e75740 1 -- 172.21.15.55:0/53049437 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d948ba800 con 0x564d949e4840 2024-04-26T21:54:36.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.016+0000 7f701e59c640 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564d949e4840 0x564d949e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.017+0000 7f7002ff5640 10 client.15630 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:36.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.017+0000 7f701e59c640 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564d949e4840 0x564d949e6d00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd311c06118ce0c2 server_cookie=ab75617820d151c0 in_seq=0 out_seq=0 2024-04-26T21:54:36.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7002ff5640 1 -- 172.21.15.55:0/53049437 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564d948ba800 con 0x564d949e4840 2024-04-26T21:54:36.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7002ff5640 10 client.15630 handle_command_reply: tid=0 2024-04-26T21:54:36.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 client.15630 shutdown 2024-04-26T21:54:36.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 -- 172.21.15.55:0/53049437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 msgr2=0x564d94965c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94965c40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f70100ee9e0 tx=0x7f701003bc70 comp rx=0 tx=0).stop 2024-04-26T21:54:36.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 -- 172.21.15.55:0/53049437 shutdown_connections 2024-04-26T21:54:36.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d9494fa70 0x564d94967120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564d9494f0b0 0x564d94966180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564d949e4840 0x564d949e6d00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 --2- 172.21.15.55:0/53049437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564d9494dee0 0x564d94965c40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7027e75740 1 -- 172.21.15.55:0/53049437 >> 172.21.15.55:0/53049437 conn(0x564d949493d0 msgr2=0x564d9494c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.021+0000 7f7002ff5640 0 client.15630 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.022+0000 7f7027e75740 1 -- 172.21.15.55:0/53049437 shutdown_connections 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.022+0000 7f7027e75740 1 -- 172.21.15.55:0/53049437 wait complete. 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.022+0000 7f7027e75740 20 client.15630 trim_cache size 0 max 16384 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:54:36.025 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:54:36.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 -- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70080a3cc0 msgr2=0x7f70080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70080a3cc0 0x7f70080a6180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f702006ba90 tx=0x7f7014006c40 comp rx=0 tx=0).stop 2024-04-26T21:54:36.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 -- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 msgr2=0x7f7020198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 0x7f7020198d90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7010031500 tx=0x7f7010003d50 comp rx=0 tx=0).stop 2024-04-26T21:54:36.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 -- 172.21.15.55:0/3773034145 shutdown_connections 2024-04-26T21:54:36.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70080a3cc0 0x7f70080a6180 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7020108640 0x7f70201a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7020105d10 0x7f70201992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 --2- 172.21.15.55:0/3773034145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70201033e0 0x7f7020198d90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.023+0000 7f702515f640 1 -- 172.21.15.55:0/3773034145 >> 172.21.15.55:0/3773034145 conn(0x7f70200fcdd0 msgr2=0x7f7020103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.024+0000 7f702515f640 1 -- 172.21.15.55:0/3773034145 shutdown_connections 2024-04-26T21:54:36.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.024+0000 7f702515f640 1 -- 172.21.15.55:0/3773034145 wait complete. 2024-04-26T21:54:36.049 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:54:36.049 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:54:36.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.206+0000 7fa71d8c8640 1 -- 172.21.15.55:0/2704586635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 msgr2=0x7fa718101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.206+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/2704586635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 0x7fa718101c90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa708009900 tx=0x7fa70802a440 comp rx=0 tx=0).stop 2024-04-26T21:54:36.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.207+0000 7fa71d8c8640 1 -- 172.21.15.55:0/2704586635 shutdown_connections 2024-04-26T21:54:36.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.207+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/2704586635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa718103420 0x7fa718109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.207+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/2704586635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa718102a60 0x7fa718102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.207+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/2704586635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 0x7fa718101c90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.207+0000 7fa71d8c8640 1 -- 172.21.15.55:0/2704586635 >> 172.21.15.55:0/2704586635 conn(0x7fa7180fcde0 msgr2=0x7fa7180ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.207+0000 7fa71d8c8640 1 -- 172.21.15.55:0/2704586635 shutdown_connections 2024-04-26T21:54:36.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.207+0000 7fa71d8c8640 1 -- 172.21.15.55:0/2704586635 wait complete. 2024-04-26T21:54:36.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.208+0000 7fa71d8c8640 1 Processor -- start 2024-04-26T21:54:36.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.208+0000 7fa71d8c8640 1 -- start start 2024-04-26T21:54:36.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa71d8c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 0x7fa71806a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa71d8c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa718102a60 0x7fa71819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa71d8c8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa718103420 0x7fa7181a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa716ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 0x7fa71806a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa716ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 0x7fa71806a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58586/0 (socket says 172.21.15.55:58586) 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa716ffd640 1 -- 172.21.15.55:0/3551660020 learned_addr learned my addr 172.21.15.55:0/3551660020 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa71810d500 con 0x7fa718101870 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa71810d380 con 0x7fa718102a60 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.209+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa71810d680 con 0x7fa718103420 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.210+0000 7fa7177fe640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa718103420 0x7fa7181a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.210+0000 7fa716ffd640 1 -- 172.21.15.55:0/3551660020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa718103420 msgr2=0x7fa7181a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.210+0000 7fa716ffd640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa718103420 0x7fa7181a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.210+0000 7fa716ffd640 1 -- 172.21.15.55:0/3551660020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa718102a60 msgr2=0x7fa71819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.210+0000 7fa716ffd640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa718102a60 0x7fa71819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.210+0000 7fa716ffd640 1 -- 172.21.15.55:0/3551660020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7181a09e0 con 0x7fa718101870 2024-04-26T21:54:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.210+0000 7fa7177fe640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa718103420 0x7fa7181a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:36.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.210+0000 7fa716ffd640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 0x7fa71806a650 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa708009850 tx=0x7fa708003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.211+0000 7fa71c8c6640 1 -- 172.21.15.55:0/3551660020 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70803f070 con 0x7fa718101870 2024-04-26T21:54:36.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.211+0000 7fa71c8c6640 1 -- 172.21.15.55:0/3551660020 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa70803b460 con 0x7fa718101870 2024-04-26T21:54:36.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.211+0000 7fa71c8c6640 1 -- 172.21.15.55:0/3551660020 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708035030 con 0x7fa718101870 2024-04-26T21:54:36.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.211+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7181a0c70 con 0x7fa718101870 2024-04-26T21:54:36.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.211+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181a1110 con 0x7fa718101870 2024-04-26T21:54:36.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.213+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718108cb0 con 0x7fa718101870 2024-04-26T21:54:36.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.214+0000 7fa71c8c6640 1 -- 172.21.15.55:0/3551660020 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa708042070 con 0x7fa718101870 2024-04-26T21:54:36.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.215+0000 7fa71c8c6640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa7000a3d00 0x7fa7000a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:36.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.216+0000 7fa7167fc640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa7000a3d00 0x7fa7000a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:36.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.216+0000 7fa71c8c6640 1 -- 172.21.15.55:0/3551660020 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa7080f1120 con 0x7fa718101870 2024-04-26T21:54:36.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.216+0000 7fa7167fc640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa7000a3d00 0x7fa7000a61c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa71806a3c0 tx=0x7fa70c006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:36.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.221+0000 7fa71c8c6640 1 -- 172.21.15.55:0/3551660020 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa7080bd960 con 0x7fa718101870 2024-04-26T21:54:36.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.759+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa71802dad0 con 0x7fa718101870 2024-04-26T21:54:36.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.766+0000 7fa71c8c6640 1 -- 172.21.15.55:0/3551660020 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11863 (secure 0 0 0) 0x7fa7080bd260 con 0x7fa718101870 2024-04-26T21:54:36.767 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:54:36.767 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:52:54.328649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24502,24509]},"id":1}]} 2024-04-26T21:54:36.767 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 21 2024-04-26T21:54:36.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.777+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa7000a3d00 msgr2=0x7fa7000a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.778+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa7000a3d00 0x7fa7000a61c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa71806a3c0 tx=0x7fa70c006c40 comp rx=0 tx=0).stop 2024-04-26T21:54:36.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.778+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 msgr2=0x7fa71806a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:36.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.778+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 0x7fa71806a650 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa708009850 tx=0x7fa708003d60 comp rx=0 tx=0).stop 2024-04-26T21:54:36.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.778+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 shutdown_connections 2024-04-26T21:54:36.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.779+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa7000a3d00 0x7fa7000a61c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.779+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa718103420 0x7fa7181a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.779+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa718102a60 0x7fa71819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.779+0000 7fa71d8c8640 1 --2- 172.21.15.55:0/3551660020 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa718101870 0x7fa71806a650 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:36.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.779+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 >> 172.21.15.55:0/3551660020 conn(0x7fa7180fcde0 msgr2=0x7fa718105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:36.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.782+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 shutdown_connections 2024-04-26T21:54:36.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:36.782+0000 7fa71d8c8640 1 -- 172.21.15.55:0/3551660020 wait complete. 2024-04-26T21:54:36.840 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:54:37.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:36 smithi064 ceph-mon[32765]: pgmap v402: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 430 MiB/s wr, 352 op/s 2024-04-26T21:54:37.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:36 smithi165 ceph-mon[33296]: pgmap v402: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 430 MiB/s wr, 352 op/s 2024-04-26T21:54:37.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:36 smithi055 ceph-mon[29221]: pgmap v402: 97 pgs: 97 active+clean; 5.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 430 MiB/s wr, 352 op/s 2024-04-26T21:54:37.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.220+0000 7fb588a6d640 1 -- 172.21.15.55:0/803363429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 msgr2=0x7fb58410bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.220+0000 7fb588a6d640 1 --2- 172.21.15.55:0/803363429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 0x7fb58410bd60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb578009900 tx=0x7fb57802a440 comp rx=0 tx=0).stop 2024-04-26T21:54:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.224+0000 7fb588a6d640 1 -- 172.21.15.55:0/803363429 shutdown_connections 2024-04-26T21:54:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.224+0000 7fb588a6d640 1 --2- 172.21.15.55:0/803363429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58410ec70 0x7fb58410f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.224+0000 7fb588a6d640 1 --2- 172.21.15.55:0/803363429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58410c2a0 0x7fb58410e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.224+0000 7fb588a6d640 1 --2- 172.21.15.55:0/803363429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 0x7fb58410bd60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.224+0000 7fb588a6d640 1 -- 172.21.15.55:0/803363429 >> 172.21.15.55:0/803363429 conn(0x7fb58406f7c0 msgr2=0x7fb584071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.235+0000 7fb588a6d640 1 -- 172.21.15.55:0/803363429 shutdown_connections 2024-04-26T21:54:37.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.235+0000 7fb588a6d640 1 -- 172.21.15.55:0/803363429 wait complete. 2024-04-26T21:54:37.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.238+0000 7fb588a6d640 1 Processor -- start 2024-04-26T21:54:37.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.238+0000 7fb588a6d640 1 -- start start 2024-04-26T21:54:37.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.238+0000 7fb588a6d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 0x7fb584116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.239+0000 7fb588a6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58410c2a0 0x7fb584116670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.239+0000 7fb588a6d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58410ec70 0x7fb5841b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.239+0000 7fb588a6d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb58411d2d0 con 0x7fb584109930 2024-04-26T21:54:37.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.239+0000 7fb588a6d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb58411d150 con 0x7fb58410c2a0 2024-04-26T21:54:37.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.239+0000 7fb588a6d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb58411d450 con 0x7fb58410ec70 2024-04-26T21:54:37.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.239+0000 7fb58259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 0x7fb584116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.239+0000 7fb58259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 0x7fb584116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58616/0 (socket says 172.21.15.55:58616) 2024-04-26T21:54:37.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.239+0000 7fb58259c640 1 -- 172.21.15.55:0/3980675295 learned_addr learned my addr 172.21.15.55:0/3980675295 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.241+0000 7fb58259c640 1 -- 172.21.15.55:0/3980675295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58410ec70 msgr2=0x7fb5841b79e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:54:37.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.241+0000 7fb58259c640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58410ec70 0x7fb5841b79e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.241+0000 7fb58259c640 1 -- 172.21.15.55:0/3980675295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58410c2a0 msgr2=0x7fb584116670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.241+0000 7fb58259c640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58410c2a0 0x7fb584116670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.241+0000 7fb58259c640 1 -- 172.21.15.55:0/3980675295 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5841b7f20 con 0x7fb584109930 2024-04-26T21:54:37.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.242+0000 7fb58259c640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 0x7fb584116130 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb578004ba0 tx=0x7fb578004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.244+0000 7fb5737fe640 1 -- 172.21.15.55:0/3980675295 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578040030 con 0x7fb584109930 2024-04-26T21:54:37.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.244+0000 7fb5737fe640 1 -- 172.21.15.55:0/3980675295 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5780050f0 con 0x7fb584109930 2024-04-26T21:54:37.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.244+0000 7fb5737fe640 1 -- 172.21.15.55:0/3980675295 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57803c940 con 0x7fb584109930 2024-04-26T21:54:37.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.244+0000 7fb588a6d640 1 -- 172.21.15.55:0/3980675295 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5841b80f0 con 0x7fb584109930 2024-04-26T21:54:37.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.244+0000 7fb588a6d640 1 -- 172.21.15.55:0/3980675295 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5841b8560 con 0x7fb584109930 2024-04-26T21:54:37.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.253+0000 7fb58b786740 1 Processor -- start 2024-04-26T21:54:37.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.254+0000 7fb58b786740 1 -- start start 2024-04-26T21:54:37.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.254+0000 7fb58b786740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdd820 0x563e91cddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.255+0000 7fb58b786740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91cdee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.255+0000 7fb58b786740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdf3b0 0x563e91ce5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.255+0000 7fb58b786740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563e91c41d80 con 0x563e91cdf3b0 2024-04-26T21:54:37.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.255+0000 7fb58b786740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563e91bfba20 con 0x563e91cde9f0 2024-04-26T21:54:37.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.255+0000 7fb58b786740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563e91c3dd40 con 0x563e91cdd820 2024-04-26T21:54:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.256+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdd820 0x563e91cddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.256+0000 7fb5737fe640 1 -- 172.21.15.55:0/3980675295 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb578043030 con 0x7fb584109930 2024-04-26T21:54:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.256+0000 7fb581d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdf3b0 0x563e91ce5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.256+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdd820 0x563e91cddc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48376/0 (socket says 172.21.15.55:48376) 2024-04-26T21:54:37.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.256+0000 7fb582d9d640 1 -- 172.21.15.55:0/4066684972 learned_addr learned my addr 172.21.15.55:0/4066684972 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.257+0000 7fb582d9d640 1 -- 172.21.15.55:0/4066684972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 msgr2=0x563e91cdee70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:54:37.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.257+0000 7fb582d9d640 1 --2- 172.21.15.55:0/4066684972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91cdee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.257+0000 7fb582d9d640 1 -- 172.21.15.55:0/4066684972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdf3b0 msgr2=0x563e91ce5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.257+0000 7fb582d9d640 1 --2- 172.21.15.55:0/4066684972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdf3b0 0x563e91ce5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.257+0000 7fb582d9d640 1 -- 172.21.15.55:0/4066684972 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e91c06c30 con 0x563e91cdd820 2024-04-26T21:54:37.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.258+0000 7fb5737fe640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb56c0a3e40 0x7fb56c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.258+0000 7fb582d9d640 1 --2- 172.21.15.55:0/4066684972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdd820 0x563e91cddc40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009cd0 tx=0x7fb57c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2064a48d5d4c5c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.259+0000 7fb5737fe640 1 -- 172.21.15.55:0/3980675295 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb5780f1d80 con 0x7fb584109930 2024-04-26T21:54:37.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.260+0000 7fb570ff9640 1 -- 172.21.15.55:0/4066684972 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c016070 con 0x563e91cdd820 2024-04-26T21:54:37.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.261+0000 7fb570ff9640 1 -- 172.21.15.55:0/4066684972 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb57c005020 con 0x563e91cdd820 2024-04-26T21:54:37.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.261+0000 7fb570ff9640 1 -- 172.21.15.55:0/4066684972 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0116d0 con 0x563e91cdd820 2024-04-26T21:54:37.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.264+0000 7fb58b786740 1 -- 172.21.15.55:0/4066684972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdd820 msgr2=0x563e91cddc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.264+0000 7fb58b786740 1 --2- 172.21.15.55:0/4066684972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdd820 0x563e91cddc40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009cd0 tx=0x7fb57c00e6c0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.265+0000 7fb58259c640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb56c0a3e40 0x7fb56c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.272+0000 7fb58b786740 1 -- 172.21.15.55:0/4066684972 shutdown_connections 2024-04-26T21:54:37.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.272+0000 7fb58b786740 1 --2- 172.21.15.55:0/4066684972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdf3b0 0x563e91ce5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.272+0000 7fb58b786740 1 --2- 172.21.15.55:0/4066684972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91cdee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.272+0000 7fb58b786740 1 --2- 172.21.15.55:0/4066684972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdd820 0x563e91cddc40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.272+0000 7fb58b786740 1 -- 172.21.15.55:0/4066684972 >> 172.21.15.55:0/4066684972 conn(0x563e91c1e910 msgr2=0x563e91cd9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.273+0000 7fb58259c640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb56c0a3e40 0x7fb56c0a6300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb57802aa30 tx=0x7fb578009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.278+0000 7fb58b786740 1 -- 172.21.15.55:0/4066684972 shutdown_connections 2024-04-26T21:54:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.284+0000 7fb58b786740 1 -- 172.21.15.55:0/4066684972 wait complete. 2024-04-26T21:54:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.284+0000 7fb58b786740 1 -- start start 2024-04-26T21:54:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.285+0000 7fb58b786740 1 Processor -- start 2024-04-26T21:54:37.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb58b786740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb58b786740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91ce8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb58b786740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 0x563e91cef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb58b786740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563e91ce63c0 con 0x563e91cdd820 2024-04-26T21:54:37.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb58b786740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563e91bc8dc0 con 0x563e91cde9f0 2024-04-26T21:54:37.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb58b786740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563e91ce6540 con 0x563e91cdf3b0 2024-04-26T21:54:37.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb58b786740 10 client.0 _ll_register_callbacks cb 0x563e91be4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:37.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58646/0 (socket says 172.21.15.55:58646) 2024-04-26T21:54:37.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.286+0000 7fb582d9d640 1 -- 172.21.15.55:0/428809216 learned_addr learned my addr 172.21.15.55:0/428809216 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.290+0000 7fb581d9b640 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91ce8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.290+0000 7fb582d9d640 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 0x563e91cef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.290+0000 7fb582d9d640 1 -- 172.21.15.55:0/428809216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 msgr2=0x563e91cef0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.290+0000 7fb582d9d640 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 0x563e91cef0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.290+0000 7fb582d9d640 1 -- 172.21.15.55:0/428809216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 msgr2=0x563e91ce8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.290+0000 7fb582d9d640 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91ce8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.290+0000 7fb582d9d640 1 -- 172.21.15.55:0/428809216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e919533f0 con 0x563e91cdd820 2024-04-26T21:54:37.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.298+0000 7fb582d9d640 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7b50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb57c006010 tx=0x7fb57c009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc16c9b4c7d15f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.303+0000 7fb569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:37.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.303+0000 7fb569ffb640 1 -- 172.21.15.55:0/428809216 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c016070 con 0x563e91cdd820 2024-04-26T21:54:37.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.303+0000 7fb58b786740 1 -- 172.21.15.55:0/428809216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57c005020 con 0x563e91cdd820 2024-04-26T21:54:37.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.309+0000 7fb569ffb640 1 -- 172.21.15.55:0/428809216 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb57c0053d0 con 0x563e91cdd820 2024-04-26T21:54:37.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.310+0000 7fb569ffb640 1 -- 172.21.15.55:0/428809216 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb57c023040 con 0x563e91cdd820 2024-04-26T21:54:37.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.314+0000 7fb58b786740 10 client.15648 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:37.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.314+0000 7fb58b786740 1 -- 172.21.15.55:0/428809216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563e91a1d7d0 con 0x563e91cdd820 2024-04-26T21:54:37.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.320+0000 7fb569ffb640 1 -- 172.21.15.55:0/428809216 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb57c005020 con 0x563e91cdd820 2024-04-26T21:54:37.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.320+0000 7fb569ffb640 1 -- 172.21.15.55:0/428809216 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb57c00f040 con 0x563e91cdd820 2024-04-26T21:54:37.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.320+0000 7fb58b786740 10 client.15648 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:37.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.320+0000 7fb58b786740 10 client.15648 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:54:37.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.320+0000 7fb58b786740 20 client.15648 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:37.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.320+0000 7fb58b786740 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563e91cf3cc0 0x563e91cf6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.320+0000 7fb58b786740 4 client.15648 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:37.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.320+0000 7fb58b786740 1 -- 172.21.15.55:0/428809216 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e91c4f990 con 0x563e91cf3cc0 2024-04-26T21:54:37.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.329+0000 7fb581d9b640 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563e91cf3cc0 0x563e91cf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.337+0000 7fb581d9b640 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563e91cf3cc0 0x563e91cf6100 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b60ae430b1edd116 server_cookie=c27bf43325ce07b9 in_seq=0 out_seq=0 2024-04-26T21:54:37.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.339+0000 7fb569ffb640 10 client.15648 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:54:37.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.343+0000 7fb569ffb640 1 -- 172.21.15.55:0/428809216 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e91c4f990 con 0x563e91cf3cc0 2024-04-26T21:54:37.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.343+0000 7fb569ffb640 10 client.15648 handle_command_reply: tid=0 2024-04-26T21:54:37.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.343+0000 7fb58b786740 1 client.15648 shutdown 2024-04-26T21:54:37.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.345+0000 7fb58b786740 1 -- 172.21.15.55:0/428809216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 msgr2=0x563e91ce7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.345+0000 7fb58b786740 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7b50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb57c006010 tx=0x7fb57c009de0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.347+0000 7fb58b786740 1 -- 172.21.15.55:0/428809216 shutdown_connections 2024-04-26T21:54:37.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.347+0000 7fb58b786740 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563e91cf3cc0 0x563e91cf6100 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.347+0000 7fb58b786740 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 0x563e91cef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.347+0000 7fb58b786740 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91ce8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.347+0000 7fb58b786740 1 --2- 172.21.15.55:0/428809216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7b50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.347+0000 7fb58b786740 1 -- 172.21.15.55:0/428809216 >> 172.21.15.55:0/428809216 conn(0x563e91c1e910 msgr2=0x563e91cd8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.349+0000 7fb569ffb640 0 client.15648 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:54:37.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.349+0000 7fb58b786740 1 -- 172.21.15.55:0/428809216 shutdown_connections 2024-04-26T21:54:37.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.350+0000 7fb58b786740 1 -- 172.21.15.55:0/428809216 wait complete. 2024-04-26T21:54:37.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.351+0000 7fb58b786740 20 client.15648 trim_cache size 0 max 16384 2024-04-26T21:54:37.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.407+0000 7fb58b786740 1 Processor -- start 2024-04-26T21:54:37.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.408+0000 7fb58b786740 1 -- start start 2024-04-26T21:54:37.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.408+0000 7fb58b786740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.409+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.409+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58656/0 (socket says 172.21.15.55:58656) 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.409+0000 7fb58b786740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cde9f0 0x563e91cef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.409+0000 7fb582d9d640 1 -- 172.21.15.55:0/2657307190 learned_addr learned my addr 172.21.15.55:0/2657307190 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.409+0000 7fb58b786740 1 --2- 172.21.15.55:0/2657307190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cdf3b0 0x563e91cfaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.409+0000 7fb58b786740 1 -- 172.21.15.55:0/2657307190 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb564010eb0 con 0x563e91cdd820 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.409+0000 7fb58b786740 1 -- 172.21.15.55:0/2657307190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400e780 con 0x563e91cdf3b0 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.409+0000 7fb58b786740 1 -- 172.21.15.55:0/2657307190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400c5d0 con 0x563e91cde9f0 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.410+0000 7fb582d9d640 1 -- 172.21.15.55:0/2657307190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cde9f0 msgr2=0x563e91cef060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.410+0000 7fb582d9d640 1 --2- 172.21.15.55:0/2657307190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cde9f0 0x563e91cef060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.410+0000 7fb582d9d640 1 -- 172.21.15.55:0/2657307190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cdf3b0 msgr2=0x563e91cfaa00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:54:37.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.410+0000 7fb582d9d640 1 --2- 172.21.15.55:0/2657307190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cdf3b0 0x563e91cfaa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.410+0000 7fb582d9d640 1 -- 172.21.15.55:0/2657307190 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e91ce55c0 con 0x563e91cdd820 2024-04-26T21:54:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.411+0000 7fb582d9d640 1 --2- 172.21.15.55:0/2657307190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7e70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb57c002230 tx=0x7fb57c003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ed201ea238367 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.411+0000 7fb56bfff640 1 -- 172.21.15.55:0/2657307190 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c016070 con 0x563e91cdd820 2024-04-26T21:54:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.412+0000 7fb56bfff640 1 -- 172.21.15.55:0/2657307190 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb57c02a740 con 0x563e91cdd820 2024-04-26T21:54:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.412+0000 7fb56bfff640 1 -- 172.21.15.55:0/2657307190 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0041e0 con 0x563e91cdd820 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 -- 172.21.15.55:0/2657307190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 msgr2=0x563e91ce7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 --2- 172.21.15.55:0/2657307190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7e70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb57c002230 tx=0x7fb57c003c70 comp rx=0 tx=0).stop 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 -- 172.21.15.55:0/2657307190 shutdown_connections 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 --2- 172.21.15.55:0/2657307190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cdf3b0 0x563e91cfaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 --2- 172.21.15.55:0/2657307190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cde9f0 0x563e91cef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 --2- 172.21.15.55:0/2657307190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ce7e70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 -- 172.21.15.55:0/2657307190 >> 172.21.15.55:0/2657307190 conn(0x563e91c1e910 msgr2=0x563e91ce7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 -- 172.21.15.55:0/2657307190 shutdown_connections 2024-04-26T21:54:37.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.414+0000 7fb58b786740 1 -- 172.21.15.55:0/2657307190 wait complete. 2024-04-26T21:54:37.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.415+0000 7fb58b786740 1 -- start start 2024-04-26T21:54:37.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.415+0000 7fb58b786740 1 Processor -- start 2024-04-26T21:54:37.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.416+0000 7fb58b786740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ceee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.416+0000 7fb58b786740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91ceb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.416+0000 7fb58b786740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 0x563e91cebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.416+0000 7fb58b786740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400cc40 con 0x563e91cdd820 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.416+0000 7fb58b786740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400e0b0 con 0x563e91cde9f0 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.416+0000 7fb58b786740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb56400d310 con 0x563e91cdf3b0 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ceee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ceee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58668/0 (socket says 172.21.15.55:58668) 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb582d9d640 1 -- 172.21.15.55:0/4208891292 learned_addr learned my addr 172.21.15.55:0/4208891292 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb58b786740 10 client.0 _ll_register_callbacks cb 0x7fb57c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb582d9d640 1 -- 172.21.15.55:0/4208891292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 msgr2=0x563e91cebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb582d9d640 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 0x563e91cebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb582d9d640 1 -- 172.21.15.55:0/4208891292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 msgr2=0x563e91ceb930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:54:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb582d9d640 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91ceb930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.417+0000 7fb582d9d640 1 -- 172.21.15.55:0/4208891292 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e91cec460 con 0x563e91cdd820 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.418+0000 7fb582d9d640 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ceee40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009a60 tx=0x7fb57c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2fb1b68dbb2769 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.418+0000 7fb569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.418+0000 7fb569ffb640 1 -- 172.21.15.55:0/4208891292 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c016040 con 0x563e91cdd820 2024-04-26T21:54:37.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.419+0000 7fb58b786740 1 -- 172.21.15.55:0/4208891292 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57c02a740 con 0x563e91cdd820 2024-04-26T21:54:37.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.419+0000 7fb569ffb640 1 -- 172.21.15.55:0/4208891292 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb57c02a8f0 con 0x563e91cdd820 2024-04-26T21:54:37.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.420+0000 7fb569ffb640 1 -- 172.21.15.55:0/4208891292 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb57c02abe0 con 0x563e91cdd820 2024-04-26T21:54:37.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.420+0000 7fb58b786740 10 client.15654 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:37.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.420+0000 7fb58b786740 1 -- 172.21.15.55:0/4208891292 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563e91d592e0 con 0x563e91cdd820 2024-04-26T21:54:37.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.421+0000 7fb569ffb640 1 -- 172.21.15.55:0/4208891292 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb57c02a740 con 0x563e91cdd820 2024-04-26T21:54:37.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.421+0000 7fb569ffb640 1 -- 172.21.15.55:0/4208891292 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fb57c0116e0 con 0x563e91cdd820 2024-04-26T21:54:37.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.421+0000 7fb58b786740 10 client.15654 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.421+0000 7fb58b786740 10 client.15654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.421+0000 7fb58b786740 20 client.15654 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.421+0000 7fb58b786740 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563e91d59c30 0x563e91d5c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.422+0000 7fb58b786740 4 client.15654 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:54:37.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.422+0000 7fb58b786740 1 -- 172.21.15.55:0/4208891292 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e91c49800 con 0x563e91d59c30 2024-04-26T21:54:37.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.425+0000 7fb581d9b640 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563e91d59c30 0x563e91d5c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.432+0000 7fb581d9b640 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563e91d59c30 0x563e91d5c050 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bff7179eee8ebb2 server_cookie=ca23ae743099295d in_seq=0 out_seq=0 2024-04-26T21:54:37.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.432+0000 7fb569ffb640 10 client.15654 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:54:37.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.434+0000 7fb569ffb640 1 -- 172.21.15.55:0/4208891292 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e91c49800 con 0x563e91d59c30 2024-04-26T21:54:37.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.434+0000 7fb569ffb640 10 client.15654 handle_command_reply: tid=0 2024-04-26T21:54:37.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.434+0000 7fb58b786740 1 client.15654 shutdown 2024-04-26T21:54:37.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.434+0000 7fb58b786740 1 -- 172.21.15.55:0/4208891292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 msgr2=0x563e91ceee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.434+0000 7fb58b786740 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ceee40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009a60 tx=0x7fb57c005d30 comp rx=0 tx=0).stop 2024-04-26T21:54:37.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 1 -- 172.21.15.55:0/4208891292 shutdown_connections 2024-04-26T21:54:37.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563e91d59c30 0x563e91d5c050 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e91cdf3b0 0x563e91cebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb569ffb640 0 client.15654 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:54:37.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563e91cde9f0 0x563e91ceb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 1 --2- 172.21.15.55:0/4208891292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563e91cdd820 0x563e91ceee40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 1 -- 172.21.15.55:0/4208891292 >> 172.21.15.55:0/4208891292 conn(0x563e91c1e910 msgr2=0x563e91ce7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 1 -- 172.21.15.55:0/4208891292 shutdown_connections 2024-04-26T21:54:37.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 1 -- 172.21.15.55:0/4208891292 wait complete. 2024-04-26T21:54:37.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.435+0000 7fb58b786740 20 client.15654 trim_cache size 0 max 16384 2024-04-26T21:54:37.438 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:54:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 -- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb56c0a3e40 msgr2=0x7fb56c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb56c0a3e40 0x7fb56c0a6300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb57802aa30 tx=0x7fb578009260 comp rx=0 tx=0).stop 2024-04-26T21:54:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 -- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 msgr2=0x7fb584116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 0x7fb584116130 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb578004ba0 tx=0x7fb578004320 comp rx=0 tx=0).stop 2024-04-26T21:54:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 -- 172.21.15.55:0/3980675295 shutdown_connections 2024-04-26T21:54:37.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb56c0a3e40 0x7fb56c0a6300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58410ec70 0x7fb5841b79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb58410c2a0 0x7fb584116670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 --2- 172.21.15.55:0/3980675295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb584109930 0x7fb584116130 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.437+0000 7fb588a6d640 1 -- 172.21.15.55:0/3980675295 >> 172.21.15.55:0/3980675295 conn(0x7fb58406f7c0 msgr2=0x7fb58410c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.438+0000 7fb588a6d640 1 -- 172.21.15.55:0/3980675295 shutdown_connections 2024-04-26T21:54:37.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.438+0000 7fb588a6d640 1 -- 172.21.15.55:0/3980675295 wait complete. 2024-04-26T21:54:37.469 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:54:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.781+0000 7ff058f8e640 1 -- 172.21.15.55:0/2430339555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0541032f0 msgr2=0x7ff054069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.781+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2430339555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0541032f0 0x7ff054069ac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009be0 tx=0x7ff04c02a6d0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.781+0000 7ff058f8e640 1 -- 172.21.15.55:0/2430339555 shutdown_connections 2024-04-26T21:54:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.781+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2430339555 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0541032f0 0x7ff054069ac0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.781+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2430339555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff054102930 0x7ff054102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.781+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2430339555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff054101740 0x7ff054101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.781+0000 7ff058f8e640 1 -- 172.21.15.55:0/2430339555 >> 172.21.15.55:0/2430339555 conn(0x7ff0540fccd0 msgr2=0x7ff0540ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.781+0000 7ff058f8e640 1 -- 172.21.15.55:0/2430339555 shutdown_connections 2024-04-26T21:54:37.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.782+0000 7ff058f8e640 1 -- 172.21.15.55:0/2430339555 wait complete. 2024-04-26T21:54:37.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.786+0000 7ff058f8e640 1 Processor -- start 2024-04-26T21:54:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.786+0000 7ff058f8e640 1 -- start start 2024-04-26T21:54:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff058f8e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff054101740 0x7ff05419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff058f8e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff054102930 0x7ff05419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff058f8e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0541032f0 0x7ff0541a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff058f8e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff054110580 con 0x7ff054101740 2024-04-26T21:54:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff058f8e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff054110400 con 0x7ff054102930 2024-04-26T21:54:37.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff058f8e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff054110700 con 0x7ff0541032f0 2024-04-26T21:54:37.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff0537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff054101740 0x7ff05419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff0537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff054101740 0x7ff05419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58678/0 (socket says 172.21.15.55:58678) 2024-04-26T21:54:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.787+0000 7ff0537fe640 1 -- 172.21.15.55:0/2646328450 learned_addr learned my addr 172.21.15.55:0/2646328450 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.789+0000 7ff053fff640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0541032f0 0x7ff0541a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.789+0000 7ff052ffd640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff054102930 0x7ff05419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.789+0000 7ff0537fe640 1 -- 172.21.15.55:0/2646328450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0541032f0 msgr2=0x7ff0541a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.789+0000 7ff0537fe640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0541032f0 0x7ff0541a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.790+0000 7ff0537fe640 1 -- 172.21.15.55:0/2646328450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff054102930 msgr2=0x7ff05419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.790+0000 7ff0537fe640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff054102930 0x7ff05419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.790+0000 7ff0537fe640 1 -- 172.21.15.55:0/2646328450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0541a27f0 con 0x7ff054101740 2024-04-26T21:54:37.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.790+0000 7ff0537fe640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff054101740 0x7ff05419abd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff04400d770 tx=0x7ff04400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.791+0000 7ff050ff9640 1 -- 172.21.15.55:0/2646328450 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044017070 con 0x7ff054101740 2024-04-26T21:54:37.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.791+0000 7ff050ff9640 1 -- 172.21.15.55:0/2646328450 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff044009c30 con 0x7ff054101740 2024-04-26T21:54:37.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.791+0000 7ff050ff9640 1 -- 172.21.15.55:0/2646328450 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044003cc0 con 0x7ff054101740 2024-04-26T21:54:37.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.791+0000 7ff058f8e640 1 -- 172.21.15.55:0/2646328450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0541a2ae0 con 0x7ff054101740 2024-04-26T21:54:37.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.791+0000 7ff058f8e640 1 -- 172.21.15.55:0/2646328450 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0541a2fe0 con 0x7ff054101740 2024-04-26T21:54:37.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.795+0000 7ff05cfe6740 1 Processor -- start 2024-04-26T21:54:37.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.795+0000 7ff05cfe6740 1 -- start start 2024-04-26T21:54:37.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.796+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.796+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee2b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.796+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2b590 0x560a9ee31e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.796+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560a9ed8dd80 con 0x560a9ee2abd0 2024-04-26T21:54:37.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.796+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a9ed47a20 con 0x560a9ee2b590 2024-04-26T21:54:37.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.796+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560a9ed89d40 con 0x560a9ee29a00 2024-04-26T21:54:37.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.796+0000 7ff050ff9640 1 -- 172.21.15.55:0/2646328450 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff044003e60 con 0x7ff054101740 2024-04-26T21:54:37.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.797+0000 7ff0537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee2b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.797+0000 7ff0537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee2b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58690/0 (socket says 172.21.15.55:58690) 2024-04-26T21:54:37.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.797+0000 7ff0537fe640 1 -- 172.21.15.55:0/2463098328 learned_addr learned my addr 172.21.15.55:0/2463098328 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.797+0000 7ff050ff9640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff03c0a3d40 0x7ff03c0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.797+0000 7ff053fff640 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff052ffd640 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2b590 0x560a9ee31e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff053fff640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff03c0a3d40 0x7ff03c0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff050ff9640 1 -- 172.21.15.55:0/2646328450 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff0440cfff0 con 0x7ff054101740 2024-04-26T21:54:37.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff0537fe640 1 -- 172.21.15.55:0/2463098328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 msgr2=0x560a9ee29e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff0537fe640 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee29e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff0537fe640 1 -- 172.21.15.55:0/2463098328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2b590 msgr2=0x560a9ee31e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff0537fe640 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2b590 0x560a9ee31e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff0537fe640 1 -- 172.21.15.55:0/2463098328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a9ed52c30 con 0x560a9ee2abd0 2024-04-26T21:54:37.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff053fff640 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:37.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.798+0000 7ff053fff640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff03c0a3d40 0x7ff03c0a6200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff04c02abf0 tx=0x7ff04c004a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.799+0000 7ff0537fe640 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee2b050 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff04409bdd0 tx=0x7ff0440983f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32d7870afcb030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.799+0000 7ff039ffb640 1 -- 172.21.15.55:0/2463098328 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044020070 con 0x560a9ee2abd0 2024-04-26T21:54:37.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.800+0000 7ff039ffb640 1 -- 172.21.15.55:0/2463098328 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04409c7d0 con 0x560a9ee2abd0 2024-04-26T21:54:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.800+0000 7ff039ffb640 1 -- 172.21.15.55:0/2463098328 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0440cdb50 con 0x560a9ee2abd0 2024-04-26T21:54:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2463098328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 msgr2=0x560a9ee2b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee2b050 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff04409bdd0 tx=0x7ff0440983f0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2463098328 shutdown_connections 2024-04-26T21:54:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2b590 0x560a9ee31e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee2b050 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2463098328 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2463098328 >> 172.21.15.55:0/2463098328 conn(0x560a9ed6a910 msgr2=0x560a9ee252b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2463098328 shutdown_connections 2024-04-26T21:54:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.801+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2463098328 wait complete. 2024-04-26T21:54:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.802+0000 7ff05cfe6740 1 -- start start 2024-04-26T21:54:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.802+0000 7ff05cfe6740 1 Processor -- start 2024-04-26T21:54:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 0x560a9ee33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee2abd0 0x560a9ee34220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee3b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560a9ee325a0 con 0x560a9ee2b590 2024-04-26T21:54:37.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a9ed14dc0 con 0x560a9ee29a00 2024-04-26T21:54:37.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560a9ee32720 con 0x560a9ee2abd0 2024-04-26T21:54:37.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff05cfe6740 10 client.0 _ll_register_callbacks cb 0x560a9ed30600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:37.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff053fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee3b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff053fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee3b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58704/0 (socket says 172.21.15.55:58704) 2024-04-26T21:54:37.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff053fff640 1 -- 172.21.15.55:0/2561638109 learned_addr learned my addr 172.21.15.55:0/2561638109 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff053fff640 1 -- 172.21.15.55:0/2561638109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee2abd0 msgr2=0x560a9ee34220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:54:37.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff053fff640 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee2abd0 0x560a9ee34220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff053fff640 1 -- 172.21.15.55:0/2561638109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 msgr2=0x560a9ee33ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:54:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.803+0000 7ff053fff640 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 0x560a9ee33ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.804+0000 7ff053fff640 1 -- 172.21.15.55:0/2561638109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a9ea9f3f0 con 0x560a9ee2b590 2024-04-26T21:54:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.804+0000 7ff053fff640 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee3b1f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff04c004af0 tx=0x7ff04c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c945f076f85101b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.804+0000 7ff032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.804+0000 7ff032ffd640 1 -- 172.21.15.55:0/2561638109 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c04d030 con 0x560a9ee2b590 2024-04-26T21:54:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.804+0000 7ff032ffd640 1 -- 172.21.15.55:0/2561638109 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff04c003e00 con 0x560a9ee2b590 2024-04-26T21:54:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.804+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2561638109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff04409c7d0 con 0x560a9ee2b590 2024-04-26T21:54:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.805+0000 7ff032ffd640 1 -- 172.21.15.55:0/2561638109 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff04c043070 con 0x560a9ee2b590 2024-04-26T21:54:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.805+0000 7ff05cfe6740 10 client.15666 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.805+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2561638109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560a9eb697d0 con 0x560a9ee2b590 2024-04-26T21:54:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.806+0000 7ff032ffd640 1 -- 172.21.15.55:0/2561638109 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c0044f0 con 0x560a9ee2b590 2024-04-26T21:54:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.806+0000 7ff032ffd640 1 -- 172.21.15.55:0/2561638109 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff04c03c040 con 0x560a9ee2b590 2024-04-26T21:54:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.806+0000 7ff05cfe6740 10 client.15666 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.806+0000 7ff05cfe6740 10 client.15666 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:54:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.806+0000 7ff05cfe6740 20 client.15666 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.806+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x560a9ee50fb0 0x560a9ee53470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.806+0000 7ff05cfe6740 4 client.15666 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.806+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2561638109 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff04409c7d0 con 0x560a9ee50fb0 2024-04-26T21:54:37.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.810+0000 7ff0537fe640 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x560a9ee50fb0 0x560a9ee53470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.817+0000 7ff032ffd640 10 client.15666 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:54:37.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.817+0000 7ff0537fe640 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x560a9ee50fb0 0x560a9ee53470 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aafbdea54fbf8745 server_cookie=b27604f5faad293 in_seq=0 out_seq=0 2024-04-26T21:54:37.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff032ffd640 1 -- 172.21.15.55:0/2561638109 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff04409c7d0 con 0x560a9ee50fb0 2024-04-26T21:54:37.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff032ffd640 10 client.15666 handle_command_reply: tid=0 2024-04-26T21:54:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 client.15666 shutdown 2024-04-26T21:54:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2561638109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 msgr2=0x560a9ee3b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee3b1f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff04c004af0 tx=0x7ff04c040000 comp rx=0 tx=0).stop 2024-04-26T21:54:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2561638109 shutdown_connections 2024-04-26T21:54:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x560a9ee50fb0 0x560a9ee53470 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee3b1f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee2abd0 0x560a9ee34220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/2561638109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 0x560a9ee33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2561638109 >> 172.21.15.55:0/2561638109 conn(0x560a9ed6a910 msgr2=0x560a9ee24450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.821+0000 7ff032ffd640 0 client.15666 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:54:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.822+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2561638109 shutdown_connections 2024-04-26T21:54:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.822+0000 7ff05cfe6740 1 -- 172.21.15.55:0/2561638109 wait complete. 2024-04-26T21:54:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.822+0000 7ff05cfe6740 20 client.15666 trim_cache size 0 max 16384 2024-04-26T21:54:37.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.843+0000 7ff05cfe6740 1 Processor -- start 2024-04-26T21:54:37.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.843+0000 7ff05cfe6740 1 -- start start 2024-04-26T21:54:37.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.843+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee20240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.843+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2abd0 0x560a9ee3b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.843+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.843+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541a5f90 con 0x560a9ee2b590 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.843+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541a3860 con 0x560a9ee2abd0 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.843+0000 7ff05cfe6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff054072f40 con 0x560a9ee29a00 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.844+0000 7ff053fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee33ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.844+0000 7ff053fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee33ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58706/0 (socket says 172.21.15.55:58706) 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.844+0000 7ff053fff640 1 -- 172.21.15.55:0/3073488654 learned_addr learned my addr 172.21.15.55:0/3073488654 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.844+0000 7ff0537fe640 1 --2- 172.21.15.55:0/3073488654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee20240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.845+0000 7ff053fff640 1 -- 172.21.15.55:0/3073488654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 msgr2=0x560a9ee20240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.845+0000 7ff053fff640 1 --2- 172.21.15.55:0/3073488654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee20240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.845+0000 7ff053fff640 1 -- 172.21.15.55:0/3073488654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2abd0 msgr2=0x560a9ee3b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.845+0000 7ff053fff640 1 --2- 172.21.15.55:0/3073488654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2abd0 0x560a9ee3b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.845+0000 7ff053fff640 1 -- 172.21.15.55:0/3073488654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a9ee1f820 con 0x560a9ee2b590 2024-04-26T21:54:37.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.845+0000 7ff053fff640 1 --2- 172.21.15.55:0/3073488654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee33ce0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0308a0 tx=0x7ff04c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24e2ee8ec9d3c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.846+0000 7ff0397fa640 1 -- 172.21.15.55:0/3073488654 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c04d070 con 0x560a9ee2b590 2024-04-26T21:54:37.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.846+0000 7ff0397fa640 1 -- 172.21.15.55:0/3073488654 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c03c040 con 0x560a9ee2b590 2024-04-26T21:54:37.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.846+0000 7ff0397fa640 1 -- 172.21.15.55:0/3073488654 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c036040 con 0x560a9ee2b590 2024-04-26T21:54:37.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.847+0000 7ff05cfe6740 1 -- 172.21.15.55:0/3073488654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 msgr2=0x560a9ee33ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.847+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/3073488654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee33ce0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0308a0 tx=0x7ff04c02a6d0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.847+0000 7ff05cfe6740 1 -- 172.21.15.55:0/3073488654 shutdown_connections 2024-04-26T21:54:37.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.847+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/3073488654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2b590 0x560a9ee33ce0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.847+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/3073488654 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee2abd0 0x560a9ee3b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.847+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/3073488654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee29a00 0x560a9ee20240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.847+0000 7ff05cfe6740 1 -- 172.21.15.55:0/3073488654 >> 172.21.15.55:0/3073488654 conn(0x560a9ed6a910 msgr2=0x560a9ee2e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.848+0000 7ff05cfe6740 1 -- 172.21.15.55:0/3073488654 shutdown_connections 2024-04-26T21:54:37.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.848+0000 7ff05cfe6740 1 -- 172.21.15.55:0/3073488654 wait complete. 2024-04-26T21:54:37.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.848+0000 7ff05cfe6740 1 -- start start 2024-04-26T21:54:37.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.849+0000 7ff05cfe6740 1 Processor -- start 2024-04-26T21:54:37.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.850+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 0x560a9ee38200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.850+0000 7ff05cfe6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee38740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.850+0000 7ff052ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee38740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.850+0000 7ff052ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee38740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58714/0 (socket says 172.21.15.55:58714) 2024-04-26T21:54:37.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.850+0000 7ff052ffd640 1 -- 172.21.15.55:0/766201668 learned_addr learned my addr 172.21.15.55:0/766201668 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:37.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.851+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee2b590 0x560a9eeb55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.851+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff0540735b0 con 0x560a9ee2abd0 2024-04-26T21:54:37.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.851+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0541a32d0 con 0x560a9ee29a00 2024-04-26T21:54:37.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.851+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff054073c80 con 0x560a9ee2b590 2024-04-26T21:54:37.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.851+0000 7ff0537fe640 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 0x560a9ee38200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.852+0000 7ff052ffd640 1 -- 172.21.15.55:0/766201668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee2b590 msgr2=0x560a9eeb55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.852+0000 7ff052ffd640 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee2b590 0x560a9eeb55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.852+0000 7ff052ffd640 1 -- 172.21.15.55:0/766201668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 msgr2=0x560a9ee38200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:37.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.852+0000 7ff052ffd640 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 0x560a9ee38200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.852+0000 7ff052ffd640 1 -- 172.21.15.55:0/766201668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a9eeb5b10 con 0x560a9ee2abd0 2024-04-26T21:54:37.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.852+0000 7ff05cfe6740 10 client.0 _ll_register_callbacks cb 0x7ff04c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:37.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.852+0000 7ff052ffd640 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee38740 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff048005fd0 tx=0x7ff048012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a6a632df388afd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:37.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.852+0000 7ff032ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:37.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.853+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff04c03c040 con 0x560a9ee2abd0 2024-04-26T21:54:37.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.853+0000 7ff032ffd640 1 -- 172.21.15.55:0/766201668 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048005440 con 0x560a9ee2abd0 2024-04-26T21:54:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.854+0000 7ff032ffd640 1 -- 172.21.15.55:0/766201668 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff048004730 con 0x560a9ee2abd0 2024-04-26T21:54:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.855+0000 7ff032ffd640 1 -- 172.21.15.55:0/766201668 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff0480055e0 con 0x560a9ee2abd0 2024-04-26T21:54:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.855+0000 7ff05cfe6740 10 client.15672 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.855+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=21}) -- 0x560a9eeb6e40 con 0x560a9ee2abd0 2024-04-26T21:54:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.855+0000 7ff032ffd640 1 -- 172.21.15.55:0/766201668 <== mon.0 v2:172.21.15.55:3300/0 4 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff048019020 con 0x560a9ee2abd0 2024-04-26T21:54:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.855+0000 7ff05cfe6740 10 client.15672 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.855+0000 7ff05cfe6740 10 client.15672 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:54:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.855+0000 7ff05cfe6740 20 client.15672 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.855+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x560a9eeb7640 0x560a9eeb9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.856+0000 7ff05cfe6740 4 client.15672 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:54:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.856+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a9ed9b990 con 0x560a9eeb7640 2024-04-26T21:54:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.857+0000 7ff0537fe640 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x560a9eeb7640 0x560a9eeb9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:37.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.862+0000 7ff0537fe640 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x560a9eeb7640 0x560a9eeb9a60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=751a8b39eeecad21 server_cookie=cab3ed612a7ea173 in_seq=0 out_seq=0 2024-04-26T21:54:37.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.862+0000 7ff032ffd640 10 client.15672 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:54:37.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff032ffd640 1 -- 172.21.15.55:0/766201668 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a9ed9b990 con 0x560a9eeb7640 2024-04-26T21:54:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff032ffd640 10 client.15672 handle_command_reply: tid=0 2024-04-26T21:54:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff05cfe6740 1 client.15672 shutdown 2024-04-26T21:54:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 msgr2=0x560a9ee38740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee38740 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff048005fd0 tx=0x7ff048012820 comp rx=0 tx=0).stop 2024-04-26T21:54:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 shutdown_connections 2024-04-26T21:54:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x560a9eeb7640 0x560a9eeb9a60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560a9ee2b590 0x560a9eeb55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560a9ee2abd0 0x560a9ee38740 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.863+0000 7ff05cfe6740 1 --2- 172.21.15.55:0/766201668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a9ee29a00 0x560a9ee38200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.864+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 >> 172.21.15.55:0/766201668 conn(0x560a9ed6a910 msgr2=0x560a9ee33ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.864+0000 7ff032ffd640 0 client.15672 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:54:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.864+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 shutdown_connections 2024-04-26T21:54:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.864+0000 7ff05cfe6740 1 -- 172.21.15.55:0/766201668 wait complete. 2024-04-26T21:54:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.864+0000 7ff05cfe6740 20 client.15672 trim_cache size 0 max 16384 2024-04-26T21:54:37.867 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:54:37.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.865+0000 7ff058f8e640 1 -- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff03c0a3d40 msgr2=0x7ff03c0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.865+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff03c0a3d40 0x7ff03c0a6200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff04c02abf0 tx=0x7ff04c004a80 comp rx=0 tx=0).stop 2024-04-26T21:54:37.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.865+0000 7ff058f8e640 1 -- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff054101740 msgr2=0x7ff05419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:37.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff054101740 0x7ff05419abd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff04400d770 tx=0x7ff04400dc30 comp rx=0 tx=0).stop 2024-04-26T21:54:37.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 -- 172.21.15.55:0/2646328450 shutdown_connections 2024-04-26T21:54:37.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff03c0a3d40 0x7ff03c0a6200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0541032f0 0x7ff0541a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff054102930 0x7ff05419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 --2- 172.21.15.55:0/2646328450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff054101740 0x7ff05419abd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:37.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 -- 172.21.15.55:0/2646328450 >> 172.21.15.55:0/2646328450 conn(0x7ff0540fccd0 msgr2=0x7ff0540fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:37.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 -- 172.21.15.55:0/2646328450 shutdown_connections 2024-04-26T21:54:37.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:37.866+0000 7ff058f8e640 1 -- 172.21.15.55:0/2646328450 wait complete. 2024-04-26T21:54:37.886 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:54:38.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:37 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3551660020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:38.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:37 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3551660020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:38.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.240+0000 7fe02d859640 1 -- 172.21.15.55:0/38458066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe02810a840 msgr2=0x7fe02810ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.240+0000 7fe02d859640 1 --2- 172.21.15.55:0/38458066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe02810a840 0x7fe02810ccd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe020009ca0 tx=0x7fe02002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.240+0000 7fe02d859640 1 -- 172.21.15.55:0/38458066 shutdown_connections 2024-04-26T21:54:38.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.240+0000 7fe02d859640 1 --2- 172.21.15.55:0/38458066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe02810a840 0x7fe02810ccd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.240+0000 7fe02d859640 1 --2- 172.21.15.55:0/38458066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe028075390 0x7fe028075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.240+0000 7fe02d859640 1 --2- 172.21.15.55:0/38458066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe028076970 0x7fe028074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.240+0000 7fe02d859640 1 -- 172.21.15.55:0/38458066 >> 172.21.15.55:0/38458066 conn(0x7fe0280fdf10 msgr2=0x7fe028100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.240+0000 7fe02d859640 1 -- 172.21.15.55:0/38458066 shutdown_connections 2024-04-26T21:54:38.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.241+0000 7fe02d859640 1 -- 172.21.15.55:0/38458066 wait complete. 2024-04-26T21:54:38.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.242+0000 7fe02d859640 1 Processor -- start 2024-04-26T21:54:38.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.243+0000 7fe02d859640 1 -- start start 2024-04-26T21:54:38.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.244+0000 7fe02d859640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe028075390 0x7fe02819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.244+0000 7fe02d859640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe028076970 0x7fe02819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.244+0000 7fe02d859640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe02810a840 0x7fe0281a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.244+0000 7fe02d859640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281107c0 con 0x7fe028075390 2024-04-26T21:54:38.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.244+0000 7fe02d859640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe028110640 con 0x7fe028076970 2024-04-26T21:54:38.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.245+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe028075390 0x7fe02819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.245+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe028075390 0x7fe02819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58736/0 (socket says 172.21.15.55:58736) 2024-04-26T21:54:38.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.245+0000 7fe026ffd640 1 -- 172.21.15.55:0/1658830930 learned_addr learned my addr 172.21.15.55:0/1658830930 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:38.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.245+0000 7fe0277fe640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe02810a840 0x7fe0281a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.245+0000 7fe0267fc640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe028076970 0x7fe02819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.245+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe028110940 con 0x7fe02810a840 2024-04-26T21:54:38.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.246+0000 7fe026ffd640 1 -- 172.21.15.55:0/1658830930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe02810a840 msgr2=0x7fe0281a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.246+0000 7fe026ffd640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe02810a840 0x7fe0281a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.246+0000 7fe026ffd640 1 -- 172.21.15.55:0/1658830930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe028076970 msgr2=0x7fe02819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.246+0000 7fe026ffd640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe028076970 0x7fe02819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.246+0000 7fe026ffd640 1 -- 172.21.15.55:0/1658830930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0281a2ae0 con 0x7fe028075390 2024-04-26T21:54:38.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.247+0000 7fe026ffd640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe028075390 0x7fe02819aec0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe01800b4a0 tx=0x7fe01800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.248+0000 7fe02c857640 1 -- 172.21.15.55:0/1658830930 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018015020 con 0x7fe028075390 2024-04-26T21:54:38.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.248+0000 7fe02c857640 1 -- 172.21.15.55:0/1658830930 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe018011ce0 con 0x7fe028075390 2024-04-26T21:54:38.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.248+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0281a2dd0 con 0x7fe028075390 2024-04-26T21:54:38.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.248+0000 7fe02c857640 1 -- 172.21.15.55:0/1658830930 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe018010dc0 con 0x7fe028075390 2024-04-26T21:54:38.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.249+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0281a3310 con 0x7fe028075390 2024-04-26T21:54:38.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.255+0000 7fe02c857640 1 -- 172.21.15.55:0/1658830930 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe018018020 con 0x7fe028075390 2024-04-26T21:54:38.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.256+0000 7fe030573740 1 Processor -- start 2024-04-26T21:54:38.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.256+0000 7fe030573740 1 -- start start 2024-04-26T21:54:38.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:37 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3551660020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.257+0000 7fe030573740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d2622820 0x55a3d2622c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.257+0000 7fe030573740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26239f0 0x55a3d2623e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.257+0000 7fe030573740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d262ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.257+0000 7fe030573740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a3d2586d80 con 0x55a3d26243b0 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.257+0000 7fe030573740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a3d2540a20 con 0x55a3d2622820 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.257+0000 7fe030573740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a3d2582d40 con 0x55a3d26239f0 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe02c857640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0100a3d40 0x7fe0100a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe0277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d2622820 0x55a3d2622c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe0277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d2622820 0x55a3d2622c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:33158/0 (socket says 172.21.15.55:33158) 2024-04-26T21:54:38.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe0277fe640 1 -- 172.21.15.55:0/234955075 learned_addr learned my addr 172.21.15.55:0/234955075 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:38.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe0267fc640 1 --2- 172.21.15.55:0/234955075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d262ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe0277fe640 1 -- 172.21.15.55:0/234955075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26239f0 msgr2=0x55a3d2623e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe0277fe640 1 --2- 172.21.15.55:0/234955075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26239f0 0x55a3d2623e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe0277fe640 1 -- 172.21.15.55:0/234955075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 msgr2=0x55a3d262ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.258+0000 7fe0277fe640 1 --2- 172.21.15.55:0/234955075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d262ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.259+0000 7fe0277fe640 1 -- 172.21.15.55:0/234955075 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d254bc30 con 0x55a3d2622820 2024-04-26T21:54:38.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.259+0000 7fe0277fe640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0100a3d40 0x7fe0100a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.259+0000 7fe02c857640 1 -- 172.21.15.55:0/1658830930 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe0180cfce0 con 0x7fe028075390 2024-04-26T21:54:38.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.260+0000 7fe0277fe640 1 --2- 172.21.15.55:0/234955075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d2622820 0x55a3d2622c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe020009ca0 tx=0x7fe020032c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884e75c78cf80a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.260+0000 7fe0277fe640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0100a3d40 0x7fe0100a6200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe020035000 tx=0x7fe020047000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.260+0000 7fe00dffb640 1 -- 172.21.15.55:0/234955075 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020003c20 con 0x55a3d2622820 2024-04-26T21:54:38.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.261+0000 7fe00dffb640 1 -- 172.21.15.55:0/234955075 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe020036040 con 0x55a3d2622820 2024-04-26T21:54:38.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.261+0000 7fe00dffb640 1 -- 172.21.15.55:0/234955075 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020030c20 con 0x55a3d2622820 2024-04-26T21:54:38.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 -- 172.21.15.55:0/234955075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d2622820 msgr2=0x55a3d2622c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 --2- 172.21.15.55:0/234955075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d2622820 0x55a3d2622c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe020009ca0 tx=0x7fe020032c00 comp rx=0 tx=0).stop 2024-04-26T21:54:38.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 -- 172.21.15.55:0/234955075 shutdown_connections 2024-04-26T21:54:38.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 --2- 172.21.15.55:0/234955075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d262ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 --2- 172.21.15.55:0/234955075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26239f0 0x55a3d2623e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 --2- 172.21.15.55:0/234955075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d2622820 0x55a3d2622c40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 -- 172.21.15.55:0/234955075 >> 172.21.15.55:0/234955075 conn(0x55a3d2563910 msgr2=0x55a3d261e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 -- 172.21.15.55:0/234955075 shutdown_connections 2024-04-26T21:54:38.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.263+0000 7fe030573740 1 -- 172.21.15.55:0/234955075 wait complete. 2024-04-26T21:54:38.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.264+0000 7fe030573740 1 -- start start 2024-04-26T21:54:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.265+0000 7fe030573740 1 Processor -- start 2024-04-26T21:54:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.265+0000 7fe030573740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 0x55a3d262c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.265+0000 7fe030573740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.265+0000 7fe030573740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26243b0 0x55a3d2633f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.265+0000 7fe030573740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a3d262b1e0 con 0x55a3d26239f0 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.266+0000 7fe030573740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a3d250ddc0 con 0x55a3d26243b0 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.266+0000 7fe030573740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a3d262b360 con 0x55a3d2622820 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.266+0000 7fe030573740 10 client.0 _ll_register_callbacks cb 0x55a3d2529600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.266+0000 7fe0267fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.266+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 0x55a3d262c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.266+0000 7fe0267fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58760/0 (socket says 172.21.15.55:58760) 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.266+0000 7fe0267fc640 1 -- 172.21.15.55:0/1220326241 learned_addr learned my addr 172.21.15.55:0/1220326241 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe0267fc640 1 -- 172.21.15.55:0/1220326241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 msgr2=0x55a3d262c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe0267fc640 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 0x55a3d262c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe0267fc640 1 -- 172.21.15.55:0/1220326241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26243b0 msgr2=0x55a3d2633f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe0267fc640 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26243b0 0x55a3d2633f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe0267fc640 1 -- 172.21.15.55:0/1220326241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d22983f0 con 0x55a3d26239f0 2024-04-26T21:54:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe0267fc640 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009f30 tx=0x7fe01c0116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab10a852f906ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe006ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe006ffd640 1 -- 172.21.15.55:0/1220326241 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c019070 con 0x55a3d26239f0 2024-04-26T21:54:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe006ffd640 1 -- 172.21.15.55:0/1220326241 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe01c015870 con 0x55a3d26239f0 2024-04-26T21:54:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.267+0000 7fe030573740 1 -- 172.21.15.55:0/1220326241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe020036040 con 0x55a3d26239f0 2024-04-26T21:54:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.268+0000 7fe006ffd640 1 -- 172.21.15.55:0/1220326241 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe01c025050 con 0x55a3d26239f0 2024-04-26T21:54:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.268+0000 7fe030573740 10 client.15684 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.268+0000 7fe030573740 1 -- 172.21.15.55:0/1220326241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a3d23627d0 con 0x55a3d26239f0 2024-04-26T21:54:38.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.268+0000 7fe006ffd640 1 -- 172.21.15.55:0/1220326241 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe020036040 con 0x55a3d26239f0 2024-04-26T21:54:38.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.268+0000 7fe006ffd640 1 -- 172.21.15.55:0/1220326241 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe01c014a40 con 0x55a3d26239f0 2024-04-26T21:54:38.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.269+0000 7fe030573740 10 client.15684 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:38.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.269+0000 7fe030573740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:38.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.269+0000 7fe030573740 20 client.15684 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:38.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.269+0000 7fe030573740 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55a3d2638880 0x55a3d263acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.269+0000 7fe030573740 4 client.15684 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:38.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.269+0000 7fe030573740 1 -- 172.21.15.55:0/1220326241 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3d2594990 con 0x55a3d2638880 2024-04-26T21:54:38.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.344+0000 7fe026ffd640 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55a3d2638880 0x55a3d263acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.345+0000 7fe006ffd640 10 client.15684 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:38.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.345+0000 7fe026ffd640 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55a3d2638880 0x55a3d263acc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b2b990dc9f2bbf server_cookie=65310490d582f34f in_seq=0 out_seq=0 2024-04-26T21:54:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.349+0000 7fe006ffd640 1 -- 172.21.15.55:0/1220326241 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3d2594990 con 0x55a3d2638880 2024-04-26T21:54:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.349+0000 7fe006ffd640 10 client.15684 handle_command_reply: tid=0 2024-04-26T21:54:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.349+0000 7fe030573740 1 client.15684 shutdown 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.349+0000 7fe030573740 1 -- 172.21.15.55:0/1220326241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26239f0 msgr2=0x55a3d262cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.349+0000 7fe030573740 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009f30 tx=0x7fe01c0116f0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 1 -- 172.21.15.55:0/1220326241 shutdown_connections 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26243b0 0x55a3d2633f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55a3d2638880 0x55a3d263acc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 1 --2- 172.21.15.55:0/1220326241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 0x55a3d262c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 1 -- 172.21.15.55:0/1220326241 >> 172.21.15.55:0/1220326241 conn(0x55a3d2563910 msgr2=0x55a3d261d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe006ffd640 0 client.15684 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 1 -- 172.21.15.55:0/1220326241 shutdown_connections 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 1 -- 172.21.15.55:0/1220326241 wait complete. 2024-04-26T21:54:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.350+0000 7fe030573740 20 client.15684 trim_cache size 0 max 16384 2024-04-26T21:54:38.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.378+0000 7fe030573740 1 Processor -- start 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.379+0000 7fe030573740 1 -- start start 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.379+0000 7fe030573740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 0x55a3d262c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.379+0000 7fe030573740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.379+0000 7fe030573740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d2633ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.380+0000 7fe030573740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281aeb70 con 0x55a3d26243b0 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.380+0000 7fe030573740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281ac440 con 0x55a3d26239f0 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.380+0000 7fe030573740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281aa290 con 0x55a3d2622820 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.380+0000 7fe0277fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d2633ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.380+0000 7fe0277fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d2633ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58766/0 (socket says 172.21.15.55:58766) 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.380+0000 7fe0277fe640 1 -- 172.21.15.55:0/2105021107 learned_addr learned my addr 172.21.15.55:0/2105021107 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:38.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.380+0000 7fe0267fc640 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.380+0000 7fe026ffd640 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 0x55a3d262c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.381+0000 7fe0277fe640 1 -- 172.21.15.55:0/2105021107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 msgr2=0x55a3d262c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.381+0000 7fe0277fe640 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 0x55a3d262c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.381+0000 7fe0277fe640 1 -- 172.21.15.55:0/2105021107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 msgr2=0x55a3d262cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.381+0000 7fe0277fe640 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.381+0000 7fe0277fe640 1 -- 172.21.15.55:0/2105021107 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3d262d850 con 0x55a3d26243b0 2024-04-26T21:54:38.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.381+0000 7fe0277fe640 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d2633ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe02002ade0 tx=0x7fe020003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee57a0b7bcae3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.383+0000 7fe00d7fa640 1 -- 172.21.15.55:0/2105021107 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02005d070 con 0x55a3d26243b0 2024-04-26T21:54:38.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.384+0000 7fe00d7fa640 1 -- 172.21.15.55:0/2105021107 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe020054040 con 0x55a3d26243b0 2024-04-26T21:54:38.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.384+0000 7fe00d7fa640 1 -- 172.21.15.55:0/2105021107 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020030a20 con 0x55a3d26243b0 2024-04-26T21:54:38.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.386+0000 7fe030573740 1 -- 172.21.15.55:0/2105021107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 msgr2=0x55a3d2633ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.386+0000 7fe030573740 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d2633ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe02002ade0 tx=0x7fe020003a20 comp rx=0 tx=0).stop 2024-04-26T21:54:38.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.386+0000 7fe030573740 1 -- 172.21.15.55:0/2105021107 shutdown_connections 2024-04-26T21:54:38.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.386+0000 7fe030573740 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d26243b0 0x55a3d2633ef0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.386+0000 7fe030573740 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d262cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.386+0000 7fe030573740 1 --2- 172.21.15.55:0/2105021107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d2622820 0x55a3d262c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.387+0000 7fe030573740 1 -- 172.21.15.55:0/2105021107 >> 172.21.15.55:0/2105021107 conn(0x55a3d2563910 msgr2=0x55a3d26190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.387+0000 7fe030573740 1 -- 172.21.15.55:0/2105021107 shutdown_connections 2024-04-26T21:54:38.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.387+0000 7fe030573740 1 -- 172.21.15.55:0/2105021107 wait complete. 2024-04-26T21:54:38.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.388+0000 7fe030573740 1 -- start start 2024-04-26T21:54:38.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.388+0000 7fe030573740 1 Processor -- start 2024-04-26T21:54:38.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.389+0000 7fe030573740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d2622820 0x55a3d2633da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.389+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d2622820 0x55a3d2633da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.389+0000 7fe026ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d2622820 0x55a3d2633da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58778/0 (socket says 172.21.15.55:58778) 2024-04-26T21:54:38.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.389+0000 7fe026ffd640 1 -- 172.21.15.55:0/2169509655 learned_addr learned my addr 172.21.15.55:0/2169509655 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:38.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.389+0000 7fe030573740 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d2630890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.389+0000 7fe030573740 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26243b0 0x55a3d2630dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.390+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281aa900 con 0x55a3d2622820 2024-04-26T21:54:38.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.390+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281abd70 con 0x55a3d26239f0 2024-04-26T21:54:38.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.390+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0281aafd0 con 0x55a3d26243b0 2024-04-26T21:54:38.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.390+0000 7fe030573740 10 client.0 _ll_register_callbacks cb 0x7fe01c0094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:38.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.390+0000 7fe0267fc640 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d2630890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.390+0000 7fe0277fe640 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26243b0 0x55a3d2630dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.391+0000 7fe0267fc640 1 -- 172.21.15.55:0/2169509655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26243b0 msgr2=0x55a3d2630dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.391+0000 7fe0267fc640 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26243b0 0x55a3d2630dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.391+0000 7fe0267fc640 1 -- 172.21.15.55:0/2169509655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d2622820 msgr2=0x55a3d2633da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:38.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.391+0000 7fe0267fc640 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d2622820 0x55a3d2633da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.391+0000 7fe0267fc640 1 -- 172.21.15.55:0/2169509655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3d269cdf0 con 0x55a3d26239f0 2024-04-26T21:54:38.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.392+0000 7fe0267fc640 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d2630890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe01c014e90 tx=0x7fe01c011d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ab2bfe5cfefc67 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:38.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.392+0000 7fe006ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:54:38.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.392+0000 7fe006ffd640 1 -- 172.21.15.55:0/2169509655 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c02f070 con 0x55a3d26239f0 2024-04-26T21:54:38.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.392+0000 7fe006ffd640 1 -- 172.21.15.55:0/2169509655 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe01c005dd0 con 0x55a3d26239f0 2024-04-26T21:54:38.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.393+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe020054040 con 0x55a3d26239f0 2024-04-26T21:54:38.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.396+0000 7fe006ffd640 1 -- 172.21.15.55:0/2169509655 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe01c039050 con 0x55a3d26239f0 2024-04-26T21:54:38.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.397+0000 7fe030573740 10 client.25295 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:38.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.397+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a3d269e000 con 0x55a3d26239f0 2024-04-26T21:54:38.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.400+0000 7fe006ffd640 1 -- 172.21.15.55:0/2169509655 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe01c011e60 con 0x55a3d26239f0 2024-04-26T21:54:38.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.400+0000 7fe006ffd640 1 -- 172.21.15.55:0/2169509655 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe01c015bf0 con 0x55a3d26239f0 2024-04-26T21:54:38.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.401+0000 7fe030573740 10 client.25295 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:38.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.401+0000 7fe030573740 10 client.25295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:38.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.401+0000 7fe030573740 20 client.25295 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:38.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.401+0000 7fe030573740 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55a3d269e9e0 0x55a3d26a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.401+0000 7fe030573740 4 client.25295 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:54:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.401+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe020054040 con 0x55a3d269e9e0 2024-04-26T21:54:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.415+0000 7fe026ffd640 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55a3d269e9e0 0x55a3d26a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.417+0000 7fe026ffd640 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55a3d269e9e0 0x55a3d26a0e20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd834e59791c146 server_cookie=e72a9cbca089833e in_seq=0 out_seq=0 2024-04-26T21:54:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.421+0000 7fe006ffd640 10 client.25295 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.421+0000 7fe006ffd640 1 -- 172.21.15.55:0/2169509655 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe020054040 con 0x55a3d269e9e0 2024-04-26T21:54:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.421+0000 7fe006ffd640 10 client.25295 handle_command_reply: tid=0 2024-04-26T21:54:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.421+0000 7fe030573740 1 client.25295 shutdown 2024-04-26T21:54:38.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.422+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 msgr2=0x55a3d2630890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.422+0000 7fe030573740 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d2630890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe01c014e90 tx=0x7fe01c011d30 comp rx=0 tx=0).stop 2024-04-26T21:54:38.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.428+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 shutdown_connections 2024-04-26T21:54:38.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.428+0000 7fe030573740 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a3d26243b0 0x55a3d2630dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.428+0000 7fe030573740 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a3d26239f0 0x55a3d2630890 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.428+0000 7fe030573740 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55a3d269e9e0 0x55a3d26a0e20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.429+0000 7fe030573740 1 --2- 172.21.15.55:0/2169509655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a3d2622820 0x55a3d2633da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.429+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 >> 172.21.15.55:0/2169509655 conn(0x55a3d2563910 msgr2=0x55a3d262c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.429+0000 7fe006ffd640 0 client.25295 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:38.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.430+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 shutdown_connections 2024-04-26T21:54:38.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.430+0000 7fe030573740 1 -- 172.21.15.55:0/2169509655 wait complete. 2024-04-26T21:54:38.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.430+0000 7fe030573740 20 client.25295 trim_cache size 0 max 16384 2024-04-26T21:54:38.433 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:54:38.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.450+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0100a3d40 msgr2=0x7fe0100a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.450+0000 7fe02d859640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0100a3d40 0x7fe0100a6200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe020035000 tx=0x7fe020047000 comp rx=0 tx=0).stop 2024-04-26T21:54:38.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.451+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe028075390 msgr2=0x7fe02819aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:38.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.451+0000 7fe02d859640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe028075390 0x7fe02819aec0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe01800b4a0 tx=0x7fe01800b8a0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.452+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 shutdown_connections 2024-04-26T21:54:38.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.452+0000 7fe02d859640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0100a3d40 0x7fe0100a6200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.452+0000 7fe02d859640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe02810a840 0x7fe0281a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.452+0000 7fe02d859640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe028076970 0x7fe02819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.452+0000 7fe02d859640 1 --2- 172.21.15.55:0/1658830930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe028075390 0x7fe02819aec0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:38.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.452+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 >> 172.21.15.55:0/1658830930 conn(0x7fe0280fdf10 msgr2=0x7fe028106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:38.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.457+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 shutdown_connections 2024-04-26T21:54:38.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:38.458+0000 7fe02d859640 1 -- 172.21.15.55:0/1658830930 wait complete. 2024-04-26T21:54:38.942 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:38 smithi055 ceph-mon[29221]: pgmap v403: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 615 MiB/s wr, 651 op/s 2024-04-26T21:54:39.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:38 smithi165 ceph-mon[33296]: pgmap v403: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 615 MiB/s wr, 651 op/s 2024-04-26T21:54:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:38 smithi064 ceph-mon[32765]: pgmap v403: 97 pgs: 97 active+clean; 7.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 615 MiB/s wr, 651 op/s 2024-04-26T21:54:39.530 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:54:39.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:54:39.530 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:54:39.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.905+0000 7f174359e640 1 -- 172.21.15.55:0/955271061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f174410a9b0 msgr2=0x7f174410f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.905+0000 7f174359e640 1 --2- 172.21.15.55:0/955271061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f174410a9b0 0x7f174410f270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1738009900 tx=0x7f173802a440 comp rx=0 tx=0).stop 2024-04-26T21:54:39.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.906+0000 7f174359e640 1 -- 172.21.15.55:0/955271061 shutdown_connections 2024-04-26T21:54:39.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.906+0000 7f174359e640 1 --2- 172.21.15.55:0/955271061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f174410a9b0 0x7f174410f270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.906+0000 7f174359e640 1 --2- 172.21.15.55:0/955271061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f174410a010 0x7f174410a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.906+0000 7f174359e640 1 --2- 172.21.15.55:0/955271061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1744073d70 0x7f1744074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.906+0000 7f174359e640 1 -- 172.21.15.55:0/955271061 >> 172.21.15.55:0/955271061 conn(0x7f174406f840 msgr2=0x7f1744071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.907+0000 7f174359e640 1 -- 172.21.15.55:0/955271061 shutdown_connections 2024-04-26T21:54:39.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.908+0000 7f174359e640 1 -- 172.21.15.55:0/955271061 wait complete. 2024-04-26T21:54:39.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.911+0000 7f174359e640 1 Processor -- start 2024-04-26T21:54:39.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.912+0000 7f174359e640 1 -- start start 2024-04-26T21:54:39.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.912+0000 7f174359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1744073d70 0x7f17441a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.912+0000 7f174359e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174410a010 0x7f17441a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.912+0000 7f174359e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f174410a9b0 0x7f17441aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.912+0000 7f174359e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f174411d2c0 con 0x7f174410a010 2024-04-26T21:54:39.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.912+0000 7f174359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f174411d140 con 0x7f1744073d70 2024-04-26T21:54:39.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.912+0000 7f174359e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f174411d440 con 0x7f174410a9b0 2024-04-26T21:54:39.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.913+0000 7f1741d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174410a010 0x7f17441a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.913+0000 7f1741d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174410a010 0x7f17441a7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58794/0 (socket says 172.21.15.55:58794) 2024-04-26T21:54:39.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.913+0000 7f1741d9b640 1 -- 172.21.15.55:0/16378973 learned_addr learned my addr 172.21.15.55:0/16378973 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:39.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.913+0000 7f1742d9d640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f174410a9b0 0x7f17441aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.914+0000 7f174259c640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1744073d70 0x7f17441a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.916+0000 7f1741d9b640 1 -- 172.21.15.55:0/16378973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f174410a9b0 msgr2=0x7f17441aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.916+0000 7f1741d9b640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f174410a9b0 0x7f17441aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.916+0000 7f1741d9b640 1 -- 172.21.15.55:0/16378973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1744073d70 msgr2=0x7f17441a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.916+0000 7f1741d9b640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1744073d70 0x7f17441a79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.916+0000 7f1741d9b640 1 -- 172.21.15.55:0/16378973 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17441af5f0 con 0x7f174410a010 2024-04-26T21:54:39.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.917+0000 7f174259c640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1744073d70 0x7f17441a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:39.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.919+0000 7f1741d9b640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174410a010 0x7f17441a7f10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f173c00ba50 tx=0x7f173c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.921+0000 7f17337fe640 1 -- 172.21.15.55:0/16378973 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c019070 con 0x7f174410a010 2024-04-26T21:54:39.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.921+0000 7f17337fe640 1 -- 172.21.15.55:0/16378973 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f173c00cbe0 con 0x7f174410a010 2024-04-26T21:54:39.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.921+0000 7f17337fe640 1 -- 172.21.15.55:0/16378973 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c0075d0 con 0x7f174410a010 2024-04-26T21:54:39.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.923+0000 7f174359e640 1 -- 172.21.15.55:0/16378973 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17441af8e0 con 0x7f174410a010 2024-04-26T21:54:39.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.923+0000 7f174359e640 1 -- 172.21.15.55:0/16378973 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17441afe30 con 0x7f174410a010 2024-04-26T21:54:39.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.931+0000 7f174b7a8740 1 Processor -- start 2024-04-26T21:54:39.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.937+0000 7f174b7a8740 1 -- start start 2024-04-26T21:54:39.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.942+0000 7f17337fe640 1 -- 172.21.15.55:0/16378973 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f173c014440 con 0x7f174410a010 2024-04-26T21:54:39.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.943+0000 7f17337fe640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f172c0a3e10 0x7f172c0a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.943+0000 7f17337fe640 1 -- 172.21.15.55:0/16378973 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f173c0cf9b0 con 0x7f174410a010 2024-04-26T21:54:39.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.946+0000 7f1742d9d640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f172c0a3e10 0x7f172c0a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.947+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c9ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.947+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671ca1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.947+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 0x55e671ca4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.947+0000 7f174b7a8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e671bf0b00 con 0x55e671ca25f0 2024-04-26T21:54:39.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.947+0000 7f174b7a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e671b8c920 con 0x55e671c9d480 2024-04-26T21:54:39.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.947+0000 7f174b7a8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e671becac0 con 0x55e671c9ca90 2024-04-26T21:54:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.949+0000 7f1741d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671ca1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.951+0000 7f1741d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671ca1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:33194/0 (socket says 172.21.15.55:33194) 2024-04-26T21:54:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.951+0000 7f1741d9b640 1 -- 172.21.15.55:0/569156150 learned_addr learned my addr 172.21.15.55:0/569156150 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.952+0000 7f174259c640 1 --2- 172.21.15.55:0/569156150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c9ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.952+0000 7f174259c640 1 -- 172.21.15.55:0/569156150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 msgr2=0x55e671ca1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.952+0000 7f174259c640 1 --2- 172.21.15.55:0/569156150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671ca1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.952+0000 7f174259c640 1 -- 172.21.15.55:0/569156150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 msgr2=0x55e671ca4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.952+0000 7f174259c640 1 --2- 172.21.15.55:0/569156150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 0x55e671ca4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.952+0000 7f174259c640 1 -- 172.21.15.55:0/569156150 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6719ca7e0 con 0x55e671c9ca90 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.953+0000 7f174259c640 1 --2- 172.21.15.55:0/569156150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c9ceb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1734009d50 tx=0x7f173400e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d0c4d8db13f7b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.953+0000 7f1730ff9640 1 -- 172.21.15.55:0/569156150 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734017070 con 0x55e671c9ca90 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.954+0000 7f1730ff9640 1 -- 172.21.15.55:0/569156150 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1734013920 con 0x55e671c9ca90 2024-04-26T21:54:39.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.954+0000 7f1730ff9640 1 -- 172.21.15.55:0/569156150 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734012820 con 0x55e671c9ca90 2024-04-26T21:54:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.955+0000 7f174b7a8740 1 -- 172.21.15.55:0/569156150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 msgr2=0x55e671c9ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.955+0000 7f174b7a8740 1 --2- 172.21.15.55:0/569156150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c9ceb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1734009d50 tx=0x7f173400e5a0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.958+0000 7f1742d9d640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f172c0a3e10 0x7f172c0a62d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1738006010 tx=0x7f1738005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.961+0000 7f174b7a8740 1 -- 172.21.15.55:0/569156150 shutdown_connections 2024-04-26T21:54:39.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.961+0000 7f174b7a8740 1 --2- 172.21.15.55:0/569156150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 0x55e671ca4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.961+0000 7f174b7a8740 1 --2- 172.21.15.55:0/569156150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671ca1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.961+0000 7f174b7a8740 1 --2- 172.21.15.55:0/569156150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c9ceb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.961+0000 7f174b7a8740 1 -- 172.21.15.55:0/569156150 >> 172.21.15.55:0/569156150 conn(0x55e671c98100 msgr2=0x55e671c98500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:39.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.971+0000 7f174b7a8740 1 -- 172.21.15.55:0/569156150 shutdown_connections 2024-04-26T21:54:39.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.973+0000 7f174b7a8740 1 -- 172.21.15.55:0/569156150 wait complete. 2024-04-26T21:54:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.973+0000 7f174b7a8740 1 -- start start 2024-04-26T21:54:39.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.974+0000 7f174b7a8740 1 Processor -- start 2024-04-26T21:54:39.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.974+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9ca90 0x55e671ca6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.974+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9d480 0x55e671ca7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.974+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cae420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.974+0000 7f174b7a8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e671ca55a0 con 0x55e671c9ca90 2024-04-26T21:54:39.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.974+0000 7f174b7a8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e671b8a5d0 con 0x55e671ca25f0 2024-04-26T21:54:39.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.974+0000 7f174b7a8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e671ca5720 con 0x55e671c9d480 2024-04-26T21:54:39.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.974+0000 7f174b7a8740 10 client.0 _ll_register_callbacks cb 0x55e671b78080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:39.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.979+0000 7f174259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9ca90 0x55e671ca6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.979+0000 7f174259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9ca90 0x55e671ca6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58808/0 (socket says 172.21.15.55:58808) 2024-04-26T21:54:39.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.979+0000 7f174259c640 1 -- 172.21.15.55:0/3210579919 learned_addr learned my addr 172.21.15.55:0/3210579919 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:39.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.979+0000 7f1742d9d640 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.979+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9d480 0x55e671ca7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:39.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.986+0000 7f1742d9d640 1 -- 172.21.15.55:0/3210579919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9d480 msgr2=0x55e671ca7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.986+0000 7f1742d9d640 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9d480 0x55e671ca7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.986+0000 7f1742d9d640 1 -- 172.21.15.55:0/3210579919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9ca90 msgr2=0x55e671ca6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:39.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.986+0000 7f1742d9d640 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9ca90 0x55e671ca6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:39.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.986+0000 7f1742d9d640 1 -- 172.21.15.55:0/3210579919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e671a2af70 con 0x55e671ca25f0 2024-04-26T21:54:39.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.988+0000 7f1742d9d640 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cae420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f173802a960 tx=0x7f173802a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efecf92f9e071e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:39.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.988+0000 7f1729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:54:39.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.989+0000 7f1729ffb640 1 -- 172.21.15.55:0/3210579919 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738035050 con 0x55e671ca25f0 2024-04-26T21:54:39.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.989+0000 7f1729ffb640 1 -- 172.21.15.55:0/3210579919 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f17380319b0 con 0x55e671ca25f0 2024-04-26T21:54:39.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.989+0000 7f174b7a8740 1 -- 172.21.15.55:0/3210579919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1734013920 con 0x55e671ca25f0 2024-04-26T21:54:39.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.990+0000 7f1729ffb640 1 -- 172.21.15.55:0/3210579919 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1738031bb0 con 0x55e671ca25f0 2024-04-26T21:54:39.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.992+0000 7f174b7a8740 10 client.25304 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:39.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.992+0000 7f174b7a8740 1 -- 172.21.15.55:0/3210579919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e6718c2170 con 0x55e671ca25f0 2024-04-26T21:54:39.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.994+0000 7f1729ffb640 1 -- 172.21.15.55:0/3210579919 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1734013920 con 0x55e671ca25f0 2024-04-26T21:54:39.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.995+0000 7f1729ffb640 1 -- 172.21.15.55:0/3210579919 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1738053440 con 0x55e671ca25f0 2024-04-26T21:54:39.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.995+0000 7f174b7a8740 10 client.25304 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:39.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.995+0000 7f174b7a8740 10 client.25304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:39.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.995+0000 7f174b7a8740 20 client.25304 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:39.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.995+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e671cb2cb0 0x55e671cb5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:39.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.995+0000 7f174b7a8740 4 client.25304 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:39.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.995+0000 7f174b7a8740 1 -- 172.21.15.55:0/3210579919 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e671bfe710 con 0x55e671cb2cb0 2024-04-26T21:54:40.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:39.999+0000 7f174259c640 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e671cb2cb0 0x55e671cb5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.014+0000 7f174259c640 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e671cb2cb0 0x55e671cb5140 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5168e513358c4a98 server_cookie=51cae51777e299ca in_seq=0 out_seq=0 2024-04-26T21:54:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.015+0000 7f1729ffb640 10 client.25304 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:40.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.025+0000 7f1729ffb640 1 -- 172.21.15.55:0/3210579919 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e671bfe710 con 0x55e671cb2cb0 2024-04-26T21:54:40.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.026+0000 7f1729ffb640 10 client.25304 handle_command_reply: tid=0 2024-04-26T21:54:40.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.026+0000 7f174b7a8740 1 client.25304 shutdown 2024-04-26T21:54:40.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.027+0000 7f174b7a8740 1 -- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 msgr2=0x55e671cae420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:40.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.027+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cae420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f173802a960 tx=0x7f173802a440 comp rx=0 tx=0).stop 2024-04-26T21:54:40.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.028+0000 7f174b7a8740 1 -- 172.21.15.55:0/3210579919 shutdown_connections 2024-04-26T21:54:40.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.028+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cae420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.028+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9d480 0x55e671ca7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.028+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e671cb2cb0 0x55e671cb5140 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.028+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3210579919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9ca90 0x55e671ca6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.028+0000 7f174b7a8740 1 -- 172.21.15.55:0/3210579919 >> 172.21.15.55:0/3210579919 conn(0x55e671c98100 msgr2=0x55e671c862f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:40.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.029+0000 7f1729ffb640 0 client.25304 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:40.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.033+0000 7f174b7a8740 1 -- 172.21.15.55:0/3210579919 shutdown_connections 2024-04-26T21:54:40.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.033+0000 7f174b7a8740 1 -- 172.21.15.55:0/3210579919 wait complete. 2024-04-26T21:54:40.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.035+0000 7f174b7a8740 20 client.25304 trim_cache size 0 max 16384 2024-04-26T21:54:40.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.099+0000 7f174b7a8740 1 Processor -- start 2024-04-26T21:54:40.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.104+0000 7f174b7a8740 1 -- start start 2024-04-26T21:54:40.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.104+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c85a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:40.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.104+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671cae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:40.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.105+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 0x55e671c85fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:40.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.105+0000 7f174259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c85a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.105+0000 7f174259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c85a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48536/0 (socket says 172.21.15.55:48536) 2024-04-26T21:54:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.105+0000 7f174259c640 1 -- 172.21.15.55:0/3329510559 learned_addr learned my addr 172.21.15.55:0/3329510559 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.105+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671cae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.106+0000 7f174b7a8740 1 -- 172.21.15.55:0/3329510559 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f17441b2c20 con 0x55e671ca25f0 2024-04-26T21:54:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.106+0000 7f174b7a8740 1 -- 172.21.15.55:0/3329510559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17441b04f0 con 0x55e671c9d480 2024-04-26T21:54:40.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.106+0000 7f174b7a8740 1 -- 172.21.15.55:0/3329510559 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1744117bc0 con 0x55e671c9ca90 2024-04-26T21:54:40.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.106+0000 7f1742d9d640 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 0x55e671c85fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.107+0000 7f1741d9b640 1 -- 172.21.15.55:0/3329510559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 msgr2=0x55e671c85a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.107+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c85a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.107+0000 7f1741d9b640 1 -- 172.21.15.55:0/3329510559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 msgr2=0x55e671c85fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.107+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 0x55e671c85fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.107+0000 7f1741d9b640 1 -- 172.21.15.55:0/3329510559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e671cb57c0 con 0x55e671c9d480 2024-04-26T21:54:40.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.108+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671cae3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f173c0980b0 tx=0x7f173c0cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e712a914d7ee1c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:40.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.111+0000 7f172bfff640 1 -- 172.21.15.55:0/3329510559 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c09ff50 con 0x55e671c9d480 2024-04-26T21:54:40.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.111+0000 7f172bfff640 1 -- 172.21.15.55:0/3329510559 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f173c0a4810 con 0x55e671c9d480 2024-04-26T21:54:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.111+0000 7f172bfff640 1 -- 172.21.15.55:0/3329510559 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c0cd510 con 0x55e671c9d480 2024-04-26T21:54:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.113+0000 7f174b7a8740 1 -- 172.21.15.55:0/3329510559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 msgr2=0x55e671cae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.113+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671cae3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f173c0980b0 tx=0x7f173c0cf040 comp rx=0 tx=0).stop 2024-04-26T21:54:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.113+0000 7f174b7a8740 1 -- 172.21.15.55:0/3329510559 shutdown_connections 2024-04-26T21:54:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.113+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671ca25f0 0x55e671c85fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.113+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671c9d480 0x55e671cae3e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.113+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3329510559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671c85a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.113+0000 7f174b7a8740 1 -- 172.21.15.55:0/3329510559 >> 172.21.15.55:0/3329510559 conn(0x55e671c98100 msgr2=0x55e671c9e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:40.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.117+0000 7f174b7a8740 1 -- 172.21.15.55:0/3329510559 shutdown_connections 2024-04-26T21:54:40.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.117+0000 7f174b7a8740 1 -- 172.21.15.55:0/3329510559 wait complete. 2024-04-26T21:54:40.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.118+0000 7f174b7a8740 1 -- start start 2024-04-26T21:54:40.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.119+0000 7f174b7a8740 1 Processor -- start 2024-04-26T21:54:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.119+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671cac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.120+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9d480 0x55e671cac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.120+0000 7f174b7a8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.120+0000 7f174b7a8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1744118210 con 0x55e671c9d480 2024-04-26T21:54:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.120+0000 7f174259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671cac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.120+0000 7f174259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671cac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48544/0 (socket says 172.21.15.55:48544) 2024-04-26T21:54:40.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.120+0000 7f174259c640 1 -- 172.21.15.55:0/3522125750 learned_addr learned my addr 172.21.15.55:0/3522125750 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.120+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1744119680 con 0x55e671ca25f0 2024-04-26T21:54:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.120+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f17441188e0 con 0x55e671c9ca90 2024-04-26T21:54:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.121+0000 7f1742d9d640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.121+0000 7f174b7a8740 10 client.0 _ll_register_callbacks cb 0x7f1738004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:40.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.123+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9d480 0x55e671cac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:40.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.123+0000 7f174259c640 1 -- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 msgr2=0x55e671cad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:40.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.123+0000 7f174259c640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.123+0000 7f174259c640 1 -- 172.21.15.55:0/3522125750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9d480 msgr2=0x55e671cac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:40.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.123+0000 7f174259c640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9d480 0x55e671cac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:40.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.123+0000 7f174259c640 1 -- 172.21.15.55:0/3522125750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e671c97f30 con 0x55e671c9ca90 2024-04-26T21:54:40.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.123+0000 7f1742d9d640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:54:40.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.123+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9d480 0x55e671cac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:40.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.127+0000 7f174259c640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671cac2a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1734006fd0 tx=0x7f1734004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bef4953e5e5079 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:40.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.127+0000 7f1729ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:54:40.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.130+0000 7f1729ffb640 1 -- 172.21.15.55:0/3522125750 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734017040 con 0x55e671c9ca90 2024-04-26T21:54:40.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.130+0000 7f1729ffb640 1 -- 172.21.15.55:0/3522125750 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f17340045c0 con 0x55e671c9ca90 2024-04-26T21:54:40.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.130+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f173c0a4810 con 0x55e671c9ca90 2024-04-26T21:54:40.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.132+0000 7f1729ffb640 1 -- 172.21.15.55:0/3522125750 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1734037050 con 0x55e671c9ca90 2024-04-26T21:54:40.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.132+0000 7f174b7a8740 10 client.25255 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:40.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.132+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e671d198e0 con 0x55e671c9ca90 2024-04-26T21:54:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.133+0000 7f1729ffb640 1 -- 172.21.15.55:0/3522125750 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f173c0a4810 con 0x55e671c9ca90 2024-04-26T21:54:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.133+0000 7f1729ffb640 1 -- 172.21.15.55:0/3522125750 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f1734013470 con 0x55e671c9ca90 2024-04-26T21:54:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.133+0000 7f174b7a8740 10 client.25255 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.133+0000 7f174b7a8740 10 client.25255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.133+0000 7f174b7a8740 20 client.25255 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.133+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e671d1a080 0x55e671d1c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.133+0000 7f174b7a8740 4 client.25255 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:54:40.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.133+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e671bf8580 con 0x55e671d1a080 2024-04-26T21:54:40.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.136+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e671d1a080 0x55e671d1c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:40.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.140+0000 7f1729ffb640 10 client.25255 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:40.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:40.140+0000 7f1741d9b640 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e671d1a080 0x55e671d1c4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89e628fea3af830 server_cookie=b9f15ef2435e5585 in_seq=0 out_seq=0 2024-04-26T21:54:41.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:40 smithi165 ceph-mon[33296]: pgmap v404: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 776 MiB/s wr, 1.07k op/s 2024-04-26T21:54:41.191 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:40 smithi055 ceph-mon[29221]: pgmap v404: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 776 MiB/s wr, 1.07k op/s 2024-04-26T21:54:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:40 smithi064 ceph-mon[32765]: pgmap v404: 97 pgs: 97 active+clean; 8.6 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 776 MiB/s wr, 1.07k op/s 2024-04-26T21:54:41.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.684+0000 7f1729ffb640 1 -- 172.21.15.55:0/3522125750 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e671bf8580 con 0x55e671d1a080 2024-04-26T21:54:41.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.684+0000 7f1729ffb640 10 client.25255 handle_command_reply: tid=0 2024-04-26T21:54:41.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.685+0000 7f174b7a8740 1 client.25255 shutdown 2024-04-26T21:54:41.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 msgr2=0x55e671cac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:41.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671cac2a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1734006fd0 tx=0x7f1734004040 comp rx=0 tx=0).stop 2024-04-26T21:54:41.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 shutdown_connections 2024-04-26T21:54:41.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e671ca25f0 0x55e671cad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:41.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e671d1a080 0x55e671d1c4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:41.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e671c9d480 0x55e671cac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:41.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f174b7a8740 1 --2- 172.21.15.55:0/3522125750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e671c9ca90 0x55e671cac2a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:41.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 >> 172.21.15.55:0/3522125750 conn(0x55e671c98100 msgr2=0x55e671c85a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:41.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.691+0000 7f1729ffb640 0 client.25255 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:41.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.692+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 shutdown_connections 2024-04-26T21:54:41.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.692+0000 7f174b7a8740 1 -- 172.21.15.55:0/3522125750 wait complete. 2024-04-26T21:54:41.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.692+0000 7f174b7a8740 20 client.25255 trim_cache size 0 max 16384 2024-04-26T21:54:41.696 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:54:41.696 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:54:41.697 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "4fd2155a-da1a-42a5-9552-550d11ae1ade", 2024-04-26T21:54:41.697 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:54:41.697 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:54:41.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.708+0000 7f174359e640 1 -- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f172c0a3e10 msgr2=0x7f172c0a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:41.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.708+0000 7f174359e640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f172c0a3e10 0x7f172c0a62d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1738006010 tx=0x7f1738005e80 comp rx=0 tx=0).stop 2024-04-26T21:54:41.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.708+0000 7f174359e640 1 -- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174410a010 msgr2=0x7f17441a7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:41.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.708+0000 7f174359e640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174410a010 0x7f17441a7f10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f173c00ba50 tx=0x7f173c00bf10 comp rx=0 tx=0).stop 2024-04-26T21:54:41.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.714+0000 7f174359e640 1 -- 172.21.15.55:0/16378973 shutdown_connections 2024-04-26T21:54:41.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.714+0000 7f174359e640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f172c0a3e10 0x7f172c0a62d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:41.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.714+0000 7f174359e640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f174410a9b0 0x7f17441aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:41.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.714+0000 7f174359e640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f174410a010 0x7f17441a7f10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:41.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.714+0000 7f174359e640 1 --2- 172.21.15.55:0/16378973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1744073d70 0x7f17441a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:41.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.714+0000 7f174359e640 1 -- 172.21.15.55:0/16378973 >> 172.21.15.55:0/16378973 conn(0x7f174406f840 msgr2=0x7f17440714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:41.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.718+0000 7f174359e640 1 -- 172.21.15.55:0/16378973 shutdown_connections 2024-04-26T21:54:41.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:41.718+0000 7f174359e640 1 -- 172.21.15.55:0/16378973 wait complete. 2024-04-26T21:54:41.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:54:41.786 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:54:42.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.166+0000 7f6b334e4640 1 -- 172.21.15.55:0/3589773119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c10c380 msgr2=0x7f6b2c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.166+0000 7f6b334e4640 1 --2- 172.21.15.55:0/3589773119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c10c380 0x7f6b2c10e790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00af00 tx=0x7f6b1c032790 comp rx=0 tx=0).stop 2024-04-26T21:54:42.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.167+0000 7f6b334e4640 1 -- 172.21.15.55:0/3589773119 shutdown_connections 2024-04-26T21:54:42.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.167+0000 7f6b334e4640 1 --2- 172.21.15.55:0/3589773119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2c10ee20 0x7f6b2c1112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.167+0000 7f6b334e4640 1 --2- 172.21.15.55:0/3589773119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c10c380 0x7f6b2c10e790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.167+0000 7f6b334e4640 1 --2- 172.21.15.55:0/3589773119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b2c1099c0 0x7f6b2c10bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.167+0000 7f6b334e4640 1 -- 172.21.15.55:0/3589773119 >> 172.21.15.55:0/3589773119 conn(0x7f6b2c06f990 msgr2=0x7f6b2c071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:42.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.168+0000 7f6b334e4640 1 -- 172.21.15.55:0/3589773119 shutdown_connections 2024-04-26T21:54:42.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.171+0000 7f6b334e4640 1 -- 172.21.15.55:0/3589773119 wait complete. 2024-04-26T21:54:42.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.172+0000 7f6b334e4640 1 Processor -- start 2024-04-26T21:54:42.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.172+0000 7f6b334e4640 1 -- start start 2024-04-26T21:54:42.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.173+0000 7f6b334e4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2c1099c0 0x7f6b2c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.173+0000 7f6b334e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c10c380 0x7f6b2c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.173+0000 7f6b334e4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b2c10ee20 0x7f6b2c1a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.173+0000 7f6b334e4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c114870 con 0x7f6b2c10ee20 2024-04-26T21:54:42.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.173+0000 7f6b334e4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c1146f0 con 0x7f6b2c10c380 2024-04-26T21:54:42.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.173+0000 7f6b334e4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c1149f0 con 0x7f6b2c1099c0 2024-04-26T21:54:42.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.175+0000 7f6b32ce3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b2c10ee20 0x7f6b2c1a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.175+0000 7f6b32ce3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b2c10ee20 0x7f6b2c1a6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48994/0 (socket says 172.21.15.55:48994) 2024-04-26T21:54:42.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.175+0000 7f6b32ce3640 1 -- 172.21.15.55:0/328327377 learned_addr learned my addr 172.21.15.55:0/328327377 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:42.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.179+0000 7f6b32ce3640 1 -- 172.21.15.55:0/328327377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2c1099c0 msgr2=0x7f6b2c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.179+0000 7f6b32ce3640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2c1099c0 0x7f6b2c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.179+0000 7f6b32ce3640 1 -- 172.21.15.55:0/328327377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c10c380 msgr2=0x7f6b2c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.179+0000 7f6b32ce3640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c10c380 0x7f6b2c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.179+0000 7f6b32ce3640 1 -- 172.21.15.55:0/328327377 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c1a6ba0 con 0x7f6b2c10ee20 2024-04-26T21:54:42.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.180+0000 7f6b32ce3640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b2c10ee20 0x7f6b2c1a6530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400eaa0 tx=0x7f6b2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:42.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.181+0000 7f6b237fe640 1 -- 172.21.15.55:0/328327377 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400cdb0 con 0x7f6b2c10ee20 2024-04-26T21:54:42.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.181+0000 7f6b334e4640 1 -- 172.21.15.55:0/328327377 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c1a6e90 con 0x7f6b2c10ee20 2024-04-26T21:54:42.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.181+0000 7f6b334e4640 1 -- 172.21.15.55:0/328327377 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c1a73b0 con 0x7f6b2c10ee20 2024-04-26T21:54:42.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.182+0000 7f6b237fe640 1 -- 172.21.15.55:0/328327377 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b24005020 con 0x7f6b2c10ee20 2024-04-26T21:54:42.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.182+0000 7f6b237fe640 1 -- 172.21.15.55:0/328327377 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b240116d0 con 0x7f6b2c10ee20 2024-04-26T21:54:42.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.191+0000 7f6b237fe640 1 -- 172.21.15.55:0/328327377 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b24011870 con 0x7f6b2c10ee20 2024-04-26T21:54:42.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.191+0000 7f6b37464740 1 Processor -- start 2024-04-26T21:54:42.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.192+0000 7f6b37464740 1 -- start start 2024-04-26T21:54:42.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.192+0000 7f6b37464740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bc0b0 0x5568fd5bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.192+0000 7f6b37464740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.192+0000 7f6b37464740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bdc40 0x5568fd5c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.192+0000 7f6b37464740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5568fd520d80 con 0x5568fd5bc0b0 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.192+0000 7f6b37464740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568fd4da8b0 con 0x5568fd5bdc40 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.192+0000 7f6b37464740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5568fd51cd40 con 0x5568fd5bd280 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.193+0000 7f6b324e2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.193+0000 7f6b324e2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42436/0 (socket says 172.21.15.55:42436) 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.193+0000 7f6b324e2640 1 -- 172.21.15.55:0/3180511972 learned_addr learned my addr 172.21.15.55:0/3180511972 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.193+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/3180511972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bdc40 0x5568fd5c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.193+0000 7f6b237fe640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b180a3e10 0x7f6b180a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.194+0000 7f6b32ce3640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b180a3e10 0x7f6b180a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.196+0000 7f6b237fe640 1 -- 172.21.15.55:0/328327377 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f6b240d1700 con 0x7f6b2c10ee20 2024-04-26T21:54:42.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.201+0000 7f6b324e2640 1 -- 172.21.15.55:0/3180511972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bdc40 msgr2=0x5568fd5c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:42.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.201+0000 7f6b324e2640 1 --2- 172.21.15.55:0/3180511972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bdc40 0x5568fd5c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.201+0000 7f6b324e2640 1 -- 172.21.15.55:0/3180511972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bc0b0 msgr2=0x5568fd5bc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:42.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.201+0000 7f6b324e2640 1 --2- 172.21.15.55:0/3180511972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bc0b0 0x5568fd5bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.201+0000 7f6b324e2640 1 -- 172.21.15.55:0/3180511972 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568fd4aba40 con 0x5568fd5bd280 2024-04-26T21:54:42.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.204+0000 7f6b32ce3640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b180a3e10 0x7f6b180a62d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b24016000 tx=0x7f6b24013000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:42.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.209+0000 7f6b324e2640 1 --2- 172.21.15.55:0/3180511972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5bd700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800baf0 tx=0x7f6b2800a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1a5fa9269367b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:42.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.209+0000 7f6b20ff9640 1 -- 172.21.15.55:0/3180511972 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800ac90 con 0x5568fd5bd280 2024-04-26T21:54:42.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.209+0000 7f6b20ff9640 1 -- 172.21.15.55:0/3180511972 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b2800ae30 con 0x5568fd5bd280 2024-04-26T21:54:42.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.209+0000 7f6b20ff9640 1 -- 172.21.15.55:0/3180511972 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28014670 con 0x5568fd5bd280 2024-04-26T21:54:42.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.211+0000 7f6b37464740 1 -- 172.21.15.55:0/3180511972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 msgr2=0x5568fd5bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.211+0000 7f6b37464740 1 --2- 172.21.15.55:0/3180511972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5bd700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800baf0 tx=0x7f6b2800a250 comp rx=0 tx=0).stop 2024-04-26T21:54:42.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.211+0000 7f6b37464740 1 -- 172.21.15.55:0/3180511972 shutdown_connections 2024-04-26T21:54:42.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.211+0000 7f6b37464740 1 --2- 172.21.15.55:0/3180511972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bdc40 0x5568fd5c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.211+0000 7f6b37464740 1 --2- 172.21.15.55:0/3180511972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5bd700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.211+0000 7f6b37464740 1 --2- 172.21.15.55:0/3180511972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bc0b0 0x5568fd5bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.211+0000 7f6b37464740 1 -- 172.21.15.55:0/3180511972 >> 172.21.15.55:0/3180511972 conn(0x5568fd5b75a0 msgr2=0x5568fd5b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:42.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.215+0000 7f6b37464740 1 -- 172.21.15.55:0/3180511972 shutdown_connections 2024-04-26T21:54:42.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.215+0000 7f6b37464740 1 -- 172.21.15.55:0/3180511972 wait complete. 2024-04-26T21:54:42.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 1 -- start start 2024-04-26T21:54:42.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 1 Processor -- start 2024-04-26T21:54:42.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bc0b0 0x5568fd5c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bd280 0x5568fd5c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5568fd5c4c50 con 0x5568fd5bdc40 2024-04-26T21:54:42.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568fd4a7d10 con 0x5568fd5bd280 2024-04-26T21:54:42.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5568fd5c4dd0 con 0x5568fd5bc0b0 2024-04-26T21:54:42.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.216+0000 7f6b37464740 10 client.0 _ll_register_callbacks cb 0x5568fd4d4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:42.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.217+0000 7f6b31ce1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.217+0000 7f6b31ce1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49020/0 (socket says 172.21.15.55:49020) 2024-04-26T21:54:42.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.217+0000 7f6b31ce1640 1 -- 172.21.15.55:0/3800500582 learned_addr learned my addr 172.21.15.55:0/3800500582 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:42.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.217+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bc0b0 0x5568fd5c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.217+0000 7f6b324e2640 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bd280 0x5568fd5c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.217+0000 7f6b31ce1640 1 -- 172.21.15.55:0/3800500582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bc0b0 msgr2=0x5568fd5c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:42.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.217+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bc0b0 0x5568fd5c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.218+0000 7f6b31ce1640 1 -- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bd280 msgr2=0x5568fd5c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:42.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.218+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bd280 0x5568fd5c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.218+0000 7f6b31ce1640 1 -- 172.21.15.55:0/3800500582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568fd2fc7d0 con 0x5568fd5bdc40 2024-04-26T21:54:42.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.218+0000 7f6b324e2640 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bd280 0x5568fd5c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:54:42.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.218+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cdac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0748b0 tx=0x7f6b1c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab911700d6d1017 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:42.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.218+0000 7f6b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:54:42.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.219+0000 7f6b15ffb640 1 -- 172.21.15.55:0/3800500582 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c04f060 con 0x5568fd5bdc40 2024-04-26T21:54:42.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.219+0000 7f6b37464740 1 -- 172.21.15.55:0/3800500582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b2800ae30 con 0x5568fd5bdc40 2024-04-26T21:54:42.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.219+0000 7f6b15ffb640 1 -- 172.21.15.55:0/3800500582 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f6b1c004230 con 0x5568fd5bdc40 2024-04-26T21:54:42.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.219+0000 7f6b15ffb640 1 -- 172.21.15.55:0/3800500582 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6b1c004430 con 0x5568fd5bdc40 2024-04-26T21:54:42.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.219+0000 7f6b37464740 10 client.15708 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:42.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.219+0000 7f6b37464740 1 -- 172.21.15.55:0/3800500582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5568fd2323f0 con 0x5568fd5bdc40 2024-04-26T21:54:42.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.220+0000 7f6b15ffb640 1 -- 172.21.15.55:0/3800500582 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b2800ae30 con 0x5568fd5bdc40 2024-04-26T21:54:42.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.220+0000 7f6b15ffb640 1 -- 172.21.15.55:0/3800500582 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6b1c003710 con 0x5568fd5bdc40 2024-04-26T21:54:42.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.221+0000 7f6b37464740 10 client.15708 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:42.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.221+0000 7f6b37464740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:42.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.221+0000 7f6b37464740 20 client.15708 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:42.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.221+0000 7f6b37464740 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5568fd5d2430 0x5568fd5d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.221+0000 7f6b37464740 4 client.15708 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:54:42.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.221+0000 7f6b37464740 1 -- 172.21.15.55:0/3800500582 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568fd52e990 con 0x5568fd5d2430 2024-04-26T21:54:42.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.225+0000 7f6b324e2640 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5568fd5d2430 0x5568fd5d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.229+0000 7f6b15ffb640 10 client.15708 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:42.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.229+0000 7f6b324e2640 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5568fd5d2430 0x5568fd5d48f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe523ab517d3aff9 server_cookie=a81c7654a8ca2b8e in_seq=0 out_seq=0 2024-04-26T21:54:42.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.243+0000 7f6b15ffb640 1 -- 172.21.15.55:0/3800500582 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568fd52e990 con 0x5568fd5d2430 2024-04-26T21:54:42.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.243+0000 7f6b15ffb640 10 client.15708 handle_command_reply: tid=0 2024-04-26T21:54:42.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.243+0000 7f6b37464740 1 client.15708 shutdown 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.244+0000 7f6b37464740 1 -- 172.21.15.55:0/3800500582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 msgr2=0x5568fd5cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.244+0000 7f6b37464740 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cdac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0748b0 tx=0x7f6b1c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.246+0000 7f6b37464740 1 -- 172.21.15.55:0/3800500582 shutdown_connections 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.246+0000 7f6b37464740 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5568fd5d2430 0x5568fd5d48f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.246+0000 7f6b37464740 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cdac0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.246+0000 7f6b37464740 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bd280 0x5568fd5c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.246+0000 7f6b37464740 1 --2- 172.21.15.55:0/3800500582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bc0b0 0x5568fd5c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.246+0000 7f6b37464740 1 -- 172.21.15.55:0/3800500582 >> 172.21.15.55:0/3800500582 conn(0x5568fd5b75a0 msgr2=0x5568fd5b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.246+0000 7f6b15ffb640 0 client.15708 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.247+0000 7f6b37464740 1 -- 172.21.15.55:0/3800500582 shutdown_connections 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.248+0000 7f6b37464740 1 -- 172.21.15.55:0/3800500582 wait complete. 2024-04-26T21:54:42.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.248+0000 7f6b37464740 20 client.15708 trim_cache size 0 max 16384 2024-04-26T21:54:42.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.318+0000 7f6b37464740 1 Processor -- start 2024-04-26T21:54:42.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.318+0000 7f6b37464740 1 -- start start 2024-04-26T21:54:42.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.318+0000 7f6b37464740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 0x5568fd5b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.318+0000 7f6b37464740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bd280 0x5568fd5cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.319+0000 7f6b324e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bd280 0x5568fd5cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.319+0000 7f6b324e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bd280 0x5568fd5cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49030/0 (socket says 172.21.15.55:49030) 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.319+0000 7f6b324e2640 1 -- 172.21.15.55:0/1855367336 learned_addr learned my addr 172.21.15.55:0/1855367336 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.319+0000 7f6b37464740 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bdc40 0x5568fd5d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.319+0000 7f6b37464740 1 -- 172.21.15.55:0/1855367336 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10010f90 con 0x5568fd5bd280 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.319+0000 7f6b37464740 1 -- 172.21.15.55:0/1855367336 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1000e860 con 0x5568fd5bc0b0 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.319+0000 7f6b37464740 1 -- 172.21.15.55:0/1855367336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1000c6b0 con 0x5568fd5bdc40 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.320+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bdc40 0x5568fd5d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.320+0000 7f6b324e2640 1 -- 172.21.15.55:0/1855367336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bdc40 msgr2=0x5568fd5d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.320+0000 7f6b324e2640 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bdc40 0x5568fd5d91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.320+0000 7f6b324e2640 1 -- 172.21.15.55:0/1855367336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 msgr2=0x5568fd5b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:42.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.320+0000 7f6b324e2640 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 0x5568fd5b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.320+0000 7f6b324e2640 1 -- 172.21.15.55:0/1855367336 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568fd1f20f0 con 0x5568fd5bd280 2024-04-26T21:54:42.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.321+0000 7f6b324e2640 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bd280 0x5568fd5cda50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800baf0 tx=0x7f6b28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1942af846d196f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:42.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.321+0000 7f6b17fff640 1 -- 172.21.15.55:0/1855367336 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2800ae30 con 0x5568fd5bd280 2024-04-26T21:54:42.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.321+0000 7f6b17fff640 1 -- 172.21.15.55:0/1855367336 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b28004040 con 0x5568fd5bd280 2024-04-26T21:54:42.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.322+0000 7f6b17fff640 1 -- 172.21.15.55:0/1855367336 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b280050d0 con 0x5568fd5bd280 2024-04-26T21:54:42.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.324+0000 7f6b37464740 1 -- 172.21.15.55:0/1855367336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bd280 msgr2=0x5568fd5cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.324+0000 7f6b37464740 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bd280 0x5568fd5cda50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800baf0 tx=0x7f6b28000bc0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.328+0000 7f6b37464740 1 -- 172.21.15.55:0/1855367336 shutdown_connections 2024-04-26T21:54:42.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.328+0000 7f6b37464740 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bdc40 0x5568fd5d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.328+0000 7f6b37464740 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bd280 0x5568fd5cda50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.328+0000 7f6b37464740 1 --2- 172.21.15.55:0/1855367336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 0x5568fd5b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.328+0000 7f6b37464740 1 -- 172.21.15.55:0/1855367336 >> 172.21.15.55:0/1855367336 conn(0x5568fd5b75a0 msgr2=0x5568fd5b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.330+0000 7f6b37464740 1 -- 172.21.15.55:0/1855367336 shutdown_connections 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.330+0000 7f6b37464740 1 -- 172.21.15.55:0/1855367336 wait complete. 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.331+0000 7f6b37464740 1 -- start start 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.331+0000 7f6b37464740 1 Processor -- start 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b37464740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 0x5568fd5cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b37464740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b37464740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b37464740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1000cd20 con 0x5568fd5bdc40 2024-04-26T21:54:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b37464740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1000e190 con 0x5568fd5bc0b0 2024-04-26T21:54:42.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b37464740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1000d3f0 con 0x5568fd5bd280 2024-04-26T21:54:42.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b37464740 10 client.0 _ll_register_callbacks cb 0x7f6b1c0083f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:54:42.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b324e2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b324e2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42464/0 (socket says 172.21.15.55:42464) 2024-04-26T21:54:42.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.332+0000 7f6b324e2640 1 -- 172.21.15.55:0/2951374050 learned_addr learned my addr 172.21.15.55:0/2951374050 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:54:42.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.333+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 0x5568fd5cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.333+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.334+0000 7f6b324e2640 1 -- 172.21.15.55:0/2951374050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 msgr2=0x5568fd5cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:42.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.334+0000 7f6b324e2640 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 0x5568fd5cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.334+0000 7f6b324e2640 1 -- 172.21.15.55:0/2951374050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 msgr2=0x5568fd5cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:54:42.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.334+0000 7f6b324e2640 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.334+0000 7f6b324e2640 1 -- 172.21.15.55:0/2951374050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568fd4e14b0 con 0x5568fd5bd280 2024-04-26T21:54:42.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.335+0000 7f6b324e2640 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5cb7f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b28016040 tx=0x7f6b28005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e23f1afbb4da14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.336+0000 7f6b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.336+0000 7f6b15ffb640 1 -- 172.21.15.55:0/2951374050 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28045040 con 0x5568fd5bd280 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.336+0000 7f6b37464740 1 -- 172.21.15.55:0/2951374050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b28004040 con 0x5568fd5bd280 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.336+0000 7f6b15ffb640 1 -- 172.21.15.55:0/2951374050 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f6b28011040 con 0x5568fd5bd280 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.337+0000 7f6b15ffb640 1 -- 172.21.15.55:0/2951374050 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6b280049b0 con 0x5568fd5bd280 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.337+0000 7f6b37464740 10 client.25270 fetch_fsmap learned FSMap version 21 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.337+0000 7f6b37464740 1 -- 172.21.15.55:0/2951374050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5568fd632a40 con 0x5568fd5bd280 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.338+0000 7f6b15ffb640 1 -- 172.21.15.55:0/2951374050 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b28004040 con 0x5568fd5bd280 2024-04-26T21:54:42.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.338+0000 7f6b15ffb640 1 -- 172.21.15.55:0/2951374050 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6b280035f0 con 0x5568fd5bd280 2024-04-26T21:54:42.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.338+0000 7f6b37464740 10 client.25270 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:54:42.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.338+0000 7f6b37464740 10 client.25270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:54:42.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.338+0000 7f6b37464740 20 client.25270 populate_metadata read hostname 'smithi055' 2024-04-26T21:54:42.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.338+0000 7f6b37464740 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5568fd633340 0x5568fd635760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:54:42.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.338+0000 7f6b37464740 4 client.25270 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:54:42.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.338+0000 7f6b37464740 1 -- 172.21.15.55:0/2951374050 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568fd528800 con 0x5568fd633340 2024-04-26T21:54:42.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.339+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5568fd633340 0x5568fd635760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:54:42.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.343+0000 7f6b31ce1640 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5568fd633340 0x5568fd635760 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6ea396f3b3599b server_cookie=27eddb7b42120ecf in_seq=0 out_seq=0 2024-04-26T21:54:42.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.343+0000 7f6b15ffb640 10 client.25270 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:42.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.349+0000 7f6b15ffb640 1 -- 172.21.15.55:0/2951374050 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x5568fd528800 con 0x5568fd633340 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.349+0000 7f6b15ffb640 10 client.25270 handle_command_reply: tid=0 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.349+0000 7f6b37464740 1 client.25270 shutdown 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b37464740 1 -- 172.21.15.55:0/2951374050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 msgr2=0x5568fd5cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b37464740 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5cb7f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b28016040 tx=0x7f6b28005fb0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b37464740 1 -- 172.21.15.55:0/2951374050 shutdown_connections 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b37464740 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5568fd633340 0x5568fd635760 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b37464740 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5568fd5bdc40 0x5568fd5cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b37464740 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5568fd5bd280 0x5568fd5cb7f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b37464740 1 --2- 172.21.15.55:0/2951374050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568fd5bc0b0 0x5568fd5cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b37464740 1 -- 172.21.15.55:0/2951374050 >> 172.21.15.55:0/2951374050 conn(0x5568fd5b75a0 msgr2=0x5568fd5b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:42.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.350+0000 7f6b15ffb640 0 client.25270 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:54:42.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.364+0000 7f6b37464740 1 -- 172.21.15.55:0/2951374050 shutdown_connections 2024-04-26T21:54:42.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.367+0000 7f6b37464740 1 -- 172.21.15.55:0/2951374050 wait complete. 2024-04-26T21:54:42.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.369+0000 7f6b37464740 20 client.25270 trim_cache size 0 max 16384 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (3 inodes in the stack)", 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout: "4fd2155a-da1a-42a5-9552-550d11ae1ade": { 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "4fd2155a-da1a-42a5-9552-550d11ae1ade", 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:54:42.374 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:54:42.375 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:54:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.376+0000 7f6b334e4640 1 -- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b180a3e10 msgr2=0x7f6b180a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.376+0000 7f6b334e4640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b180a3e10 0x7f6b180a62d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b24016000 tx=0x7f6b24013000 comp rx=0 tx=0).stop 2024-04-26T21:54:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.376+0000 7f6b334e4640 1 -- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b2c10ee20 msgr2=0x7f6b2c1a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:54:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.376+0000 7f6b334e4640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b2c10ee20 0x7f6b2c1a6530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400eaa0 tx=0x7f6b2400ef60 comp rx=0 tx=0).stop 2024-04-26T21:54:42.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.378+0000 7f6b334e4640 1 -- 172.21.15.55:0/328327377 shutdown_connections 2024-04-26T21:54:42.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.378+0000 7f6b334e4640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6b180a3e10 0x7f6b180a62d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.378+0000 7f6b334e4640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6b2c10ee20 0x7f6b2c1a6530 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.378+0000 7f6b334e4640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6b2c10c380 0x7f6b2c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.378+0000 7f6b334e4640 1 --2- 172.21.15.55:0/328327377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b2c1099c0 0x7f6b2c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:54:42.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.378+0000 7f6b334e4640 1 -- 172.21.15.55:0/328327377 >> 172.21.15.55:0/328327377 conn(0x7f6b2c06f990 msgr2=0x7f6b2c10a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:54:42.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.380+0000 7f6b334e4640 1 -- 172.21.15.55:0/328327377 shutdown_connections 2024-04-26T21:54:42.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:54:42.380+0000 7f6b334e4640 1 -- 172.21.15.55:0/328327377 wait complete. 2024-04-26T21:54:42.439 INFO:tasks.cephfs.filesystem:scrub status for tag:4fd2155a-da1a-42a5-9552-550d11ae1ade - {'path': '/', 'tag': '4fd2155a-da1a-42a5-9552-550d11ae1ade', 'options': 'recursive,force'} 2024-04-26T21:54:43.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:42 smithi165 ceph-mon[33296]: pgmap v405: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 825 MiB/s wr, 1.45k op/s 2024-04-26T21:54:43.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:42 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:54:43.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:42 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:43.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:42 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:54:43.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:42 smithi055 ceph-mon[29221]: pgmap v405: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 825 MiB/s wr, 1.45k op/s 2024-04-26T21:54:43.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:42 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:54:43.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:42 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:43.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:42 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:54:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:42 smithi064 ceph-mon[32765]: pgmap v405: 97 pgs: 97 active+clean; 9.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 825 MiB/s wr, 1.45k op/s 2024-04-26T21:54:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:42 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:54:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:42 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:42 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:54:45.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:44 smithi165 ceph-mon[33296]: pgmap v406: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 786 MiB/s wr, 1.68k op/s 2024-04-26T21:54:45.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:44 smithi055 ceph-mon[29221]: pgmap v406: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 786 MiB/s wr, 1.68k op/s 2024-04-26T21:54:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:44 smithi064 ceph-mon[32765]: pgmap v406: 97 pgs: 97 active+clean; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 786 MiB/s wr, 1.68k op/s 2024-04-26T21:54:47.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:46 smithi165 ceph-mon[33296]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 853 MiB/s wr, 2.07k op/s 2024-04-26T21:54:47.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:46 smithi055 ceph-mon[29221]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 853 MiB/s wr, 2.07k op/s 2024-04-26T21:54:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:46 smithi064 ceph-mon[32765]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 853 MiB/s wr, 2.07k op/s 2024-04-26T21:54:49.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:48 smithi055 ceph-mon[29221]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 894 MiB/s wr, 2.62k op/s 2024-04-26T21:54:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:48 smithi064 ceph-mon[32765]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 894 MiB/s wr, 2.62k op/s 2024-04-26T21:54:49.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:48 smithi165 ceph-mon[33296]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 894 MiB/s wr, 2.62k op/s 2024-04-26T21:54:50.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:49 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:50.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:49 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:50.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:49 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:54:51.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:50 smithi055 ceph-mon[29221]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 2.85k op/s 2024-04-26T21:54:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:50 smithi064 ceph-mon[32765]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 2.85k op/s 2024-04-26T21:54:51.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:50 smithi165 ceph-mon[33296]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 2.85k op/s 2024-04-26T21:54:52.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:51 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:54:52.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:51 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:54:52.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:51 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:54:53.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:52 smithi055 ceph-mon[29221]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 2.75k op/s 2024-04-26T21:54:53.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:52 smithi064 ceph-mon[32765]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 2.75k op/s 2024-04-26T21:54:53.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:52 smithi165 ceph-mon[33296]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 2.75k op/s 2024-04-26T21:54:55.191 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:54 smithi055 ceph-mon[29221]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 803 MiB/s wr, 2.44k op/s 2024-04-26T21:54:55.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:54 smithi064 ceph-mon[32765]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 803 MiB/s wr, 2.44k op/s 2024-04-26T21:54:55.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:54 smithi165 ceph-mon[33296]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 803 MiB/s wr, 2.44k op/s 2024-04-26T21:54:57.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:56 smithi055 ceph-mon[29221]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 866 MiB/s wr, 2.50k op/s 2024-04-26T21:54:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:56 smithi064 ceph-mon[32765]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 866 MiB/s wr, 2.50k op/s 2024-04-26T21:54:57.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:56 smithi165 ceph-mon[33296]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 866 MiB/s wr, 2.50k op/s 2024-04-26T21:54:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:54:58 smithi055 ceph-mon[29221]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 900 MiB/s wr, 2.39k op/s 2024-04-26T21:54:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:54:58 smithi064 ceph-mon[32765]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 900 MiB/s wr, 2.39k op/s 2024-04-26T21:54:59.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:54:58 smithi165 ceph-mon[33296]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 900 MiB/s wr, 2.39k op/s 2024-04-26T21:55:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:00 smithi055 ceph-mon[29221]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 880 MiB/s wr, 1.99k op/s 2024-04-26T21:55:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:00 smithi064 ceph-mon[32765]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 880 MiB/s wr, 1.99k op/s 2024-04-26T21:55:01.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:00 smithi165 ceph-mon[33296]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 880 MiB/s wr, 1.99k op/s 2024-04-26T21:55:03.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:02 smithi055 ceph-mon[29221]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 1.61k op/s 2024-04-26T21:55:03.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:02 smithi064 ceph-mon[32765]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 1.61k op/s 2024-04-26T21:55:03.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:02 smithi165 ceph-mon[33296]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 1.61k op/s 2024-04-26T21:55:05.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:04 smithi055 ceph-mon[29221]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 1.38k op/s 2024-04-26T21:55:05.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:04 smithi064 ceph-mon[32765]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 1.38k op/s 2024-04-26T21:55:05.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:04 smithi165 ceph-mon[33296]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 1.38k op/s 2024-04-26T21:55:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:06 smithi064 ceph-mon[32765]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 852 MiB/s wr, 1.40k op/s 2024-04-26T21:55:07.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:06 smithi165 ceph-mon[33296]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 852 MiB/s wr, 1.40k op/s 2024-04-26T21:55:07.454 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:06 smithi055 ceph-mon[29221]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 852 MiB/s wr, 1.40k op/s 2024-04-26T21:55:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:08 smithi064 ceph-mon[32765]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 896 MiB/s wr, 1.56k op/s 2024-04-26T21:55:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:08 smithi165 ceph-mon[33296]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 896 MiB/s wr, 1.56k op/s 2024-04-26T21:55:09.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:08 smithi055 ceph-mon[29221]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 896 MiB/s wr, 1.56k op/s 2024-04-26T21:55:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:10 smithi064 ceph-mon[32765]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 883 MiB/s wr, 1.46k op/s 2024-04-26T21:55:11.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:10 smithi165 ceph-mon[33296]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 883 MiB/s wr, 1.46k op/s 2024-04-26T21:55:11.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:10 smithi055 ceph-mon[29221]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 883 MiB/s wr, 1.46k op/s 2024-04-26T21:55:12.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:55:12.440 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:55:12.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 -- 172.21.15.55:0/3374512980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 msgr2=0x7f4b10109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 --2- 172.21.15.55:0/3374512980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 0x7f4b10109cf0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009ca0 tx=0x7f4b0802a8e0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 -- 172.21.15.55:0/3374512980 shutdown_connections 2024-04-26T21:55:12.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 --2- 172.21.15.55:0/3374512980 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 0x7f4b10109cf0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 --2- 172.21.15.55:0/3374512980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b10102aa0 0x7f4b10102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 --2- 172.21.15.55:0/3374512980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b10101d90 0x7f4b101021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 -- 172.21.15.55:0/3374512980 >> 172.21.15.55:0/3374512980 conn(0x7f4b100fc560 msgr2=0x7f4b100fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 -- 172.21.15.55:0/3374512980 shutdown_connections 2024-04-26T21:55:12.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.778+0000 7f4b1807e640 1 -- 172.21.15.55:0/3374512980 wait complete. 2024-04-26T21:55:12.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.779+0000 7f4b1807e640 1 Processor -- start 2024-04-26T21:55:12.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.779+0000 7f4b1807e640 1 -- start start 2024-04-26T21:55:12.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1807e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b10101d90 0x7f4b101014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1807e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b10102aa0 0x7f4b1019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1807e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 0x7f4b101a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1661b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 0x7f4b101a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1661b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 0x7f4b101a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43278/0 (socket says 172.21.15.55:43278) 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1661b640 1 -- 172.21.15.55:0/508386661 learned_addr learned my addr 172.21.15.55:0/508386661 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010d4a0 con 0x7f4b10103460 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010d320 con 0x7f4b10102aa0 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.780+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010d620 con 0x7f4b10101d90 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.781+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b10101d90 0x7f4b101014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.781+0000 7f4b1661b640 1 -- 172.21.15.55:0/508386661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b10101d90 msgr2=0x7f4b101014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.781+0000 7f4b1661b640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b10101d90 0x7f4b101014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.781+0000 7f4b1661b640 1 -- 172.21.15.55:0/508386661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b10102aa0 msgr2=0x7f4b1019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.781+0000 7f4b15619640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b10102aa0 0x7f4b1019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.781+0000 7f4b1661b640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b10102aa0 0x7f4b1019da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.781+0000 7f4b1661b640 1 -- 172.21.15.55:0/508386661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b101a0950 con 0x7f4b10103460 2024-04-26T21:55:12.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.781+0000 7f4b1661b640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 0x7f4b101a0090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009dc0 tx=0x7f4b08003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.782+0000 7f4b06ffd640 1 -- 172.21.15.55:0/508386661 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08004340 con 0x7f4b10103460 2024-04-26T21:55:12.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.782+0000 7f4b06ffd640 1 -- 172.21.15.55:0/508386661 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b080305e0 con 0x7f4b10103460 2024-04-26T21:55:12.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.782+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b101a0be0 con 0x7f4b10103460 2024-04-26T21:55:12.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.782+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b101a10c0 con 0x7f4b10103460 2024-04-26T21:55:12.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.782+0000 7f4b06ffd640 1 -- 172.21.15.55:0/508386661 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0803a660 con 0x7f4b10103460 2024-04-26T21:55:12.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.786+0000 7f4b1ad9a740 1 Processor -- start 2024-04-26T21:55:12.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.787+0000 7f4b06ffd640 1 -- 172.21.15.55:0/508386661 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4b08030a40 con 0x7f4b10103460 2024-04-26T21:55:12.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.787+0000 7f4b1ad9a740 1 -- start start 2024-04-26T21:55:12.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.787+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d50080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.787+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.787+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d56c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.787+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561ef2caad80 con 0x561ef2d4fc60 2024-04-26T21:55:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.787+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561ef2c648b0 con 0x561ef2d4f390 2024-04-26T21:55:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.787+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ef2ca6d40 con 0x561ef2d4e9d0 2024-04-26T21:55:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.788+0000 7f4b15619640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d50080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.788+0000 7f4b15619640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d50080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43292/0 (socket says 172.21.15.55:43292) 2024-04-26T21:55:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.788+0000 7f4b15619640 1 -- 172.21.15.55:0/1247470749 learned_addr learned my addr 172.21.15.55:0/1247470749 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:12.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.788+0000 7f4b15619640 1 -- 172.21.15.55:0/1247470749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 msgr2=0x561ef2d4ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.788+0000 7f4b15619640 1 --2- 172.21.15.55:0/1247470749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.788+0000 7f4b15619640 1 -- 172.21.15.55:0/1247470749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 msgr2=0x561ef2d56c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:55:12.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.788+0000 7f4b15619640 1 --2- 172.21.15.55:0/1247470749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d56c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.788+0000 7f4b15619640 1 -- 172.21.15.55:0/1247470749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef2c35a40 con 0x561ef2d4fc60 2024-04-26T21:55:12.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.789+0000 7f4b15619640 1 --2- 172.21.15.55:0/1247470749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d50080 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000aac0 tx=0x7f4b0000af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5a8ed4f3480e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.789+0000 7f4b06ffd640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4afc0a3d40 0x7f4afc0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.789+0000 7f4afbfff640 1 -- 172.21.15.55:0/1247470749 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00015070 con 0x561ef2d4fc60 2024-04-26T21:55:12.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.790+0000 7f4afbfff640 1 -- 172.21.15.55:0/1247470749 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b000113f0 con 0x561ef2d4fc60 2024-04-26T21:55:12.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.790+0000 7f4b06ffd640 1 -- 172.21.15.55:0/508386661 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4b080f1e30 con 0x7f4b10103460 2024-04-26T21:55:12.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.790+0000 7f4afbfff640 1 -- 172.21.15.55:0/1247470749 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000185c0 con 0x561ef2d4fc60 2024-04-26T21:55:12.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.791+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4afc0a3d40 0x7f4afc0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.792+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1247470749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 msgr2=0x561ef2d50080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:12.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.792+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1247470749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d50080 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000aac0 tx=0x7f4b0000af80 comp rx=0 tx=0).stop 2024-04-26T21:55:12.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.792+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1247470749 shutdown_connections 2024-04-26T21:55:12.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.792+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1247470749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d56c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.792+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1247470749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.792+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1247470749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d50080 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.792+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1247470749 >> 172.21.15.55:0/1247470749 conn(0x561ef2d41350 msgr2=0x561ef2d41750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:12.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.793+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4afc0a3d40 0x7f4afc0a6200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c006fd0 tx=0x7f4b0c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.794+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1247470749 shutdown_connections 2024-04-26T21:55:12.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.794+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1247470749 wait complete. 2024-04-26T21:55:12.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.795+0000 7f4b1ad9a740 1 -- start start 2024-04-26T21:55:12.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.796+0000 7f4b1ad9a740 1 Processor -- start 2024-04-26T21:55:12.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.797+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.797+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d6a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.798+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.798+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561ef2d68330 con 0x561ef2d4fc60 2024-04-26T21:55:12.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.798+0000 7f4b15e1a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.798+0000 7f4b15e1a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d69c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:58124/0 (socket says 172.21.15.55:58124) 2024-04-26T21:55:12.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.798+0000 7f4b15e1a640 1 -- 172.21.15.55:0/1012364813 learned_addr learned my addr 172.21.15.55:0/1012364813 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:12.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.798+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561ef2c31d10 con 0x561ef2d4f390 2024-04-26T21:55:12.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.799+0000 7f4b15619640 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d6a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.799+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ef2d684b0 con 0x561ef2d4e9d0 2024-04-26T21:55:12.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.799+0000 7f4b1661b640 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:12.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.799+0000 7f4b1ad9a740 10 client.0 _ll_register_callbacks cb 0x561ef2c5e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:12.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.801+0000 7f4b1661b640 1 -- 172.21.15.55:0/1012364813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 msgr2=0x561ef2d69c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.801+0000 7f4b1661b640 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d69c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.801+0000 7f4b1661b640 1 -- 172.21.15.55:0/1012364813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 msgr2=0x561ef2d6a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:12.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.801+0000 7f4b1661b640 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d6a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:12.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.801+0000 7f4b1661b640 1 -- 172.21.15.55:0/1012364813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef2a867d0 con 0x561ef2d4fc60 2024-04-26T21:55:12.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.802+0000 7f4af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:12.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.802+0000 7f4b1661b640 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d711c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4b080ef850 tx=0x7f4b08042380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0d349e51bfa7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:12.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.803+0000 7f4af97fa640 1 -- 172.21.15.55:0/1012364813 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080c2460 con 0x561ef2d4fc60 2024-04-26T21:55:12.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.804+0000 7f4af97fa640 1 -- 172.21.15.55:0/1012364813 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4b08030790 con 0x561ef2d4fc60 2024-04-26T21:55:12.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.804+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b000113f0 con 0x561ef2d4fc60 2024-04-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.804+0000 7f4af97fa640 1 -- 172.21.15.55:0/1012364813 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b080c6c90 con 0x561ef2d4fc60 2024-04-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.804+0000 7f4b1ad9a740 10 client.15726 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.805+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561ef29bc3f0 con 0x561ef2d4fc60 2024-04-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.805+0000 7f4af97fa640 1 -- 172.21.15.55:0/1012364813 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b08036070 con 0x561ef2d4fc60 2024-04-26T21:55:12.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.805+0000 7f4af97fa640 1 -- 172.21.15.55:0/1012364813 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b080ef990 con 0x561ef2d4fc60 2024-04-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.806+0000 7f4b1ad9a740 10 client.15726 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.806+0000 7f4b1ad9a740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.806+0000 7f4b1ad9a740 20 client.15726 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.806+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561ef2d756f0 0x561ef2d77b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.806+0000 7f4b1ad9a740 4 client.15726 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:12.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.806+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4b000113f0 con 0x561ef2d756f0 2024-04-26T21:55:12.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:12.807+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561ef2d756f0 0x561ef2d77b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.026+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561ef2d756f0 0x561ef2d77b20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efcd1131c1175691 server_cookie=74e8d487f010c77a in_seq=0 out_seq=0 2024-04-26T21:55:13.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.026+0000 7f4af97fa640 10 client.15726 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:13.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.037+0000 7f4af97fa640 1 -- 172.21.15.55:0/1012364813 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f4b000113f0 con 0x561ef2d756f0 2024-04-26T21:55:13.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.037+0000 7f4af97fa640 10 client.15726 handle_command_reply: tid=0 2024-04-26T21:55:13.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.037+0000 7f4b1ad9a740 1 client.15726 shutdown 2024-04-26T21:55:13.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 msgr2=0x561ef2d711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:13.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d711c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4b080ef850 tx=0x7f4b08042380 comp rx=0 tx=0).stop 2024-04-26T21:55:13.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 shutdown_connections 2024-04-26T21:55:13.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561ef2d756f0 0x561ef2d77b20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d711c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d6a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/1012364813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d69c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 >> 172.21.15.55:0/1012364813 conn(0x561ef2d41350 msgr2=0x561ef2d4e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:13.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.042+0000 7f4af97fa640 0 client.15726 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:13.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.043+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 shutdown_connections 2024-04-26T21:55:13.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.043+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/1012364813 wait complete. 2024-04-26T21:55:13.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.043+0000 7f4b1ad9a740 20 client.15726 trim_cache size 0 max 16384 2024-04-26T21:55:13.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.104+0000 7f4b1ad9a740 1 Processor -- start 2024-04-26T21:55:13.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.104+0000 7f4b1ad9a740 1 -- start start 2024-04-26T21:55:13.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.105+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.105+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4f390 0x561ef2d71180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.105+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4fc60 0x561ef2d7c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.105+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101a1ef0 con 0x561ef2d4e9d0 2024-04-26T21:55:13.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.105+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10072f70 con 0x561ef2d4fc60 2024-04-26T21:55:13.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.105+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10070dc0 con 0x561ef2d4f390 2024-04-26T21:55:13.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.106+0000 7f4b15e1a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.106+0000 7f4b15e1a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43324/0 (socket says 172.21.15.55:43324) 2024-04-26T21:55:13.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.106+0000 7f4b15e1a640 1 -- 172.21.15.55:0/2453564633 learned_addr learned my addr 172.21.15.55:0/2453564633 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:13.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.107+0000 7f4b15e1a640 1 -- 172.21.15.55:0/2453564633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4f390 msgr2=0x561ef2d71180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:13.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.107+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/2453564633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4f390 0x561ef2d71180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.107+0000 7f4b15e1a640 1 -- 172.21.15.55:0/2453564633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4fc60 msgr2=0x561ef2d7c420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:13.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.107+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/2453564633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4fc60 0x561ef2d7c420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.107+0000 7f4b15e1a640 1 -- 172.21.15.55:0/2453564633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef297c0f0 con 0x561ef2d4e9d0 2024-04-26T21:55:13.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.108+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/2453564633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4e0d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x561ef2d783a0 tx=0x7f4b0c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ad6995eb2c061d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:13.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.108+0000 7f4afb7fe640 1 -- 172.21.15.55:0/2453564633 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c025160 con 0x561ef2d4e9d0 2024-04-26T21:55:13.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.108+0000 7f4afb7fe640 1 -- 172.21.15.55:0/2453564633 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0c02b070 con 0x561ef2d4e9d0 2024-04-26T21:55:13.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.109+0000 7f4afb7fe640 1 -- 172.21.15.55:0/2453564633 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0149b0 con 0x561ef2d4e9d0 2024-04-26T21:55:13.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.109+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2453564633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4e9d0 msgr2=0x561ef2d4e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:13.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.109+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2453564633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4e0d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x561ef2d783a0 tx=0x7f4b0c013000 comp rx=0 tx=0).stop 2024-04-26T21:55:13.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.110+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2453564633 shutdown_connections 2024-04-26T21:55:13.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.110+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2453564633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4fc60 0x561ef2d7c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.110+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2453564633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4f390 0x561ef2d71180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.110+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2453564633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4e9d0 0x561ef2d4e0d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.110+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2453564633 >> 172.21.15.55:0/2453564633 conn(0x561ef2d41350 msgr2=0x561ef2d4d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:13.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.110+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2453564633 shutdown_connections 2024-04-26T21:55:13.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.110+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2453564633 wait complete. 2024-04-26T21:55:13.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.111+0000 7f4b1ad9a740 1 -- start start 2024-04-26T21:55:13.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.111+0000 7f4b1ad9a740 1 Processor -- start 2024-04-26T21:55:13.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.112+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d6f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.112+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d6f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.112+0000 7f4b1ad9a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d70700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.112+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10071430 con 0x561ef2d4fc60 2024-04-26T21:55:13.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.112+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b100728a0 con 0x561ef2d4f390 2024-04-26T21:55:13.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.112+0000 7f4b1ad9a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10071b00 con 0x561ef2d4e9d0 2024-04-26T21:55:13.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.112+0000 7f4b1ad9a740 10 client.0 _ll_register_callbacks cb 0x7f4b080b9c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:13.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.113+0000 7f4b1661b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d70700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.113+0000 7f4b1661b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d70700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43336/0 (socket says 172.21.15.55:43336) 2024-04-26T21:55:13.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.113+0000 7f4b1661b640 1 -- 172.21.15.55:0/2522384330 learned_addr learned my addr 172.21.15.55:0/2522384330 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:13.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.114+0000 7f4b1661b640 1 -- 172.21.15.55:0/2522384330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 msgr2=0x561ef2d6f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:13.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.114+0000 7f4b1661b640 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d6f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.114+0000 7f4b15619640 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d6f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.114+0000 7f4b1661b640 1 -- 172.21.15.55:0/2522384330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 msgr2=0x561ef2d6f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:13.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.114+0000 7f4b1661b640 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d6f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.114+0000 7f4b1661b640 1 -- 172.21.15.55:0/2522384330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef2c6b4b0 con 0x561ef2d4fc60 2024-04-26T21:55:13.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.115+0000 7f4b1661b640 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d70700 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4b08042530 tx=0x7f4b080bd590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fbdbe7a8587159 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:13.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.117+0000 7f4af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:13.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.117+0000 7f4af97fa640 1 -- 172.21.15.55:0/2522384330 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080bda10 con 0x561ef2d4fc60 2024-04-26T21:55:13.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.117+0000 7f4af97fa640 1 -- 172.21.15.55:0/2522384330 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4b080c20c0 con 0x561ef2d4fc60 2024-04-26T21:55:13.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.117+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2522384330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b0c02b070 con 0x561ef2d4fc60 2024-04-26T21:55:13.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.118+0000 7f4af97fa640 1 -- 172.21.15.55:0/2522384330 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b080bde80 con 0x561ef2d4fc60 2024-04-26T21:55:13.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.118+0000 7f4b1ad9a740 10 client.15732 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:13.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.118+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2522384330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561ef2ddc370 con 0x561ef2d4fc60 2024-04-26T21:55:13.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.120+0000 7f4af97fa640 1 -- 172.21.15.55:0/2522384330 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0c02b070 con 0x561ef2d4fc60 2024-04-26T21:55:13.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.120+0000 7f4af97fa640 1 -- 172.21.15.55:0/2522384330 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f4b080044e0 con 0x561ef2d4fc60 2024-04-26T21:55:13.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.120+0000 7f4b1ad9a740 10 client.15732 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:13.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.120+0000 7f4b1ad9a740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:13.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.120+0000 7f4b1ad9a740 20 client.15732 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:13.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.120+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561ef2ddcb30 0x561ef2ddef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.120+0000 7f4b1ad9a740 4 client.15732 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:55:13.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.120+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2522384330 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ef2cb8990 con 0x561ef2ddcb30 2024-04-26T21:55:13.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.124+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561ef2ddcb30 0x561ef2ddef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.134+0000 7f4b15e1a640 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561ef2ddcb30 0x561ef2ddef50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4868916ff0a7ab3f server_cookie=ff8119866b650c99 in_seq=0 out_seq=0 2024-04-26T21:55:13.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.134+0000 7f4af97fa640 10 client.15732 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:13.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.139+0000 7f4af97fa640 1 -- 172.21.15.55:0/2522384330 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561ef2cb8990 con 0x561ef2ddcb30 2024-04-26T21:55:13.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.139+0000 7f4af97fa640 10 client.15732 handle_command_reply: tid=0 2024-04-26T21:55:13.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.139+0000 7f4b1ad9a740 1 client.15732 shutdown 2024-04-26T21:55:13.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.141+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2522384330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 msgr2=0x561ef2d70700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:13.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.141+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d70700 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4b08042530 tx=0x7f4b080bd590 comp rx=0 tx=0).stop 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.143+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2522384330 shutdown_connections 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.143+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561ef2ddcb30 0x561ef2ddef50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.143+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561ef2d4fc60 0x561ef2d70700 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.143+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef2d4f390 0x561ef2d6f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.143+0000 7f4b1ad9a740 1 --2- 172.21.15.55:0/2522384330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ef2d4e9d0 0x561ef2d6f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.143+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2522384330 >> 172.21.15.55:0/2522384330 conn(0x561ef2d41350 msgr2=0x561ef2d4e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.143+0000 7f4af97fa640 0 client.15732 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.144+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2522384330 shutdown_connections 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.144+0000 7f4b1ad9a740 1 -- 172.21.15.55:0/2522384330 wait complete. 2024-04-26T21:55:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.144+0000 7f4b1ad9a740 20 client.15732 trim_cache size 0 max 16384 2024-04-26T21:55:13.148 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:55:13.148 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:55:13.148 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:55:13.148 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:55:13.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.147+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4afc0a3d40 msgr2=0x7f4afc0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:13.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.147+0000 7f4b1807e640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4afc0a3d40 0x7f4afc0a6200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c006fd0 tx=0x7f4b0c008040 comp rx=0 tx=0).stop 2024-04-26T21:55:13.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.147+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 msgr2=0x7f4b101a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:13.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.147+0000 7f4b1807e640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 0x7f4b101a0090 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009dc0 tx=0x7f4b08003f90 comp rx=0 tx=0).stop 2024-04-26T21:55:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.148+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 shutdown_connections 2024-04-26T21:55:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.148+0000 7f4b1807e640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4afc0a3d40 0x7f4afc0a6200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.148+0000 7f4b1807e640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4b10103460 0x7f4b101a0090 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.148+0000 7f4b1807e640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b10102aa0 0x7f4b1019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.148+0000 7f4b1807e640 1 --2- 172.21.15.55:0/508386661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b10101d90 0x7f4b101014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.148+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 >> 172.21.15.55:0/508386661 conn(0x7f4b100fc560 msgr2=0x7f4b10105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:13.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.148+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 shutdown_connections 2024-04-26T21:55:13.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.148+0000 7f4b1807e640 1 -- 172.21.15.55:0/508386661 wait complete. 2024-04-26T21:55:13.178 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:55:13.178 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:55:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:12 smithi064 ceph-mon[32765]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 1.36k op/s 2024-04-26T21:55:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:12 smithi165 ceph-mon[33296]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 1.36k op/s 2024-04-26T21:55:13.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 -- 172.21.15.55:0/2386326228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0cc1086d0 msgr2=0x7fa0cc10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:13.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 --2- 172.21.15.55:0/2386326228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0cc1086d0 0x7fa0cc10aac0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009ca0 tx=0x7fa0c402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 -- 172.21.15.55:0/2386326228 shutdown_connections 2024-04-26T21:55:13.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 --2- 172.21.15.55:0/2386326228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0cc1086d0 0x7fa0cc10aac0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 --2- 172.21.15.55:0/2386326228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc105da0 0x7fa0cc108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 --2- 172.21.15.55:0/2386326228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0cc103470 0x7fa0cc105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 -- 172.21.15.55:0/2386326228 >> 172.21.15.55:0/2386326228 conn(0x7fa0cc0fce20 msgr2=0x7fa0cc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:13.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 -- 172.21.15.55:0/2386326228 shutdown_connections 2024-04-26T21:55:13.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.365+0000 7fa0d2209640 1 -- 172.21.15.55:0/2386326228 wait complete. 2024-04-26T21:55:13.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.367+0000 7fa0d2209640 1 Processor -- start 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.368+0000 7fa0d2209640 1 -- start start 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.369+0000 7fa0d2209640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc103470 0x7fa0cc197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.369+0000 7fa0d2209640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0cc105da0 0x7fa0cc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.369+0000 7fa0d2209640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0cc1086d0 0x7fa0cc194170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.369+0000 7fa0d2209640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc10e5b0 con 0x7fa0cc105da0 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.369+0000 7fa0d2209640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc10e430 con 0x7fa0cc103470 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.369+0000 7fa0d2209640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa0cc10e730 con 0x7fa0cc1086d0 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.369+0000 7fa0cbfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0cc1086d0 0x7fa0cc194170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.370+0000 7fa0cbfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0cc1086d0 0x7fa0cc194170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:58168/0 (socket says 172.21.15.55:58168) 2024-04-26T21:55:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.370+0000 7fa0cbfff640 1 -- 172.21.15.55:0/1845091492 learned_addr learned my addr 172.21.15.55:0/1845091492 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.373+0000 7fa0cb7fe640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc103470 0x7fa0cc197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.373+0000 7fa0caffd640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0cc105da0 0x7fa0cc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.378+0000 7fa0cbfff640 1 -- 172.21.15.55:0/1845091492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc103470 msgr2=0x7fa0cc197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.378+0000 7fa0cbfff640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc103470 0x7fa0cc197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.378+0000 7fa0cbfff640 1 -- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0cc105da0 msgr2=0x7fa0cc197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.378+0000 7fa0cbfff640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0cc105da0 0x7fa0cc197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:13.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.378+0000 7fa0cbfff640 1 -- 172.21.15.55:0/1845091492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0cc194a30 con 0x7fa0cc1086d0 2024-04-26T21:55:13.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.386+0000 7fa0cbfff640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0cc1086d0 0x7fa0cc194170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4031a20 tx=0x7fa0c4003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:13.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.391+0000 7fa0c8ff9640 1 -- 172.21.15.55:0/1845091492 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4004240 con 0x7fa0cc1086d0 2024-04-26T21:55:13.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.391+0000 7fa0d2209640 1 -- 172.21.15.55:0/1845091492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0cc19ab60 con 0x7fa0cc1086d0 2024-04-26T21:55:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.391+0000 7fa0d2209640 1 -- 172.21.15.55:0/1845091492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0cc19b130 con 0x7fa0cc1086d0 2024-04-26T21:55:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.392+0000 7fa0c8ff9640 1 -- 172.21.15.55:0/1845091492 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c4003e10 con 0x7fa0cc1086d0 2024-04-26T21:55:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.393+0000 7fa0c8ff9640 1 -- 172.21.15.55:0/1845091492 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c402fdc0 con 0x7fa0cc1086d0 2024-04-26T21:55:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.407+0000 7fa0c8ff9640 1 -- 172.21.15.55:0/1845091492 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa0c40043e0 con 0x7fa0cc1086d0 2024-04-26T21:55:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.409+0000 7fa0c8ff9640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0b40a3bf0 0x7fa0b40a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.411+0000 7fa0c8ff9640 1 -- 172.21.15.55:0/1845091492 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa0c40f1c90 con 0x7fa0cc1086d0 2024-04-26T21:55:13.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.412+0000 7fa0cb7fe640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0b40a3bf0 0x7fa0b40a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.414+0000 7fa0cb7fe640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0b40a3bf0 0x7fa0b40a60b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009990 tx=0x7fa0bc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.422+0000 7fa0d2209640 1 -- 172.21.15.55:0/1845091492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b8005290 con 0x7fa0cc1086d0 2024-04-26T21:55:13.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:13.440+0000 7fa0c8ff9640 1 -- 172.21.15.55:0/1845091492 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa0c40be580 con 0x7fa0cc1086d0 2024-04-26T21:55:13.450 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:12 smithi055 ceph-mon[29221]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 1.36k op/s 2024-04-26T21:55:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.193+0000 7fa0d2209640 1 -- 172.21.15.55:0/1845091492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0b8005850 con 0x7fa0cc1086d0 2024-04-26T21:55:14.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.197+0000 7fa0c8ff9640 1 -- 172.21.15.55:0/1845091492 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11863 (secure 0 0 0) 0x7fa0c40c0090 con 0x7fa0cc1086d0 2024-04-26T21:55:14.200 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:55:14.200 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:52:54.328649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24502,24509]},"id":1}]} 2024-04-26T21:55:14.200 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 21 2024-04-26T21:55:14.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.209+0000 7fa0b27fc640 1 -- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0b40a3bf0 msgr2=0x7fa0b40a60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.209+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0b40a3bf0 0x7fa0b40a60b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009990 tx=0x7fa0bc009110 comp rx=0 tx=0).stop 2024-04-26T21:55:14.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.209+0000 7fa0b27fc640 1 -- 172.21.15.55:0/1845091492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0cc1086d0 msgr2=0x7fa0cc194170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.209+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0cc1086d0 0x7fa0cc194170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4031a20 tx=0x7fa0c4003c20 comp rx=0 tx=0).stop 2024-04-26T21:55:14.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.214+0000 7fa0b27fc640 1 -- 172.21.15.55:0/1845091492 shutdown_connections 2024-04-26T21:55:14.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.214+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa0b40a3bf0 0x7fa0b40a60b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.214+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0cc1086d0 0x7fa0cc194170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.214+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0cc105da0 0x7fa0cc197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.214+0000 7fa0b27fc640 1 --2- 172.21.15.55:0/1845091492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0cc103470 0x7fa0cc197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.214+0000 7fa0b27fc640 1 -- 172.21.15.55:0/1845091492 >> 172.21.15.55:0/1845091492 conn(0x7fa0cc0fce20 msgr2=0x7fa0cc102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:14.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.218+0000 7fa0b27fc640 1 -- 172.21.15.55:0/1845091492 shutdown_connections 2024-04-26T21:55:14.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.218+0000 7fa0b27fc640 1 -- 172.21.15.55:0/1845091492 wait complete. 2024-04-26T21:55:14.252 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:55:14.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.542+0000 7fe010c21640 1 -- 172.21.15.55:0/3154021964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe00c073e50 msgr2=0x7fe00c0742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.542+0000 7fe010c21640 1 --2- 172.21.15.55:0/3154021964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe00c073e50 0x7fe00c0742d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009900 tx=0x7fdffc02e820 comp rx=0 tx=0).stop 2024-04-26T21:55:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.547+0000 7fe010c21640 1 -- 172.21.15.55:0/3154021964 shutdown_connections 2024-04-26T21:55:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.548+0000 7fe010c21640 1 --2- 172.21.15.55:0/3154021964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe00c110c70 0x7fe00c113060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.548+0000 7fe010c21640 1 --2- 172.21.15.55:0/3154021964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe00c073e50 0x7fe00c0742d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.548+0000 7fe010c21640 1 --2- 172.21.15.55:0/3154021964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c10a510 0x7fe00c073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.548+0000 7fe010c21640 1 -- 172.21.15.55:0/3154021964 >> 172.21.15.55:0/3154021964 conn(0x7fe00c06f780 msgr2=0x7fe00c071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:14.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.561+0000 7fe010c21640 1 -- 172.21.15.55:0/3154021964 shutdown_connections 2024-04-26T21:55:14.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.561+0000 7fe010c21640 1 -- 172.21.15.55:0/3154021964 wait complete. 2024-04-26T21:55:14.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.562+0000 7fe010c21640 1 Processor -- start 2024-04-26T21:55:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.563+0000 7fe010c21640 1 -- start start 2024-04-26T21:55:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.563+0000 7fe010c21640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe00c073e50 0x7fe00c1a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.563+0000 7fe010c21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c10a510 0x7fe00c1a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.563+0000 7fe010c21640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe00c110c70 0x7fe00c1aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.564+0000 7fe010c21640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c114970 con 0x7fe00c073e50 2024-04-26T21:55:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.564+0000 7fe010c21640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c1147f0 con 0x7fe00c10a510 2024-04-26T21:55:14.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.564+0000 7fe010c21640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c114af0 con 0x7fe00c110c70 2024-04-26T21:55:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.565+0000 7fe00affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c10a510 0x7fe00c1a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.565+0000 7fe00affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c10a510 0x7fe00c1a7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35028/0 (socket says 172.21.15.55:35028) 2024-04-26T21:55:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.565+0000 7fe00affd640 1 -- 172.21.15.55:0/3797209756 learned_addr learned my addr 172.21.15.55:0/3797209756 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.566+0000 7fe00affd640 1 -- 172.21.15.55:0/3797209756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe00c110c70 msgr2=0x7fe00c1aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.566+0000 7fe00affd640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe00c110c70 0x7fe00c1aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.566+0000 7fe00affd640 1 -- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe00c073e50 msgr2=0x7fe00c1a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.566+0000 7fe00affd640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe00c073e50 0x7fe00c1a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.566+0000 7fe00affd640 1 -- 172.21.15.55:0/3797209756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c1af410 con 0x7fe00c10a510 2024-04-26T21:55:14.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.569+0000 7fe00affd640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c10a510 0x7fe00c1a7d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdffc000c00 tx=0x7fdffc003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.576+0000 7fe008ff9640 1 -- 172.21.15.55:0/3797209756 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc003990 con 0x7fe00c10a510 2024-04-26T21:55:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.576+0000 7fe008ff9640 1 -- 172.21.15.55:0/3797209756 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdffc03e400 con 0x7fe00c10a510 2024-04-26T21:55:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.576+0000 7fe008ff9640 1 -- 172.21.15.55:0/3797209756 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00f3c0 con 0x7fe00c10a510 2024-04-26T21:55:14.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.576+0000 7fe010c21640 1 -- 172.21.15.55:0/3797209756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c1af6a0 con 0x7fe00c10a510 2024-04-26T21:55:14.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.576+0000 7fe010c21640 1 -- 172.21.15.55:0/3797209756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c1afb40 con 0x7fe00c10a510 2024-04-26T21:55:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.587+0000 7fe008ff9640 1 -- 172.21.15.55:0/3797209756 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdffc045020 con 0x7fe00c10a510 2024-04-26T21:55:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.587+0000 7fe014ba1740 1 Processor -- start 2024-04-26T21:55:14.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.587+0000 7fe014ba1740 1 -- start start 2024-04-26T21:55:14.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.588+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aedfa30 0x563f4aedfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.588+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aee1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.588+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aee15c0 0x563f4aee7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.589+0000 7fe014ba1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563f4ae43d80 con 0x563f4aee15c0 2024-04-26T21:55:14.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.589+0000 7fe014ba1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563f4adfda20 con 0x563f4aee0c00 2024-04-26T21:55:14.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.589+0000 7fe014ba1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563f4ae3fd40 con 0x563f4aedfa30 2024-04-26T21:55:14.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.591+0000 7fe00bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aedfa30 0x563f4aedfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.593+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aee1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.593+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aee1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35036/0 (socket says 172.21.15.55:35036) 2024-04-26T21:55:14.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.593+0000 7fe00b7fe640 1 -- 172.21.15.55:0/3991937379 learned_addr learned my addr 172.21.15.55:0/3991937379 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:14.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.593+0000 7fe00affd640 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aee15c0 0x563f4aee7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.595+0000 7fe00b7fe640 1 -- 172.21.15.55:0/3991937379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aedfa30 msgr2=0x563f4aedfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.595+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aedfa30 0x563f4aedfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.595+0000 7fe00b7fe640 1 -- 172.21.15.55:0/3991937379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aee15c0 msgr2=0x563f4aee7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:14.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.595+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aee15c0 0x563f4aee7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.595+0000 7fe00b7fe640 1 -- 172.21.15.55:0/3991937379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f4ae08c30 con 0x563f4aee0c00 2024-04-26T21:55:14.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.595+0000 7fe00affd640 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aee15c0 0x563f4aee7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:14.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.597+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aee1080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe004007ce0 tx=0x7fe00400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840f661e0db731ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.600+0000 7fdff1ffb640 1 -- 172.21.15.55:0/3991937379 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004014070 con 0x563f4aee0c00 2024-04-26T21:55:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.600+0000 7fdff1ffb640 1 -- 172.21.15.55:0/3991937379 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004003b70 con 0x563f4aee0c00 2024-04-26T21:55:14.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.600+0000 7fdff1ffb640 1 -- 172.21.15.55:0/3991937379 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00400f6b0 con 0x563f4aee0c00 2024-04-26T21:55:14.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.600+0000 7fe008ff9640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdff40a3e40 0x7fdff40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.601+0000 7fe008ff9640 1 -- 172.21.15.55:0/3797209756 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fdffc0f0c60 con 0x7fe00c10a510 2024-04-26T21:55:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.602+0000 7fe014ba1740 1 -- 172.21.15.55:0/3991937379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 msgr2=0x563f4aee1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.602+0000 7fe00bfff640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdff40a3e40 0x7fdff40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.604+0000 7fe014ba1740 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aee1080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe004007ce0 tx=0x7fe00400c6a0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.606+0000 7fe014ba1740 1 -- 172.21.15.55:0/3991937379 shutdown_connections 2024-04-26T21:55:14.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.606+0000 7fe014ba1740 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aee15c0 0x563f4aee7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.607+0000 7fe014ba1740 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aee1080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.607+0000 7fe014ba1740 1 --2- 172.21.15.55:0/3991937379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aedfa30 0x563f4aedfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.607+0000 7fe014ba1740 1 -- 172.21.15.55:0/3991937379 >> 172.21.15.55:0/3991937379 conn(0x563f4ae20910 msgr2=0x563f4aedb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.607+0000 7fe014ba1740 1 -- 172.21.15.55:0/3991937379 shutdown_connections 2024-04-26T21:55:14.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.607+0000 7fe014ba1740 1 -- 172.21.15.55:0/3991937379 wait complete. 2024-04-26T21:55:14.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.610+0000 7fe00bfff640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdff40a3e40 0x7fdff40a6300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x563f4aee0b00 tx=0x7fe000008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:14.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.611+0000 7fe014ba1740 1 -- start start 2024-04-26T21:55:14.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.612+0000 7fe014ba1740 1 Processor -- start 2024-04-26T21:55:14.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.613+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aee9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.613+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aeea030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.613+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 0x563f4aef1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.614+0000 7fe014ba1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563f4aee85d0 con 0x563f4aedfa30 2024-04-26T21:55:14.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.614+0000 7fe014ba1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563f4adcadc0 con 0x563f4aee0c00 2024-04-26T21:55:14.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.614+0000 7fe014ba1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563f4aee8750 con 0x563f4aee15c0 2024-04-26T21:55:14.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.614+0000 7fe014ba1740 10 client.0 _ll_register_callbacks cb 0x563f4ade6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:14.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.617+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aee9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.617+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aee9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43384/0 (socket says 172.21.15.55:43384) 2024-04-26T21:55:14.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.617+0000 7fe00b7fe640 1 -- 172.21.15.55:0/852226516 learned_addr learned my addr 172.21.15.55:0/852226516 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.618+0000 7fe00bfff640 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 0x563f4aef1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.618+0000 7fe00affd640 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aeea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.618+0000 7fe00b7fe640 1 -- 172.21.15.55:0/852226516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 msgr2=0x563f4aef1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.619+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 0x563f4aef1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.619+0000 7fe00b7fe640 1 -- 172.21.15.55:0/852226516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 msgr2=0x563f4aeea030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.619+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aeea030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.619+0000 7fe00b7fe640 1 -- 172.21.15.55:0/852226516 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4ab553f0 con 0x563f4aedfa30 2024-04-26T21:55:14.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.620+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aee9af0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe004007a50 tx=0x7fe004003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b3c1f33836643f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.623+0000 7fdfeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.623+0000 7fdfeaffd640 1 -- 172.21.15.55:0/852226516 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004026060 con 0x563f4aedfa30 2024-04-26T21:55:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.623+0000 7fe014ba1740 1 -- 172.21.15.55:0/852226516 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe004003b70 con 0x563f4aedfa30 2024-04-26T21:55:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.623+0000 7fdfeaffd640 1 -- 172.21.15.55:0/852226516 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe00400f450 con 0x563f4aedfa30 2024-04-26T21:55:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.624+0000 7fdfeaffd640 1 -- 172.21.15.55:0/852226516 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe004014040 con 0x563f4aedfa30 2024-04-26T21:55:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.624+0000 7fe014ba1740 10 client.15747 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.624+0000 7fe014ba1740 1 -- 172.21.15.55:0/852226516 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f4ac1f7d0 con 0x563f4aedfa30 2024-04-26T21:55:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.625+0000 7fdfeaffd640 1 -- 172.21.15.55:0/852226516 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004003b70 con 0x563f4aedfa30 2024-04-26T21:55:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.625+0000 7fdfeaffd640 1 -- 172.21.15.55:0/852226516 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fe00402c770 con 0x563f4aedfa30 2024-04-26T21:55:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.626+0000 7fe014ba1740 10 client.15747 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.626+0000 7fe014ba1740 10 client.15747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:55:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.626+0000 7fe014ba1740 20 client.15747 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.626+0000 7fe014ba1740 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563f4aef5590 0x563f4aef7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.626+0000 7fe014ba1740 4 client.15747 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.626+0000 7fe014ba1740 1 -- 172.21.15.55:0/852226516 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f4ae51990 con 0x563f4aef5590 2024-04-26T21:55:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.629+0000 7fe00affd640 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563f4aef5590 0x563f4aef7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.643+0000 7fe00affd640 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563f4aef5590 0x563f4aef7980 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b109c2632b5e894 server_cookie=82b40362e5c6b395 in_seq=0 out_seq=0 2024-04-26T21:55:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.643+0000 7fdfeaffd640 10 client.15747 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:55:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.648+0000 7fdfeaffd640 1 -- 172.21.15.55:0/852226516 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f4ae51990 con 0x563f4aef5590 2024-04-26T21:55:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.648+0000 7fdfeaffd640 10 client.15747 handle_command_reply: tid=0 2024-04-26T21:55:14.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.649+0000 7fe014ba1740 1 client.15747 shutdown 2024-04-26T21:55:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.651+0000 7fe014ba1740 1 -- 172.21.15.55:0/852226516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 msgr2=0x563f4aee9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.651+0000 7fe014ba1740 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aee9af0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe004007a50 tx=0x7fe004003970 comp rx=0 tx=0).stop 2024-04-26T21:55:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.651+0000 7fe014ba1740 1 -- 172.21.15.55:0/852226516 shutdown_connections 2024-04-26T21:55:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.651+0000 7fe014ba1740 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563f4aef5590 0x563f4aef7980 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.654+0000 7fe014ba1740 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 0x563f4aef1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.654+0000 7fe014ba1740 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aeea030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.654+0000 7fe014ba1740 1 --2- 172.21.15.55:0/852226516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aee9af0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.654+0000 7fe014ba1740 1 -- 172.21.15.55:0/852226516 >> 172.21.15.55:0/852226516 conn(0x563f4ae20910 msgr2=0x563f4aeda3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.656+0000 7fdfeaffd640 0 client.15747 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:55:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.656+0000 7fe014ba1740 1 -- 172.21.15.55:0/852226516 shutdown_connections 2024-04-26T21:55:14.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.656+0000 7fe014ba1740 1 -- 172.21.15.55:0/852226516 wait complete. 2024-04-26T21:55:14.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.656+0000 7fe014ba1740 20 client.15747 trim_cache size 0 max 16384 2024-04-26T21:55:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.695+0000 7fe014ba1740 1 Processor -- start 2024-04-26T21:55:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.695+0000 7fe014ba1740 1 -- start start 2024-04-26T21:55:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.695+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aedb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.695+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aef1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.695+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 0x563f4aed9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.695+0000 7fe014ba1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdff8010f90 con 0x563f4aedfa30 2024-04-26T21:55:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.695+0000 7fe014ba1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdff800e860 con 0x563f4aee0c00 2024-04-26T21:55:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.695+0000 7fe014ba1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdff800c6b0 con 0x563f4aee15c0 2024-04-26T21:55:14.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.696+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aedb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.696+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aedb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43388/0 (socket says 172.21.15.55:43388) 2024-04-26T21:55:14.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.696+0000 7fe00b7fe640 1 -- 172.21.15.55:0/1346163297 learned_addr learned my addr 172.21.15.55:0/1346163297 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:14.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.697+0000 7fe00bfff640 1 --2- 172.21.15.55:0/1346163297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 0x563f4aed9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.697+0000 7fe00b7fe640 1 -- 172.21.15.55:0/1346163297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 msgr2=0x563f4aed9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.697+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/1346163297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 0x563f4aed9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.697+0000 7fe00b7fe640 1 -- 172.21.15.55:0/1346163297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 msgr2=0x563f4aef1030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.697+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/1346163297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aef1030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.697+0000 7fe00b7fe640 1 -- 172.21.15.55:0/1346163297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f4aeea8b0 con 0x563f4aedfa30 2024-04-26T21:55:14.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.701+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/1346163297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aedb4c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe004007c30 tx=0x7fe004004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dee0ec958974b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:14.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.702+0000 7fdff17fa640 1 -- 172.21.15.55:0/1346163297 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004026050 con 0x563f4aedfa30 2024-04-26T21:55:14.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.702+0000 7fdff17fa640 1 -- 172.21.15.55:0/1346163297 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00402c8d0 con 0x563f4aedfa30 2024-04-26T21:55:14.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.702+0000 7fdff17fa640 1 -- 172.21.15.55:0/1346163297 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004018610 con 0x563f4aedfa30 2024-04-26T21:55:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.704+0000 7fe014ba1740 1 -- 172.21.15.55:0/1346163297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 msgr2=0x563f4aedb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.704+0000 7fe014ba1740 1 --2- 172.21.15.55:0/1346163297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aedb4c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe004007c30 tx=0x7fe004004930 comp rx=0 tx=0).stop 2024-04-26T21:55:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.705+0000 7fe014ba1740 1 -- 172.21.15.55:0/1346163297 shutdown_connections 2024-04-26T21:55:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.705+0000 7fe014ba1740 1 --2- 172.21.15.55:0/1346163297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee15c0 0x563f4aed9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.705+0000 7fe014ba1740 1 --2- 172.21.15.55:0/1346163297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee0c00 0x563f4aef1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.705+0000 7fe014ba1740 1 --2- 172.21.15.55:0/1346163297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aedb4c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.705+0000 7fe014ba1740 1 -- 172.21.15.55:0/1346163297 >> 172.21.15.55:0/1346163297 conn(0x563f4ae20910 msgr2=0x563f4aedadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:14.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.706+0000 7fe014ba1740 1 -- 172.21.15.55:0/1346163297 shutdown_connections 2024-04-26T21:55:14.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.707+0000 7fe014ba1740 1 -- 172.21.15.55:0/1346163297 wait complete. 2024-04-26T21:55:14.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.708+0000 7fe014ba1740 1 -- start start 2024-04-26T21:55:14.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.708+0000 7fe014ba1740 1 Processor -- start 2024-04-26T21:55:14.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.709+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aeebee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.709+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee0c00 0x563f4aeec420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.709+0000 7fe014ba1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee15c0 0x563f4af599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.709+0000 7fe014ba1740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdff800cd20 con 0x563f4aedfa30 2024-04-26T21:55:14.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.709+0000 7fe014ba1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdff800e190 con 0x563f4aee15c0 2024-04-26T21:55:14.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.709+0000 7fe014ba1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdff800d3f0 con 0x563f4aee0c00 2024-04-26T21:55:14.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.709+0000 7fe014ba1740 10 client.0 _ll_register_callbacks cb 0x7fe004007b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.710+0000 7fe00affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee0c00 0x563f4aeec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.710+0000 7fe00affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee0c00 0x563f4aeec420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:58198/0 (socket says 172.21.15.55:58198) 2024-04-26T21:55:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.710+0000 7fe00affd640 1 -- 172.21.15.55:0/955962371 learned_addr learned my addr 172.21.15.55:0/955962371 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.710+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aeebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.711+0000 7fe00affd640 1 -- 172.21.15.55:0/955962371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee15c0 msgr2=0x563f4af599f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.711+0000 7fe00affd640 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee15c0 0x563f4af599f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.711+0000 7fe00affd640 1 -- 172.21.15.55:0/955962371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 msgr2=0x563f4aeebee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:14.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.711+0000 7fe00affd640 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aeebee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.711+0000 7fe00affd640 1 -- 172.21.15.55:0/955962371 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f4af59f30 con 0x563f4aee0c00 2024-04-26T21:55:14.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.711+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aeebee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:14.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.712+0000 7fe00affd640 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee0c00 0x563f4aeec420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0bccd0 tx=0x7fdffc02e820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e353b6768c951d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:14.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.712+0000 7fdfeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:55:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.712+0000 7fdfeaffd640 1 -- 172.21.15.55:0/955962371 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0bc6a0 con 0x563f4aee0c00 2024-04-26T21:55:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.713+0000 7fdfeaffd640 1 -- 172.21.15.55:0/955962371 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fdffc0b9320 con 0x563f4aee0c00 2024-04-26T21:55:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.713+0000 7fe014ba1740 1 -- 172.21.15.55:0/955962371 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe00402c8d0 con 0x563f4aee0c00 2024-04-26T21:55:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.714+0000 7fdfeaffd640 1 -- 172.21.15.55:0/955962371 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdffc036810 con 0x563f4aee0c00 2024-04-26T21:55:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.714+0000 7fe014ba1740 10 client.25285 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:14.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.714+0000 7fe014ba1740 1 -- 172.21.15.55:0/955962371 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563f4af5b220 con 0x563f4aee0c00 2024-04-26T21:55:14.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.719+0000 7fdfeaffd640 1 -- 172.21.15.55:0/955962371 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00402c8d0 con 0x563f4aee0c00 2024-04-26T21:55:14.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.719+0000 7fdfeaffd640 1 -- 172.21.15.55:0/955962371 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdffc00d340 con 0x563f4aee0c00 2024-04-26T21:55:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.721+0000 7fe014ba1740 10 client.25285 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.721+0000 7fe014ba1740 10 client.25285 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:55:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.721+0000 7fe014ba1740 20 client.25285 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.721+0000 7fe014ba1740 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563f4af5bb30 0x563f4af5df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.721+0000 7fe014ba1740 4 client.25285 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:55:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.721+0000 7fe014ba1740 1 -- 172.21.15.55:0/955962371 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f4ae4b800 con 0x563f4af5bb30 2024-04-26T21:55:14.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.727+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563f4af5bb30 0x563f4af5df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:14.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.738+0000 7fe00b7fe640 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563f4af5bb30 0x563f4af5df50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad8a0d804ed367d8 server_cookie=8d8bf9db0b53a70a in_seq=0 out_seq=0 2024-04-26T21:55:14.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.738+0000 7fdfeaffd640 10 client.25285 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:55:14.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.740+0000 7fdfeaffd640 1 -- 172.21.15.55:0/955962371 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f4ae4b800 con 0x563f4af5bb30 2024-04-26T21:55:14.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.740+0000 7fdfeaffd640 10 client.25285 handle_command_reply: tid=0 2024-04-26T21:55:14.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.741+0000 7fe014ba1740 1 client.25285 shutdown 2024-04-26T21:55:14.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.745+0000 7fe014ba1740 1 -- 172.21.15.55:0/955962371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee0c00 msgr2=0x563f4aeec420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.745+0000 7fe014ba1740 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee0c00 0x563f4aeec420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0bccd0 tx=0x7fdffc02e820 comp rx=0 tx=0).stop 2024-04-26T21:55:14.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.747+0000 7fe014ba1740 1 -- 172.21.15.55:0/955962371 shutdown_connections 2024-04-26T21:55:14.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.747+0000 7fe014ba1740 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x563f4af5bb30 0x563f4af5df50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.747+0000 7fe014ba1740 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f4aee15c0 0x563f4af599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.747+0000 7fe014ba1740 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563f4aee0c00 0x563f4aeec420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.747+0000 7fe014ba1740 1 --2- 172.21.15.55:0/955962371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563f4aedfa30 0x563f4aeebee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.747+0000 7fe014ba1740 1 -- 172.21.15.55:0/955962371 >> 172.21.15.55:0/955962371 conn(0x563f4ae20910 msgr2=0x563f4aede1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:14.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.747+0000 7fdfeaffd640 0 client.25285 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:55:14.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.747+0000 7fe014ba1740 1 -- 172.21.15.55:0/955962371 shutdown_connections 2024-04-26T21:55:14.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.748+0000 7fe014ba1740 1 -- 172.21.15.55:0/955962371 wait complete. 2024-04-26T21:55:14.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.748+0000 7fe014ba1740 20 client.25285 trim_cache size 0 max 16384 2024-04-26T21:55:14.752 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:55:14.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.754+0000 7fe010c21640 1 -- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdff40a3e40 msgr2=0x7fdff40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.754+0000 7fe010c21640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdff40a3e40 0x7fdff40a6300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x563f4aee0b00 tx=0x7fe000008040 comp rx=0 tx=0).stop 2024-04-26T21:55:14.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.754+0000 7fe010c21640 1 -- 172.21.15.55:0/3797209756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c10a510 msgr2=0x7fe00c1a7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:14.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.754+0000 7fe010c21640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c10a510 0x7fe00c1a7d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdffc000c00 tx=0x7fdffc003820 comp rx=0 tx=0).stop 2024-04-26T21:55:14.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.754+0000 7fe010c21640 1 -- 172.21.15.55:0/3797209756 shutdown_connections 2024-04-26T21:55:14.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.754+0000 7fe010c21640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdff40a3e40 0x7fdff40a6300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.754+0000 7fe010c21640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe00c110c70 0x7fe00c1aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.754+0000 7fe010c21640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c10a510 0x7fe00c1a7d30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.755+0000 7fe010c21640 1 --2- 172.21.15.55:0/3797209756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe00c073e50 0x7fe00c1a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:14.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.755+0000 7fe010c21640 1 -- 172.21.15.55:0/3797209756 >> 172.21.15.55:0/3797209756 conn(0x7fe00c06f780 msgr2=0x7fe00c10cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:14.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.755+0000 7fe010c21640 1 -- 172.21.15.55:0/3797209756 shutdown_connections 2024-04-26T21:55:14.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:14.755+0000 7fe010c21640 1 -- 172.21.15.55:0/3797209756 wait complete. 2024-04-26T21:55:14.785 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:55:15.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.046+0000 7fa719dc3640 1 -- 172.21.15.55:0/595294046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa71410a010 msgr2=0x7fa71410a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.046+0000 7fa719dc3640 1 --2- 172.21.15.55:0/595294046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa71410a010 0x7fa71410a470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009be0 tx=0x7fa70c02a710 comp rx=0 tx=0).stop 2024-04-26T21:55:15.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.046+0000 7fa719dc3640 1 -- 172.21.15.55:0/595294046 shutdown_connections 2024-04-26T21:55:15.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.046+0000 7fa719dc3640 1 --2- 172.21.15.55:0/595294046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa71410a9b0 0x7fa71410f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.046+0000 7fa719dc3640 1 --2- 172.21.15.55:0/595294046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa71410a010 0x7fa71410a470 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009be0 tx=0x7fa70c02a710 comp rx=0 tx=0).stop 2024-04-26T21:55:15.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.046+0000 7fa719dc3640 1 --2- 172.21.15.55:0/595294046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa714073d70 0x7fa714074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.046+0000 7fa719dc3640 1 -- 172.21.15.55:0/595294046 >> 172.21.15.55:0/595294046 conn(0x7fa71406f840 msgr2=0x7fa714071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.051+0000 7fa719dc3640 1 -- 172.21.15.55:0/595294046 shutdown_connections 2024-04-26T21:55:15.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.056+0000 7fa719dc3640 1 -- 172.21.15.55:0/595294046 wait complete. 2024-04-26T21:55:15.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.057+0000 7fa719dc3640 1 Processor -- start 2024-04-26T21:55:15.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.061+0000 7fa719dc3640 1 -- start start 2024-04-26T21:55:15.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.061+0000 7fa719dc3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa714073d70 0x7fa7141a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.061+0000 7fa719dc3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71410a9b0 0x7fa7141a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.061+0000 7fa719dc3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7141acf90 0x7fa7141af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.061+0000 7fa719dc3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa71411d130 con 0x7fa71410a9b0 2024-04-26T21:55:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.061+0000 7fa719dc3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa71411cfb0 con 0x7fa714073d70 2024-04-26T21:55:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.061+0000 7fa719dc3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa71411d2b0 con 0x7fa7141acf90 2024-04-26T21:55:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.062+0000 7fa713fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71410a9b0 0x7fa7141a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.062+0000 7fa713fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71410a9b0 0x7fa7141a7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43424/0 (socket says 172.21.15.55:43424) 2024-04-26T21:55:15.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.062+0000 7fa713fff640 1 -- 172.21.15.55:0/3973146301 learned_addr learned my addr 172.21.15.55:0/3973146301 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.063+0000 7fa7195c2640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7141acf90 0x7fa7141af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.063+0000 7fa7195c2640 1 -- 172.21.15.55:0/3973146301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa714073d70 msgr2=0x7fa7141a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.063+0000 7fa7195c2640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa714073d70 0x7fa7141a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.063+0000 7fa7195c2640 1 -- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71410a9b0 msgr2=0x7fa7141a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.063+0000 7fa7195c2640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71410a9b0 0x7fa7141a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.063+0000 7fa7195c2640 1 -- 172.21.15.55:0/3973146301 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7141af9f0 con 0x7fa7141acf90 2024-04-26T21:55:15.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.064+0000 7fa7195c2640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7141acf90 0x7fa7141af380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa708009fd0 tx=0x7fa70800eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.065+0000 7fa711ffb640 1 -- 172.21.15.55:0/3973146301 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708004230 con 0x7fa7141acf90 2024-04-26T21:55:15.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.065+0000 7fa711ffb640 1 -- 172.21.15.55:0/3973146301 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7080043d0 con 0x7fa7141acf90 2024-04-26T21:55:15.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.065+0000 7fa711ffb640 1 -- 172.21.15.55:0/3973146301 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080047c0 con 0x7fa7141acf90 2024-04-26T21:55:15.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.065+0000 7fa719dc3640 1 -- 172.21.15.55:0/3973146301 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7141afce0 con 0x7fa7141acf90 2024-04-26T21:55:15.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.065+0000 7fa719dc3640 1 -- 172.21.15.55:0/3973146301 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7141b01b0 con 0x7fa7141acf90 2024-04-26T21:55:15.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.075+0000 7fa71de19740 1 Processor -- start 2024-04-26T21:55:15.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.075+0000 7fa71de19740 1 -- start start 2024-04-26T21:55:15.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.076+0000 7fa71de19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77fda00 0x5652e77fde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.076+0000 7fa71de19740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 0x5652e77ff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.077+0000 7fa71de19740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e7805e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.077+0000 7fa71de19740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5652e7761d80 con 0x5652e77febd0 2024-04-26T21:55:15.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.077+0000 7fa71de19740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5652e771ba20 con 0x5652e77ff590 2024-04-26T21:55:15.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.077+0000 7fa71de19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5652e775dd40 con 0x5652e77fda00 2024-04-26T21:55:15.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.077+0000 7fa711ffb640 1 -- 172.21.15.55:0/3973146301 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa708004a00 con 0x7fa7141acf90 2024-04-26T21:55:15.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.079+0000 7fa713fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77fda00 0x5652e77fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.079+0000 7fa713fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77fda00 0x5652e77fde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:58228/0 (socket says 172.21.15.55:58228) 2024-04-26T21:55:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.079+0000 7fa713fff640 1 -- 172.21.15.55:0/653160430 learned_addr learned my addr 172.21.15.55:0/653160430 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.080+0000 7fa713fff640 1 --2- 172.21.15.55:0/653160430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e7805e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.081+0000 7fa713fff640 1 -- 172.21.15.55:0/653160430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 msgr2=0x5652e7805e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.081+0000 7fa713fff640 1 --2- 172.21.15.55:0/653160430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e7805e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.081+0000 7fa711ffb640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6fc0a3e10 0x7fa6fc0a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.081+0000 7fa713fff640 1 -- 172.21.15.55:0/653160430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 msgr2=0x5652e77ff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.081+0000 7fa713fff640 1 --2- 172.21.15.55:0/653160430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 0x5652e77ff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.081+0000 7fa713fff640 1 -- 172.21.15.55:0/653160430 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e7726c30 con 0x5652e77fda00 2024-04-26T21:55:15.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.082+0000 7fa711ffb640 1 -- 172.21.15.55:0/3973146301 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa7080d1550 con 0x7fa7141acf90 2024-04-26T21:55:15.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.082+0000 7fa713fff640 1 --2- 172.21.15.55:0/653160430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77fda00 0x5652e77fde20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa70c004af0 tx=0x7fa70c005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad28980a538c1158 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.083+0000 7fa7195c2640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6fc0a3e10 0x7fa6fc0a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.088+0000 7fa7195c2640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6fc0a3e10 0x7fa6fc0a62d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa708009510 tx=0x7fa708011000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.090+0000 7fa6faffd640 1 -- 172.21.15.55:0/653160430 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c047070 con 0x5652e77fda00 2024-04-26T21:55:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.090+0000 7fa6faffd640 1 -- 172.21.15.55:0/653160430 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa70c003e20 con 0x5652e77fda00 2024-04-26T21:55:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.090+0000 7fa6faffd640 1 -- 172.21.15.55:0/653160430 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c036070 con 0x5652e77fda00 2024-04-26T21:55:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.091+0000 7fa71de19740 1 -- 172.21.15.55:0/653160430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77fda00 msgr2=0x5652e77fde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.091+0000 7fa71de19740 1 --2- 172.21.15.55:0/653160430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77fda00 0x5652e77fde20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa70c004af0 tx=0x7fa70c005e30 comp rx=0 tx=0).stop 2024-04-26T21:55:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.091+0000 7fa71de19740 1 -- 172.21.15.55:0/653160430 shutdown_connections 2024-04-26T21:55:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.091+0000 7fa71de19740 1 --2- 172.21.15.55:0/653160430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e7805e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.092+0000 7fa71de19740 1 --2- 172.21.15.55:0/653160430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 0x5652e77ff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.092+0000 7fa71de19740 1 --2- 172.21.15.55:0/653160430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77fda00 0x5652e77fde20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.092+0000 7fa71de19740 1 -- 172.21.15.55:0/653160430 >> 172.21.15.55:0/653160430 conn(0x5652e773e910 msgr2=0x5652e77f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.092+0000 7fa71de19740 1 -- 172.21.15.55:0/653160430 shutdown_connections 2024-04-26T21:55:15.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.094+0000 7fa71de19740 1 -- 172.21.15.55:0/653160430 wait complete. 2024-04-26T21:55:15.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.094+0000 7fa71de19740 1 -- start start 2024-04-26T21:55:15.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.095+0000 7fa71de19740 1 Processor -- start 2024-04-26T21:55:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.096+0000 7fa71de19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77febd0 0x5652e7807e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.096+0000 7fa71de19740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.096+0000 7fa71de19740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e780d3d0 0x5652e780f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.096+0000 7fa71de19740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5652e7806540 con 0x5652e780d3d0 2024-04-26T21:55:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.096+0000 7fa71de19740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5652e76e8dc0 con 0x5652e77ff590 2024-04-26T21:55:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.096+0000 7fa71de19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5652e78066c0 con 0x5652e77febd0 2024-04-26T21:55:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.096+0000 7fa71de19740 10 client.0 _ll_register_callbacks cb 0x5652e7704600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:15.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.100+0000 7fa718dc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.100+0000 7fa718dc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35096/0 (socket says 172.21.15.55:35096) 2024-04-26T21:55:15.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.100+0000 7fa718dc1640 1 -- 172.21.15.55:0/1238500941 learned_addr learned my addr 172.21.15.55:0/1238500941 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.103+0000 7fa713fff640 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77febd0 0x5652e7807e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.103+0000 7fa713fff640 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e780d3d0 0x5652e780f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.104+0000 7fa718dc1640 1 -- 172.21.15.55:0/1238500941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77febd0 msgr2=0x5652e7807e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.104+0000 7fa718dc1640 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77febd0 0x5652e7807e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.104+0000 7fa718dc1640 1 -- 172.21.15.55:0/1238500941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e780d3d0 msgr2=0x5652e780f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.104+0000 7fa718dc1640 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e780d3d0 0x5652e780f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.104+0000 7fa718dc1640 1 -- 172.21.15.55:0/1238500941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e74733f0 con 0x5652e77ff590 2024-04-26T21:55:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.106+0000 7fa6f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:55:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.106+0000 7fa718dc1640 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78083a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa70400bc40 tx=0x7fa70400d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=130fef84aa2c5f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.109+0000 7fa6f3fff640 1 -- 172.21.15.55:0/1238500941 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704016070 con 0x5652e77ff590 2024-04-26T21:55:15.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.109+0000 7fa71de19740 1 -- 172.21.15.55:0/1238500941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa70c003e20 con 0x5652e77ff590 2024-04-26T21:55:15.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.109+0000 7fa6f3fff640 1 -- 172.21.15.55:0/1238500941 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa704010660 con 0x5652e77ff590 2024-04-26T21:55:15.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.110+0000 7fa6f3fff640 1 -- 172.21.15.55:0/1238500941 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa704020050 con 0x5652e77ff590 2024-04-26T21:55:15.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.111+0000 7fa71de19740 10 client.25340 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:15.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.111+0000 7fa71de19740 1 -- 172.21.15.55:0/1238500941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5652e753d7d0 con 0x5652e77ff590 2024-04-26T21:55:15.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.112+0000 7fa6f3fff640 1 -- 172.21.15.55:0/1238500941 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa70c003e20 con 0x5652e77ff590 2024-04-26T21:55:15.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.112+0000 7fa6f3fff640 1 -- 172.21.15.55:0/1238500941 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa70401b9a0 con 0x5652e77ff590 2024-04-26T21:55:15.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.112+0000 7fa71de19740 10 client.25340 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.112+0000 7fa71de19740 10 client.25340 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.112+0000 7fa71de19740 20 client.25340 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.112+0000 7fa71de19740 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5652e7814570 0x5652e7816990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.112+0000 7fa71de19740 4 client.25340 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:15.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.112+0000 7fa71de19740 1 -- 172.21.15.55:0/1238500941 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652e776f990 con 0x5652e7814570 2024-04-26T21:55:15.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.122+0000 7fa713fff640 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5652e7814570 0x5652e7816990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.144+0000 7fa6f3fff640 10 client.25340 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:55:15.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.144+0000 7fa713fff640 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5652e7814570 0x5652e7816990 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72c70e3089cea97f server_cookie=3189479f9159e18 in_seq=0 out_seq=0 2024-04-26T21:55:15.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.147+0000 7fa6f3fff640 1 -- 172.21.15.55:0/1238500941 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652e776f990 con 0x5652e7814570 2024-04-26T21:55:15.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.147+0000 7fa6f3fff640 10 client.25340 handle_command_reply: tid=0 2024-04-26T21:55:15.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.147+0000 7fa71de19740 1 client.25340 shutdown 2024-04-26T21:55:15.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.149+0000 7fa71de19740 1 -- 172.21.15.55:0/1238500941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 msgr2=0x5652e78083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.149+0000 7fa71de19740 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78083a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa70400bc40 tx=0x7fa70400d6b0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.150+0000 7fa71de19740 1 -- 172.21.15.55:0/1238500941 shutdown_connections 2024-04-26T21:55:15.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.150+0000 7fa71de19740 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5652e7814570 0x5652e7816990 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.150+0000 7fa71de19740 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e780d3d0 0x5652e780f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.150+0000 7fa71de19740 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78083a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.150+0000 7fa71de19740 1 --2- 172.21.15.55:0/1238500941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e77febd0 0x5652e7807e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.150+0000 7fa71de19740 1 -- 172.21.15.55:0/1238500941 >> 172.21.15.55:0/1238500941 conn(0x5652e773e910 msgr2=0x5652e77f8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.153+0000 7fa6f3fff640 0 client.25340 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:55:15.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.161+0000 7fa71de19740 1 -- 172.21.15.55:0/1238500941 shutdown_connections 2024-04-26T21:55:15.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.161+0000 7fa71de19740 1 -- 172.21.15.55:0/1238500941 wait complete. 2024-04-26T21:55:15.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.161+0000 7fa71de19740 20 client.25340 trim_cache size 0 max 16384 2024-04-26T21:55:15.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:15 smithi055 ceph-mon[29221]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 794 MiB/s wr, 1.29k op/s 2024-04-26T21:55:15.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:15 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1845091492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:15.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.212+0000 7fa71de19740 1 Processor -- start 2024-04-26T21:55:15.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.214+0000 7fa71de19740 1 -- start start 2024-04-26T21:55:15.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.215+0000 7fa71de19740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77febd0 0x5652e7807150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.215+0000 7fa71de19740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77ff590 0x5652e7807690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.215+0000 7fa71de19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e77fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.215+0000 7fa71de19740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa700010e60 con 0x5652e77ff590 2024-04-26T21:55:15.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.215+0000 7fa71de19740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa70000e730 con 0x5652e77febd0 2024-04-26T21:55:15.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.215+0000 7fa71de19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa70000c580 con 0x5652e780d3d0 2024-04-26T21:55:15.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.215+0000 7fa713fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77febd0 0x5652e7807150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.215+0000 7fa713fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77febd0 0x5652e7807150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35106/0 (socket says 172.21.15.55:35106) 2024-04-26T21:55:15.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.216+0000 7fa713fff640 1 -- 172.21.15.55:0/491939123 learned_addr learned my addr 172.21.15.55:0/491939123 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.216+0000 7fa713fff640 1 --2- 172.21.15.55:0/491939123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e77fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.217+0000 7fa713fff640 1 -- 172.21.15.55:0/491939123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77febd0 msgr2=0x5652e7807150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.217+0000 7fa713fff640 1 --2- 172.21.15.55:0/491939123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77febd0 0x5652e7807150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.217+0000 7fa713fff640 1 -- 172.21.15.55:0/491939123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77ff590 msgr2=0x5652e7807690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:55:15.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.217+0000 7fa713fff640 1 --2- 172.21.15.55:0/491939123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77ff590 0x5652e7807690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.217+0000 7fa713fff640 1 -- 172.21.15.55:0/491939123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e77f3820 con 0x5652e780d3d0 2024-04-26T21:55:15.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.218+0000 7fa713fff640 1 --2- 172.21.15.55:0/491939123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e77fc220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa70c030860 tx=0x7fa70c043040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e229b5c4de3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.222+0000 7fa6fa7fc640 1 -- 172.21.15.55:0/491939123 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c036040 con 0x5652e780d3d0 2024-04-26T21:55:15.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.222+0000 7fa6fa7fc640 1 -- 172.21.15.55:0/491939123 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa70c003b70 con 0x5652e780d3d0 2024-04-26T21:55:15.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.222+0000 7fa6fa7fc640 1 -- 172.21.15.55:0/491939123 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c047040 con 0x5652e780d3d0 2024-04-26T21:55:15.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.224+0000 7fa71de19740 1 -- 172.21.15.55:0/491939123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 msgr2=0x5652e77fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.224+0000 7fa71de19740 1 --2- 172.21.15.55:0/491939123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e77fc220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa70c030860 tx=0x7fa70c043040 comp rx=0 tx=0).stop 2024-04-26T21:55:15.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.226+0000 7fa71de19740 1 -- 172.21.15.55:0/491939123 shutdown_connections 2024-04-26T21:55:15.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.226+0000 7fa71de19740 1 --2- 172.21.15.55:0/491939123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e77fc220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.226+0000 7fa71de19740 1 --2- 172.21.15.55:0/491939123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77ff590 0x5652e7807690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.226+0000 7fa71de19740 1 --2- 172.21.15.55:0/491939123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77febd0 0x5652e7807150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.226+0000 7fa71de19740 1 -- 172.21.15.55:0/491939123 >> 172.21.15.55:0/491939123 conn(0x5652e773e910 msgr2=0x5652e7802670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.237+0000 7fa71de19740 1 -- 172.21.15.55:0/491939123 shutdown_connections 2024-04-26T21:55:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.242+0000 7fa71de19740 1 -- 172.21.15.55:0/491939123 wait complete. 2024-04-26T21:55:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.243+0000 7fa71de19740 1 -- start start 2024-04-26T21:55:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.243+0000 7fa71de19740 1 Processor -- start 2024-04-26T21:55:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.245+0000 7fa71de19740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 0x5652e7825370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.245+0000 7fa71de19740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.245+0000 7fa71de19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e7826790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.245+0000 7fa71de19740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa70000cbf0 con 0x5652e77febd0 2024-04-26T21:55:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.245+0000 7fa71de19740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa70000e060 con 0x5652e77ff590 2024-04-26T21:55:15.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.245+0000 7fa71de19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa70000d2c0 con 0x5652e780d3d0 2024-04-26T21:55:15.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.246+0000 7fa71de19740 10 client.0 _ll_register_callbacks cb 0x7fa70400b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:15.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.250+0000 7fa713fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e7826790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.250+0000 7fa713fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 0x5652e7825370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.250+0000 7fa713fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 0x5652e7825370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43460/0 (socket says 172.21.15.55:43460) 2024-04-26T21:55:15.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.250+0000 7fa713fff640 1 -- 172.21.15.55:0/2198049245 learned_addr learned my addr 172.21.15.55:0/2198049245 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.250+0000 7fa718dc1640 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.253+0000 7fa718dc1640 1 -- 172.21.15.55:0/2198049245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 msgr2=0x5652e7826790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.253+0000 7fa718dc1640 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e7826790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.253+0000 7fa718dc1640 1 -- 172.21.15.55:0/2198049245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 msgr2=0x5652e7825370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.253+0000 7fa718dc1640 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 0x5652e7825370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.253+0000 7fa718dc1640 1 -- 172.21.15.55:0/2198049245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e7826d80 con 0x5652e77ff590 2024-04-26T21:55:15.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.260+0000 7fa718dc1640 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78258b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5652e77ff9f0 tx=0x7fa70400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b4e3a062c5542c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.265+0000 7fa6f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:55:15.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.265+0000 7fa6f3fff640 1 -- 172.21.15.55:0/2198049245 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704029070 con 0x5652e77ff590 2024-04-26T21:55:15.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.265+0000 7fa71de19740 1 -- 172.21.15.55:0/2198049245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa70c003b70 con 0x5652e77ff590 2024-04-26T21:55:15.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.265+0000 7fa6f3fff640 1 -- 172.21.15.55:0/2198049245 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa70401b850 con 0x5652e77ff590 2024-04-26T21:55:15.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.268+0000 7fa6f3fff640 1 -- 172.21.15.55:0/2198049245 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa704035050 con 0x5652e77ff590 2024-04-26T21:55:15.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.271+0000 7fa71de19740 10 client.25346 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:15.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.271+0000 7fa71de19740 1 -- 172.21.15.55:0/2198049245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5652e7828080 con 0x5652e77ff590 2024-04-26T21:55:15.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.275+0000 7fa6f3fff640 1 -- 172.21.15.55:0/2198049245 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa70c003b70 con 0x5652e77ff590 2024-04-26T21:55:15.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.275+0000 7fa6f3fff640 1 -- 172.21.15.55:0/2198049245 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa704010460 con 0x5652e77ff590 2024-04-26T21:55:15.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.275+0000 7fa71de19740 10 client.25346 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:15.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.275+0000 7fa71de19740 10 client.25346 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:55:15.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.275+0000 7fa71de19740 20 client.25346 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:15.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.275+0000 7fa71de19740 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5652e787ac40 0x5652e787d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.275+0000 7fa71de19740 4 client.25346 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:55:15.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.275+0000 7fa71de19740 1 -- 172.21.15.55:0/2198049245 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652e7769800 con 0x5652e787ac40 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.279+0000 7fa713fff640 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5652e787ac40 0x5652e787d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.281+0000 7fa713fff640 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5652e787ac40 0x5652e787d060 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea5bff8fd2621082 server_cookie=39808a83660a12cb in_seq=0 out_seq=0 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.281+0000 7fa6f3fff640 10 client.25346 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.281+0000 7fa6f3fff640 1 -- 172.21.15.55:0/2198049245 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652e7769800 con 0x5652e787ac40 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.281+0000 7fa6f3fff640 10 client.25346 handle_command_reply: tid=0 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.281+0000 7fa71de19740 1 client.25346 shutdown 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.282+0000 7fa71de19740 1 -- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 msgr2=0x5652e78258b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.282+0000 7fa71de19740 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78258b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5652e77ff9f0 tx=0x7fa70400b3b0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.282+0000 7fa71de19740 1 -- 172.21.15.55:0/2198049245 shutdown_connections 2024-04-26T21:55:15.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.282+0000 7fa71de19740 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5652e787ac40 0x5652e787d060 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.282+0000 7fa71de19740 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652e780d3d0 0x5652e7826790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.282+0000 7fa71de19740 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652e77ff590 0x5652e78258b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.282+0000 7fa71de19740 1 --2- 172.21.15.55:0/2198049245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652e77febd0 0x5652e7825370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.282+0000 7fa71de19740 1 -- 172.21.15.55:0/2198049245 >> 172.21.15.55:0/2198049245 conn(0x5652e773e910 msgr2=0x5652e7807150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.287+0000 7fa6f3fff640 0 client.25346 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:55:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.287+0000 7fa71de19740 1 -- 172.21.15.55:0/2198049245 shutdown_connections 2024-04-26T21:55:15.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.287+0000 7fa71de19740 1 -- 172.21.15.55:0/2198049245 wait complete. 2024-04-26T21:55:15.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.287+0000 7fa71de19740 20 client.25346 trim_cache size 0 max 16384 2024-04-26T21:55:15.293 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:55:15.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.295+0000 7fa719dc3640 1 -- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6fc0a3e10 msgr2=0x7fa6fc0a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.295+0000 7fa719dc3640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6fc0a3e10 0x7fa6fc0a62d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa708009510 tx=0x7fa708011000 comp rx=0 tx=0).stop 2024-04-26T21:55:15.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.295+0000 7fa719dc3640 1 -- 172.21.15.55:0/3973146301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7141acf90 msgr2=0x7fa7141af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.295+0000 7fa719dc3640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7141acf90 0x7fa7141af380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa708009fd0 tx=0x7fa70800eba0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.299+0000 7fa719dc3640 1 -- 172.21.15.55:0/3973146301 shutdown_connections 2024-04-26T21:55:15.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.299+0000 7fa719dc3640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa6fc0a3e10 0x7fa6fc0a62d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.299+0000 7fa719dc3640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7141acf90 0x7fa7141af380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.299+0000 7fa719dc3640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa71410a9b0 0x7fa7141a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.299+0000 7fa719dc3640 1 --2- 172.21.15.55:0/3973146301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa714073d70 0x7fa7141a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.299+0000 7fa719dc3640 1 -- 172.21.15.55:0/3973146301 >> 172.21.15.55:0/3973146301 conn(0x7fa71406f840 msgr2=0x7fa714071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.301+0000 7fa719dc3640 1 -- 172.21.15.55:0/3973146301 shutdown_connections 2024-04-26T21:55:15.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.302+0000 7fa719dc3640 1 -- 172.21.15.55:0/3973146301 wait complete. 2024-04-26T21:55:15.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:15 smithi165 ceph-mon[33296]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 794 MiB/s wr, 1.29k op/s 2024-04-26T21:55:15.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:15 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1845091492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:15.355 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:55:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:15 smithi064 ceph-mon[32765]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 794 MiB/s wr, 1.29k op/s 2024-04-26T21:55:15.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:15 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1845091492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:15.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.642+0000 7f91c23e3640 1 -- 172.21.15.55:0/537381373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91bc10a9b0 msgr2=0x7f91bc10f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.642+0000 7f91c23e3640 1 --2- 172.21.15.55:0/537381373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91bc10a9b0 0x7f91bc10f270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009860 tx=0x7f91b802a380 comp rx=0 tx=0).stop 2024-04-26T21:55:15.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.646+0000 7f91c23e3640 1 -- 172.21.15.55:0/537381373 shutdown_connections 2024-04-26T21:55:15.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.646+0000 7f91c23e3640 1 --2- 172.21.15.55:0/537381373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91bc10a9b0 0x7f91bc10f270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.646+0000 7f91c23e3640 1 --2- 172.21.15.55:0/537381373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91bc10a010 0x7f91bc10a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.646+0000 7f91c23e3640 1 --2- 172.21.15.55:0/537381373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91bc073d70 0x7f91bc074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.646+0000 7f91c23e3640 1 -- 172.21.15.55:0/537381373 >> 172.21.15.55:0/537381373 conn(0x7f91bc06f840 msgr2=0x7f91bc071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.658+0000 7f91c23e3640 1 -- 172.21.15.55:0/537381373 shutdown_connections 2024-04-26T21:55:15.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.658+0000 7f91c23e3640 1 -- 172.21.15.55:0/537381373 wait complete. 2024-04-26T21:55:15.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.659+0000 7f91c23e3640 1 Processor -- start 2024-04-26T21:55:15.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.659+0000 7f91c23e3640 1 -- start start 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.660+0000 7f91c23e3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91bc073d70 0x7f91bc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.660+0000 7f91c23e3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91bc10a010 0x7f91bc19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.660+0000 7f91c23e3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91bc10a9b0 0x7f91bc1a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.660+0000 7f91c23e3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc112ad0 con 0x7f91bc10a010 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.660+0000 7f91c23e3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc112950 con 0x7f91bc073d70 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.660+0000 7f91c23e3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc112c50 con 0x7f91bc10a9b0 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.666+0000 7f91c13e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91bc073d70 0x7f91bc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.666+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91bc10a9b0 0x7f91bc1a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.666+0000 7f91c13e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91bc073d70 0x7f91bc19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35120/0 (socket says 172.21.15.55:35120) 2024-04-26T21:55:15.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.666+0000 7f91c13e1640 1 -- 172.21.15.55:0/2047931457 learned_addr learned my addr 172.21.15.55:0/2047931457 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.667+0000 7f91c13e1640 1 -- 172.21.15.55:0/2047931457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91bc10a9b0 msgr2=0x7f91bc1a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.667+0000 7f91c13e1640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91bc10a9b0 0x7f91bc1a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.667+0000 7f91c13e1640 1 -- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91bc10a010 msgr2=0x7f91bc19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.668+0000 7f91c13e1640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91bc10a010 0x7f91bc19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.668+0000 7f91c13e1640 1 -- 172.21.15.55:0/2047931457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc1a6d10 con 0x7f91bc073d70 2024-04-26T21:55:15.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.669+0000 7f91c13e1640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91bc073d70 0x7f91bc19f0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91ac01ea20 tx=0x7f91ac01eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.672+0000 7f91b27fc640 1 -- 172.21.15.55:0/2047931457 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac028070 con 0x7f91bc073d70 2024-04-26T21:55:15.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.673+0000 7f91c23e3640 1 -- 172.21.15.55:0/2047931457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc1a6fa0 con 0x7f91bc073d70 2024-04-26T21:55:15.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.673+0000 7f91c23e3640 1 -- 172.21.15.55:0/2047931457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc1a74c0 con 0x7f91bc073d70 2024-04-26T21:55:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.680+0000 7f91c6439740 1 Processor -- start 2024-04-26T21:55:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.680+0000 7f91b27fc640 1 -- 172.21.15.55:0/2047931457 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91ac01abc0 con 0x7f91bc073d70 2024-04-26T21:55:15.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.680+0000 7f91b27fc640 1 -- 172.21.15.55:0/2047931457 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac020630 con 0x7f91bc073d70 2024-04-26T21:55:15.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.681+0000 7f91c6439740 1 -- start start 2024-04-26T21:55:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.681+0000 7f91c6439740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac217a00 0x5621ac217e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.681+0000 7f91c6439740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac218bd0 0x5621ac219050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.682+0000 7f91c6439740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 0x5621ac21fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.682+0000 7f91c6439740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5621ac17bd80 con 0x5621ac217a00 2024-04-26T21:55:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.682+0000 7f91c6439740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5621ac135a20 con 0x5621ac218bd0 2024-04-26T21:55:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.682+0000 7f91c6439740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5621ac177d40 con 0x5621ac219590 2024-04-26T21:55:15.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.685+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac217a00 0x5621ac217e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.685+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac217a00 0x5621ac217e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43498/0 (socket says 172.21.15.55:43498) 2024-04-26T21:55:15.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.685+0000 7f91c1be2640 1 -- 172.21.15.55:0/3027467302 learned_addr learned my addr 172.21.15.55:0/3027467302 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.687+0000 7f91c0be0640 1 --2- 172.21.15.55:0/3027467302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 0x5621ac21fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.688+0000 7f91c1be2640 1 -- 172.21.15.55:0/3027467302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 msgr2=0x5621ac21fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.688+0000 7f91c1be2640 1 --2- 172.21.15.55:0/3027467302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 0x5621ac21fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.688+0000 7f91c1be2640 1 -- 172.21.15.55:0/3027467302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac218bd0 msgr2=0x5621ac219050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:15.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.688+0000 7f91c1be2640 1 --2- 172.21.15.55:0/3027467302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac218bd0 0x5621ac219050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.688+0000 7f91c1be2640 1 -- 172.21.15.55:0/3027467302 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ac140c30 con 0x5621ac217a00 2024-04-26T21:55:15.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.691+0000 7f91c1be2640 1 --2- 172.21.15.55:0/3027467302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac217a00 0x5621ac217e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f91b8030420 tx=0x7f91b8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e6204d7f0dbd97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.691+0000 7f91b27fc640 1 -- 172.21.15.55:0/2047931457 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f91ac021470 con 0x7f91bc073d70 2024-04-26T21:55:15.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.695+0000 7f91a77fe640 1 -- 172.21.15.55:0/3027467302 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8041030 con 0x5621ac217a00 2024-04-26T21:55:15.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.695+0000 7f91a77fe640 1 -- 172.21.15.55:0/3027467302 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b8038040 con 0x5621ac217a00 2024-04-26T21:55:15.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.695+0000 7f91a77fe640 1 -- 172.21.15.55:0/3027467302 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8037070 con 0x5621ac217a00 2024-04-26T21:55:15.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.697+0000 7f91c6439740 1 -- 172.21.15.55:0/3027467302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac217a00 msgr2=0x5621ac217e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.697+0000 7f91c6439740 1 --2- 172.21.15.55:0/3027467302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac217a00 0x5621ac217e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f91b8030420 tx=0x7f91b8009580 comp rx=0 tx=0).stop 2024-04-26T21:55:15.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.697+0000 7f91c6439740 1 -- 172.21.15.55:0/3027467302 shutdown_connections 2024-04-26T21:55:15.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.697+0000 7f91c6439740 1 --2- 172.21.15.55:0/3027467302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 0x5621ac21fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.697+0000 7f91c6439740 1 --2- 172.21.15.55:0/3027467302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac218bd0 0x5621ac219050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.697+0000 7f91c6439740 1 --2- 172.21.15.55:0/3027467302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac217a00 0x5621ac217e20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.697+0000 7f91c6439740 1 -- 172.21.15.55:0/3027467302 >> 172.21.15.55:0/3027467302 conn(0x5621ac158910 msgr2=0x5621ac2132b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.695+0000 7f91b27fc640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f91a80a3e40 0x7f91a80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.699+0000 7f91b27fc640 1 -- 172.21.15.55:0/2047931457 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f91ac0e11f0 con 0x7f91bc073d70 2024-04-26T21:55:15.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.699+0000 7f91c13e1640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f91a80a3e40 0x7f91a80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.701+0000 7f91c6439740 1 -- 172.21.15.55:0/3027467302 shutdown_connections 2024-04-26T21:55:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.701+0000 7f91c6439740 1 -- 172.21.15.55:0/3027467302 wait complete. 2024-04-26T21:55:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.702+0000 7f91c6439740 1 -- start start 2024-04-26T21:55:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.702+0000 7f91c6439740 1 Processor -- start 2024-04-26T21:55:15.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.703+0000 7f91c13e1640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f91a80a3e40 0x7f91a80a6300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0a9730 tx=0x7f91ac0acd60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.703+0000 7f91c6439740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac221ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.703+0000 7f91c6439740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 0x5621ac222000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.703+0000 7f91c6439740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac226fc0 0x5621ac2293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.703+0000 7f91c6439740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5621ac220590 con 0x5621ac218bd0 2024-04-26T21:55:15.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.703+0000 7f91c6439740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5621ac102dc0 con 0x5621ac226fc0 2024-04-26T21:55:15.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.703+0000 7f91c6439740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5621ac220710 con 0x5621ac219590 2024-04-26T21:55:15.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.703+0000 7f91c6439740 10 client.0 _ll_register_callbacks cb 0x5621ac11e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:15.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.704+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac221ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.704+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac221ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43504/0 (socket says 172.21.15.55:43504) 2024-04-26T21:55:15.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.704+0000 7f91c1be2640 1 -- 172.21.15.55:0/3926798686 learned_addr learned my addr 172.21.15.55:0/3926798686 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.704+0000 7f91c0be0640 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 0x5621ac222000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.705+0000 7f91c1be2640 1 -- 172.21.15.55:0/3926798686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 msgr2=0x5621ac222000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.705+0000 7f91c1be2640 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 0x5621ac222000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.705+0000 7f91c1be2640 1 -- 172.21.15.55:0/3926798686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac226fc0 msgr2=0x5621ac2293b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.705+0000 7f91c1be2640 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac226fc0 0x5621ac2293b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.706+0000 7f91c1be2640 1 -- 172.21.15.55:0/3926798686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621abe8d3f0 con 0x5621ac218bd0 2024-04-26T21:55:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.707+0000 7f91c1be2640 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac221ac0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f91b8030380 tx=0x7f91b8030820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d08722976f1825 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.708+0000 7f91a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:15.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.708+0000 7f91a4ff9640 1 -- 172.21.15.55:0/3926798686 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b804c070 con 0x5621ac218bd0 2024-04-26T21:55:15.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.708+0000 7f91a4ff9640 1 -- 172.21.15.55:0/3926798686 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f91b8030da0 con 0x5621ac218bd0 2024-04-26T21:55:15.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.709+0000 7f91c6439740 1 -- 172.21.15.55:0/3926798686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91b8038040 con 0x5621ac218bd0 2024-04-26T21:55:15.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.710+0000 7f91a4ff9640 1 -- 172.21.15.55:0/3926798686 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f91b8057050 con 0x5621ac218bd0 2024-04-26T21:55:15.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.710+0000 7f91c6439740 10 client.15774 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:15.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.710+0000 7f91c6439740 1 -- 172.21.15.55:0/3926798686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5621abf577d0 con 0x5621ac218bd0 2024-04-26T21:55:15.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.712+0000 7f91a4ff9640 1 -- 172.21.15.55:0/3926798686 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b8038040 con 0x5621ac218bd0 2024-04-26T21:55:15.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.712+0000 7f91a4ff9640 1 -- 172.21.15.55:0/3926798686 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f91b80327a0 con 0x5621ac218bd0 2024-04-26T21:55:15.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.712+0000 7f91c6439740 10 client.15774 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:15.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.712+0000 7f91c6439740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:15.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.712+0000 7f91c6439740 20 client.15774 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:15.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.712+0000 7f91c6439740 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5621ac22d8a0 0x5621ac22fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.712+0000 7f91c6439740 4 client.15774 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:15.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.712+0000 7f91c6439740 1 -- 172.21.15.55:0/3926798686 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621ac189990 con 0x5621ac22d8a0 2024-04-26T21:55:15.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.715+0000 7f91c0be0640 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5621ac22d8a0 0x5621ac22fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.723+0000 7f91c0be0640 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5621ac22d8a0 0x5621ac22fc90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f351c4c32d5b547 server_cookie=15bc2e6f14289108 in_seq=0 out_seq=0 2024-04-26T21:55:15.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.726+0000 7f91a4ff9640 10 client.15774 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.745+0000 7f91a4ff9640 1 -- 172.21.15.55:0/3926798686 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621ac189990 con 0x5621ac22d8a0 2024-04-26T21:55:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.745+0000 7f91a4ff9640 10 client.15774 handle_command_reply: tid=0 2024-04-26T21:55:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.745+0000 7f91c6439740 1 client.15774 shutdown 2024-04-26T21:55:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.746+0000 7f91c6439740 1 -- 172.21.15.55:0/3926798686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 msgr2=0x5621ac221ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.746+0000 7f91c6439740 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac221ac0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f91b8030380 tx=0x7f91b8030820 comp rx=0 tx=0).stop 2024-04-26T21:55:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.746+0000 7f91c6439740 1 -- 172.21.15.55:0/3926798686 shutdown_connections 2024-04-26T21:55:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.746+0000 7f91c6439740 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac226fc0 0x5621ac2293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.746+0000 7f91c6439740 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac219590 0x5621ac222000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.746+0000 7f91c6439740 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5621ac22d8a0 0x5621ac22fc90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.746+0000 7f91c6439740 1 --2- 172.21.15.55:0/3926798686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac221ac0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.746+0000 7f91c6439740 1 -- 172.21.15.55:0/3926798686 >> 172.21.15.55:0/3926798686 conn(0x5621ac158910 msgr2=0x5621ac211bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.747+0000 7f91a4ff9640 0 client.15774 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:15.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.747+0000 7f91c6439740 1 -- 172.21.15.55:0/3926798686 shutdown_connections 2024-04-26T21:55:15.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.748+0000 7f91c6439740 1 -- 172.21.15.55:0/3926798686 wait complete. 2024-04-26T21:55:15.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.749+0000 7f91c6439740 20 client.15774 trim_cache size 0 max 16384 2024-04-26T21:55:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.799+0000 7f91c6439740 1 Processor -- start 2024-04-26T21:55:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.799+0000 7f91c6439740 1 -- start start 2024-04-26T21:55:15.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.799+0000 7f91c6439740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac213440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.799+0000 7f91c6439740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 0x5621ac21f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.799+0000 7f91c6439740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac226fc0 0x5621ac2216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.799+0000 7f91c6439740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0010fb0 con 0x5621ac218bd0 2024-04-26T21:55:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.799+0000 7f91c6439740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91a000e880 con 0x5621ac219590 2024-04-26T21:55:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.799+0000 7f91c6439740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f91a000c6d0 con 0x5621ac226fc0 2024-04-26T21:55:15.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.800+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac213440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.800+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac213440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43516/0 (socket says 172.21.15.55:43516) 2024-04-26T21:55:15.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.800+0000 7f91c1be2640 1 -- 172.21.15.55:0/1710322944 learned_addr learned my addr 172.21.15.55:0/1710322944 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.800+0000 7f91c1be2640 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac226fc0 0x5621ac2216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.802+0000 7f91c0be0640 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 0x5621ac21f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.802+0000 7f91c1be2640 1 -- 172.21.15.55:0/1710322944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac226fc0 msgr2=0x5621ac2216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.802+0000 7f91c1be2640 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac226fc0 0x5621ac2216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.802+0000 7f91c1be2640 1 -- 172.21.15.55:0/1710322944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 msgr2=0x5621ac21f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.802+0000 7f91c1be2640 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 0x5621ac21f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.802+0000 7f91c1be2640 1 -- 172.21.15.55:0/1710322944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ac213fb0 con 0x5621ac218bd0 2024-04-26T21:55:15.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.803+0000 7f91c1be2640 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac213440 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f91b8030550 tx=0x7f91b8032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5e3dedeaef64af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.804+0000 7f91a6ffd640 1 -- 172.21.15.55:0/1710322944 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b804c070 con 0x5621ac218bd0 2024-04-26T21:55:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.804+0000 7f91a6ffd640 1 -- 172.21.15.55:0/1710322944 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b8032990 con 0x5621ac218bd0 2024-04-26T21:55:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.804+0000 7f91a6ffd640 1 -- 172.21.15.55:0/1710322944 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8041070 con 0x5621ac218bd0 2024-04-26T21:55:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.805+0000 7f91c6439740 1 -- 172.21.15.55:0/1710322944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 msgr2=0x5621ac213440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.805+0000 7f91c6439740 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac213440 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f91b8030550 tx=0x7f91b8032d90 comp rx=0 tx=0).stop 2024-04-26T21:55:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.805+0000 7f91c6439740 1 -- 172.21.15.55:0/1710322944 shutdown_connections 2024-04-26T21:55:15.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.805+0000 7f91c6439740 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac226fc0 0x5621ac2216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.805+0000 7f91c6439740 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 0x5621ac21f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.805+0000 7f91c6439740 1 --2- 172.21.15.55:0/1710322944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac218bd0 0x5621ac213440 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.805+0000 7f91c6439740 1 -- 172.21.15.55:0/1710322944 >> 172.21.15.55:0/1710322944 conn(0x5621ac158910 msgr2=0x5621ac212d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.806+0000 7f91c6439740 1 -- 172.21.15.55:0/1710322944 shutdown_connections 2024-04-26T21:55:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.807+0000 7f91c6439740 1 -- 172.21.15.55:0/1710322944 wait complete. 2024-04-26T21:55:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.808+0000 7f91c6439740 1 -- start start 2024-04-26T21:55:15.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.808+0000 7f91c6439740 1 Processor -- start 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.808+0000 7f91c6439740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac218bd0 0x5621ac23cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c6439740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 0x5621ac23d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c6439740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac226fc0 0x5621ac240170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c6439740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f91a000cd40 con 0x5621ac226fc0 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c6439740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91a000e1b0 con 0x5621ac219590 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c6439740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f91a000d410 con 0x5621ac218bd0 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c6439740 10 client.0 _ll_register_callbacks cb 0x7f91b80094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac226fc0 0x5621ac240170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c1be2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac226fc0 0x5621ac240170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43526/0 (socket says 172.21.15.55:43526) 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c1be2640 1 -- 172.21.15.55:0/1818959944 learned_addr learned my addr 172.21.15.55:0/1818959944 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:15.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c0be0640 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 0x5621ac23d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.809+0000 7f91c1be2640 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac218bd0 0x5621ac23cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.810+0000 7f91c1be2640 1 -- 172.21.15.55:0/1818959944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac218bd0 msgr2=0x5621ac23cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.810+0000 7f91c1be2640 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac218bd0 0x5621ac23cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.810+0000 7f91c1be2640 1 -- 172.21.15.55:0/1818959944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 msgr2=0x5621ac23d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.810+0000 7f91c1be2640 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 0x5621ac23d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.810+0000 7f91c1be2640 1 -- 172.21.15.55:0/1818959944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ac240760 con 0x5621ac226fc0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.810+0000 7f91c1be2640 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac226fc0 0x5621ac240170 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005700 tx=0x7f91b803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a713668bb917bf64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.810+0000 7f91a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.810+0000 7f91a4ff9640 1 -- 172.21.15.55:0/1818959944 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8041070 con 0x5621ac226fc0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.811+0000 7f91a4ff9640 1 -- 172.21.15.55:0/1818959944 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f91b8030da0 con 0x5621ac226fc0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.811+0000 7f91c6439740 1 -- 172.21.15.55:0/1818959944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621ac183800 con 0x5621ac226fc0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.812+0000 7f91a4ff9640 1 -- 172.21.15.55:0/1818959944 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f91b804c040 con 0x5621ac226fc0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.812+0000 7f91c6439740 10 client.15780 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.812+0000 7f91c6439740 1 -- 172.21.15.55:0/1818959944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5621ac293cf0 con 0x5621ac226fc0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.813+0000 7f91a4ff9640 1 -- 172.21.15.55:0/1818959944 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5621ac183800 con 0x5621ac226fc0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.813+0000 7f91a4ff9640 1 -- 172.21.15.55:0/1818959944 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f91b80046c0 con 0x5621ac226fc0 2024-04-26T21:55:15.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.813+0000 7f91c6439740 10 client.15780 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.813+0000 7f91c6439740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.813+0000 7f91c6439740 20 client.15780 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.813+0000 7f91c6439740 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5621ac2944e0 0x5621ac296900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.813+0000 7f91c6439740 4 client.15780 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:55:15.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.813+0000 7f91c6439740 1 -- 172.21.15.55:0/1818959944 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621ac0fe610 con 0x5621ac2944e0 2024-04-26T21:55:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.818+0000 7f91c0be0640 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5621ac2944e0 0x5621ac296900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:15.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.824+0000 7f91a4ff9640 10 client.15780 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:15.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.824+0000 7f91c0be0640 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5621ac2944e0 0x5621ac296900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c14fd793bfc99e server_cookie=add719b5d258501e in_seq=0 out_seq=0 2024-04-26T21:55:15.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.826+0000 7f91a4ff9640 1 -- 172.21.15.55:0/1818959944 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621ac0fe610 con 0x5621ac2944e0 2024-04-26T21:55:15.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.826+0000 7f91a4ff9640 10 client.15780 handle_command_reply: tid=0 2024-04-26T21:55:15.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.826+0000 7f91c6439740 1 client.15780 shutdown 2024-04-26T21:55:15.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.828+0000 7f91c6439740 1 -- 172.21.15.55:0/1818959944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac226fc0 msgr2=0x5621ac240170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.828+0000 7f91c6439740 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac226fc0 0x5621ac240170 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005700 tx=0x7f91b803f040 comp rx=0 tx=0).stop 2024-04-26T21:55:15.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.830+0000 7f91c6439740 1 -- 172.21.15.55:0/1818959944 shutdown_connections 2024-04-26T21:55:15.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.830+0000 7f91c6439740 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5621ac2944e0 0x5621ac296900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.830+0000 7f91c6439740 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5621ac226fc0 0x5621ac240170 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.830+0000 7f91c6439740 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621ac219590 0x5621ac23d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.830+0000 7f91c6439740 1 --2- 172.21.15.55:0/1818959944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ac218bd0 0x5621ac23cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.830+0000 7f91a4ff9640 0 client.15780 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:15.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.830+0000 7f91c6439740 1 -- 172.21.15.55:0/1818959944 >> 172.21.15.55:0/1818959944 conn(0x5621ac158910 msgr2=0x5621ac212d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.831+0000 7f91c6439740 1 -- 172.21.15.55:0/1818959944 shutdown_connections 2024-04-26T21:55:15.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.832+0000 7f91c6439740 1 -- 172.21.15.55:0/1818959944 wait complete. 2024-04-26T21:55:15.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.832+0000 7f91c6439740 20 client.15780 trim_cache size 0 max 16384 2024-04-26T21:55:15.833 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.833+0000 7f91c23e3640 1 -- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f91a80a3e40 msgr2=0x7f91a80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f91a80a3e40 0x7f91a80a6300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0a9730 tx=0x7f91ac0acd60 comp rx=0 tx=0).stop 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 -- 172.21.15.55:0/2047931457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91bc073d70 msgr2=0x7f91bc19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91bc073d70 0x7f91bc19f0f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91ac01ea20 tx=0x7f91ac01eee0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 -- 172.21.15.55:0/2047931457 shutdown_connections 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f91a80a3e40 0x7f91a80a6300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91bc10a9b0 0x7f91bc1a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91bc10a010 0x7f91bc19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 --2- 172.21.15.55:0/2047931457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f91bc073d70 0x7f91bc19f0f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 -- 172.21.15.55:0/2047931457 >> 172.21.15.55:0/2047931457 conn(0x7f91bc06f840 msgr2=0x7f91bc10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 -- 172.21.15.55:0/2047931457 shutdown_connections 2024-04-26T21:55:15.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:15.834+0000 7f91c23e3640 1 -- 172.21.15.55:0/2047931457 wait complete. 2024-04-26T21:55:16.856 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:55:16.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:16.856 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:55:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 -- 172.21.15.55:0/2266997335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 msgr2=0x7f86f01057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 --2- 172.21.15.55:0/2266997335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 0x7f86f01057f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e002a410 comp rx=0 tx=0).stop 2024-04-26T21:55:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 -- 172.21.15.55:0/2266997335 shutdown_connections 2024-04-26T21:55:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 --2- 172.21.15.55:0/2266997335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f0108660 0x7f86f010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 --2- 172.21.15.55:0/2266997335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86f0105d30 0x7f86f0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 --2- 172.21.15.55:0/2266997335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 0x7f86f01057f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 -- 172.21.15.55:0/2266997335 >> 172.21.15.55:0/2266997335 conn(0x7f86f00fcdf0 msgr2=0x7f86f00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 -- 172.21.15.55:0/2266997335 shutdown_connections 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.071+0000 7f86f6446640 1 -- 172.21.15.55:0/2266997335 wait complete. 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.072+0000 7f86f6446640 1 Processor -- start 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.072+0000 7f86f6446640 1 -- start start 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.072+0000 7f86f6446640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 0x7f86f0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.072+0000 7f86f6446640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86f0105d30 0x7f86f0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.072+0000 7f86f6446640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f0108660 0x7f86f01a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.073+0000 7f86f6446640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010e5f0 con 0x7f86f0103400 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.073+0000 7f86f6446640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010e470 con 0x7f86f0105d30 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.073+0000 7f86f6446640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86f010e770 con 0x7f86f0108660 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.073+0000 7f86effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 0x7f86f0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.073+0000 7f86effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 0x7f86f0198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43546/0 (socket says 172.21.15.55:43546) 2024-04-26T21:55:17.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.073+0000 7f86effff640 1 -- 172.21.15.55:0/2993327730 learned_addr learned my addr 172.21.15.55:0/2993327730 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.073+0000 7f86e7fff640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86f0105d30 0x7f86f0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.074+0000 7f86effff640 1 -- 172.21.15.55:0/2993327730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f0108660 msgr2=0x7f86f01a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.074+0000 7f86effff640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f0108660 0x7f86f01a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.074+0000 7f86effff640 1 -- 172.21.15.55:0/2993327730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86f0105d30 msgr2=0x7f86f0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.074+0000 7f86effff640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86f0105d30 0x7f86f0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.074+0000 7f86effff640 1 -- 172.21.15.55:0/2993327730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a0960 con 0x7f86f0103400 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.074+0000 7f86e7fff640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86f0105d30 0x7f86f0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.074+0000 7f86effff640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 0x7f86f0198d50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f86e0031500 tx=0x7f86e0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.074+0000 7f86edffb640 1 -- 172.21.15.55:0/2993327730 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e003f070 con 0x7f86f0103400 2024-04-26T21:55:17.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.075+0000 7f86edffb640 1 -- 172.21.15.55:0/2993327730 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e003b460 con 0x7f86f0103400 2024-04-26T21:55:17.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.075+0000 7f86edffb640 1 -- 172.21.15.55:0/2993327730 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0035030 con 0x7f86f0103400 2024-04-26T21:55:17.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.075+0000 7f86f6446640 1 -- 172.21.15.55:0/2993327730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a0bf0 con 0x7f86f0103400 2024-04-26T21:55:17.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.075+0000 7f86f6446640 1 -- 172.21.15.55:0/2993327730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a1130 con 0x7f86f0103400 2024-04-26T21:55:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.079+0000 7f86f9233740 1 Processor -- start 2024-04-26T21:55:17.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.079+0000 7f86edffb640 1 -- 172.21.15.55:0/2993327730 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86e0042070 con 0x7f86f0103400 2024-04-26T21:55:17.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.080+0000 7f86f9233740 1 -- start start 2024-04-26T21:55:17.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.080+0000 7f86f9233740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e68b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.080+0000 7f86f9233740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e6901b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.081+0000 7f86f9233740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e692cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.081+0000 7f86f9233740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55987e5ebb00 con 0x55987e68b6b0 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.081+0000 7f86f9233740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55987e587920 con 0x55987e68acc0 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.081+0000 7f86f9233740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55987e5e7ac0 con 0x55987e690820 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.081+0000 7f86edffb640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86dc0a3cc0 0x7f86dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.081+0000 7f86f49e3640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86dc0a3cc0 0x7f86dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.081+0000 7f86f49e3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e68b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.081+0000 7f86edffb640 1 -- 172.21.15.55:0/2993327730 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f86e00f0fe0 con 0x7f86f0103400 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.082+0000 7f86effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e6901b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.082+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e692cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.082+0000 7f86effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e6901b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43550/0 (socket says 172.21.15.55:43550) 2024-04-26T21:55:17.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.082+0000 7f86effff640 1 -- 172.21.15.55:0/2230730019 learned_addr learned my addr 172.21.15.55:0/2230730019 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:17.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.083+0000 7f86effff640 1 -- 172.21.15.55:0/2230730019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 msgr2=0x55987e692cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:17.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.083+0000 7f86effff640 1 --2- 172.21.15.55:0/2230730019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e692cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.083+0000 7f86effff640 1 -- 172.21.15.55:0/2230730019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 msgr2=0x55987e68b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:17.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.083+0000 7f86effff640 1 --2- 172.21.15.55:0/2230730019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e68b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.083+0000 7f86effff640 1 -- 172.21.15.55:0/2230730019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987e3c57e0 con 0x55987e68b6b0 2024-04-26T21:55:17.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.083+0000 7f86f49e3640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86dc0a3cc0 0x7f86dc0a6180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f86e8010290 tx=0x7f86e8001880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:17.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.083+0000 7f86effff640 1 --2- 172.21.15.55:0/2230730019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e6901b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f86e00eea20 tx=0x7f86e00bb6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb318cc47763f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:17.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.084+0000 7f86e67fc640 1 -- 172.21.15.55:0/2230730019 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0049050 con 0x55987e68b6b0 2024-04-26T21:55:17.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.084+0000 7f86e67fc640 1 -- 172.21.15.55:0/2230730019 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e00bd8c0 con 0x55987e68b6b0 2024-04-26T21:55:17.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.084+0000 7f86e67fc640 1 -- 172.21.15.55:0/2230730019 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e003ed60 con 0x55987e68b6b0 2024-04-26T21:55:17.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 -- 172.21.15.55:0/2230730019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 msgr2=0x55987e6901b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:17.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 --2- 172.21.15.55:0/2230730019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e6901b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f86e00eea20 tx=0x7f86e00bb6d0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 -- 172.21.15.55:0/2230730019 shutdown_connections 2024-04-26T21:55:17.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 --2- 172.21.15.55:0/2230730019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e692cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 --2- 172.21.15.55:0/2230730019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e6901b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 --2- 172.21.15.55:0/2230730019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e68b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 -- 172.21.15.55:0/2230730019 >> 172.21.15.55:0/2230730019 conn(0x55987e686330 msgr2=0x55987e686730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:17.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 -- 172.21.15.55:0/2230730019 shutdown_connections 2024-04-26T21:55:17.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.086+0000 7f86f9233740 1 -- 172.21.15.55:0/2230730019 wait complete. 2024-04-26T21:55:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.087+0000 7f86f9233740 1 -- start start 2024-04-26T21:55:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86f9233740 1 Processor -- start 2024-04-26T21:55:17.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86f9233740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68acc0 0x55987e694e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86f9233740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68b6b0 0x55987e695370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86f9233740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86f9233740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55987e693590 con 0x55987e68acc0 2024-04-26T21:55:17.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86f9233740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55987e5855d0 con 0x55987e68b6b0 2024-04-26T21:55:17.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86f9233740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55987e693710 con 0x55987e690820 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86f9233740 10 client.0 _ll_register_callbacks cb 0x55987e573080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.088+0000 7f86effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68acc0 0x55987e694e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.089+0000 7f86effff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68acc0 0x55987e694e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43562/0 (socket says 172.21.15.55:43562) 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.089+0000 7f86effff640 1 -- 172.21.15.55:0/3584931314 learned_addr learned my addr 172.21.15.55:0/3584931314 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.089+0000 7f86effff640 1 -- 172.21.15.55:0/3584931314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 msgr2=0x55987e69c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.089+0000 7f86effff640 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.089+0000 7f86effff640 1 -- 172.21.15.55:0/3584931314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68b6b0 msgr2=0x55987e695370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.089+0000 7f86effff640 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68b6b0 0x55987e695370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.089+0000 7f86effff640 1 -- 172.21.15.55:0/3584931314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987e425f70 con 0x55987e68acc0 2024-04-26T21:55:17.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.090+0000 7f86effff640 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68acc0 0x55987e694e30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f86e00eea70 tx=0x7f86e0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316976a9c0360120 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:17.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.090+0000 7f86cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:17.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.091+0000 7f86cf7fe640 1 -- 172.21.15.55:0/3584931314 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0049050 con 0x55987e68acc0 2024-04-26T21:55:17.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.091+0000 7f86cf7fe640 1 -- 172.21.15.55:0/3584931314 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f86e00b97a0 con 0x55987e68acc0 2024-04-26T21:55:17.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.091+0000 7f86f9233740 1 -- 172.21.15.55:0/3584931314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e00bd8c0 con 0x55987e68acc0 2024-04-26T21:55:17.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.091+0000 7f86cf7fe640 1 -- 172.21.15.55:0/3584931314 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f86e00a2050 con 0x55987e68acc0 2024-04-26T21:55:17.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.091+0000 7f86f9233740 10 client.15792 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:17.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.091+0000 7f86f9233740 1 -- 172.21.15.55:0/3584931314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55987e2bd170 con 0x55987e68acc0 2024-04-26T21:55:17.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.092+0000 7f86cf7fe640 1 -- 172.21.15.55:0/3584931314 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e003ed10 con 0x55987e68acc0 2024-04-26T21:55:17.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.092+0000 7f86cf7fe640 1 -- 172.21.15.55:0/3584931314 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f86e00c1160 con 0x55987e68acc0 2024-04-26T21:55:17.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.092+0000 7f86f9233740 10 client.15792 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:17.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.092+0000 7f86f9233740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:17.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.092+0000 7f86f9233740 20 client.15792 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:17.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.092+0000 7f86f9233740 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55987e6a06e0 0x55987e6a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.092+0000 7f86f9233740 4 client.15792 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:17.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.092+0000 7f86f9233740 1 -- 172.21.15.55:0/3584931314 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f86e00bd8c0 con 0x55987e6a06e0 2024-04-26T21:55:17.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.094+0000 7f86e7fff640 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55987e6a06e0 0x55987e6a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.099+0000 7f86cf7fe640 10 client.15792 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:17.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.099+0000 7f86e7fff640 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55987e6a06e0 0x55987e6a2b10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66db4edc74c152fd server_cookie=87ee8e38ddbe5cce in_seq=0 out_seq=0 2024-04-26T21:55:17.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.103+0000 7f86cf7fe640 1 -- 172.21.15.55:0/3584931314 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f86e00bd8c0 con 0x55987e6a06e0 2024-04-26T21:55:17.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.103+0000 7f86cf7fe640 10 client.15792 handle_command_reply: tid=0 2024-04-26T21:55:17.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.104+0000 7f86f9233740 1 client.15792 shutdown 2024-04-26T21:55:17.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86f9233740 1 -- 172.21.15.55:0/3584931314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68acc0 msgr2=0x55987e694e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:17.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86f9233740 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68acc0 0x55987e694e30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f86e00eea70 tx=0x7f86e0005c00 comp rx=0 tx=0).stop 2024-04-26T21:55:17.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86f9233740 1 -- 172.21.15.55:0/3584931314 shutdown_connections 2024-04-26T21:55:17.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86f9233740 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86f9233740 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68b6b0 0x55987e695370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86f9233740 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55987e6a06e0 0x55987e6a2b10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86f9233740 1 --2- 172.21.15.55:0/3584931314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68acc0 0x55987e694e30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86f9233740 1 -- 172.21.15.55:0/3584931314 >> 172.21.15.55:0/3584931314 conn(0x55987e686330 msgr2=0x55987e6810e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:17.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.105+0000 7f86cf7fe640 0 client.15792 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:17.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.106+0000 7f86f9233740 1 -- 172.21.15.55:0/3584931314 shutdown_connections 2024-04-26T21:55:17.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.106+0000 7f86f9233740 1 -- 172.21.15.55:0/3584931314 wait complete. 2024-04-26T21:55:17.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.106+0000 7f86f9233740 20 client.15792 trim_cache size 0 max 16384 2024-04-26T21:55:17.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.156+0000 7f86f9233740 1 Processor -- start 2024-04-26T21:55:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.156+0000 7f86f9233740 1 -- start start 2024-04-26T21:55:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.157+0000 7f86f9233740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e694070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.157+0000 7f86f9233740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e68b6b0 0x55987e69c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.157+0000 7f86f9233740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e690820 0x55987e680f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.158+0000 7f86f9233740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01aa740 con 0x55987e690820 2024-04-26T21:55:17.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.158+0000 7f86f9233740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a8010 con 0x55987e68acc0 2024-04-26T21:55:17.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.158+0000 7f86f9233740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a5e60 con 0x55987e68b6b0 2024-04-26T21:55:17.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.158+0000 7f86effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e694070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.158+0000 7f86effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e694070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35214/0 (socket says 172.21.15.55:35214) 2024-04-26T21:55:17.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.158+0000 7f86effff640 1 -- 172.21.15.55:0/2685160485 learned_addr learned my addr 172.21.15.55:0/2685160485 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:17.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.158+0000 7f86f49e3640 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e690820 0x55987e680f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.158+0000 7f86e7fff640 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e68b6b0 0x55987e69c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.159+0000 7f86f49e3640 1 -- 172.21.15.55:0/2685160485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e68b6b0 msgr2=0x55987e69c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:17.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.159+0000 7f86f49e3640 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e68b6b0 0x55987e69c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.159+0000 7f86f49e3640 1 -- 172.21.15.55:0/2685160485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 msgr2=0x55987e694070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:17.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.159+0000 7f86f49e3640 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e694070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.159+0000 7f86f49e3640 1 -- 172.21.15.55:0/2685160485 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987e690570 con 0x55987e690820 2024-04-26T21:55:17.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.160+0000 7f86f49e3640 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e690820 0x55987e680f60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f86e8010b20 tx=0x7f86e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e955e8b968eff39a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:17.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.163+0000 7f86e5ffb640 1 -- 172.21.15.55:0/2685160485 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8015610 con 0x55987e690820 2024-04-26T21:55:17.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.164+0000 7f86e5ffb640 1 -- 172.21.15.55:0/2685160485 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e8011040 con 0x55987e690820 2024-04-26T21:55:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.167+0000 7f86f9233740 1 -- 172.21.15.55:0/2685160485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e690820 msgr2=0x55987e680f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:17.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.167+0000 7f86f9233740 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e690820 0x55987e680f60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f86e8010b20 tx=0x7f86e8016000 comp rx=0 tx=0).stop 2024-04-26T21:55:17.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.179+0000 7f86f9233740 1 -- 172.21.15.55:0/2685160485 shutdown_connections 2024-04-26T21:55:17.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.179+0000 7f86f9233740 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e690820 0x55987e680f60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.179+0000 7f86f9233740 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e68b6b0 0x55987e69c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.179+0000 7f86f9233740 1 --2- 172.21.15.55:0/2685160485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e694070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.179+0000 7f86f9233740 1 -- 172.21.15.55:0/2685160485 >> 172.21.15.55:0/2685160485 conn(0x55987e686330 msgr2=0x55987e680850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:17.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.180+0000 7f86f9233740 1 -- 172.21.15.55:0/2685160485 shutdown_connections 2024-04-26T21:55:17.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.180+0000 7f86f9233740 1 -- 172.21.15.55:0/2685160485 wait complete. 2024-04-26T21:55:17.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.180+0000 7f86f9233740 1 -- start start 2024-04-26T21:55:17.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.181+0000 7f86f9233740 1 Processor -- start 2024-04-26T21:55:17.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.182+0000 7f86f9233740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e6964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.182+0000 7f86f9233740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e696a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.182+0000 7f86f9233740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.182+0000 7f86f9233740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a64d0 con 0x55987e68b6b0 2024-04-26T21:55:17.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.182+0000 7f86f9233740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a7940 con 0x55987e68acc0 2024-04-26T21:55:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.182+0000 7f86f9233740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01a6ba0 con 0x55987e690820 2024-04-26T21:55:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.182+0000 7f86f9233740 10 client.0 _ll_register_callbacks cb 0x7f86e00bb3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.183+0000 7f86f49e3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.183+0000 7f86f49e3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:58326/0 (socket says 172.21.15.55:58326) 2024-04-26T21:55:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.183+0000 7f86f49e3640 1 -- 172.21.15.55:0/315638694 learned_addr learned my addr 172.21.15.55:0/315638694 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.183+0000 7f86effff640 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e6964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.184+0000 7f86f49e3640 1 -- 172.21.15.55:0/315638694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 msgr2=0x55987e6964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.184+0000 7f86f49e3640 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e6964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.184+0000 7f86f49e3640 1 -- 172.21.15.55:0/315638694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 msgr2=0x55987e696a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.184+0000 7f86f49e3640 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e696a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.184+0000 7f86f49e3640 1 -- 172.21.15.55:0/315638694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987e69bed0 con 0x55987e690820 2024-04-26T21:55:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.185+0000 7f86f49e3640 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69b8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f86e8002170 tx=0x7f86e80021a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43ade2ca8b527a0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.186+0000 7f86cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:55:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.194+0000 7f86cf7fe640 1 -- 172.21.15.55:0/315638694 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e802e060 con 0x55987e690820 2024-04-26T21:55:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.194+0000 7f86cf7fe640 1 -- 172.21.15.55:0/315638694 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f86e8003970 con 0x55987e690820 2024-04-26T21:55:17.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.196+0000 7f86f9233740 1 -- 172.21.15.55:0/315638694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e8011040 con 0x55987e690820 2024-04-26T21:55:17.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.200+0000 7f86cf7fe640 1 -- 172.21.15.55:0/315638694 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f86e8003b70 con 0x55987e690820 2024-04-26T21:55:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.214+0000 7f86f9233740 10 client.25327 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.214+0000 7f86f9233740 1 -- 172.21.15.55:0/315638694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55987e70bd80 con 0x55987e690820 2024-04-26T21:55:17.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.229+0000 7f86cf7fe640 1 -- 172.21.15.55:0/315638694 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e8011040 con 0x55987e690820 2024-04-26T21:55:17.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.230+0000 7f86cf7fe640 1 -- 172.21.15.55:0/315638694 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f86e8015720 con 0x55987e690820 2024-04-26T21:55:17.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.234+0000 7f86f9233740 10 client.25327 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:17.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.234+0000 7f86f9233740 10 client.25327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:17.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.234+0000 7f86f9233740 20 client.25327 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:17.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.234+0000 7f86f9233740 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55987e70c610 0x55987e70ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.234+0000 7f86f9233740 4 client.25327 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:55:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.234+0000 7f86f9233740 1 -- 172.21.15.55:0/315638694 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55987e5f9710 con 0x55987e70c610 2024-04-26T21:55:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.238+0000 7f86effff640 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55987e70c610 0x55987e70ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.253+0000 7f86effff640 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55987e70c610 0x55987e70ea30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57b37b71b20e6583 server_cookie=80b8a637b270c21f in_seq=0 out_seq=0 2024-04-26T21:55:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:17.253+0000 7f86cf7fe640 10 client.25327 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:17.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:17 smithi165 ceph-mon[33296]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 864 MiB/s wr, 1.38k op/s 2024-04-26T21:55:17.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:17 smithi055 ceph-mon[29221]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 864 MiB/s wr, 1.38k op/s 2024-04-26T21:55:17.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:17 smithi064 ceph-mon[32765]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 864 MiB/s wr, 1.38k op/s 2024-04-26T21:55:18.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.268+0000 7f86cf7fe640 1 -- 172.21.15.55:0/315638694 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55987e5f9710 con 0x55987e70c610 2024-04-26T21:55:18.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.268+0000 7f86cf7fe640 10 client.25327 handle_command_reply: tid=0 2024-04-26T21:55:18.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.268+0000 7f86f9233740 1 client.25327 shutdown 2024-04-26T21:55:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.275+0000 7f86f9233740 1 -- 172.21.15.55:0/315638694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 msgr2=0x55987e69b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:18.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.275+0000 7f86f9233740 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69b8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f86e8002170 tx=0x7f86e80021a0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.276+0000 7f86f9233740 1 -- 172.21.15.55:0/315638694 shutdown_connections 2024-04-26T21:55:18.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.276+0000 7f86f9233740 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55987e690820 0x55987e69b8e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.276+0000 7f86f9233740 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55987e70c610 0x55987e70ea30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.276+0000 7f86f9233740 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55987e68b6b0 0x55987e696a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.276+0000 7f86f9233740 1 --2- 172.21.15.55:0/315638694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55987e68acc0 0x55987e6964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.276+0000 7f86f9233740 1 -- 172.21.15.55:0/315638694 >> 172.21.15.55:0/315638694 conn(0x55987e686330 msgr2=0x55987e689ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.276+0000 7f86cf7fe640 0 client.25327 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.278+0000 7f86f9233740 1 -- 172.21.15.55:0/315638694 shutdown_connections 2024-04-26T21:55:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.278+0000 7f86f9233740 1 -- 172.21.15.55:0/315638694 wait complete. 2024-04-26T21:55:18.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.278+0000 7f86f9233740 20 client.25327 trim_cache size 0 max 16384 2024-04-26T21:55:18.282 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:55:18.282 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:55:18.282 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "2e51f7a0-5ecc-47bf-b91f-0946fdaee0a1", 2024-04-26T21:55:18.282 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:55:18.283 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:55:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.282+0000 7f86f6446640 1 -- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86dc0a3cc0 msgr2=0x7f86dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.282+0000 7f86f6446640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86dc0a3cc0 0x7f86dc0a6180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f86e8010290 tx=0x7f86e8001880 comp rx=0 tx=0).stop 2024-04-26T21:55:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.282+0000 7f86f6446640 1 -- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 msgr2=0x7f86f0198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:18.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.282+0000 7f86f6446640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 0x7f86f0198d50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f86e0031500 tx=0x7f86e0003d50 comp rx=0 tx=0).stop 2024-04-26T21:55:18.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.283+0000 7f86f6446640 1 -- 172.21.15.55:0/2993327730 shutdown_connections 2024-04-26T21:55:18.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.283+0000 7f86f6446640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86dc0a3cc0 0x7f86dc0a6180 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.283+0000 7f86f6446640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f0108660 0x7f86f01a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.283+0000 7f86f6446640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f86f0105d30 0x7f86f0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.283+0000 7f86f6446640 1 --2- 172.21.15.55:0/2993327730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f86f0103400 0x7f86f0198d50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.283+0000 7f86f6446640 1 -- 172.21.15.55:0/2993327730 >> 172.21.15.55:0/2993327730 conn(0x7f86f00fcdf0 msgr2=0x7f86f0103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:18.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.284+0000 7f86f6446640 1 -- 172.21.15.55:0/2993327730 shutdown_connections 2024-04-26T21:55:18.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.284+0000 7f86f6446640 1 -- 172.21.15.55:0/2993327730 wait complete. 2024-04-26T21:55:18.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:55:18.389 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:55:18.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.793+0000 7efdb2b90640 1 -- 172.21.15.55:0/441152528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac10c2a0 msgr2=0x7efdac10e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:18.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.793+0000 7efdb2b90640 1 --2- 172.21.15.55:0/441152528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac10c2a0 0x7efdac10e730 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efda0009900 tx=0x7efda002a440 comp rx=0 tx=0).stop 2024-04-26T21:55:18.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.796+0000 7efdb2b90640 1 -- 172.21.15.55:0/441152528 shutdown_connections 2024-04-26T21:55:18.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.796+0000 7efdb2b90640 1 --2- 172.21.15.55:0/441152528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac10ec70 0x7efdac10f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.796+0000 7efdb2b90640 1 --2- 172.21.15.55:0/441152528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac10c2a0 0x7efdac10e730 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.796+0000 7efdb2b90640 1 --2- 172.21.15.55:0/441152528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdac109930 0x7efdac10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.796+0000 7efdb2b90640 1 -- 172.21.15.55:0/441152528 >> 172.21.15.55:0/441152528 conn(0x7efdac06f7c0 msgr2=0x7efdac071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:18.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.806+0000 7efdb2b90640 1 -- 172.21.15.55:0/441152528 shutdown_connections 2024-04-26T21:55:18.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.809+0000 7efdb2b90640 1 -- 172.21.15.55:0/441152528 wait complete. 2024-04-26T21:55:18.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.810+0000 7efdb2b90640 1 Processor -- start 2024-04-26T21:55:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.811+0000 7efdb2b90640 1 -- start start 2024-04-26T21:55:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.811+0000 7efdb2b90640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac109930 0x7efdac1a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.811+0000 7efdb2b90640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdac10c2a0 0x7efdac1a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.811+0000 7efdb2b90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac10ec70 0x7efdac1aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.811+0000 7efdb2b90640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efdac11d370 con 0x7efdac109930 2024-04-26T21:55:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.811+0000 7efdb2b90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efdac11d1f0 con 0x7efdac10ec70 2024-04-26T21:55:18.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.811+0000 7efdb2b90640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efdac11d4f0 con 0x7efdac10c2a0 2024-04-26T21:55:18.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.812+0000 7efdb112d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac10ec70 0x7efdac1aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.812+0000 7efdb112d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac10ec70 0x7efdac1aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35252/0 (socket says 172.21.15.55:35252) 2024-04-26T21:55:18.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.812+0000 7efdb112d640 1 -- 172.21.15.55:0/1074901889 learned_addr learned my addr 172.21.15.55:0/1074901889 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:18.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.813+0000 7efdb092c640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac109930 0x7efdac1a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.817+0000 7efdb112d640 1 -- 172.21.15.55:0/1074901889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdac10c2a0 msgr2=0x7efdac1a7db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:55:18.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.817+0000 7efdb112d640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdac10c2a0 0x7efdac1a7db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.817+0000 7efdb112d640 1 -- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac109930 msgr2=0x7efdac1a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:18.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.817+0000 7efdb112d640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac109930 0x7efdac1a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.817+0000 7efdb112d640 1 -- 172.21.15.55:0/1074901889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac1af490 con 0x7efdac10ec70 2024-04-26T21:55:18.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.820+0000 7efdb112d640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac10ec70 0x7efdac1aee20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efda400e9d0 tx=0x7efda400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:18.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.824+0000 7efda9ffb640 1 -- 172.21.15.55:0/1074901889 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda400cd80 con 0x7efdac10ec70 2024-04-26T21:55:18.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.824+0000 7efda9ffb640 1 -- 172.21.15.55:0/1074901889 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda4005020 con 0x7efdac10ec70 2024-04-26T21:55:18.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.824+0000 7efda9ffb640 1 -- 172.21.15.55:0/1074901889 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda40116d0 con 0x7efdac10ec70 2024-04-26T21:55:18.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.825+0000 7efdb2b90640 1 -- 172.21.15.55:0/1074901889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac1af780 con 0x7efdac10ec70 2024-04-26T21:55:18.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.825+0000 7efdb2b90640 1 -- 172.21.15.55:0/1074901889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac1afc80 con 0x7efdac10ec70 2024-04-26T21:55:18.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.829+0000 7efdb5983740 1 Processor -- start 2024-04-26T21:55:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.834+0000 7efda9ffb640 1 -- 172.21.15.55:0/1074901889 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efda4004a40 con 0x7efdac10ec70 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efda9ffb640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efd940a3dc0 0x7efd940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efdb5983740 1 -- start start 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efdb5983740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e828300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efdb5983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e829530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efdb5983740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e830300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efdb5983740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56260e78cd80 con 0x56260e829a70 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efdb5983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56260e7468b0 con 0x56260e8290b0 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efdb5983740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56260e788d40 con 0x56260e827ee0 2024-04-26T21:55:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.837+0000 7efda9ffb640 1 -- 172.21.15.55:0/1074901889 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7efda40d0580 con 0x7efdac10ec70 2024-04-26T21:55:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.839+0000 7efdb092c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e828300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.839+0000 7efdb092c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e828300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:58362/0 (socket says 172.21.15.55:58362) 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.839+0000 7efdb092c640 1 -- 172.21.15.55:0/3146063528 learned_addr learned my addr 172.21.15.55:0/3146063528 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.840+0000 7efdb112d640 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e830300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.840+0000 7efdabfff640 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e829530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.840+0000 7efdabfff640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efd940a3dc0 0x7efd940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.840+0000 7efdb092c640 1 -- 172.21.15.55:0/3146063528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 msgr2=0x56260e829530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.840+0000 7efdb092c640 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e829530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.840+0000 7efdb092c640 1 -- 172.21.15.55:0/3146063528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 msgr2=0x56260e830300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.840+0000 7efdb092c640 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e830300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.840+0000 7efdb092c640 1 -- 172.21.15.55:0/3146063528 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260e717a40 con 0x56260e827ee0 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.841+0000 7efdb092c640 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e828300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009a40 tx=0x7efd9c009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e262d92005cabd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.842+0000 7efd92ffd640 1 -- 172.21.15.55:0/3146063528 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c017070 con 0x56260e827ee0 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.842+0000 7efd92ffd640 1 -- 172.21.15.55:0/3146063528 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c013470 con 0x56260e827ee0 2024-04-26T21:55:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.842+0000 7efd92ffd640 1 -- 172.21.15.55:0/3146063528 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c012610 con 0x56260e827ee0 2024-04-26T21:55:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.843+0000 7efdb5983740 1 -- 172.21.15.55:0/3146063528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 msgr2=0x56260e828300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.843+0000 7efdb5983740 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e828300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009a40 tx=0x7efd9c009f00 comp rx=0 tx=0).stop 2024-04-26T21:55:18.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.845+0000 7efdabfff640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efd940a3dc0 0x7efd940a6280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efda0005740 tx=0x7efda0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.851+0000 7efdb5983740 1 -- 172.21.15.55:0/3146063528 shutdown_connections 2024-04-26T21:55:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.851+0000 7efdb5983740 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e830300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.851+0000 7efdb5983740 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e829530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.851+0000 7efdb5983740 1 --2- 172.21.15.55:0/3146063528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e828300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.851+0000 7efdb5983740 1 -- 172.21.15.55:0/3146063528 >> 172.21.15.55:0/3146063528 conn(0x56260e8233d0 msgr2=0x56260e8237d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.852+0000 7efdb5983740 1 -- 172.21.15.55:0/3146063528 shutdown_connections 2024-04-26T21:55:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.852+0000 7efdb5983740 1 -- 172.21.15.55:0/3146063528 wait complete. 2024-04-26T21:55:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.853+0000 7efdb5983740 1 -- start start 2024-04-26T21:55:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.853+0000 7efdb5983740 1 Processor -- start 2024-04-26T21:55:18.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.854+0000 7efdb5983740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e832400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.854+0000 7efdb5983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e832940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.854+0000 7efdb5983740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e839980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.854+0000 7efdb5983740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56260e830a80 con 0x56260e829a70 2024-04-26T21:55:18.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.854+0000 7efdb5983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56260e713d10 con 0x56260e8290b0 2024-04-26T21:55:18.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.854+0000 7efdb5983740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56260e830c00 con 0x56260e827ee0 2024-04-26T21:55:18.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.854+0000 7efdb5983740 10 client.0 _ll_register_callbacks cb 0x56260e740210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:18.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.855+0000 7efdb112d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e839980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.855+0000 7efdb112d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e839980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43616/0 (socket says 172.21.15.55:43616) 2024-04-26T21:55:18.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.855+0000 7efdb112d640 1 -- 172.21.15.55:0/2596892487 learned_addr learned my addr 172.21.15.55:0/2596892487 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:18.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.855+0000 7efdabfff640 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e832940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.856+0000 7efdb092c640 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e832400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.858+0000 7efdb092c640 1 -- 172.21.15.55:0/2596892487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 msgr2=0x56260e832940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:18.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.858+0000 7efdb092c640 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e832940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.858+0000 7efdb092c640 1 -- 172.21.15.55:0/2596892487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 msgr2=0x56260e839980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:18.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.858+0000 7efdb092c640 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e839980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.858+0000 7efdb092c640 1 -- 172.21.15.55:0/2596892487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260e5687d0 con 0x56260e827ee0 2024-04-26T21:55:18.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.859+0000 7efdb112d640 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e839980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:55:18.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.862+0000 7efdb092c640 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e832400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c013620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bbe2c6315b07fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:18.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.863+0000 7efd8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:55:18.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.864+0000 7efd8bfff640 1 -- 172.21.15.55:0/2596892487 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c017050 con 0x56260e827ee0 2024-04-26T21:55:18.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.864+0000 7efd8bfff640 1 -- 172.21.15.55:0/2596892487 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7efd9c01c450 con 0x56260e827ee0 2024-04-26T21:55:18.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.864+0000 7efdb5983740 1 -- 172.21.15.55:0/2596892487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c013470 con 0x56260e827ee0 2024-04-26T21:55:18.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.866+0000 7efd8bfff640 1 -- 172.21.15.55:0/2596892487 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efd9c013ba0 con 0x56260e827ee0 2024-04-26T21:55:18.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.866+0000 7efdb5983740 10 client.25333 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:18.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.866+0000 7efdb5983740 1 -- 172.21.15.55:0/2596892487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56260e49e3f0 con 0x56260e827ee0 2024-04-26T21:55:18.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.871+0000 7efd8bfff640 1 -- 172.21.15.55:0/2596892487 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c013470 con 0x56260e827ee0 2024-04-26T21:55:18.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.871+0000 7efd8bfff640 1 -- 172.21.15.55:0/2596892487 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efd9c030900 con 0x56260e827ee0 2024-04-26T21:55:18.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.871+0000 7efdb5983740 10 client.25333 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:18.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.871+0000 7efdb5983740 10 client.25333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:18.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.871+0000 7efdb5983740 20 client.25333 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:18.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.871+0000 7efdb5983740 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56260e83e190 0x56260e8405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.871+0000 7efdb5983740 4 client.25333 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:18.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.871+0000 7efdb5983740 1 -- 172.21.15.55:0/2596892487 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56260e79a990 con 0x56260e83e190 2024-04-26T21:55:18.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.872+0000 7efdabfff640 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56260e83e190 0x56260e8405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.876+0000 7efdabfff640 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56260e83e190 0x56260e8405f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8582a14270815d5 server_cookie=de77ae790ce4104f in_seq=0 out_seq=0 2024-04-26T21:55:18.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.877+0000 7efd8bfff640 10 client.25333 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:18.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.880+0000 7efd8bfff640 1 -- 172.21.15.55:0/2596892487 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56260e79a990 con 0x56260e83e190 2024-04-26T21:55:18.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.880+0000 7efd8bfff640 10 client.25333 handle_command_reply: tid=0 2024-04-26T21:55:18.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.880+0000 7efdb5983740 1 client.25333 shutdown 2024-04-26T21:55:18.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.881+0000 7efdb5983740 1 -- 172.21.15.55:0/2596892487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 msgr2=0x56260e832400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:18.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.881+0000 7efdb5983740 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e832400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c013620 comp rx=0 tx=0).stop 2024-04-26T21:55:18.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.881+0000 7efdb5983740 1 -- 172.21.15.55:0/2596892487 shutdown_connections 2024-04-26T21:55:18.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.881+0000 7efdb5983740 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56260e83e190 0x56260e8405f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.881+0000 7efdb5983740 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e829a70 0x56260e839980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.881+0000 7efdb5983740 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e832940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.881+0000 7efdb5983740 1 --2- 172.21.15.55:0/2596892487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e832400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.881+0000 7efdb5983740 1 -- 172.21.15.55:0/2596892487 >> 172.21.15.55:0/2596892487 conn(0x56260e8233d0 msgr2=0x56260e822550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:18.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.882+0000 7efd8bfff640 0 client.25333 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:18.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.885+0000 7efdb5983740 1 -- 172.21.15.55:0/2596892487 shutdown_connections 2024-04-26T21:55:18.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.885+0000 7efdb5983740 1 -- 172.21.15.55:0/2596892487 wait complete. 2024-04-26T21:55:18.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.888+0000 7efdb5983740 20 client.25333 trim_cache size 0 max 16384 2024-04-26T21:55:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.984+0000 7efdb5983740 1 Processor -- start 2024-04-26T21:55:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.984+0000 7efdb5983740 1 -- start start 2024-04-26T21:55:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.984+0000 7efdb5983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8233d0 0x56260e822550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.984+0000 7efdb5983740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e827ee0 0x56260e839940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.984+0000 7efdb5983740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e8290b0 0x56260e8316f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.984+0000 7efdb5983740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efd98021fa0 con 0x56260e827ee0 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.984+0000 7efdb5983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd9801f870 con 0x56260e8233d0 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.984+0000 7efdb5983740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd9801d6c0 con 0x56260e8290b0 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.985+0000 7efdabfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e827ee0 0x56260e839940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.985+0000 7efdabfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e827ee0 0x56260e839940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43626/0 (socket says 172.21.15.55:43626) 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.985+0000 7efdabfff640 1 -- 172.21.15.55:0/1695315752 learned_addr learned my addr 172.21.15.55:0/1695315752 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.986+0000 7efdb112d640 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e8290b0 0x56260e8316f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.987+0000 7efdb092c640 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8233d0 0x56260e822550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.987+0000 7efdb112d640 1 -- 172.21.15.55:0/1695315752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8233d0 msgr2=0x56260e822550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:18.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.987+0000 7efdb112d640 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8233d0 0x56260e822550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.987+0000 7efdb112d640 1 -- 172.21.15.55:0/1695315752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e827ee0 msgr2=0x56260e839940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:18.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.987+0000 7efdb112d640 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e827ee0 0x56260e839940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.987+0000 7efdb112d640 1 -- 172.21.15.55:0/1695315752 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56260e82d260 con 0x56260e8290b0 2024-04-26T21:55:18.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.988+0000 7efdb112d640 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e8290b0 0x56260e8316f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efda40cdec0 tx=0x7efda40989d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89356b647f67d0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:18.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.989+0000 7efd927fc640 1 -- 172.21.15.55:0/1695315752 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda401f040 con 0x56260e8290b0 2024-04-26T21:55:18.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.989+0000 7efd927fc640 1 -- 172.21.15.55:0/1695315752 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda409a960 con 0x56260e8290b0 2024-04-26T21:55:18.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.989+0000 7efd927fc640 1 -- 172.21.15.55:0/1695315752 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda40983b0 con 0x56260e8290b0 2024-04-26T21:55:18.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.990+0000 7efdb5983740 1 -- 172.21.15.55:0/1695315752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e8290b0 msgr2=0x56260e8316f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:18.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.990+0000 7efdb5983740 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e8290b0 0x56260e8316f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efda40cdec0 tx=0x7efda40989d0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.993+0000 7efdb5983740 1 -- 172.21.15.55:0/1695315752 shutdown_connections 2024-04-26T21:55:18.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.993+0000 7efdb5983740 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e8290b0 0x56260e8316f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.993+0000 7efdb5983740 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e827ee0 0x56260e839940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.993+0000 7efdb5983740 1 --2- 172.21.15.55:0/1695315752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8233d0 0x56260e822550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:18.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.993+0000 7efdb5983740 1 -- 172.21.15.55:0/1695315752 >> 172.21.15.55:0/1695315752 conn(0x56260e81e7f0 msgr2=0x56260e8237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:18.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.997+0000 7efdb5983740 1 -- 172.21.15.55:0/1695315752 shutdown_connections 2024-04-26T21:55:18.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.998+0000 7efdb5983740 1 -- 172.21.15.55:0/1695315752 wait complete. 2024-04-26T21:55:18.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.999+0000 7efdb5983740 1 -- start start 2024-04-26T21:55:18.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:18.999+0000 7efdb5983740 1 Processor -- start 2024-04-26T21:55:19.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.000+0000 7efdb5983740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e8233d0 0x56260e836460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:19.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.000+0000 7efdb5983740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e8369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:19.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.000+0000 7efdb5983740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e8a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:19.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.000+0000 7efdb5983740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efd9801dd30 con 0x56260e8233d0 2024-04-26T21:55:19.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.000+0000 7efdb5983740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd9801f1a0 con 0x56260e8290b0 2024-04-26T21:55:19.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.000+0000 7efdb5983740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd9801e400 con 0x56260e827ee0 2024-04-26T21:55:19.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.000+0000 7efdb5983740 10 client.0 _ll_register_callbacks cb 0x7efd9c0098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:19.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.003+0000 7efdb092c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e8233d0 0x56260e836460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:19.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.003+0000 7efdb092c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e8233d0 0x56260e836460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43642/0 (socket says 172.21.15.55:43642) 2024-04-26T21:55:19.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.003+0000 7efdb092c640 1 -- 172.21.15.55:0/292213440 learned_addr learned my addr 172.21.15.55:0/292213440 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:19.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.004+0000 7efdb112d640 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e8a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:19.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.004+0000 7efdabfff640 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e8369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:19.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.006+0000 7efdb112d640 1 -- 172.21.15.55:0/292213440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 msgr2=0x56260e8369a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:19.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.006+0000 7efdb112d640 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e8369a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.006+0000 7efdb112d640 1 -- 172.21.15.55:0/292213440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e8233d0 msgr2=0x56260e836460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:19.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.006+0000 7efdb112d640 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e8233d0 0x56260e836460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.006+0000 7efdb112d640 1 -- 172.21.15.55:0/292213440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56260e45e0f0 con 0x56260e8290b0 2024-04-26T21:55:19.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.008+0000 7efdb112d640 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e8a2760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efda409a910 tx=0x7efda40ce2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ad81c2258f84ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:19.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.008+0000 7efd8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:55:19.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.015+0000 7efd8bfff640 1 -- 172.21.15.55:0/292213440 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda401f040 con 0x56260e8290b0 2024-04-26T21:55:19.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.015+0000 7efdb5983740 1 -- 172.21.15.55:0/292213440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda409a960 con 0x56260e8290b0 2024-04-26T21:55:19.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.015+0000 7efd8bfff640 1 -- 172.21.15.55:0/292213440 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7efda409bcc0 con 0x56260e8290b0 2024-04-26T21:55:19.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.018+0000 7efd8bfff640 1 -- 172.21.15.55:0/292213440 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efda40bb300 con 0x56260e8290b0 2024-04-26T21:55:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.018+0000 7efdb5983740 10 client.25373 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:19.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.019+0000 7efdb5983740 1 -- 172.21.15.55:0/292213440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56260e74d4b0 con 0x56260e8290b0 2024-04-26T21:55:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.031+0000 7efd8bfff640 1 -- 172.21.15.55:0/292213440 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efda409a960 con 0x56260e8290b0 2024-04-26T21:55:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.031+0000 7efd8bfff640 1 -- 172.21.15.55:0/292213440 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7efda400f040 con 0x56260e8290b0 2024-04-26T21:55:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.031+0000 7efdb5983740 10 client.25373 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.031+0000 7efdb5983740 10 client.25373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.031+0000 7efdb5983740 20 client.25373 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.032+0000 7efdb5983740 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56260e829a70 0x56260e8a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.032+0000 7efdb5983740 4 client.25373 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:55:19.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.032+0000 7efdb5983740 1 -- 172.21.15.55:0/292213440 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56260e794800 con 0x56260e829a70 2024-04-26T21:55:19.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.036+0000 7efdb092c640 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56260e829a70 0x56260e8a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:19.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.043+0000 7efdb092c640 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56260e829a70 0x56260e8a6020 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=314256a52f843085 server_cookie=d13665994fba51f6 in_seq=0 out_seq=0 2024-04-26T21:55:19.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.043+0000 7efd8bfff640 10 client.25373 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:19.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.048+0000 7efd8bfff640 1 -- 172.21.15.55:0/292213440 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x56260e794800 con 0x56260e829a70 2024-04-26T21:55:19.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.048+0000 7efd8bfff640 10 client.25373 handle_command_reply: tid=0 2024-04-26T21:55:19.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.048+0000 7efdb5983740 1 client.25373 shutdown 2024-04-26T21:55:19.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.048+0000 7efdb5983740 1 -- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 msgr2=0x56260e8a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:19.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.048+0000 7efdb5983740 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e8a2760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efda409a910 tx=0x7efda40ce2f0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.051+0000 7efdb5983740 1 -- 172.21.15.55:0/292213440 shutdown_connections 2024-04-26T21:55:19.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.051+0000 7efdb5983740 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56260e8290b0 0x56260e8a2760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.051+0000 7efdb5983740 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56260e827ee0 0x56260e8369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.051+0000 7efdb5983740 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56260e829a70 0x56260e8a6020 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.051+0000 7efdb5983740 1 --2- 172.21.15.55:0/292213440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56260e8233d0 0x56260e836460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.051+0000 7efdb5983740 1 -- 172.21.15.55:0/292213440 >> 172.21.15.55:0/292213440 conn(0x56260e81e7f0 msgr2=0x56260e822550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:19.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.052+0000 7efd8bfff640 0 client.25373 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:19.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.053+0000 7efdb5983740 1 -- 172.21.15.55:0/292213440 shutdown_connections 2024-04-26T21:55:19.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.054+0000 7efdb5983740 1 -- 172.21.15.55:0/292213440 wait complete. 2024-04-26T21:55:19.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.054+0000 7efdb5983740 20 client.25373 trim_cache size 0 max 16384 2024-04-26T21:55:19.055 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:55:19.055 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (3 inodes in the stack)", 2024-04-26T21:55:19.055 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:55:19.055 INFO:teuthology.orchestra.run.smithi055.stdout: "2e51f7a0-5ecc-47bf-b91f-0946fdaee0a1": { 2024-04-26T21:55:19.055 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:55:19.055 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "2e51f7a0-5ecc-47bf-b91f-0946fdaee0a1", 2024-04-26T21:55:19.055 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:55:19.056 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:55:19.056 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:55:19.056 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:55:19.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.060+0000 7efdb2b90640 1 -- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efd940a3dc0 msgr2=0x7efd940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:19.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.060+0000 7efdb2b90640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efd940a3dc0 0x7efd940a6280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efda0005740 tx=0x7efda0000bc0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.060+0000 7efdb2b90640 1 -- 172.21.15.55:0/1074901889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac10ec70 msgr2=0x7efdac1aee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:19.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.060+0000 7efdb2b90640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac10ec70 0x7efdac1aee20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efda400e9d0 tx=0x7efda400ee90 comp rx=0 tx=0).stop 2024-04-26T21:55:19.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.065+0000 7efdb2b90640 1 -- 172.21.15.55:0/1074901889 shutdown_connections 2024-04-26T21:55:19.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.065+0000 7efdb2b90640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efd940a3dc0 0x7efd940a6280 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.065+0000 7efdb2b90640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efdac10ec70 0x7efdac1aee20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.065+0000 7efdb2b90640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdac10c2a0 0x7efdac1a7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.065+0000 7efdb2b90640 1 --2- 172.21.15.55:0/1074901889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac109930 0x7efdac1a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:19.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.065+0000 7efdb2b90640 1 -- 172.21.15.55:0/1074901889 >> 172.21.15.55:0/1074901889 conn(0x7efdac06f7c0 msgr2=0x7efdac10ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.067+0000 7efdb2b90640 1 -- 172.21.15.55:0/1074901889 shutdown_connections 2024-04-26T21:55:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:19.068+0000 7efdb2b90640 1 -- 172.21.15.55:0/1074901889 wait complete. 2024-04-26T21:55:19.138 INFO:tasks.cephfs.filesystem:scrub status for tag:2e51f7a0-5ecc-47bf-b91f-0946fdaee0a1 - {'path': '/', 'tag': '2e51f7a0-5ecc-47bf-b91f-0946fdaee0a1', 'options': 'recursive,force'} 2024-04-26T21:55:19.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:19 smithi165 ceph-mon[33296]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 919 MiB/s wr, 1.48k op/s 2024-04-26T21:55:19.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:19 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:19.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:19 smithi055 ceph-mon[29221]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 919 MiB/s wr, 1.48k op/s 2024-04-26T21:55:19.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:19 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:19 smithi064 ceph-mon[32765]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 919 MiB/s wr, 1.48k op/s 2024-04-26T21:55:19.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:19 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:20.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:20 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:55:20.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:20 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:20.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:20 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:55:20.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:20.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:20 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:20.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:20 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:55:20.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:20 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:20.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:20 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:55:20.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:20.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:20 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:20.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:20 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:55:20.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:20 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:20.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:20 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:55:20.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:20.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:20 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:21.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:21 smithi165 ceph-mon[33296]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.23k op/s 2024-04-26T21:55:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:21 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:21 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:21 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:21 smithi055 ceph-mon[29221]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.23k op/s 2024-04-26T21:55:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:21 smithi064 ceph-mon[32765]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.23k op/s 2024-04-26T21:55:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:21 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:21 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:21 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:55:23.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:23 smithi165 ceph-mon[33296]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 MiB/s wr, 1.27k op/s 2024-04-26T21:55:23.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:23 smithi055 ceph-mon[29221]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 MiB/s wr, 1.27k op/s 2024-04-26T21:55:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:23 smithi064 ceph-mon[32765]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 MiB/s wr, 1.27k op/s 2024-04-26T21:55:25.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:25 smithi055 ceph-mon[29221]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 807 MiB/s wr, 1.40k op/s 2024-04-26T21:55:25.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:25 smithi064 ceph-mon[32765]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 807 MiB/s wr, 1.40k op/s 2024-04-26T21:55:25.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:25 smithi165 ceph-mon[33296]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 807 MiB/s wr, 1.40k op/s 2024-04-26T21:55:26.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:26 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:26.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:26 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:26.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:26 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:27.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:27 smithi055 ceph-mon[29221]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 878 MiB/s wr, 1.83k op/s 2024-04-26T21:55:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:27 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:55:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:27 smithi064 ceph-mon[32765]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 878 MiB/s wr, 1.83k op/s 2024-04-26T21:55:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:27 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:55:27.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:27 smithi165 ceph-mon[33296]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 878 MiB/s wr, 1.83k op/s 2024-04-26T21:55:27.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:27 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:55:29.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:29 smithi055 ceph-mon[29221]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 921 MiB/s wr, 2.37k op/s 2024-04-26T21:55:29.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:29 smithi064 ceph-mon[32765]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 921 MiB/s wr, 2.37k op/s 2024-04-26T21:55:29.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:29 smithi165 ceph-mon[33296]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 921 MiB/s wr, 2.37k op/s 2024-04-26T21:55:31.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:31 smithi055 ceph-mon[29221]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 882 MiB/s wr, 2.58k op/s 2024-04-26T21:55:31.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:31 smithi064 ceph-mon[32765]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 882 MiB/s wr, 2.58k op/s 2024-04-26T21:55:31.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:31 smithi165 ceph-mon[33296]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 882 MiB/s wr, 2.58k op/s 2024-04-26T21:55:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:33 smithi055 ceph-mon[29221]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 804 MiB/s wr, 2.52k op/s 2024-04-26T21:55:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:33 smithi064 ceph-mon[32765]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 804 MiB/s wr, 2.52k op/s 2024-04-26T21:55:33.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:33 smithi165 ceph-mon[33296]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 804 MiB/s wr, 2.52k op/s 2024-04-26T21:55:35.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:35 smithi055 ceph-mon[29221]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 793 MiB/s wr, 2.40k op/s 2024-04-26T21:55:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:35 smithi064 ceph-mon[32765]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 793 MiB/s wr, 2.40k op/s 2024-04-26T21:55:35.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:35 smithi165 ceph-mon[33296]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 793 MiB/s wr, 2.40k op/s 2024-04-26T21:55:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:37 smithi055 ceph-mon[29221]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 2.33k op/s 2024-04-26T21:55:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:37 smithi064 ceph-mon[32765]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 2.33k op/s 2024-04-26T21:55:37.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:37 smithi165 ceph-mon[33296]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 2.33k op/s 2024-04-26T21:55:39.444 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:39 smithi055 ceph-mon[29221]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 909 MiB/s wr, 2.06k op/s 2024-04-26T21:55:39.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:39 smithi064 ceph-mon[32765]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 909 MiB/s wr, 2.06k op/s 2024-04-26T21:55:39.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:39 smithi165 ceph-mon[33296]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 909 MiB/s wr, 2.06k op/s 2024-04-26T21:55:41.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:41 smithi055 ceph-mon[29221]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 874 MiB/s wr, 1.54k op/s 2024-04-26T21:55:41.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:41 smithi064 ceph-mon[32765]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 874 MiB/s wr, 1.54k op/s 2024-04-26T21:55:41.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:41 smithi165 ceph-mon[33296]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 874 MiB/s wr, 1.54k op/s 2024-04-26T21:55:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:43 smithi064 ceph-mon[32765]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 798 MiB/s wr, 1.14k op/s 2024-04-26T21:55:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:43 smithi165 ceph-mon[33296]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 798 MiB/s wr, 1.14k op/s 2024-04-26T21:55:43.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:43 smithi055 ceph-mon[29221]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 798 MiB/s wr, 1.14k op/s 2024-04-26T21:55:45.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:45 smithi064 ceph-mon[32765]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.12k op/s 2024-04-26T21:55:45.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:45 smithi165 ceph-mon[33296]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.12k op/s 2024-04-26T21:55:45.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:45 smithi055 ceph-mon[29221]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.12k op/s 2024-04-26T21:55:47.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:47 smithi064 ceph-mon[32765]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 860 MiB/s wr, 1.02k op/s 2024-04-26T21:55:47.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:47 smithi165 ceph-mon[33296]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 860 MiB/s wr, 1.02k op/s 2024-04-26T21:55:47.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:47 smithi055 ceph-mon[29221]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 860 MiB/s wr, 1.02k op/s 2024-04-26T21:55:49.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:55:49.140 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:55:49.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 -- 172.21.15.55:0/4171327462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c102a70 msgr2=0x7f8a6c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/4171327462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c102a70 0x7f8a6c102ef0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009900 tx=0x7f8a5c02a410 comp rx=0 tx=0).stop 2024-04-26T21:55:49.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 -- 172.21.15.55:0/4171327462 shutdown_connections 2024-04-26T21:55:49.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/4171327462 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c103430 0x7f8a6c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/4171327462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c102a70 0x7f8a6c102ef0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/4171327462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a6c101880 0x7f8a6c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 -- 172.21.15.55:0/4171327462 >> 172.21.15.55:0/4171327462 conn(0x7f8a6c0fcdf0 msgr2=0x7f8a6c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:49.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 -- 172.21.15.55:0/4171327462 shutdown_connections 2024-04-26T21:55:49.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.421+0000 7f8a72cf1640 1 -- 172.21.15.55:0/4171327462 wait complete. 2024-04-26T21:55:49.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.422+0000 7f8a72cf1640 1 Processor -- start 2024-04-26T21:55:49.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.422+0000 7f8a72cf1640 1 -- start start 2024-04-26T21:55:49.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.424+0000 7f8a72cf1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c101880 0x7f8a6c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.424+0000 7f8a72cf1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a6c102a70 0x7f8a6c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.424+0000 7f8a72cf1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c103430 0x7f8a6c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.424+0000 7f8a70a8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c101880 0x7f8a6c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.424+0000 7f8a70a8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c101880 0x7f8a6c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41664/0 (socket says 172.21.15.55:41664) 2024-04-26T21:55:49.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.424+0000 7f8a70a8d640 1 -- 172.21.15.55:0/1323414646 learned_addr learned my addr 172.21.15.55:0/1323414646 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:49.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.424+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10e5c0 con 0x7f8a6c101880 2024-04-26T21:55:49.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.425+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a6c102a70 0x7f8a6c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.425+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10e440 con 0x7f8a6c103430 2024-04-26T21:55:49.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.425+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c10e740 con 0x7f8a6c102a70 2024-04-26T21:55:49.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.425+0000 7f8a7128e640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c103430 0x7f8a6c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.426+0000 7f8a70a8d640 1 -- 172.21.15.55:0/1323414646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a6c102a70 msgr2=0x7f8a6c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.426+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a6c102a70 0x7f8a6c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.426+0000 7f8a70a8d640 1 -- 172.21.15.55:0/1323414646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c103430 msgr2=0x7f8a6c1a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.426+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c103430 0x7f8a6c1a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.426+0000 7f8a70a8d640 1 -- 172.21.15.55:0/1323414646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6c1a0890 con 0x7f8a6c101880 2024-04-26T21:55:49.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.426+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c101880 0x7f8a6c198c80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d950 tx=0x7f8a6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:49.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.427+0000 7f8a69ffb640 1 -- 172.21.15.55:0/1323414646 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60017070 con 0x7f8a6c101880 2024-04-26T21:55:49.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.427+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c1a0b80 con 0x7f8a6c101880 2024-04-26T21:55:49.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.427+0000 7f8a69ffb640 1 -- 172.21.15.55:0/1323414646 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a60009c30 con 0x7f8a6c101880 2024-04-26T21:55:49.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.427+0000 7f8a69ffb640 1 -- 172.21.15.55:0/1323414646 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000f620 con 0x7f8a6c101880 2024-04-26T21:55:49.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.427+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6c1a10d0 con 0x7f8a6c101880 2024-04-26T21:55:49.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.431+0000 7f8a75a08740 1 Processor -- start 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a75a08740 1 -- start start 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22bad300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bae0b0 0x559a22bae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22baea70 0x559a22bb5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a75a08740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559a22b11d80 con 0x559a22baea70 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a75a08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a22acb8b0 con 0x559a22bae0b0 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a75a08740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559a22b0dd40 con 0x559a22bacee0 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22baea70 0x559a22bb5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22baea70 0x559a22bb5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41676/0 (socket says 172.21.15.55:41676) 2024-04-26T21:55:49.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a6bfff640 1 -- 172.21.15.55:0/917605684 learned_addr learned my addr 172.21.15.55:0/917605684 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:49.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a69ffb640 1 -- 172.21.15.55:0/1323414646 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a6001a070 con 0x7f8a6c101880 2024-04-26T21:55:49.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a6bfff640 1 -- 172.21.15.55:0/917605684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 msgr2=0x559a22bad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:49.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a7128e640 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22bad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22bad300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a6bfff640 1 -- 172.21.15.55:0/917605684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bae0b0 msgr2=0x559a22bae530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:55:49.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bae0b0 0x559a22bae530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a6bfff640 1 -- 172.21.15.55:0/917605684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a22a9ca40 con 0x559a22baea70 2024-04-26T21:55:49.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.432+0000 7f8a7128e640 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22bad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.433+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22baea70 0x559a22bb5300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009900 tx=0x7f8a5c004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4db0c4f3ea0b220 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.433+0000 7f8a52ffd640 1 -- 172.21.15.55:0/917605684 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c035030 con 0x559a22baea70 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.433+0000 7f8a69ffb640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a540a3c40 0x7f8a540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.433+0000 7f8a52ffd640 1 -- 172.21.15.55:0/917605684 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a5c040070 con 0x559a22baea70 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.433+0000 7f8a52ffd640 1 -- 172.21.15.55:0/917605684 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c031b40 con 0x559a22baea70 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a7128e640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a540a3c40 0x7f8a540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a69ffb640 1 -- 172.21.15.55:0/1323414646 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8a600d0030 con 0x7f8a6c101880 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a7128e640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a540a3c40 0x7f8a540a6100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x559a22badfb0 tx=0x7f8a64009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a75a08740 1 -- 172.21.15.55:0/917605684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22baea70 msgr2=0x559a22bb5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a75a08740 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22baea70 0x559a22bb5300 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009900 tx=0x7f8a5c004390 comp rx=0 tx=0).stop 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a75a08740 1 -- 172.21.15.55:0/917605684 shutdown_connections 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a75a08740 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22baea70 0x559a22bb5300 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a75a08740 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bae0b0 0x559a22bae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a75a08740 1 --2- 172.21.15.55:0/917605684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22bad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.434+0000 7f8a75a08740 1 -- 172.21.15.55:0/917605684 >> 172.21.15.55:0/917605684 conn(0x559a22ba83d0 msgr2=0x559a22ba87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 -- 172.21.15.55:0/917605684 shutdown_connections 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 -- 172.21.15.55:0/917605684 wait complete. 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 -- start start 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 Processor -- start 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bacee0 0x559a22bb7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bae0b0 0x559a22bb7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 0x559a22bbee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559a22bb5a20 con 0x559a22bacee0 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a22a98d10 con 0x559a22bae0b0 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.435+0000 7f8a75a08740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559a22bb5ba0 con 0x559a22bbca90 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a75a08740 10 client.0 _ll_register_callbacks cb 0x559a22ac5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bacee0 0x559a22bb7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bacee0 0x559a22bb7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41690/0 (socket says 172.21.15.55:41690) 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 -- 172.21.15.55:0/3369430341 learned_addr learned my addr 172.21.15.55:0/3369430341 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 -- 172.21.15.55:0/3369430341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 msgr2=0x559a22bbee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 0x559a22bbee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 -- 172.21.15.55:0/3369430341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bae0b0 msgr2=0x559a22bb7a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bae0b0 0x559a22bb7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 -- 172.21.15.55:0/3369430341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a228ed7d0 con 0x559a22bacee0 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bacee0 0x559a22bb7520 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009610 tx=0x7f8a600cde20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd617242dbbc4e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:49.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a4bfff640 1 -- 172.21.15.55:0/3369430341 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60021050 con 0x559a22bacee0 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a4bfff640 1 -- 172.21.15.55:0/3369430341 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8a6009b670 con 0x559a22bacee0 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.436+0000 7f8a75a08740 1 -- 172.21.15.55:0/3369430341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a5c040070 con 0x559a22bacee0 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a4bfff640 1 -- 172.21.15.55:0/3369430341 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8a600ba050 con 0x559a22bacee0 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a75a08740 10 client.15825 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a75a08740 1 -- 172.21.15.55:0/3369430341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559a228233f0 con 0x559a22bacee0 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a4bfff640 1 -- 172.21.15.55:0/3369430341 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a5c040070 con 0x559a22bacee0 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a4bfff640 1 -- 172.21.15.55:0/3369430341 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8a6009b0e0 con 0x559a22bacee0 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a75a08740 10 client.15825 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a75a08740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a75a08740 20 client.15825 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a75a08740 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a22bc32e0 0x559a22bc5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a75a08740 4 client.15825 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:49.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.437+0000 7f8a75a08740 1 -- 172.21.15.55:0/3369430341 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a22b1f990 con 0x559a22bc32e0 2024-04-26T21:55:49.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.439+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a22bc32e0 0x559a22bc5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.442+0000 7f8a4bfff640 10 client.15825 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:49.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.442+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a22bc32e0 0x559a22bc5780 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95db0f5eea3273b8 server_cookie=1d1090bc60daab74 in_seq=0 out_seq=0 2024-04-26T21:55:49.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a4bfff640 1 -- 172.21.15.55:0/3369430341 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a22b1f990 con 0x559a22bc32e0 2024-04-26T21:55:49.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a4bfff640 10 client.15825 handle_command_reply: tid=0 2024-04-26T21:55:49.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 client.15825 shutdown 2024-04-26T21:55:49.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 -- 172.21.15.55:0/3369430341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bacee0 msgr2=0x559a22bb7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bacee0 0x559a22bb7520 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009610 tx=0x7f8a600cde20 comp rx=0 tx=0).stop 2024-04-26T21:55:49.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 -- 172.21.15.55:0/3369430341 shutdown_connections 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 0x559a22bbee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bae0b0 0x559a22bb7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a22bc32e0 0x559a22bc5780 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a4bfff640 0 client.15825 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 --2- 172.21.15.55:0/3369430341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bacee0 0x559a22bb7520 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.446+0000 7f8a75a08740 1 -- 172.21.15.55:0/3369430341 >> 172.21.15.55:0/3369430341 conn(0x559a22ba83d0 msgr2=0x559a22ba7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.447+0000 7f8a75a08740 1 -- 172.21.15.55:0/3369430341 shutdown_connections 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.447+0000 7f8a75a08740 1 -- 172.21.15.55:0/3369430341 wait complete. 2024-04-26T21:55:49.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.447+0000 7f8a75a08740 20 client.15825 trim_cache size 0 max 16384 2024-04-26T21:55:49.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.482+0000 7f8a75a08740 1 Processor -- start 2024-04-26T21:55:49.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.482+0000 7f8a75a08740 1 -- start start 2024-04-26T21:55:49.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bacee0 0x559a22bb79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22bb5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 0x559a22bab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a75a08740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1ac770 con 0x559a22bae0b0 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a75a08740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1aa040 con 0x559a22bacee0 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a75a08740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a7e90 con 0x559a22bbca90 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22bb5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22bb5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41706/0 (socket says 172.21.15.55:41706) 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a6bfff640 1 -- 172.21.15.55:0/243454061 learned_addr learned my addr 172.21.15.55:0/243454061 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a7128e640 1 --2- 172.21.15.55:0/243454061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 0x559a22bab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a6bfff640 1 -- 172.21.15.55:0/243454061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 msgr2=0x559a22bab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/243454061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 0x559a22bab2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a6bfff640 1 -- 172.21.15.55:0/243454061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bacee0 msgr2=0x559a22bb79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/243454061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bacee0 0x559a22bb79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.483+0000 7f8a6bfff640 1 -- 172.21.15.55:0/243454061 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a22bb6600 con 0x559a22bae0b0 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.484+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/243454061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22bb5fa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c006010 tx=0x7f8a5c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15cd7b4a73a6a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.484+0000 7f8a527fc640 1 -- 172.21.15.55:0/243454061 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c051650 con 0x559a22bae0b0 2024-04-26T21:55:49.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.484+0000 7f8a527fc640 1 -- 172.21.15.55:0/243454061 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a5c040070 con 0x559a22bae0b0 2024-04-26T21:55:49.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.484+0000 7f8a527fc640 1 -- 172.21.15.55:0/243454061 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c003850 con 0x559a22bae0b0 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.485+0000 7f8a75a08740 1 -- 172.21.15.55:0/243454061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 msgr2=0x559a22bb5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.486+0000 7f8a75a08740 1 --2- 172.21.15.55:0/243454061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22bb5fa0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c006010 tx=0x7f8a5c02a410 comp rx=0 tx=0).stop 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.486+0000 7f8a75a08740 1 -- 172.21.15.55:0/243454061 shutdown_connections 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.486+0000 7f8a75a08740 1 --2- 172.21.15.55:0/243454061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bbca90 0x559a22bab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.486+0000 7f8a75a08740 1 --2- 172.21.15.55:0/243454061 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22bb5fa0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.486+0000 7f8a75a08740 1 --2- 172.21.15.55:0/243454061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bacee0 0x559a22bb79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.486+0000 7f8a75a08740 1 -- 172.21.15.55:0/243454061 >> 172.21.15.55:0/243454061 conn(0x559a22ba83d0 msgr2=0x559a22bb11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.486+0000 7f8a75a08740 1 -- 172.21.15.55:0/243454061 shutdown_connections 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.486+0000 7f8a75a08740 1 -- 172.21.15.55:0/243454061 wait complete. 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.487+0000 7f8a75a08740 1 -- start start 2024-04-26T21:55:49.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.487+0000 7f8a75a08740 1 Processor -- start 2024-04-26T21:55:49.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.487+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22c26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.487+0000 7f8a75a08740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22c26a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22c26a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a6bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22c26a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41714/0 (socket says 172.21.15.55:41714) 2024-04-26T21:55:49.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a6bfff640 1 -- 172.21.15.55:0/957753994 learned_addr learned my addr 172.21.15.55:0/957753994 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:49.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22c26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a75a08740 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bbca90 0x559a22c27aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a8500 con 0x559a22bae0b0 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a9970 con 0x559a22bbca90 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6c1a8bd0 con 0x559a22bacee0 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.488+0000 7f8a75a08740 10 client.0 _ll_register_callbacks cb 0x7f8a6009c010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.489+0000 7f8a7128e640 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bbca90 0x559a22c27aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.489+0000 7f8a6bfff640 1 -- 172.21.15.55:0/957753994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 msgr2=0x559a22c26550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.489+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22c26550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.489+0000 7f8a6bfff640 1 -- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bbca90 msgr2=0x559a22c27aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.489+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bbca90 0x559a22c27aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.489+0000 7f8a6bfff640 1 -- 172.21.15.55:0/957753994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a227e30f0 con 0x559a22bae0b0 2024-04-26T21:55:49.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.489+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22c26550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:55:49.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.490+0000 7f8a6bfff640 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22c26a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe398756ed20b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:49.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.490+0000 7f8a4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:49.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.490+0000 7f8a4bfff640 1 -- 172.21.15.55:0/957753994 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c035030 con 0x559a22bae0b0 2024-04-26T21:55:49.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.490+0000 7f8a4bfff640 1 -- 172.21.15.55:0/957753994 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8a5c0315a0 con 0x559a22bae0b0 2024-04-26T21:55:49.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.490+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a5c040070 con 0x559a22bae0b0 2024-04-26T21:55:49.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.490+0000 7f8a4bfff640 1 -- 172.21.15.55:0/957753994 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8a5c0317a0 con 0x559a22bae0b0 2024-04-26T21:55:49.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.490+0000 7f8a75a08740 10 client.15831 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:49.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x559a22ad24b0 con 0x559a22bae0b0 2024-04-26T21:55:49.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a4bfff640 1 -- 172.21.15.55:0/957753994 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a5c031940 con 0x559a22bae0b0 2024-04-26T21:55:49.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a4bfff640 1 -- 172.21.15.55:0/957753994 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8a5c045470 con 0x559a22bae0b0 2024-04-26T21:55:49.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a75a08740 10 client.15831 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:49.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a75a08740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:49.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a75a08740 20 client.15831 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:49.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a75a08740 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a22bc32e0 0x559a22bd5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a75a08740 4 client.15831 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:55:49.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.491+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8a5c040070 con 0x559a22bc32e0 2024-04-26T21:55:49.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.492+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a22bc32e0 0x559a22bd5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.493+0000 7f8a4bfff640 10 client.15831 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:49.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.493+0000 7f8a70a8d640 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a22bc32e0 0x559a22bd5870 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a57435c6cb1ed5c8 server_cookie=f15acda9bb813ad9 in_seq=0 out_seq=0 2024-04-26T21:55:49.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.496+0000 7f8a4bfff640 1 -- 172.21.15.55:0/957753994 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f8a5c040070 con 0x559a22bc32e0 2024-04-26T21:55:49.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.496+0000 7f8a4bfff640 10 client.15831 handle_command_reply: tid=0 2024-04-26T21:55:49.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.496+0000 7f8a75a08740 1 client.15831 shutdown 2024-04-26T21:55:49.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 msgr2=0x559a22c26a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22c26a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c000c00 tx=0x7f8a5c009520 comp rx=0 tx=0).stop 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 shutdown_connections 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a22bbca90 0x559a22c27aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a22bc32e0 0x559a22bd5870 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a4bfff640 0 client.15831 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a22bae0b0 0x559a22c26a90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 --2- 172.21.15.55:0/957753994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a22bacee0 0x559a22c26550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 >> 172.21.15.55:0/957753994 conn(0x559a22ba83d0 msgr2=0x559a22bb7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 shutdown_connections 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 1 -- 172.21.15.55:0/957753994 wait complete. 2024-04-26T21:55:49.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.497+0000 7f8a75a08740 20 client.15831 trim_cache size 0 max 16384 2024-04-26T21:55:49.499 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:55:49.499 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:55:49.499 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:55:49.499 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.498+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a540a3c40 msgr2=0x7f8a540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.498+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a540a3c40 0x7f8a540a6100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x559a22badfb0 tx=0x7f8a64009650 comp rx=0 tx=0).stop 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.498+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c101880 msgr2=0x7f8a6c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.498+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c101880 0x7f8a6c198c80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000d950 tx=0x7f8a6000de10 comp rx=0 tx=0).stop 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.499+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 shutdown_connections 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.499+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8a540a3c40 0x7f8a540a6100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.499+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a6c103430 0x7f8a6c1a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.499+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a6c102a70 0x7f8a6c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.499+0000 7f8a72cf1640 1 --2- 172.21.15.55:0/1323414646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a6c101880 0x7f8a6c198c80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.499+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 >> 172.21.15.55:0/1323414646 conn(0x7f8a6c0fcdf0 msgr2=0x7f8a6c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:49.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.499+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 shutdown_connections 2024-04-26T21:55:49.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.499+0000 7f8a72cf1640 1 -- 172.21.15.55:0/1323414646 wait complete. 2024-04-26T21:55:49.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:49 smithi064 ceph-mon[32765]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 885 MiB/s wr, 1.04k op/s 2024-04-26T21:55:49.525 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:55:49.525 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:55:49.541 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:49 smithi055 ceph-mon[29221]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 885 MiB/s wr, 1.04k op/s 2024-04-26T21:55:49.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:49 smithi165 ceph-mon[33296]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 885 MiB/s wr, 1.04k op/s 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 -- 172.21.15.55:0/3731105404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 msgr2=0x7fccec1081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 --2- 172.21.15.55:0/3731105404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 0x7fccec1081b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd402a410 comp rx=0 tx=0).stop 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 -- 172.21.15.55:0/3731105404 shutdown_connections 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 --2- 172.21.15.55:0/3731105404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec1086f0 0x7fccec10aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 --2- 172.21.15.55:0/3731105404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 0x7fccec1081b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 --2- 172.21.15.55:0/3731105404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccec103490 0x7fccec105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 -- 172.21.15.55:0/3731105404 >> 172.21.15.55:0/3731105404 conn(0x7fccec0fce20 msgr2=0x7fccec0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 -- 172.21.15.55:0/3731105404 shutdown_connections 2024-04-26T21:55:49.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.896+0000 7fccf08f1640 1 -- 172.21.15.55:0/3731105404 wait complete. 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.897+0000 7fccf08f1640 1 Processor -- start 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.897+0000 7fccf08f1640 1 -- start start 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.897+0000 7fccf08f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec103490 0x7fccec198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.897+0000 7fccf08f1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 0x7fccec199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fccf08f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccec1086f0 0x7fccec1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fccf08f1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10e5d0 con 0x7fccec105dc0 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fccf08f1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10e450 con 0x7fccec103490 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fcce3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 0x7fccec199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fccf08f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fccec10e750 con 0x7fccec1086f0 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fcce3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 0x7fccec199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41722/0 (socket says 172.21.15.55:41722) 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fcce3fff640 1 -- 172.21.15.55:0/1334427175 learned_addr learned my addr 172.21.15.55:0/1334427175 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fccea59c640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec103490 0x7fccec198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fcce3fff640 1 -- 172.21.15.55:0/1334427175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccec1086f0 msgr2=0x7fccec1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fcce3fff640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccec1086f0 0x7fccec1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fcce3fff640 1 -- 172.21.15.55:0/1334427175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec103490 msgr2=0x7fccec198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fcce3fff640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec103490 0x7fccec198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fcce3fff640 1 -- 172.21.15.55:0/1334427175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec1a0800 con 0x7fccec105dc0 2024-04-26T21:55:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.898+0000 7fccea59c640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec103490 0x7fccec198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.899+0000 7fcce3fff640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 0x7fccec199120 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009230 tx=0x7fccd40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.899+0000 7fcce37fe640 1 -- 172.21.15.55:0/1334427175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd403f030 con 0x7fccec105dc0 2024-04-26T21:55:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.899+0000 7fcce37fe640 1 -- 172.21.15.55:0/1334427175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd4003d30 con 0x7fccec105dc0 2024-04-26T21:55:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.899+0000 7fcce37fe640 1 -- 172.21.15.55:0/1334427175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4035070 con 0x7fccec105dc0 2024-04-26T21:55:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.899+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec1a0a90 con 0x7fccec105dc0 2024-04-26T21:55:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.899+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec1a0fb0 con 0x7fccec105dc0 2024-04-26T21:55:49.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.901+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccec0fe910 con 0x7fccec105dc0 2024-04-26T21:55:49.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.904+0000 7fcce37fe640 1 -- 172.21.15.55:0/1334427175 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fccd4042030 con 0x7fccec105dc0 2024-04-26T21:55:49.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.906+0000 7fcce37fe640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccd80a3c40 0x7fccd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:49.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.907+0000 7fccea59c640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccd80a3c40 0x7fccd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:49.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.907+0000 7fcce37fe640 1 -- 172.21.15.55:0/1334427175 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fccd40f0f60 con 0x7fccec105dc0 2024-04-26T21:55:49.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.908+0000 7fccea59c640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccd80a3c40 0x7fccd80a6100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00a8b0 tx=0x7fccdc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:49.916+0000 7fcce37fe640 1 -- 172.21.15.55:0/1334427175 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fccd4049030 con 0x7fccec105dc0 2024-04-26T21:55:50.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.440+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccec02dd00 con 0x7fccec105dc0 2024-04-26T21:55:50.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.441+0000 7fcce37fe640 1 -- 172.21.15.55:0/1334427175 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11863 (secure 0 0 0) 0x7fccd40bd740 con 0x7fccec105dc0 2024-04-26T21:55:50.442 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:55:50.443 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:52:54.328649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24502,24509]},"id":1}]} 2024-04-26T21:55:50.443 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 21 2024-04-26T21:55:50.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccd80a3c40 msgr2=0x7fccd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccd80a3c40 0x7fccd80a6100 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00a8b0 tx=0x7fccdc008040 comp rx=0 tx=0).stop 2024-04-26T21:55:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 msgr2=0x7fccec199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 0x7fccec199120 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009230 tx=0x7fccd40045e0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 shutdown_connections 2024-04-26T21:55:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fccd80a3c40 0x7fccd80a6100 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fccec1086f0 0x7fccec1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccec105dc0 0x7fccec199120 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 --2- 172.21.15.55:0/1334427175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccec103490 0x7fccec198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.444+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 >> 172.21.15.55:0/1334427175 conn(0x7fccec0fce20 msgr2=0x7fccec103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:50.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.445+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 shutdown_connections 2024-04-26T21:55:50.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.445+0000 7fccf08f1640 1 -- 172.21.15.55:0/1334427175 wait complete. 2024-04-26T21:55:50.466 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:55:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.830+0000 7f5373ed9640 1 -- 172.21.15.55:0/232078116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c106340 msgr2=0x7f536c1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.830+0000 7f5373ed9640 1 --2- 172.21.15.55:0/232078116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c106340 0x7f536c1067c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f536401acb0 tx=0x7f536403b920 comp rx=0 tx=0).stop 2024-04-26T21:55:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.830+0000 7f5373ed9640 1 -- 172.21.15.55:0/232078116 shutdown_connections 2024-04-26T21:55:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.830+0000 7f5373ed9640 1 --2- 172.21.15.55:0/232078116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c106340 0x7f536c1067c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.830+0000 7f5373ed9640 1 --2- 172.21.15.55:0/232078116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f536c103990 0x7f536c105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.830+0000 7f5373ed9640 1 --2- 172.21.15.55:0/232078116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c101020 0x7f536c103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.830+0000 7f5373ed9640 1 -- 172.21.15.55:0/232078116 >> 172.21.15.55:0/232078116 conn(0x7f536c0fcc80 msgr2=0x7f536c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.832+0000 7f5373ed9640 1 -- 172.21.15.55:0/232078116 shutdown_connections 2024-04-26T21:55:50.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.832+0000 7f5373ed9640 1 -- 172.21.15.55:0/232078116 wait complete. 2024-04-26T21:55:50.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.833+0000 7f5373ed9640 1 Processor -- start 2024-04-26T21:55:50.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.834+0000 7f5373ed9640 1 -- start start 2024-04-26T21:55:50.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.834+0000 7f5373ed9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f536c101020 0x7f536c196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.834+0000 7f5373ed9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c103990 0x7f536c196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.834+0000 7f5373ed9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c106340 0x7f536c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.834+0000 7f5373ed9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f536c10a210 con 0x7f536c103990 2024-04-26T21:55:50.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.834+0000 7f5373ed9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f536c10a090 con 0x7f536c106340 2024-04-26T21:55:50.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.834+0000 7f5373ed9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f536c10a390 con 0x7f536c101020 2024-04-26T21:55:50.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.836+0000 7f5371474640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c103990 0x7f536c196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.836+0000 7f5371c75640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f536c101020 0x7f536c196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.836+0000 7f5371c75640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f536c101020 0x7f536c196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33176/0 (socket says 172.21.15.55:33176) 2024-04-26T21:55:50.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.836+0000 7f5371c75640 1 -- 172.21.15.55:0/173869646 learned_addr learned my addr 172.21.15.55:0/173869646 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.837+0000 7f5371474640 1 -- 172.21.15.55:0/173869646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f536c101020 msgr2=0x7f536c196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.837+0000 7f5372476640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c106340 0x7f536c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.837+0000 7f5371474640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f536c101020 0x7f536c196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.838+0000 7f5371474640 1 -- 172.21.15.55:0/173869646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c106340 msgr2=0x7f536c19dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.838+0000 7f5371474640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c106340 0x7f536c19dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.838+0000 7f5371474640 1 -- 172.21.15.55:0/173869646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c06c480 con 0x7f536c103990 2024-04-26T21:55:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.838+0000 7f5372476640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c106340 0x7f536c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.838+0000 7f5371c75640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f536c101020 0x7f536c196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:55:50.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.838+0000 7f5371474640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c103990 0x7f536c196d70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f535c009fd0 tx=0x7f535c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:50.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.839+0000 7f5362ffd640 1 -- 172.21.15.55:0/173869646 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c015020 con 0x7f536c103990 2024-04-26T21:55:50.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.839+0000 7f5373ed9640 1 -- 172.21.15.55:0/173869646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c06c770 con 0x7f536c103990 2024-04-26T21:55:50.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.839+0000 7f5373ed9640 1 -- 172.21.15.55:0/173869646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c06cd90 con 0x7f536c103990 2024-04-26T21:55:50.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.839+0000 7f5362ffd640 1 -- 172.21.15.55:0/173869646 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c011ce0 con 0x7f536c103990 2024-04-26T21:55:50.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.840+0000 7f5362ffd640 1 -- 172.21.15.55:0/173869646 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c010de0 con 0x7f536c103990 2024-04-26T21:55:50.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.844+0000 7f5376bf0740 1 Processor -- start 2024-04-26T21:55:50.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.844+0000 7f5376bf0740 1 -- start start 2024-04-26T21:55:50.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.844+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630319e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.845+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363031b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.845+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x563630321e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.845+0000 7f5376bf0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56363027dd80 con 0x56363031abd0 2024-04-26T21:55:50.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.845+0000 7f5372476640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630319e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.845+0000 7f5372476640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630319e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49074/0 (socket says 172.21.15.55:49074) 2024-04-26T21:55:50.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.845+0000 7f5372476640 1 -- 172.21.15.55:0/2451112315 learned_addr learned my addr 172.21.15.55:0/2451112315 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:50.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.845+0000 7f5376bf0740 1 -- 172.21.15.55:0/2451112315 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563630237a20 con 0x563630319a00 2024-04-26T21:55:50.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.845+0000 7f5376bf0740 1 -- 172.21.15.55:0/2451112315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563630279d40 con 0x56363031b590 2024-04-26T21:55:50.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.846+0000 7f5362ffd640 1 -- 172.21.15.55:0/173869646 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f535c018020 con 0x7f536c103990 2024-04-26T21:55:50.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.847+0000 7f5371c75640 1 --2- 172.21.15.55:0/2451112315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363031b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.847+0000 7f5371c75640 1 -- 172.21.15.55:0/2451112315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 msgr2=0x563630321e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:50.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.847+0000 7f5371c75640 1 --2- 172.21.15.55:0/2451112315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x563630321e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.847+0000 7f5371c75640 1 -- 172.21.15.55:0/2451112315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 msgr2=0x563630319e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:50.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.847+0000 7f5371c75640 1 --2- 172.21.15.55:0/2451112315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630319e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.847+0000 7f5371c75640 1 -- 172.21.15.55:0/2451112315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563630242c30 con 0x56363031abd0 2024-04-26T21:55:50.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.848+0000 7f5362ffd640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f53580a3cf0 0x7f53580a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.848+0000 7f5371c75640 1 --2- 172.21.15.55:0/2451112315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363031b050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5368009e10 tx=0x7f536800e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b5681744b737f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:50.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.849+0000 7f5362ffd640 1 -- 172.21.15.55:0/173869646 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f535c0cfb50 con 0x7f536c103990 2024-04-26T21:55:50.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.849+0000 7f5372476640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f53580a3cf0 0x7f53580a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.849+0000 7f5357fff640 1 -- 172.21.15.55:0/2451112315 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368017070 con 0x56363031abd0 2024-04-26T21:55:50.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.849+0000 7f5357fff640 1 -- 172.21.15.55:0/2451112315 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368013920 con 0x56363031abd0 2024-04-26T21:55:50.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.850+0000 7f5357fff640 1 -- 172.21.15.55:0/2451112315 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368012a30 con 0x56363031abd0 2024-04-26T21:55:50.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.850+0000 7f5372476640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f53580a3cf0 0x7f53580a61b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5364041040 tx=0x7f5364040f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:50.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.851+0000 7f5376bf0740 1 -- 172.21.15.55:0/2451112315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 msgr2=0x56363031b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.851+0000 7f5376bf0740 1 --2- 172.21.15.55:0/2451112315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363031b050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5368009e10 tx=0x7f536800e5a0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.851+0000 7f5376bf0740 1 -- 172.21.15.55:0/2451112315 shutdown_connections 2024-04-26T21:55:50.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.851+0000 7f5376bf0740 1 --2- 172.21.15.55:0/2451112315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x563630321e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.851+0000 7f5376bf0740 1 --2- 172.21.15.55:0/2451112315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363031b050 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.851+0000 7f5376bf0740 1 --2- 172.21.15.55:0/2451112315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630319e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.851+0000 7f5376bf0740 1 -- 172.21.15.55:0/2451112315 >> 172.21.15.55:0/2451112315 conn(0x56363025a910 msgr2=0x5636303152b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:50.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.852+0000 7f5376bf0740 1 -- 172.21.15.55:0/2451112315 shutdown_connections 2024-04-26T21:55:50.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.852+0000 7f5376bf0740 1 -- 172.21.15.55:0/2451112315 wait complete. 2024-04-26T21:55:50.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.852+0000 7f5376bf0740 1 -- start start 2024-04-26T21:55:50.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.852+0000 7f5376bf0740 1 Processor -- start 2024-04-26T21:55:50.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630323d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x5636303242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x56363032b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5376bf0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5636303225f0 con 0x56363031abd0 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5376bf0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563630204dc0 con 0x563630319a00 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5376bf0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563630322770 con 0x56363031b590 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5371474640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x5636303242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5376bf0740 10 client.0 _ll_register_callbacks cb 0x563630220600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5371474640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x5636303242c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60554/0 (socket says 172.21.15.55:60554) 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.853+0000 7f5371474640 1 -- 172.21.15.55:0/4248059473 learned_addr learned my addr 172.21.15.55:0/4248059473 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.854+0000 7f5372476640 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x56363032b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.854+0000 7f5371c75640 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630323d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.854+0000 7f5371474640 1 -- 172.21.15.55:0/4248059473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 msgr2=0x56363032b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:50.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.854+0000 7f5371474640 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x56363032b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.854+0000 7f5371474640 1 -- 172.21.15.55:0/4248059473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 msgr2=0x563630323d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:50.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.854+0000 7f5371474640 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630323d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.854+0000 7f5371474640 1 -- 172.21.15.55:0/4248059473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362ff8f3f0 con 0x56363031abd0 2024-04-26T21:55:50.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.855+0000 7f53557fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:50.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.855+0000 7f5371474640 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x5636303242c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f535c00bd70 tx=0x7f535c09b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a467027e3c2a93b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.855+0000 7f53557fa640 1 -- 172.21.15.55:0/4248059473 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c09f070 con 0x56363031abd0 2024-04-26T21:55:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.855+0000 7f53557fa640 1 -- 172.21.15.55:0/4248059473 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f535c0cd710 con 0x56363031abd0 2024-04-26T21:55:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.855+0000 7f5376bf0740 1 -- 172.21.15.55:0/4248059473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5368013920 con 0x56363031abd0 2024-04-26T21:55:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.856+0000 7f53557fa640 1 -- 172.21.15.55:0/4248059473 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f535c0a4a10 con 0x56363031abd0 2024-04-26T21:55:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.856+0000 7f5376bf0740 10 client.15849 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:50.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.856+0000 7f5376bf0740 1 -- 172.21.15.55:0/4248059473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5636300597d0 con 0x56363031abd0 2024-04-26T21:55:50.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.856+0000 7f53557fa640 1 -- 172.21.15.55:0/4248059473 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368013920 con 0x56363031abd0 2024-04-26T21:55:50.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.857+0000 7f53557fa640 1 -- 172.21.15.55:0/4248059473 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f535c09b1f0 con 0x56363031abd0 2024-04-26T21:55:50.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.857+0000 7f5376bf0740 10 client.15849 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:50.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.857+0000 7f5376bf0740 10 client.15849 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:55:50.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.857+0000 7f5376bf0740 20 client.15849 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:50.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.857+0000 7f5376bf0740 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5636303300b0 0x5636303324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.857+0000 7f5376bf0740 4 client.15849 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:50.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.857+0000 7f5376bf0740 1 -- 172.21.15.55:0/4248059473 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363028b990 con 0x5636303300b0 2024-04-26T21:55:50.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.858+0000 7f5371c75640 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5636303300b0 0x5636303324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.860+0000 7f5371c75640 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5636303300b0 0x5636303324f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a48b80c5733aff90 server_cookie=4089e44337ab8d2a in_seq=0 out_seq=0 2024-04-26T21:55:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.860+0000 7f53557fa640 10 client.15849 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:55:50.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f53557fa640 1 -- 172.21.15.55:0/4248059473 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56363028b990 con 0x5636303300b0 2024-04-26T21:55:50.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f53557fa640 10 client.15849 handle_command_reply: tid=0 2024-04-26T21:55:50.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 client.15849 shutdown 2024-04-26T21:55:50.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 -- 172.21.15.55:0/4248059473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 msgr2=0x5636303242c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x5636303242c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f535c00bd70 tx=0x7f535c09b7c0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 -- 172.21.15.55:0/4248059473 shutdown_connections 2024-04-26T21:55:50.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5636303300b0 0x5636303324f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f53557fa640 0 client.15849 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:55:50.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x56363032b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x5636303242c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 --2- 172.21.15.55:0/4248059473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x563630323d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 -- 172.21.15.55:0/4248059473 >> 172.21.15.55:0/4248059473 conn(0x56363025a910 msgr2=0x563630314450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:50.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.862+0000 7f5376bf0740 1 -- 172.21.15.55:0/4248059473 shutdown_connections 2024-04-26T21:55:50.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.863+0000 7f5376bf0740 1 -- 172.21.15.55:0/4248059473 wait complete. 2024-04-26T21:55:50.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.863+0000 7f5376bf0740 20 client.15849 trim_cache size 0 max 16384 2024-04-26T21:55:50.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.878+0000 7f5376bf0740 1 Processor -- start 2024-04-26T21:55:50.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.878+0000 7f5376bf0740 1 -- start start 2024-04-26T21:55:50.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.878+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563630319a00 0x563630321d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.878+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363032b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.878+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363031b590 0x563630323070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5376bf0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5350010d50 con 0x56363031abd0 2024-04-26T21:55:50.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5376bf0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f535000e620 con 0x56363031b590 2024-04-26T21:55:50.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5376bf0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f535000c470 con 0x563630319a00 2024-04-26T21:55:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5371474640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363032b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5371474640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363032b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60570/0 (socket says 172.21.15.55:60570) 2024-04-26T21:55:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5371474640 1 -- 172.21.15.55:0/1669468326 learned_addr learned my addr 172.21.15.55:0/1669468326 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5372476640 1 --2- 172.21.15.55:0/1669468326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363031b590 0x563630323070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5371474640 1 -- 172.21.15.55:0/1669468326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563630319a00 msgr2=0x563630321d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5371474640 1 --2- 172.21.15.55:0/1669468326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563630319a00 0x563630321d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5371474640 1 -- 172.21.15.55:0/1669468326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363031b590 msgr2=0x563630323070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5371474640 1 --2- 172.21.15.55:0/1669468326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363031b590 0x563630323070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.879+0000 7f5371474640 1 -- 172.21.15.55:0/1669468326 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363030f820 con 0x56363031abd0 2024-04-26T21:55:50.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.880+0000 7f5371474640 1 --2- 172.21.15.55:0/1669468326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363032b220 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f535c0cd650 tx=0x7f535c0b0ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff4916b031c7e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:50.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.880+0000 7f53577fe640 1 -- 172.21.15.55:0/1669468326 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c09f050 con 0x56363031abd0 2024-04-26T21:55:50.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.880+0000 7f53577fe640 1 -- 172.21.15.55:0/1669468326 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c0cf920 con 0x56363031abd0 2024-04-26T21:55:50.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.880+0000 7f53577fe640 1 -- 172.21.15.55:0/1669468326 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c0b0d10 con 0x56363031abd0 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.881+0000 7f5376bf0740 1 -- 172.21.15.55:0/1669468326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 msgr2=0x56363032b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.882+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1669468326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363032b220 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f535c0cd650 tx=0x7f535c0b0ca0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.882+0000 7f5376bf0740 1 -- 172.21.15.55:0/1669468326 shutdown_connections 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.882+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1669468326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363031b590 0x563630323070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.882+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1669468326 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x56363032b220 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.882+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1669468326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563630319a00 0x563630321d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.882+0000 7f5376bf0740 1 -- 172.21.15.55:0/1669468326 >> 172.21.15.55:0/1669468326 conn(0x56363025a910 msgr2=0x56363031e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.882+0000 7f5376bf0740 1 -- 172.21.15.55:0/1669468326 shutdown_connections 2024-04-26T21:55:50.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.882+0000 7f5376bf0740 1 -- 172.21.15.55:0/1669468326 wait complete. 2024-04-26T21:55:50.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.883+0000 7f5376bf0740 1 -- start start 2024-04-26T21:55:50.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.884+0000 7f5376bf0740 1 Processor -- start 2024-04-26T21:55:50.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.884+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x56363032afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.885+0000 7f5376bf0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x563630327a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.885+0000 7f5371474640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x563630327a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.885+0000 7f5371474640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x563630327a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60582/0 (socket says 172.21.15.55:60582) 2024-04-26T21:55:50.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.885+0000 7f5371474640 1 -- 172.21.15.55:0/1729968903 learned_addr learned my addr 172.21.15.55:0/1729968903 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:50.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.885+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x563630327f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.885+0000 7f5371c75640 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x56363032afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.885+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f535000cae0 con 0x56363031abd0 2024-04-26T21:55:50.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.886+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f535000df50 con 0x563630319a00 2024-04-26T21:55:50.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.886+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f535000d1b0 con 0x56363031b590 2024-04-26T21:55:50.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.886+0000 7f5376bf0740 10 client.0 _ll_register_callbacks cb 0x7f535c010f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:50.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.887+0000 7f5371c75640 1 -- 172.21.15.55:0/1729968903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 msgr2=0x563630327f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:50.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.887+0000 7f5371c75640 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x563630327f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.887+0000 7f5371c75640 1 -- 172.21.15.55:0/1729968903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 msgr2=0x563630327a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:50.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.887+0000 7f5371c75640 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x563630327a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.887+0000 7f5371c75640 1 -- 172.21.15.55:0/1729968903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563630394660 con 0x563630319a00 2024-04-26T21:55:50.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.887+0000 7f5371c75640 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x56363032afb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5368009de0 tx=0x7f5368006d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b688860ca011abf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:50.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.887+0000 7f53557fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:55:50.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.888+0000 7f53557fa640 1 -- 172.21.15.55:0/1729968903 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368017040 con 0x563630319a00 2024-04-26T21:55:50.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.888+0000 7f53557fa640 1 -- 172.21.15.55:0/1729968903 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5368005970 con 0x563630319a00 2024-04-26T21:55:50.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.888+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f535c0cf920 con 0x563630319a00 2024-04-26T21:55:50.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.889+0000 7f53557fa640 1 -- 172.21.15.55:0/1729968903 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f536802b050 con 0x563630319a00 2024-04-26T21:55:50.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.889+0000 7f5376bf0740 10 client.25391 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:50.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.889+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5636303957d0 con 0x563630319a00 2024-04-26T21:55:50.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.890+0000 7f53557fa640 1 -- 172.21.15.55:0/1729968903 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c0cf920 con 0x563630319a00 2024-04-26T21:55:50.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.890+0000 7f53557fa640 1 -- 172.21.15.55:0/1729968903 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f5368003690 con 0x563630319a00 2024-04-26T21:55:50.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.890+0000 7f5376bf0740 10 client.25391 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:50.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.890+0000 7f5376bf0740 10 client.25391 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:55:50.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.890+0000 7f5376bf0740 20 client.25391 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.890+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5636303300b0 0x563630397f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.890+0000 7f5376bf0740 4 client.25391 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:55:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.890+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563630285800 con 0x5636303300b0 2024-04-26T21:55:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.891+0000 7f5371474640 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5636303300b0 0x563630397f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.893+0000 7f5371474640 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5636303300b0 0x563630397f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7defa59a0905b796 server_cookie=d857e1f5820b5fde in_seq=0 out_seq=0 2024-04-26T21:55:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.893+0000 7f53557fa640 10 client.25391 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:55:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.893+0000 7f53557fa640 1 -- 172.21.15.55:0/1729968903 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563630285800 con 0x5636303300b0 2024-04-26T21:55:50.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.893+0000 7f53557fa640 10 client.25391 handle_command_reply: tid=0 2024-04-26T21:55:50.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.893+0000 7f5376bf0740 1 client.25391 shutdown 2024-04-26T21:55:50.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.893+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 msgr2=0x56363032afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.893+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x56363032afb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5368009de0 tx=0x7f5368006d00 comp rx=0 tx=0).stop 2024-04-26T21:55:50.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 shutdown_connections 2024-04-26T21:55:50.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5636303300b0 0x563630397f00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f53557fa640 0 client.25391 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:55:50.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56363031b590 0x563630327f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56363031abd0 0x563630327a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 1 --2- 172.21.15.55:0/1729968903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563630319a00 0x56363032afb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 >> 172.21.15.55:0/1729968903 conn(0x56363025a910 msgr2=0x563630323070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:50.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 shutdown_connections 2024-04-26T21:55:50.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 1 -- 172.21.15.55:0/1729968903 wait complete. 2024-04-26T21:55:50.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.894+0000 7f5376bf0740 20 client.25391 trim_cache size 0 max 16384 2024-04-26T21:55:50.896 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 -- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f53580a3cf0 msgr2=0x7f53580a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f53580a3cf0 0x7f53580a61b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5364041040 tx=0x7f5364040f30 comp rx=0 tx=0).stop 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 -- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c103990 msgr2=0x7f536c196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c103990 0x7f536c196d70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f535c009fd0 tx=0x7f535c00b870 comp rx=0 tx=0).stop 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 -- 172.21.15.55:0/173869646 shutdown_connections 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f53580a3cf0 0x7f53580a61b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f536c106340 0x7f536c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f536c103990 0x7f536c196d70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 --2- 172.21.15.55:0/173869646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f536c101020 0x7f536c196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 -- 172.21.15.55:0/173869646 >> 172.21.15.55:0/173869646 conn(0x7f536c0fcc80 msgr2=0x7f536c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 -- 172.21.15.55:0/173869646 shutdown_connections 2024-04-26T21:55:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:50.895+0000 7f5373ed9640 1 -- 172.21.15.55:0/173869646 wait complete. 2024-04-26T21:55:50.919 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:55:51.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.259+0000 7ff953ebd640 1 -- 172.21.15.55:0/3612921232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff94c10ec70 msgr2=0x7ff94c10f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.259+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3612921232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff94c10ec70 0x7ff94c10f120 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff944009ca0 tx=0x7ff94402a8f0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.260+0000 7ff953ebd640 1 -- 172.21.15.55:0/3612921232 shutdown_connections 2024-04-26T21:55:51.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.260+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3612921232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff94c10ec70 0x7ff94c10f120 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.260+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3612921232 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94c10c2a0 0x7ff94c10e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.260+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3612921232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff94c109930 0x7ff94c10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.260+0000 7ff953ebd640 1 -- 172.21.15.55:0/3612921232 >> 172.21.15.55:0/3612921232 conn(0x7ff94c06f7c0 msgr2=0x7ff94c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.261+0000 7ff953ebd640 1 -- 172.21.15.55:0/3612921232 shutdown_connections 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.263+0000 7ff953ebd640 1 -- 172.21.15.55:0/3612921232 wait complete. 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff953ebd640 1 Processor -- start 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff953ebd640 1 -- start start 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff953ebd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94c109930 0x7ff94c115fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff953ebd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff94c10c2a0 0x7ff94c116520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff953ebd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff94c10ec70 0x7ff94c1b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff953ebd640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c11d3f0 con 0x7ff94c10c2a0 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff953ebd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c11d270 con 0x7ff94c109930 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff953ebd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c11d570 con 0x7ff94c10ec70 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff95245a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff94c10ec70 0x7ff94c1b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff95245a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff94c10ec70 0x7ff94c1b7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33232/0 (socket says 172.21.15.55:33232) 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.264+0000 7ff95245a640 1 -- 172.21.15.55:0/3180318568 learned_addr learned my addr 172.21.15.55:0/3180318568 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.265+0000 7ff95245a640 1 -- 172.21.15.55:0/3180318568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94c109930 msgr2=0x7ff94c115fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.265+0000 7ff95245a640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94c109930 0x7ff94c115fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.265+0000 7ff95245a640 1 -- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff94c10c2a0 msgr2=0x7ff94c116520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.265+0000 7ff95245a640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff94c10c2a0 0x7ff94c116520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.265+0000 7ff95245a640 1 -- 172.21.15.55:0/3180318568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c116c00 con 0x7ff94c10ec70 2024-04-26T21:55:51.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.265+0000 7ff95245a640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff94c10ec70 0x7ff94c1b7b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff944006010 tx=0x7ff944030ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.273+0000 7ff942ffd640 1 -- 172.21.15.55:0/3180318568 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94403b810 con 0x7ff94c10ec70 2024-04-26T21:55:51.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.273+0000 7ff942ffd640 1 -- 172.21.15.55:0/3180318568 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff94402d070 con 0x7ff94c10ec70 2024-04-26T21:55:51.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.273+0000 7ff942ffd640 1 -- 172.21.15.55:0/3180318568 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94403ad90 con 0x7ff94c10ec70 2024-04-26T21:55:51.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.273+0000 7ff953ebd640 1 -- 172.21.15.55:0/3180318568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c1b8100 con 0x7ff94c10ec70 2024-04-26T21:55:51.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.274+0000 7ff953ebd640 1 -- 172.21.15.55:0/3180318568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c1b8680 con 0x7ff94c10ec70 2024-04-26T21:55:51.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.282+0000 7ff942ffd640 1 -- 172.21.15.55:0/3180318568 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff94403a490 con 0x7ff94c10ec70 2024-04-26T21:55:51.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.283+0000 7ff956bd0740 1 Processor -- start 2024-04-26T21:55:51.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.284+0000 7ff956bd0740 1 -- start start 2024-04-26T21:55:51.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.284+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b0ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.284+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b0be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.284+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b12c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.285+0000 7ff956bd0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556b40a6ed80 con 0x556b40b0b9f0 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.285+0000 7ff956bd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556b40a28a20 con 0x556b40b0a820 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.285+0000 7ff956bd0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b40a6ad40 con 0x556b40b0c3b0 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.285+0000 7ff951458640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b0ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.285+0000 7ff951458640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b0ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49118/0 (socket says 172.21.15.55:49118) 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.285+0000 7ff951458640 1 -- 172.21.15.55:0/12102666 learned_addr learned my addr 172.21.15.55:0/12102666 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:51.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.285+0000 7ff951c59640 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b0be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.286+0000 7ff951458640 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b12c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.288+0000 7ff951458640 1 -- 172.21.15.55:0/12102666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 msgr2=0x556b40b12c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.288+0000 7ff951458640 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b12c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.288+0000 7ff951458640 1 -- 172.21.15.55:0/12102666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 msgr2=0x556b40b0be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.288+0000 7ff951458640 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b0be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.288+0000 7ff951458640 1 -- 172.21.15.55:0/12102666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b40a33c30 con 0x556b40b0a820 2024-04-26T21:55:51.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.289+0000 7ff951458640 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b0ac40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff948001e90 tx=0x7ff9480011a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a49cf4f7e6c4a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.290+0000 7ff942ffd640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff9380a3e40 0x7ff9380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.290+0000 7ff942ffd640 1 -- 172.21.15.55:0/3180318568 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff9440f16a0 con 0x7ff94c10ec70 2024-04-26T21:55:51.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.297+0000 7ff951c59640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff9380a3e40 0x7ff9380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.300+0000 7ff951c59640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff9380a3e40 0x7ff9380a6300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff93c006fd0 tx=0x7ff93c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.302+0000 7ff937fff640 1 -- 172.21.15.55:0/12102666 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94800ed00 con 0x556b40b0a820 2024-04-26T21:55:51.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.302+0000 7ff937fff640 1 -- 172.21.15.55:0/12102666 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff948008e30 con 0x556b40b0a820 2024-04-26T21:55:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.305+0000 7ff956bd0740 1 -- 172.21.15.55:0/12102666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 msgr2=0x556b40b0ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.305+0000 7ff956bd0740 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b0ac40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff948001e90 tx=0x7ff9480011a0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.307+0000 7ff956bd0740 1 -- 172.21.15.55:0/12102666 shutdown_connections 2024-04-26T21:55:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.307+0000 7ff956bd0740 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b12c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.307+0000 7ff956bd0740 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b0be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.307+0000 7ff956bd0740 1 --2- 172.21.15.55:0/12102666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b0ac40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.307+0000 7ff956bd0740 1 -- 172.21.15.55:0/12102666 >> 172.21.15.55:0/12102666 conn(0x556b40a4b910 msgr2=0x556b40b06120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.307+0000 7ff956bd0740 1 -- 172.21.15.55:0/12102666 shutdown_connections 2024-04-26T21:55:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.307+0000 7ff956bd0740 1 -- 172.21.15.55:0/12102666 wait complete. 2024-04-26T21:55:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.308+0000 7ff956bd0740 1 -- start start 2024-04-26T21:55:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.309+0000 7ff956bd0740 1 Processor -- start 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.309+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0a820 0x556b40b14b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.309+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0b9f0 0x556b40b15050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.309+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b1c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.309+0000 7ff956bd0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556b40b13410 con 0x556b40b0a820 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.311+0000 7ff956bd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556b409f5dc0 con 0x556b40b0b9f0 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.311+0000 7ff956bd0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b40b13590 con 0x556b40b0c3b0 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.311+0000 7ff956bd0740 10 client.0 _ll_register_callbacks cb 0x556b40a11600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.312+0000 7ff951458640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0b9f0 0x556b40b15050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.312+0000 7ff951458640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0b9f0 0x556b40b15050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49120/0 (socket says 172.21.15.55:49120) 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.312+0000 7ff951458640 1 -- 172.21.15.55:0/4004214581 learned_addr learned my addr 172.21.15.55:0/4004214581 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:51.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.313+0000 7ff95245a640 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b1c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.314+0000 7ff951c59640 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0a820 0x556b40b14b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.314+0000 7ff951458640 1 -- 172.21.15.55:0/4004214581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 msgr2=0x556b40b1c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.314+0000 7ff951458640 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b1c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.314+0000 7ff951458640 1 -- 172.21.15.55:0/4004214581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0a820 msgr2=0x556b40b14b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.314+0000 7ff951458640 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0a820 0x556b40b14b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.314+0000 7ff951458640 1 -- 172.21.15.55:0/4004214581 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b407803f0 con 0x556b40b0b9f0 2024-04-26T21:55:51.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.315+0000 7ff951458640 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0b9f0 0x556b40b15050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff948009000 tx=0x7ff94800ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9759ecf64443098c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.323+0000 7ff9357fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.323+0000 7ff9357fa640 1 -- 172.21.15.55:0/4004214581 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948000ed0 con 0x556b40b0b9f0 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.323+0000 7ff956bd0740 1 -- 172.21.15.55:0/4004214581 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff948008e30 con 0x556b40b0b9f0 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.323+0000 7ff9357fa640 1 -- 172.21.15.55:0/4004214581 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff948007240 con 0x556b40b0b9f0 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.324+0000 7ff9357fa640 1 -- 172.21.15.55:0/4004214581 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff94802a050 con 0x556b40b0b9f0 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.327+0000 7ff956bd0740 10 client.25397 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.327+0000 7ff956bd0740 1 -- 172.21.15.55:0/4004214581 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556b4084a7d0 con 0x556b40b0b9f0 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.328+0000 7ff9357fa640 1 -- 172.21.15.55:0/4004214581 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff948008e30 con 0x556b40b0b9f0 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.328+0000 7ff9357fa640 1 -- 172.21.15.55:0/4004214581 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff948004af0 con 0x556b40b0b9f0 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.328+0000 7ff956bd0740 10 client.25397 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.328+0000 7ff956bd0740 10 client.25397 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.328+0000 7ff956bd0740 20 client.25397 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.328+0000 7ff956bd0740 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556b40b20c70 0x556b40b23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.328+0000 7ff956bd0740 4 client.25397 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:51.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.328+0000 7ff956bd0740 1 -- 172.21.15.55:0/4004214581 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b40a7c990 con 0x556b40b20c70 2024-04-26T21:55:51.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.329+0000 7ff951c59640 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556b40b20c70 0x556b40b23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.336+0000 7ff9357fa640 10 client.25397 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:55:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.337+0000 7ff951c59640 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556b40b20c70 0x556b40b23090 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c26cbd85d870433 server_cookie=bb30db9a9c829902 in_seq=0 out_seq=0 2024-04-26T21:55:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.339+0000 7ff9357fa640 1 -- 172.21.15.55:0/4004214581 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b40a7c990 con 0x556b40b20c70 2024-04-26T21:55:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.340+0000 7ff9357fa640 10 client.25397 handle_command_reply: tid=0 2024-04-26T21:55:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.340+0000 7ff956bd0740 1 client.25397 shutdown 2024-04-26T21:55:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.344+0000 7ff956bd0740 1 -- 172.21.15.55:0/4004214581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0b9f0 msgr2=0x556b40b15050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.344+0000 7ff956bd0740 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0b9f0 0x556b40b15050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff948009000 tx=0x7ff94800ef30 comp rx=0 tx=0).stop 2024-04-26T21:55:51.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.347+0000 7ff956bd0740 1 -- 172.21.15.55:0/4004214581 shutdown_connections 2024-04-26T21:55:51.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.347+0000 7ff956bd0740 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556b40b20c70 0x556b40b23090 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.347+0000 7ff956bd0740 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b1c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.347+0000 7ff956bd0740 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0b9f0 0x556b40b15050 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.347+0000 7ff956bd0740 1 --2- 172.21.15.55:0/4004214581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0a820 0x556b40b14b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.347+0000 7ff956bd0740 1 -- 172.21.15.55:0/4004214581 >> 172.21.15.55:0/4004214581 conn(0x556b40a4b910 msgr2=0x556b40b05280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.347+0000 7ff9357fa640 0 client.25397 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:55:51.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.356+0000 7ff956bd0740 1 -- 172.21.15.55:0/4004214581 shutdown_connections 2024-04-26T21:55:51.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.356+0000 7ff956bd0740 1 -- 172.21.15.55:0/4004214581 wait complete. 2024-04-26T21:55:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.358+0000 7ff956bd0740 20 client.25397 trim_cache size 0 max 16384 2024-04-26T21:55:51.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.413+0000 7ff956bd0740 1 Processor -- start 2024-04-26T21:55:51.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.416+0000 7ff956bd0740 1 -- start start 2024-04-26T21:55:51.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.417+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b22c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.417+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b1c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.417+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b13e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.417+0000 7ff956bd0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1c3e70 con 0x556b40b0b9f0 2024-04-26T21:55:51.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.417+0000 7ff956bd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1c1740 con 0x556b40b0a820 2024-04-26T21:55:51.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.417+0000 7ff956bd0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1bf590 con 0x556b40b0c3b0 2024-04-26T21:55:51.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.417+0000 7ff95245a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b13e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.418+0000 7ff95245a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b13e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33262/0 (socket says 172.21.15.55:33262) 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.418+0000 7ff95245a640 1 -- 172.21.15.55:0/3370931705 learned_addr learned my addr 172.21.15.55:0/3370931705 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.418+0000 7ff95245a640 1 -- 172.21.15.55:0/3370931705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 msgr2=0x556b40b22c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.418+0000 7ff95245a640 1 --2- 172.21.15.55:0/3370931705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b22c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.418+0000 7ff95245a640 1 -- 172.21.15.55:0/3370931705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 msgr2=0x556b40b1c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.418+0000 7ff95245a640 1 --2- 172.21.15.55:0/3370931705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b1c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.419+0000 7ff95245a640 1 -- 172.21.15.55:0/3370931705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b40b01ad0 con 0x556b40b0c3b0 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.419+0000 7ff95245a640 1 --2- 172.21.15.55:0/3370931705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b13e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff944047040 tx=0x7ff94403a830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8687d4816503c5d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.421+0000 7ff9377fe640 1 -- 172.21.15.55:0/3370931705 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944031dc0 con 0x556b40b0c3b0 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.421+0000 7ff9377fe640 1 -- 172.21.15.55:0/3370931705 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff944036040 con 0x556b40b0c3b0 2024-04-26T21:55:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.421+0000 7ff9377fe640 1 -- 172.21.15.55:0/3370931705 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440ef200 con 0x556b40b0c3b0 2024-04-26T21:55:51.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.423+0000 7ff956bd0740 1 -- 172.21.15.55:0/3370931705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 msgr2=0x556b40b13e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.423+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3370931705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b13e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff944047040 tx=0x7ff94403a830 comp rx=0 tx=0).stop 2024-04-26T21:55:51.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.429+0000 7ff956bd0740 1 -- 172.21.15.55:0/3370931705 shutdown_connections 2024-04-26T21:55:51.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.429+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3370931705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0c3b0 0x556b40b13e00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.429+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3370931705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b1c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.429+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3370931705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0a820 0x556b40b22c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.429+0000 7ff956bd0740 1 -- 172.21.15.55:0/3370931705 >> 172.21.15.55:0/3370931705 conn(0x556b40a4b910 msgr2=0x556b40b0f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.432+0000 7ff956bd0740 1 -- 172.21.15.55:0/3370931705 shutdown_connections 2024-04-26T21:55:51.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.436+0000 7ff956bd0740 1 -- 172.21.15.55:0/3370931705 wait complete. 2024-04-26T21:55:51.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.436+0000 7ff956bd0740 1 -- start start 2024-04-26T21:55:51.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.436+0000 7ff956bd0740 1 Processor -- start 2024-04-26T21:55:51.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.437+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0a820 0x556b40b1bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.437+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b18990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.437+0000 7ff956bd0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0c3b0 0x556b40b18ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.437+0000 7ff956bd0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1bfc00 con 0x556b40b0b9f0 2024-04-26T21:55:51.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.437+0000 7ff956bd0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1c1070 con 0x556b40b0c3b0 2024-04-26T21:55:51.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.437+0000 7ff956bd0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1c02d0 con 0x556b40b0a820 2024-04-26T21:55:51.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.437+0000 7ff956bd0740 10 client.0 _ll_register_callbacks cb 0x7ff948001860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.441+0000 7ff951c59640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0a820 0x556b40b1bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.441+0000 7ff95245a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0c3b0 0x556b40b18ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.443+0000 7ff951458640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b18990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.443+0000 7ff951458640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b18990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60628/0 (socket says 172.21.15.55:60628) 2024-04-26T21:55:51.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.444+0000 7ff951458640 1 -- 172.21.15.55:0/3837900603 learned_addr learned my addr 172.21.15.55:0/3837900603 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:51.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.446+0000 7ff951458640 1 -- 172.21.15.55:0/3837900603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0a820 msgr2=0x556b40b1bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.446+0000 7ff951458640 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0a820 0x556b40b1bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.446+0000 7ff951458640 1 -- 172.21.15.55:0/3837900603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0c3b0 msgr2=0x556b40b18ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.446+0000 7ff951458640 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0c3b0 0x556b40b18ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.446+0000 7ff951458640 1 -- 172.21.15.55:0/3837900603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b40b851c0 con 0x556b40b0b9f0 2024-04-26T21:55:51.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.448+0000 7ff951458640 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b18990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff948001070 tx=0x7ff948019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ef27f41a9526ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.448+0000 7ff9357fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:51.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.449+0000 7ff9357fa640 1 -- 172.21.15.55:0/3837900603 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948001910 con 0x556b40b0b9f0 2024-04-26T21:55:51.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.450+0000 7ff956bd0740 1 -- 172.21.15.55:0/3837900603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff944036040 con 0x556b40b0b9f0 2024-04-26T21:55:51.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.450+0000 7ff9357fa640 1 -- 172.21.15.55:0/3837900603 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff948007450 con 0x556b40b0b9f0 2024-04-26T21:55:51.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.451+0000 7ff9357fa640 1 -- 172.21.15.55:0/3837900603 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff948009d40 con 0x556b40b0b9f0 2024-04-26T21:55:51.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.451+0000 7ff956bd0740 10 client.15867 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:51.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.451+0000 7ff956bd0740 1 -- 172.21.15.55:0/3837900603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x556b40b863c0 con 0x556b40b0b9f0 2024-04-26T21:55:51.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.454+0000 7ff9357fa640 1 -- 172.21.15.55:0/3837900603 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff944036040 con 0x556b40b0b9f0 2024-04-26T21:55:51.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.454+0000 7ff9357fa640 1 -- 172.21.15.55:0/3837900603 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff948009740 con 0x556b40b0b9f0 2024-04-26T21:55:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.456+0000 7ff956bd0740 10 client.15867 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.456+0000 7ff956bd0740 10 client.15867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:55:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.456+0000 7ff956bd0740 20 client.15867 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.456+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556b40b86d10 0x556b40b89130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.456+0000 7ff956bd0740 4 client.15867 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:55:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.456+0000 7ff956bd0740 1 -- 172.21.15.55:0/3837900603 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b40a76800 con 0x556b40b86d10 2024-04-26T21:55:51.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.469+0000 7ff951c59640 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556b40b86d10 0x556b40b89130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.481+0000 7ff951c59640 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556b40b86d10 0x556b40b89130 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7305b19c2989e952 server_cookie=ec133f1916ddcc84 in_seq=0 out_seq=0 2024-04-26T21:55:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.481+0000 7ff9357fa640 10 client.15867 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:55:51.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.484+0000 7ff9357fa640 1 -- 172.21.15.55:0/3837900603 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b40a76800 con 0x556b40b86d10 2024-04-26T21:55:51.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.484+0000 7ff9357fa640 10 client.15867 handle_command_reply: tid=0 2024-04-26T21:55:51.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.485+0000 7ff956bd0740 1 client.15867 shutdown 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.486+0000 7ff956bd0740 1 -- 172.21.15.55:0/3837900603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 msgr2=0x556b40b18990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.486+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b18990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff948001070 tx=0x7ff948019640 comp rx=0 tx=0).stop 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.487+0000 7ff956bd0740 1 -- 172.21.15.55:0/3837900603 shutdown_connections 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.487+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x556b40b86d10 0x556b40b89130 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.487+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b40b0c3b0 0x556b40b18ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.487+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b40b0b9f0 0x556b40b18990 secure :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff948001070 tx=0x7ff948019640 comp rx=0 tx=0).stop 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.487+0000 7ff956bd0740 1 --2- 172.21.15.55:0/3837900603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b40b0a820 0x556b40b1bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.487+0000 7ff956bd0740 1 -- 172.21.15.55:0/3837900603 >> 172.21.15.55:0/3837900603 conn(0x556b40a4b910 msgr2=0x556b40b13e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.487+0000 7ff9357fa640 0 client.15867 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:55:51.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.492+0000 7ff956bd0740 1 -- 172.21.15.55:0/3837900603 shutdown_connections 2024-04-26T21:55:51.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.496+0000 7ff956bd0740 1 -- 172.21.15.55:0/3837900603 wait complete. 2024-04-26T21:55:51.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.500+0000 7ff956bd0740 20 client.15867 trim_cache size 0 max 16384 2024-04-26T21:55:51.504 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:55:51.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.506+0000 7ff953ebd640 1 -- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff9380a3e40 msgr2=0x7ff9380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.506+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff9380a3e40 0x7ff9380a6300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff93c006fd0 tx=0x7ff93c008040 comp rx=0 tx=0).stop 2024-04-26T21:55:51.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.506+0000 7ff953ebd640 1 -- 172.21.15.55:0/3180318568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff94c10ec70 msgr2=0x7ff94c1b7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.506+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff94c10ec70 0x7ff94c1b7b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff944006010 tx=0x7ff944030ec0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.508+0000 7ff953ebd640 1 -- 172.21.15.55:0/3180318568 shutdown_connections 2024-04-26T21:55:51.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.508+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff9380a3e40 0x7ff9380a6300 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.508+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff94c10ec70 0x7ff94c1b7b00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.508+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff94c10c2a0 0x7ff94c116520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.508+0000 7ff953ebd640 1 --2- 172.21.15.55:0/3180318568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94c109930 0x7ff94c115fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.508+0000 7ff953ebd640 1 -- 172.21.15.55:0/3180318568 >> 172.21.15.55:0/3180318568 conn(0x7ff94c06f7c0 msgr2=0x7ff94c10e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.512+0000 7ff953ebd640 1 -- 172.21.15.55:0/3180318568 shutdown_connections 2024-04-26T21:55:51.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.517+0000 7ff953ebd640 1 -- 172.21.15.55:0/3180318568 wait complete. 2024-04-26T21:55:51.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:51 smithi165 ceph-mon[33296]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 856 MiB/s wr, 961 op/s 2024-04-26T21:55:51.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:51 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1334427175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:51.579 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:51 smithi055 ceph-mon[29221]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 856 MiB/s wr, 961 op/s 2024-04-26T21:55:51.579 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:51 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1334427175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:51.581 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:55:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:51 smithi064 ceph-mon[32765]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 856 MiB/s wr, 961 op/s 2024-04-26T21:55:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:51 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1334427175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:55:51.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.853+0000 7f3121696640 1 -- 172.21.15.55:0/4009567170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 msgr2=0x7f311c113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.853+0000 7f3121696640 1 --2- 172.21.15.55:0/4009567170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 0x7f311c113160 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3114009ca0 tx=0x7f311402a980 comp rx=0 tx=0).stop 2024-04-26T21:55:51.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.865+0000 7f3121696640 1 -- 172.21.15.55:0/4009567170 shutdown_connections 2024-04-26T21:55:51.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.865+0000 7f3121696640 1 --2- 172.21.15.55:0/4009567170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 0x7f311c113160 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.865+0000 7f3121696640 1 --2- 172.21.15.55:0/4009567170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f311c073fa0 0x7f311c074400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.865+0000 7f3121696640 1 --2- 172.21.15.55:0/4009567170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c10a5e0 0x7f311c0739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.865+0000 7f3121696640 1 -- 172.21.15.55:0/4009567170 >> 172.21.15.55:0/4009567170 conn(0x7f311c06f840 msgr2=0x7f311c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.869+0000 7f3121696640 1 -- 172.21.15.55:0/4009567170 shutdown_connections 2024-04-26T21:55:51.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.870+0000 7f3121696640 1 -- 172.21.15.55:0/4009567170 wait complete. 2024-04-26T21:55:51.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.872+0000 7f3121696640 1 Processor -- start 2024-04-26T21:55:51.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.878+0000 7f3121696640 1 -- start start 2024-04-26T21:55:51.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.884+0000 7f3121696640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f311c073fa0 0x7f311c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.884+0000 7f3121696640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c10a5e0 0x7f311c1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.884+0000 7f3121696640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 0x7f311c1a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.884+0000 7f3121696640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f311c116c70 con 0x7f311c110cc0 2024-04-26T21:55:51.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.884+0000 7f3121696640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f311c116af0 con 0x7f311c073fa0 2024-04-26T21:55:51.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.884+0000 7f3121696640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f311c116df0 con 0x7f311c10a5e0 2024-04-26T21:55:51.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.888+0000 7f311bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f311c073fa0 0x7f311c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.888+0000 7f311bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f311c073fa0 0x7f311c1a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49136/0 (socket says 172.21.15.55:49136) 2024-04-26T21:55:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.889+0000 7f311bfff640 1 -- 172.21.15.55:0/533350156 learned_addr learned my addr 172.21.15.55:0/533350156 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:51.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.888+0000 7f311b7fe640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c10a5e0 0x7f311c1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.889+0000 7f3120e95640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 0x7f311c1a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.893+0000 7f3120e95640 1 -- 172.21.15.55:0/533350156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c10a5e0 msgr2=0x7f311c1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.893+0000 7f3120e95640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c10a5e0 0x7f311c1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.893+0000 7f3120e95640 1 -- 172.21.15.55:0/533350156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f311c073fa0 msgr2=0x7f311c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.893+0000 7f3120e95640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f311c073fa0 0x7f311c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.893+0000 7f3120e95640 1 -- 172.21.15.55:0/533350156 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311c1a8fe0 con 0x7f311c110cc0 2024-04-26T21:55:51.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.895+0000 7f3120e95640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 0x7f311c1a8970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3114009bf0 tx=0x7f3114003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.896+0000 7f31197fa640 1 -- 172.21.15.55:0/533350156 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114034030 con 0x7f311c110cc0 2024-04-26T21:55:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.896+0000 7f3121696640 1 -- 172.21.15.55:0/533350156 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311c1a9270 con 0x7f311c110cc0 2024-04-26T21:55:51.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.897+0000 7f3121696640 1 -- 172.21.15.55:0/533350156 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311c1a97e0 con 0x7f311c110cc0 2024-04-26T21:55:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.902+0000 7f312560f740 1 Processor -- start 2024-04-26T21:55:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.902+0000 7f312560f740 1 -- start start 2024-04-26T21:55:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.902+0000 7f312560f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d63de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.902+0000 7f312560f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.903+0000 7f312560f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d6be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.903+0000 7f312560f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0cc7d80 con 0x55e5a0d65580 2024-04-26T21:55:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.903+0000 7f312560f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0c81a20 con 0x55e5a0d639c0 2024-04-26T21:55:51.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.903+0000 7f312560f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0cc3d40 con 0x55e5a0d64be0 2024-04-26T21:55:51.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.903+0000 7f31197fa640 1 -- 172.21.15.55:0/533350156 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3114004040 con 0x7f311c110cc0 2024-04-26T21:55:51.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.903+0000 7f31197fa640 1 -- 172.21.15.55:0/533350156 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114039680 con 0x7f311c110cc0 2024-04-26T21:55:51.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.907+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d63de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.907+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d63de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49148/0 (socket says 172.21.15.55:49148) 2024-04-26T21:55:51.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.907+0000 7f311b7fe640 1 -- 172.21.15.55:0/3878996883 learned_addr learned my addr 172.21.15.55:0/3878996883 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.912+0000 7f31197fa640 1 -- 172.21.15.55:0/533350156 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3114037020 con 0x7f311c110cc0 2024-04-26T21:55:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.912+0000 7f311bfff640 1 --2- 172.21.15.55:0/3878996883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d65040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.912+0000 7f311b7fe640 1 -- 172.21.15.55:0/3878996883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 msgr2=0x55e5a0d65040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.912+0000 7f311b7fe640 1 --2- 172.21.15.55:0/3878996883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d65040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.912+0000 7f311b7fe640 1 -- 172.21.15.55:0/3878996883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 msgr2=0x55e5a0d6be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.912+0000 7f311b7fe640 1 --2- 172.21.15.55:0/3878996883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d6be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.913+0000 7f311b7fe640 1 -- 172.21.15.55:0/3878996883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a0c8cc30 con 0x55e5a0d639c0 2024-04-26T21:55:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.914+0000 7f31197fa640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31040a3e40 0x7f31040a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.915+0000 7f31197fa640 1 -- 172.21.15.55:0/533350156 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f31140f14e0 con 0x7f311c110cc0 2024-04-26T21:55:51.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.916+0000 7f311bfff640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31040a3e40 0x7f31040a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.917+0000 7f311bfff640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31040a3e40 0x7f31040a6300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3110009a50 tx=0x7f3110008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.929+0000 7f311b7fe640 1 --2- 172.21.15.55:0/3878996883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d63de0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f310c00a8d0 tx=0x7f310c00ad90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421583d49caf234f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.930+0000 7f3121696640 1 -- 172.21.15.55:0/3878996883 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c00f9c0 con 0x55e5a0d639c0 2024-04-26T21:55:51.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.930+0000 7f3121696640 1 -- 172.21.15.55:0/3878996883 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310c018070 con 0x55e5a0d639c0 2024-04-26T21:55:51.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.932+0000 7f312560f740 1 -- 172.21.15.55:0/3878996883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 msgr2=0x55e5a0d63de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.932+0000 7f312560f740 1 --2- 172.21.15.55:0/3878996883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d63de0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f310c00a8d0 tx=0x7f310c00ad90 comp rx=0 tx=0).stop 2024-04-26T21:55:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.937+0000 7f312560f740 1 -- 172.21.15.55:0/3878996883 shutdown_connections 2024-04-26T21:55:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.937+0000 7f312560f740 1 --2- 172.21.15.55:0/3878996883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d6be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.937+0000 7f312560f740 1 --2- 172.21.15.55:0/3878996883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d65040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.937+0000 7f312560f740 1 --2- 172.21.15.55:0/3878996883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d63de0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.938+0000 7f312560f740 1 -- 172.21.15.55:0/3878996883 >> 172.21.15.55:0/3878996883 conn(0x55e5a0ca4910 msgr2=0x55e5a0d5f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.940+0000 7f312560f740 1 -- 172.21.15.55:0/3878996883 shutdown_connections 2024-04-26T21:55:51.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.940+0000 7f312560f740 1 -- 172.21.15.55:0/3878996883 wait complete. 2024-04-26T21:55:51.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.941+0000 7f312560f740 1 -- start start 2024-04-26T21:55:51.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.941+0000 7f312560f740 1 Processor -- start 2024-04-26T21:55:51.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.942+0000 7f312560f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d6fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.942+0000 7f312560f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d70140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.942+0000 7f312560f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d6c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.942+0000 7f312560f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0d70a10 con 0x55e5a0d65580 2024-04-26T21:55:51.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.942+0000 7f312560f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0c4edc0 con 0x55e5a0d639c0 2024-04-26T21:55:51.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.942+0000 7f312560f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e5a0d70b90 con 0x55e5a0d64be0 2024-04-26T21:55:51.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.942+0000 7f312560f740 10 client.0 _ll_register_callbacks cb 0x55e5a0c6a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:51.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.944+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d70140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.944+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d70140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33302/0 (socket says 172.21.15.55:33302) 2024-04-26T21:55:51.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.944+0000 7f311b7fe640 1 -- 172.21.15.55:0/1181810336 learned_addr learned my addr 172.21.15.55:0/1181810336 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:51.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.945+0000 7f3120e95640 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d6c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.947+0000 7f311b7fe640 1 -- 172.21.15.55:0/1181810336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 msgr2=0x55e5a0d6fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.947+0000 7f311b7fe640 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d6fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.947+0000 7f311b7fe640 1 -- 172.21.15.55:0/1181810336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 msgr2=0x55e5a0d6c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.947+0000 7f311b7fe640 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d6c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.948+0000 7f311b7fe640 1 -- 172.21.15.55:0/1181810336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a09d93f0 con 0x55e5a0d64be0 2024-04-26T21:55:51.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.954+0000 7f311b7fe640 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d70140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f310c00fa20 tx=0x7f310c00fa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db88d8a47c892bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:51.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.954+0000 7f30fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:55:51.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.959+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1181810336 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c021070 con 0x55e5a0d64be0 2024-04-26T21:55:51.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.959+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1181810336 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f310c004270 con 0x55e5a0d64be0 2024-04-26T21:55:51.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.960+0000 7f312560f740 1 -- 172.21.15.55:0/1181810336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310c018070 con 0x55e5a0d64be0 2024-04-26T21:55:51.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.960+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1181810336 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f310c02b050 con 0x55e5a0d64be0 2024-04-26T21:55:51.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.966+0000 7f312560f740 10 client.25378 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:51.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.966+0000 7f312560f740 1 -- 172.21.15.55:0/1181810336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e5a0aa37d0 con 0x55e5a0d64be0 2024-04-26T21:55:51.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.968+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1181810336 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310c018070 con 0x55e5a0d64be0 2024-04-26T21:55:51.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.969+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1181810336 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f310c012540 con 0x55e5a0d64be0 2024-04-26T21:55:51.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.969+0000 7f312560f740 10 client.25378 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:51.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.970+0000 7f312560f740 10 client.25378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:51.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.970+0000 7f312560f740 20 client.25378 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:51.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.970+0000 7f312560f740 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e5a0d81fe0 0x55e5a0d84450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:51.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.970+0000 7f312560f740 4 client.25378 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:51.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.970+0000 7f312560f740 1 -- 172.21.15.55:0/1181810336 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5a0cd5990 con 0x55e5a0d81fe0 2024-04-26T21:55:51.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.970+0000 7f311bfff640 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e5a0d81fe0 0x55e5a0d84450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:51.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.972+0000 7f311bfff640 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e5a0d81fe0 0x55e5a0d84450 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bdd788f42b86158 server_cookie=66fd33c2d6af2f9b in_seq=0 out_seq=0 2024-04-26T21:55:51.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.972+0000 7f30fb7fe640 10 client.25378 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:51.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.976+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1181810336 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5a0cd5990 con 0x55e5a0d81fe0 2024-04-26T21:55:51.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.977+0000 7f30fb7fe640 10 client.25378 handle_command_reply: tid=0 2024-04-26T21:55:51.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.977+0000 7f312560f740 1 client.25378 shutdown 2024-04-26T21:55:51.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f312560f740 1 -- 172.21.15.55:0/1181810336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 msgr2=0x55e5a0d70140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:51.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f312560f740 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d70140 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f310c00fa20 tx=0x7f310c00fa50 comp rx=0 tx=0).stop 2024-04-26T21:55:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f312560f740 1 -- 172.21.15.55:0/1181810336 shutdown_connections 2024-04-26T21:55:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f312560f740 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e5a0d81fe0 0x55e5a0d84450 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f312560f740 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d6c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f312560f740 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d64be0 0x55e5a0d70140 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f310c00fa20 tx=0x7f310c00fa50 comp rx=0 tx=0).stop 2024-04-26T21:55:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f312560f740 1 --2- 172.21.15.55:0/1181810336 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d639c0 0x55e5a0d6fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f312560f740 1 -- 172.21.15.55:0/1181810336 >> 172.21.15.55:0/1181810336 conn(0x55e5a0ca4910 msgr2=0x55e5a0d5e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.978+0000 7f30fb7fe640 0 client.25378 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:51.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.980+0000 7f312560f740 1 -- 172.21.15.55:0/1181810336 shutdown_connections 2024-04-26T21:55:51.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.980+0000 7f312560f740 1 -- 172.21.15.55:0/1181810336 wait complete. 2024-04-26T21:55:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:51.983+0000 7f312560f740 20 client.25378 trim_cache size 0 max 16384 2024-04-26T21:55:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.043+0000 7f312560f740 1 Processor -- start 2024-04-26T21:55:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.044+0000 7f312560f740 1 -- start start 2024-04-26T21:55:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.044+0000 7f312560f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d639c0 0x55e5a0d716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.044+0000 7f312560f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d65580 0x55e5a0d624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.044+0000 7f312560f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d62a20 0x55e5a0d62e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.044+0000 7f312560f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3108010ef0 con 0x55e5a0d62a20 2024-04-26T21:55:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.044+0000 7f312560f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f310800e7c0 con 0x55e5a0d65580 2024-04-26T21:55:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.045+0000 7f312560f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f310800c5d0 con 0x55e5a0d639c0 2024-04-26T21:55:52.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.046+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d65580 0x55e5a0d624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:52.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.047+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d65580 0x55e5a0d624e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49154/0 (socket says 172.21.15.55:49154) 2024-04-26T21:55:52.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.047+0000 7f311b7fe640 1 -- 172.21.15.55:0/848346095 learned_addr learned my addr 172.21.15.55:0/848346095 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:52.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.047+0000 7f311bfff640 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d639c0 0x55e5a0d716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:52.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.048+0000 7f3120e95640 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d62a20 0x55e5a0d62e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.049+0000 7f311bfff640 1 -- 172.21.15.55:0/848346095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d65580 msgr2=0x55e5a0d624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.049+0000 7f311bfff640 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d65580 0x55e5a0d624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.049+0000 7f311bfff640 1 -- 172.21.15.55:0/848346095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d62a20 msgr2=0x55e5a0d62e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.049+0000 7f311bfff640 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d62a20 0x55e5a0d62e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.049+0000 7f311bfff640 1 -- 172.21.15.55:0/848346095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a0d5afa0 con 0x55e5a0d639c0 2024-04-26T21:55:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.050+0000 7f311bfff640 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d639c0 0x55e5a0d716f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55e5a0d84c60 tx=0x7f3110004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c1e341015858f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.051+0000 7f3101ffb640 1 -- 172.21.15.55:0/848346095 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311001c040 con 0x55e5a0d639c0 2024-04-26T21:55:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.051+0000 7f3101ffb640 1 -- 172.21.15.55:0/848346095 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3110026e50 con 0x55e5a0d639c0 2024-04-26T21:55:52.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.052+0000 7f3101ffb640 1 -- 172.21.15.55:0/848346095 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311000f040 con 0x55e5a0d639c0 2024-04-26T21:55:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.053+0000 7f312560f740 1 -- 172.21.15.55:0/848346095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d639c0 msgr2=0x55e5a0d716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.053+0000 7f312560f740 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d639c0 0x55e5a0d716f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55e5a0d84c60 tx=0x7f3110004750 comp rx=0 tx=0).stop 2024-04-26T21:55:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.054+0000 7f312560f740 1 -- 172.21.15.55:0/848346095 shutdown_connections 2024-04-26T21:55:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.054+0000 7f312560f740 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d62a20 0x55e5a0d62e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.054+0000 7f312560f740 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d65580 0x55e5a0d624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.054+0000 7f312560f740 1 --2- 172.21.15.55:0/848346095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d639c0 0x55e5a0d716f0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55e5a0d84c60 tx=0x7f3110004750 comp rx=0 tx=0).stop 2024-04-26T21:55:52.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.054+0000 7f312560f740 1 -- 172.21.15.55:0/848346095 >> 172.21.15.55:0/848346095 conn(0x55e5a0ca4910 msgr2=0x55e5a0d70fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:52.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.055+0000 7f312560f740 1 -- 172.21.15.55:0/848346095 shutdown_connections 2024-04-26T21:55:52.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.056+0000 7f312560f740 1 -- 172.21.15.55:0/848346095 wait complete. 2024-04-26T21:55:52.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.057+0000 7f312560f740 1 -- start start 2024-04-26T21:55:52.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.057+0000 7f312560f740 1 Processor -- start 2024-04-26T21:55:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.058+0000 7f312560f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d62a20 0x55e5a0d922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.058+0000 7f312560f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d94830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.058+0000 7f312560f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d94d70 0x55e5a0d951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.058+0000 7f312560f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f310800cc80 con 0x55e5a0d65580 2024-04-26T21:55:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.058+0000 7f312560f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f310800e0f0 con 0x55e5a0d94d70 2024-04-26T21:55:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.058+0000 7f312560f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f310800d350 con 0x55e5a0d62a20 2024-04-26T21:55:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.058+0000 7f312560f740 10 client.0 _ll_register_callbacks cb 0x7f310c00a7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:52.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.059+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d94830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.059+0000 7f311b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d94830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60686/0 (socket says 172.21.15.55:60686) 2024-04-26T21:55:52.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.059+0000 7f311b7fe640 1 -- 172.21.15.55:0/1445645203 learned_addr learned my addr 172.21.15.55:0/1445645203 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.060+0000 7f3120e95640 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d94d70 0x55e5a0d951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.061+0000 7f311b7fe640 1 -- 172.21.15.55:0/1445645203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d62a20 msgr2=0x55e5a0d922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.061+0000 7f311b7fe640 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d62a20 0x55e5a0d922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.061+0000 7f311b7fe640 1 -- 172.21.15.55:0/1445645203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d94d70 msgr2=0x55e5a0d951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.061+0000 7f311b7fe640 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d94d70 0x55e5a0d951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.061+0000 7f311b7fe640 1 -- 172.21.15.55:0/1445645203 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a0d95730 con 0x55e5a0d65580 2024-04-26T21:55:52.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.063+0000 7f311b7fe640 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d94830 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f310c000c00 tx=0x7f310c00fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1987c8c9dcffc43b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:52.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.063+0000 7f30fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:52.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.064+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1445645203 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c02b050 con 0x55e5a0d65580 2024-04-26T21:55:52.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.065+0000 7f312560f740 1 -- 172.21.15.55:0/1445645203 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3110026e50 con 0x55e5a0d65580 2024-04-26T21:55:52.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.065+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1445645203 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f310c012710 con 0x55e5a0d65580 2024-04-26T21:55:52.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.065+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1445645203 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f310c03f050 con 0x55e5a0d65580 2024-04-26T21:55:52.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.065+0000 7f312560f740 10 client.15882 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:52.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.065+0000 7f312560f740 1 -- 172.21.15.55:0/1445645203 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e5a0d749f0 con 0x55e5a0d65580 2024-04-26T21:55:52.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.066+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1445645203 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3110026e50 con 0x55e5a0d65580 2024-04-26T21:55:52.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.067+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1445645203 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f310c003a00 con 0x55e5a0d65580 2024-04-26T21:55:52.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.067+0000 7f312560f740 10 client.15882 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:52.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.067+0000 7f312560f740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:52.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.067+0000 7f312560f740 20 client.15882 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:52.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.068+0000 7f312560f740 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e5a0d752f0 0x55e5a0d776e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:52.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.068+0000 7f312560f740 4 client.15882 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:55:52.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.068+0000 7f312560f740 1 -- 172.21.15.55:0/1445645203 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5a0ccf800 con 0x55e5a0d752f0 2024-04-26T21:55:52.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.069+0000 7f311bfff640 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e5a0d752f0 0x55e5a0d776e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:52.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.072+0000 7f311bfff640 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e5a0d752f0 0x55e5a0d776e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7d10becfb8a72d6 server_cookie=c029a119597bb1d3 in_seq=0 out_seq=0 2024-04-26T21:55:52.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.072+0000 7f30fb7fe640 10 client.15882 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.077+0000 7f30fb7fe640 1 -- 172.21.15.55:0/1445645203 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5a0ccf800 con 0x55e5a0d752f0 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.077+0000 7f30fb7fe640 10 client.15882 handle_command_reply: tid=0 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 client.15882 shutdown 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 -- 172.21.15.55:0/1445645203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 msgr2=0x55e5a0d94830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d94830 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f310c000c00 tx=0x7f310c00fac0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 -- 172.21.15.55:0/1445645203 shutdown_connections 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e5a0d94d70 0x55e5a0d951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e5a0d752f0 0x55e5a0d776e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e5a0d65580 0x55e5a0d94830 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 --2- 172.21.15.55:0/1445645203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e5a0d62a20 0x55e5a0d922e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 -- 172.21.15.55:0/1445645203 >> 172.21.15.55:0/1445645203 conn(0x55e5a0ca4910 msgr2=0x55e5a0d70fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:52.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f30fb7fe640 0 client.15882 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:52.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 -- 172.21.15.55:0/1445645203 shutdown_connections 2024-04-26T21:55:52.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.078+0000 7f312560f740 1 -- 172.21.15.55:0/1445645203 wait complete. 2024-04-26T21:55:52.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.079+0000 7f312560f740 20 client.15882 trim_cache size 0 max 16384 2024-04-26T21:55:52.080 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:55:52.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.079+0000 7f3102ffd640 1 -- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31040a3e40 msgr2=0x7f31040a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:52.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.079+0000 7f3102ffd640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31040a3e40 0x7f31040a6300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3110009a50 tx=0x7f3110008040 comp rx=0 tx=0).stop 2024-04-26T21:55:52.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.079+0000 7f3102ffd640 1 -- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 msgr2=0x7f311c1a8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:52.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.079+0000 7f3102ffd640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 0x7f311c1a8970 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3114009bf0 tx=0x7f3114003c10 comp rx=0 tx=0).stop 2024-04-26T21:55:52.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.080+0000 7f3102ffd640 1 -- 172.21.15.55:0/533350156 shutdown_connections 2024-04-26T21:55:52.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.080+0000 7f3102ffd640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f31040a3e40 0x7f31040a6300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.080+0000 7f3102ffd640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f311c110cc0 0x7f311c1a8970 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.080+0000 7f3102ffd640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c10a5e0 0x7f311c1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.080+0000 7f3102ffd640 1 --2- 172.21.15.55:0/533350156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f311c073fa0 0x7f311c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:52.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.080+0000 7f3102ffd640 1 -- 172.21.15.55:0/533350156 >> 172.21.15.55:0/533350156 conn(0x7f311c06f840 msgr2=0x7f311c10cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:52.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.080+0000 7f3102ffd640 1 -- 172.21.15.55:0/533350156 shutdown_connections 2024-04-26T21:55:52.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:52.081+0000 7f3102ffd640 1 -- 172.21.15.55:0/533350156 wait complete. 2024-04-26T21:55:53.102 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:55:53.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:55:53.102 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:55:53.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.392+0000 7f41083f0640 1 -- 172.21.15.55:0/2938194745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f410010c2a0 msgr2=0x7f410010e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.392+0000 7f41083f0640 1 --2- 172.21.15.55:0/2938194745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f410010c2a0 0x7f410010e730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009860 tx=0x7f40fc02a380 comp rx=0 tx=0).stop 2024-04-26T21:55:53.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.395+0000 7f41083f0640 1 -- 172.21.15.55:0/2938194745 shutdown_connections 2024-04-26T21:55:53.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.395+0000 7f41083f0640 1 --2- 172.21.15.55:0/2938194745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f410010ec70 0x7f410010f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.395+0000 7f41083f0640 1 --2- 172.21.15.55:0/2938194745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f410010c2a0 0x7f410010e730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.395+0000 7f41083f0640 1 --2- 172.21.15.55:0/2938194745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4100109930 0x7f410010bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.395+0000 7f41083f0640 1 -- 172.21.15.55:0/2938194745 >> 172.21.15.55:0/2938194745 conn(0x7f410006f7c0 msgr2=0x7f4100071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.399+0000 7f41083f0640 1 -- 172.21.15.55:0/2938194745 shutdown_connections 2024-04-26T21:55:53.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.399+0000 7f41083f0640 1 -- 172.21.15.55:0/2938194745 wait complete. 2024-04-26T21:55:53.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.400+0000 7f41083f0640 1 Processor -- start 2024-04-26T21:55:53.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.400+0000 7f41083f0640 1 -- start start 2024-04-26T21:55:53.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.401+0000 7f41083f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4100109930 0x7f41001a7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.401+0000 7f41083f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f410010c2a0 0x7f41001a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.401+0000 7f41083f0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f410010ec70 0x7f41001aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.401+0000 7f41083f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f410011d320 con 0x7f410010c2a0 2024-04-26T21:55:53.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.401+0000 7f41083f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f410011d1a0 con 0x7f4100109930 2024-04-26T21:55:53.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.401+0000 7f41083f0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f410011d4a0 con 0x7f410010ec70 2024-04-26T21:55:53.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.401+0000 7f410698d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f410010ec70 0x7f41001aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.401+0000 7f410618c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4100109930 0x7f41001a7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.402+0000 7f410618c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4100109930 0x7f41001a7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49190/0 (socket says 172.21.15.55:49190) 2024-04-26T21:55:53.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.402+0000 7f410618c640 1 -- 172.21.15.55:0/2169957292 learned_addr learned my addr 172.21.15.55:0/2169957292 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:53.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.402+0000 7f410598b640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f410010c2a0 0x7f41001a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.402+0000 7f410618c640 1 -- 172.21.15.55:0/2169957292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f410010ec70 msgr2=0x7f41001aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.402+0000 7f410618c640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f410010ec70 0x7f41001aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.402+0000 7f410618c640 1 -- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f410010c2a0 msgr2=0x7f41001a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.402+0000 7f410618c640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f410010c2a0 0x7f41001a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.402+0000 7f410618c640 1 -- 172.21.15.55:0/2169957292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41001af650 con 0x7f4100109930 2024-04-26T21:55:53.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.403+0000 7f410598b640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f410010c2a0 0x7f41001a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:55:53.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.403+0000 7f410618c640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4100109930 0x7f41001a7a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f40f800ebc0 tx=0x7f40f800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.404+0000 7f40f77fe640 1 -- 172.21.15.55:0/2169957292 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8010630 con 0x7f4100109930 2024-04-26T21:55:53.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.404+0000 7f40f77fe640 1 -- 172.21.15.55:0/2169957292 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f8010c80 con 0x7f4100109930 2024-04-26T21:55:53.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.404+0000 7f40f77fe640 1 -- 172.21.15.55:0/2169957292 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8018b60 con 0x7f4100109930 2024-04-26T21:55:53.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.404+0000 7f41083f0640 1 -- 172.21.15.55:0/2169957292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41001af940 con 0x7f4100109930 2024-04-26T21:55:53.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.404+0000 7f41083f0640 1 -- 172.21.15.55:0/2169957292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41001afe40 con 0x7f4100109930 2024-04-26T21:55:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.409+0000 7f410b1e7740 1 Processor -- start 2024-04-26T21:55:53.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.412+0000 7f410b1e7740 1 -- start start 2024-04-26T21:55:53.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.412+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad48b0 0x5604d8ad4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.412+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 0x5604d8ad9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.412+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ada410 0x5604d8adc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.412+0000 7f410b1e7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5604d8a39b00 con 0x5604d8ad48b0 2024-04-26T21:55:53.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.412+0000 7f410b1e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5604d89d5920 con 0x5604d8ad52a0 2024-04-26T21:55:53.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.412+0000 7f410b1e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604d8a35ac0 con 0x5604d8ada410 2024-04-26T21:55:53.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.413+0000 7f410698d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad48b0 0x5604d8ad4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.413+0000 7f410698d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad48b0 0x5604d8ad4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60704/0 (socket says 172.21.15.55:60704) 2024-04-26T21:55:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.413+0000 7f410698d640 1 -- 172.21.15.55:0/1052267905 learned_addr learned my addr 172.21.15.55:0/1052267905 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.414+0000 7f410698d640 1 -- 172.21.15.55:0/1052267905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ada410 msgr2=0x5604d8adc8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:55:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.414+0000 7f410698d640 1 --2- 172.21.15.55:0/1052267905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ada410 0x5604d8adc8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.414+0000 7f410698d640 1 -- 172.21.15.55:0/1052267905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 msgr2=0x5604d8ad9da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:55:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.414+0000 7f410698d640 1 --2- 172.21.15.55:0/1052267905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 0x5604d8ad9da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.414+0000 7f410698d640 1 -- 172.21.15.55:0/1052267905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d88137e0 con 0x5604d8ad48b0 2024-04-26T21:55:53.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.414+0000 7f410698d640 1 --2- 172.21.15.55:0/1052267905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad48b0 0x5604d8ad4cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009ac0 tx=0x7f40f0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4675e324b1ee5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.415+0000 7f40f4bfe640 1 -- 172.21.15.55:0/1052267905 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0010d30 con 0x5604d8ad48b0 2024-04-26T21:55:53.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.415+0000 7f40f4bfe640 1 -- 172.21.15.55:0/1052267905 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f001b930 con 0x5604d8ad48b0 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.415+0000 7f40f77fe640 1 -- 172.21.15.55:0/2169957292 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f40f8018d00 con 0x7f4100109930 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.416+0000 7f40f77fe640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ec0a3e10 0x7f40ec0a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.416+0000 7f410b1e7740 1 -- 172.21.15.55:0/1052267905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad48b0 msgr2=0x5604d8ad4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.416+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1052267905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad48b0 0x5604d8ad4cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009ac0 tx=0x7f40f0009f80 comp rx=0 tx=0).stop 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.416+0000 7f410b1e7740 1 -- 172.21.15.55:0/1052267905 shutdown_connections 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.416+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1052267905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ada410 0x5604d8adc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.416+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1052267905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 0x5604d8ad9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.416+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1052267905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad48b0 0x5604d8ad4cd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.416+0000 7f410b1e7740 1 -- 172.21.15.55:0/1052267905 >> 172.21.15.55:0/1052267905 conn(0x5604d8acff20 msgr2=0x5604d8ad0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.417+0000 7f410b1e7740 1 -- 172.21.15.55:0/1052267905 shutdown_connections 2024-04-26T21:55:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.417+0000 7f410b1e7740 1 -- 172.21.15.55:0/1052267905 wait complete. 2024-04-26T21:55:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.417+0000 7f40f77fe640 1 -- 172.21.15.55:0/2169957292 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f40f80d0c60 con 0x7f4100109930 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.417+0000 7f410618c640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ec0a3e10 0x7f40ec0a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.417+0000 7f410b1e7740 1 -- start start 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.418+0000 7f410b1e7740 1 Processor -- start 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.418+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad52a0 0x5604d8adedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.418+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ada410 0x5604d8adf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.418+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ae4320 0x5604d8ae6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.418+0000 7f410b1e7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5604d8add410 con 0x5604d8ad52a0 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.418+0000 7f410b1e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5604d89d35d0 con 0x5604d8ada410 2024-04-26T21:55:53.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.418+0000 7f410b1e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604d8add590 con 0x5604d8ae4320 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.418+0000 7f410b1e7740 10 client.0 _ll_register_callbacks cb 0x5604d89c1080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.419+0000 7f410618c640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ec0a3e10 0x7f40ec0a62d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5604d8ad56a0 tx=0x7f40f8018600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.420+0000 7f410698d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad52a0 0x5604d8adedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.420+0000 7f410698d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad52a0 0x5604d8adedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60716/0 (socket says 172.21.15.55:60716) 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.420+0000 7f410698d640 1 -- 172.21.15.55:0/1893666053 learned_addr learned my addr 172.21.15.55:0/1893666053 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.420+0000 7f410698d640 1 -- 172.21.15.55:0/1893666053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ae4320 msgr2=0x5604d8ae6710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.420+0000 7f410698d640 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ae4320 0x5604d8ae6710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.420+0000 7f410698d640 1 -- 172.21.15.55:0/1893666053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ada410 msgr2=0x5604d8adf2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.420+0000 7f410698d640 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ada410 0x5604d8adf2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.421+0000 7f410698d640 1 -- 172.21.15.55:0/1893666053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d8873f70 con 0x5604d8ad52a0 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.422+0000 7f410698d640 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad52a0 0x5604d8adedb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005c80 tx=0x7f40f0010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb218976e1207f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.422+0000 7f40e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.423+0000 7f40e9ffb640 1 -- 172.21.15.55:0/1893666053 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0005070 con 0x5604d8ad52a0 2024-04-26T21:55:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.423+0000 7f410b1e7740 1 -- 172.21.15.55:0/1893666053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5604d8a47710 con 0x5604d8ad52a0 2024-04-26T21:55:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.424+0000 7f40e9ffb640 1 -- 172.21.15.55:0/1893666053 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f40f0005210 con 0x5604d8ad52a0 2024-04-26T21:55:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.425+0000 7f40e9ffb640 1 -- 172.21.15.55:0/1893666053 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f40f0005410 con 0x5604d8ad52a0 2024-04-26T21:55:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.425+0000 7f410b1e7740 10 client.15891 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.425+0000 7f410b1e7740 1 -- 172.21.15.55:0/1893666053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5604d870b170 con 0x5604d8ad52a0 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.426+0000 7f40e9ffb640 1 -- 172.21.15.55:0/1893666053 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5604d8a47710 con 0x5604d8ad52a0 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.426+0000 7f40e9ffb640 1 -- 172.21.15.55:0/1893666053 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f40f000f040 con 0x5604d8ad52a0 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.426+0000 7f410b1e7740 10 client.15891 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.426+0000 7f410b1e7740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.426+0000 7f410b1e7740 20 client.15891 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.426+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5604d8ae8470 0x5604d8aea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.426+0000 7f410b1e7740 4 client.15891 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.426+0000 7f410b1e7740 1 -- 172.21.15.55:0/1893666053 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604d8a41580 con 0x5604d8ae8470 2024-04-26T21:55:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.428+0000 7f410598b640 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5604d8ae8470 0x5604d8aea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.434+0000 7f410598b640 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5604d8ae8470 0x5604d8aea8a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59e30d472b960d server_cookie=c6f044267459121 in_seq=0 out_seq=0 2024-04-26T21:55:53.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.434+0000 7f40e9ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:53.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:53 smithi055 ceph-mon[29221]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 788 MiB/s wr, 854 op/s 2024-04-26T21:55:53.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.439+0000 7f40e9ffb640 1 -- 172.21.15.55:0/1893666053 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604d8a41580 con 0x5604d8ae8470 2024-04-26T21:55:53.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.439+0000 7f40e9ffb640 10 client.15891 handle_command_reply: tid=0 2024-04-26T21:55:53.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.442+0000 7f410b1e7740 1 client.15891 shutdown 2024-04-26T21:55:53.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.443+0000 7f410b1e7740 1 -- 172.21.15.55:0/1893666053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad52a0 msgr2=0x5604d8adedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.443+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad52a0 0x5604d8adedb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005c80 tx=0x7f40f0010dd0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.447+0000 7f410b1e7740 1 -- 172.21.15.55:0/1893666053 shutdown_connections 2024-04-26T21:55:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.447+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ae4320 0x5604d8ae6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.447+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ada410 0x5604d8adf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.447+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5604d8ae8470 0x5604d8aea8a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.447+0000 7f410b1e7740 1 --2- 172.21.15.55:0/1893666053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ad52a0 0x5604d8adedb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.447+0000 7f410b1e7740 1 -- 172.21.15.55:0/1893666053 >> 172.21.15.55:0/1893666053 conn(0x5604d8acff20 msgr2=0x5604d8ad0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.447+0000 7f40e9ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:53.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.454+0000 7f410b1e7740 1 -- 172.21.15.55:0/1893666053 shutdown_connections 2024-04-26T21:55:53.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.456+0000 7f410b1e7740 1 -- 172.21.15.55:0/1893666053 wait complete. 2024-04-26T21:55:53.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.457+0000 7f410b1e7740 20 client.15891 trim_cache size 0 max 16384 2024-04-26T21:55:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.496+0000 7f410b1e7740 1 Processor -- start 2024-04-26T21:55:53.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.496+0000 7f410b1e7740 1 -- start start 2024-04-26T21:55:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.496+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ad52a0 0x5604d8ad4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.497+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ada410 0x5604d8ad6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.497+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ae4320 0x5604d8ad6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.497+0000 7f410b1e7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f41001b2f30 con 0x5604d8ae4320 2024-04-26T21:55:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.497+0000 7f410b1e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f41001b0800 con 0x5604d8ada410 2024-04-26T21:55:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.497+0000 7f410b1e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4100117d80 con 0x5604d8ad52a0 2024-04-26T21:55:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.497+0000 7f410698d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ae4320 0x5604d8ad6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.497+0000 7f410698d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ae4320 0x5604d8ad6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60718/0 (socket says 172.21.15.55:60718) 2024-04-26T21:55:53.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.497+0000 7f410698d640 1 -- 172.21.15.55:0/3951468494 learned_addr learned my addr 172.21.15.55:0/3951468494 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:53.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.498+0000 7f410698d640 1 --2- 172.21.15.55:0/3951468494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ad52a0 0x5604d8ad4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.498+0000 7f410698d640 1 -- 172.21.15.55:0/3951468494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ad52a0 msgr2=0x5604d8ad4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:53.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.498+0000 7f410698d640 1 --2- 172.21.15.55:0/3951468494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ad52a0 0x5604d8ad4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.498+0000 7f410698d640 1 -- 172.21.15.55:0/3951468494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ada410 msgr2=0x5604d8ad6700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:53.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.498+0000 7f410698d640 1 --2- 172.21.15.55:0/3951468494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ada410 0x5604d8ad6700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.498+0000 7f410698d640 1 -- 172.21.15.55:0/3951468494 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d8ad44f0 con 0x5604d8ae4320 2024-04-26T21:55:53.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.499+0000 7f410698d640 1 --2- 172.21.15.55:0/3951468494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ae4320 0x5604d8ad6c40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f40f00098d0 tx=0x7f40f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fadb1498a01fce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.499+0000 7f40ebfff640 1 -- 172.21.15.55:0/3951468494 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0005cb0 con 0x5604d8ae4320 2024-04-26T21:55:53.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.499+0000 7f40ebfff640 1 -- 172.21.15.55:0/3951468494 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f001bbe0 con 0x5604d8ae4320 2024-04-26T21:55:53.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.499+0000 7f40ebfff640 1 -- 172.21.15.55:0/3951468494 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0003d20 con 0x5604d8ae4320 2024-04-26T21:55:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 -- 172.21.15.55:0/3951468494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ae4320 msgr2=0x5604d8ad6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3951468494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ae4320 0x5604d8ad6c40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f40f00098d0 tx=0x7f40f0009260 comp rx=0 tx=0).stop 2024-04-26T21:55:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 -- 172.21.15.55:0/3951468494 shutdown_connections 2024-04-26T21:55:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3951468494 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ae4320 0x5604d8ad6c40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3951468494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ada410 0x5604d8ad6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3951468494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ad52a0 0x5604d8ad4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 -- 172.21.15.55:0/3951468494 >> 172.21.15.55:0/3951468494 conn(0x5604d8acff20 msgr2=0x5604d8ad0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 -- 172.21.15.55:0/3951468494 shutdown_connections 2024-04-26T21:55:53.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.501+0000 7f410b1e7740 1 -- 172.21.15.55:0/3951468494 wait complete. 2024-04-26T21:55:53.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.502+0000 7f410b1e7740 1 -- start start 2024-04-26T21:55:53.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.503+0000 7f410b1e7740 1 Processor -- start 2024-04-26T21:55:53.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.503+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 0x5604d8ad8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.503+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ada410 0x5604d8ad8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.503+0000 7f410b1e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ae4320 0x5604d8afed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.503+0000 7f410b1e7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f41001183f0 con 0x5604d8ada410 2024-04-26T21:55:53.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.503+0000 7f410b1e7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f41001b0130 con 0x5604d8ad52a0 2024-04-26T21:55:53.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.503+0000 7f410b1e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4100118ac0 con 0x5604d8ae4320 2024-04-26T21:55:53.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.504+0000 7f410b1e7740 10 client.0 _ll_register_callbacks cb 0x7f40f00097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:53.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.504+0000 7f410698d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 0x5604d8ad8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.504+0000 7f410598b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ada410 0x5604d8ad8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.504+0000 7f410698d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 0x5604d8ad8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49216/0 (socket says 172.21.15.55:49216) 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.504+0000 7f410598b640 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ada410 0x5604d8ad8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60722/0 (socket says 172.21.15.55:60722) 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.504+0000 7f410698d640 1 -- 172.21.15.55:0/3189289943 learned_addr learned my addr 172.21.15.55:0/3189289943 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f410598b640 1 -- 172.21.15.55:0/3189289943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ae4320 msgr2=0x5604d8afed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f410598b640 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ae4320 0x5604d8afed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f410598b640 1 -- 172.21.15.55:0/3189289943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 msgr2=0x5604d8ad8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f410598b640 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 0x5604d8ad8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f410598b640 1 -- 172.21.15.55:0/3189289943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d8aff2f0 con 0x5604d8ada410 2024-04-26T21:55:53.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f410598b640 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ada410 0x5604d8ad8d60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f40fc02a970 tx=0x7f40fc005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f2f51e3face451 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f40e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f40e9ffb640 1 -- 172.21.15.55:0/3189289943 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc046070 con 0x5604d8ada410 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f40e9ffb640 1 -- 172.21.15.55:0/3189289943 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f40fc0307c0 con 0x5604d8ada410 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.505+0000 7f410b1e7740 1 -- 172.21.15.55:0/3189289943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f001bbe0 con 0x5604d8ada410 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.506+0000 7f40e9ffb640 1 -- 172.21.15.55:0/3189289943 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f40fc037040 con 0x5604d8ada410 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.506+0000 7f410b1e7740 10 client.15897 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.506+0000 7f410b1e7740 1 -- 172.21.15.55:0/3189289943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5604d8b50920 con 0x5604d8ada410 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.507+0000 7f40e9ffb640 1 -- 172.21.15.55:0/3189289943 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f001bbe0 con 0x5604d8ada410 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.507+0000 7f40e9ffb640 1 -- 172.21.15.55:0/3189289943 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f40fc049020 con 0x5604d8ada410 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.507+0000 7f410b1e7740 10 client.15897 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:53.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.507+0000 7f410b1e7740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.507+0000 7f410b1e7740 20 client.15897 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.507+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5604d8b51370 0x5604d8b53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.507+0000 7f410b1e7740 4 client.15897 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:55:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.507+0000 7f410b1e7740 1 -- 172.21.15.55:0/3189289943 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5604d89bcac0 con 0x5604d8b51370 2024-04-26T21:55:53.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.508+0000 7f410698d640 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5604d8b51370 0x5604d8b53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.511+0000 7f40e9ffb640 10 client.15897 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:53.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.511+0000 7f410698d640 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5604d8b51370 0x5604d8b53790 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f265f1d83c3d7cdc server_cookie=9641bb7b11f28839 in_seq=0 out_seq=0 2024-04-26T21:55:53.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.557+0000 7f40e9ffb640 1 -- 172.21.15.55:0/3189289943 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5604d89bcac0 con 0x5604d8b51370 2024-04-26T21:55:53.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.557+0000 7f40e9ffb640 10 client.15897 handle_command_reply: tid=0 2024-04-26T21:55:53.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.557+0000 7f410b1e7740 1 client.15897 shutdown 2024-04-26T21:55:53.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.557+0000 7f410b1e7740 1 -- 172.21.15.55:0/3189289943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ada410 msgr2=0x5604d8ad8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.557+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ada410 0x5604d8ad8d60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f40fc02a970 tx=0x7f40fc005b20 comp rx=0 tx=0).stop 2024-04-26T21:55:53.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.558+0000 7f410b1e7740 1 -- 172.21.15.55:0/3189289943 shutdown_connections 2024-04-26T21:55:53.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.558+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604d8ae4320 0x5604d8afed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.558+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5604d8b51370 0x5604d8b53790 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.558+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5604d8ada410 0x5604d8ad8d60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.558+0000 7f410b1e7740 1 --2- 172.21.15.55:0/3189289943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5604d8ad52a0 0x5604d8ad8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.558+0000 7f410b1e7740 1 -- 172.21.15.55:0/3189289943 >> 172.21.15.55:0/3189289943 conn(0x5604d8acff20 msgr2=0x5604d8ad4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.561+0000 7f40e9ffb640 0 client.15897 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:53.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.561+0000 7f410b1e7740 1 -- 172.21.15.55:0/3189289943 shutdown_connections 2024-04-26T21:55:53.567 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:55:53.567 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:55:53.567 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "e04c89dd-e537-4d39-9457-1f51bd5dd736", 2024-04-26T21:55:53.567 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:55:53.567 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:55:53.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.564+0000 7f410b1e7740 1 -- 172.21.15.55:0/3189289943 wait complete. 2024-04-26T21:55:53.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.565+0000 7f410b1e7740 20 client.15897 trim_cache size 0 max 16384 2024-04-26T21:55:53.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.569+0000 7f41083f0640 1 -- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ec0a3e10 msgr2=0x7f40ec0a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.569+0000 7f41083f0640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ec0a3e10 0x7f40ec0a62d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5604d8ad56a0 tx=0x7f40f8018600 comp rx=0 tx=0).stop 2024-04-26T21:55:53.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.569+0000 7f41083f0640 1 -- 172.21.15.55:0/2169957292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4100109930 msgr2=0x7f41001a7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.569+0000 7f41083f0640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4100109930 0x7f41001a7a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f40f800ebc0 tx=0x7f40f800c6a0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:53 smithi165 ceph-mon[33296]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 788 MiB/s wr, 854 op/s 2024-04-26T21:55:53.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.582+0000 7f41083f0640 1 -- 172.21.15.55:0/2169957292 shutdown_connections 2024-04-26T21:55:53.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.582+0000 7f41083f0640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f40ec0a3e10 0x7f40ec0a62d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.583+0000 7f41083f0640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f410010ec70 0x7f41001aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.583+0000 7f41083f0640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f410010c2a0 0x7f41001a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.583+0000 7f41083f0640 1 --2- 172.21.15.55:0/2169957292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4100109930 0x7f41001a7a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.583+0000 7f41083f0640 1 -- 172.21.15.55:0/2169957292 >> 172.21.15.55:0/2169957292 conn(0x7f410006f7c0 msgr2=0x7f4100109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.590+0000 7f41083f0640 1 -- 172.21.15.55:0/2169957292 shutdown_connections 2024-04-26T21:55:53.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.595+0000 7f41083f0640 1 -- 172.21.15.55:0/2169957292 wait complete. 2024-04-26T21:55:53.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:55:53.654 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:55:53.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:53 smithi064 ceph-mon[32765]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 788 MiB/s wr, 854 op/s 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 -- 172.21.15.55:0/2368204035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 msgr2=0x7fa620102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2368204035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 0x7fa620102da0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa614009900 tx=0x7fa61402a410 comp rx=0 tx=0).stop 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 -- 172.21.15.55:0/2368204035 shutdown_connections 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2368204035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa620104530 0x7fa62010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2368204035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa620103b70 0x7fa620103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2368204035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 0x7fa620102da0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 -- 172.21.15.55:0/2368204035 >> 172.21.15.55:0/2368204035 conn(0x7fa6200fdf10 msgr2=0x7fa620100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 -- 172.21.15.55:0/2368204035 shutdown_connections 2024-04-26T21:55:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.942+0000 7fa627e5b640 1 -- 172.21.15.55:0/2368204035 wait complete. 2024-04-26T21:55:53.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa627e5b640 1 Processor -- start 2024-04-26T21:55:53.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa627e5b640 1 -- start start 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa627e5b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 0x7fa620076690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa627e5b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa620103b70 0x7fa620074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa627e5b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa620104530 0x7fa620075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa627e5b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa62010e5f0 con 0x7fa620102980 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa627e5b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa62010e470 con 0x7fa620104530 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa627e5b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa62010e770 con 0x7fa620103b70 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa625bf7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 0x7fa620076690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa625bf7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 0x7fa620076690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60740/0 (socket says 172.21.15.55:60740) 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.943+0000 7fa625bf7640 1 -- 172.21.15.55:0/2960396622 learned_addr learned my addr 172.21.15.55:0/2960396622 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.944+0000 7fa6263f8640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa620104530 0x7fa620075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.944+0000 7fa625bf7640 1 -- 172.21.15.55:0/2960396622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa620103b70 msgr2=0x7fa620074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.944+0000 7fa625bf7640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa620103b70 0x7fa620074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.944+0000 7fa625bf7640 1 -- 172.21.15.55:0/2960396622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa620104530 msgr2=0x7fa620075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.944+0000 7fa625bf7640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa620104530 0x7fa620075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.944+0000 7fa625bf7640 1 -- 172.21.15.55:0/2960396622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62019cd70 con 0x7fa620102980 2024-04-26T21:55:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.945+0000 7fa625bf7640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 0x7fa620076690 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa614031500 tx=0x7fa614003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.945+0000 7fa612ffd640 1 -- 172.21.15.55:0/2960396622 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61403f070 con 0x7fa620102980 2024-04-26T21:55:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.945+0000 7fa612ffd640 1 -- 172.21.15.55:0/2960396622 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61403b460 con 0x7fa620102980 2024-04-26T21:55:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.945+0000 7fa612ffd640 1 -- 172.21.15.55:0/2960396622 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614035030 con 0x7fa620102980 2024-04-26T21:55:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.945+0000 7fa627e5b640 1 -- 172.21.15.55:0/2960396622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62019d000 con 0x7fa620102980 2024-04-26T21:55:53.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.946+0000 7fa627e5b640 1 -- 172.21.15.55:0/2960396622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62019d4a0 con 0x7fa620102980 2024-04-26T21:55:53.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.949+0000 7fa612ffd640 1 -- 172.21.15.55:0/2960396622 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa614042070 con 0x7fa620102980 2024-04-26T21:55:53.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.950+0000 7fa612ffd640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa60c0a3d90 0x7fa60c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.950+0000 7fa612ffd640 1 -- 172.21.15.55:0/2960396622 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa6140f0f30 con 0x7fa620102980 2024-04-26T21:55:53.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.950+0000 7fa6253f6640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa60c0a3d90 0x7fa60c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.951+0000 7fa6253f6640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa60c0a3d90 0x7fa60c0a6250 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa620076400 tx=0x7fa61c006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.954+0000 7fa62ab70740 1 Processor -- start 2024-04-26T21:55:53.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.954+0000 7fa62ab70740 1 -- start start 2024-04-26T21:55:53.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa6253f6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa6253f6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60744/0 (socket says 172.21.15.55:60744) 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa62ab70740 1 -- 172.21.15.55:0/27703761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dd5102cd80 con 0x55dd510c90b0 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa6253f6640 1 -- 172.21.15.55:0/27703761 learned_addr learned my addr 172.21.15.55:0/27703761 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa62ab70740 1 -- 172.21.15.55:0/27703761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dd50fe68b0 con 0x55dd510c7ee0 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.955+0000 7fa62ab70740 1 -- 172.21.15.55:0/27703761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dd51028d40 con 0x55dd510c9a70 2024-04-26T21:55:53.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.956+0000 7fa625bf7640 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.956+0000 7fa6263f8640 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa6253f6640 1 -- 172.21.15.55:0/27703761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 msgr2=0x55dd510d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa6253f6640 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa6253f6640 1 -- 172.21.15.55:0/27703761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 msgr2=0x55dd510c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa6253f6640 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa6253f6640 1 -- 172.21.15.55:0/27703761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd50fb7a40 con 0x55dd510c90b0 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa6253f6640 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510c9530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa61c015810 tx=0x7fa61c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8177f705d04841b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa60bfff640 1 -- 172.21.15.55:0/27703761 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c023070 con 0x55dd510c90b0 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa60bfff640 1 -- 172.21.15.55:0/27703761 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c00b930 con 0x55dd510c90b0 2024-04-26T21:55:53.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.957+0000 7fa60bfff640 1 -- 172.21.15.55:0/27703761 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c01eb60 con 0x55dd510c90b0 2024-04-26T21:55:53.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.959+0000 7fa62ab70740 1 -- 172.21.15.55:0/27703761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 msgr2=0x55dd510c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.959+0000 7fa62ab70740 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510c9530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa61c015810 tx=0x7fa61c00e000 comp rx=0 tx=0).stop 2024-04-26T21:55:53.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.959+0000 7fa62ab70740 1 -- 172.21.15.55:0/27703761 shutdown_connections 2024-04-26T21:55:53.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.959+0000 7fa62ab70740 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.959+0000 7fa62ab70740 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510c9530 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.959+0000 7fa62ab70740 1 --2- 172.21.15.55:0/27703761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.959+0000 7fa62ab70740 1 -- 172.21.15.55:0/27703761 >> 172.21.15.55:0/27703761 conn(0x55dd510c33d0 msgr2=0x55dd510c37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.960+0000 7fa62ab70740 1 -- 172.21.15.55:0/27703761 shutdown_connections 2024-04-26T21:55:53.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.960+0000 7fa62ab70740 1 -- 172.21.15.55:0/27703761 wait complete. 2024-04-26T21:55:53.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.961+0000 7fa62ab70740 1 -- start start 2024-04-26T21:55:53.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.961+0000 7fa62ab70740 1 Processor -- start 2024-04-26T21:55:53.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.963+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c7ee0 0x55dd510d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.963+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 0x55dd510d26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.963+0000 7fa625bf7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c7ee0 0x55dd510d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.963+0000 7fa625bf7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c7ee0 0x55dd510d2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60746/0 (socket says 172.21.15.55:60746) 2024-04-26T21:55:53.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.963+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.963+0000 7fa625bf7640 1 -- 172.21.15.55:0/618309747 learned_addr learned my addr 172.21.15.55:0/618309747 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:53.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.963+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dd510d0a80 con 0x55dd510c7ee0 2024-04-26T21:55:53.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.964+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dd50fb3d10 con 0x55dd510c90b0 2024-04-26T21:55:53.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.964+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dd510d0c00 con 0x55dd510c9a70 2024-04-26T21:55:53.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.964+0000 7fa62ab70740 10 client.0 _ll_register_callbacks cb 0x55dd50fe0210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:53.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.964+0000 7fa6253f6640 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 0x55dd510d26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.964+0000 7fa625bf7640 1 -- 172.21.15.55:0/618309747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 msgr2=0x55dd510d9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.964+0000 7fa625bf7640 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.964+0000 7fa625bf7640 1 -- 172.21.15.55:0/618309747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 msgr2=0x55dd510d26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.964+0000 7fa625bf7640 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 0x55dd510d26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.965+0000 7fa625bf7640 1 -- 172.21.15.55:0/618309747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd50e087d0 con 0x55dd510c7ee0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.965+0000 7fa6097fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.965+0000 7fa625bf7640 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c7ee0 0x55dd510d2190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6140b94d0 tx=0x7fa6140bcf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229363596a71e97d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.966+0000 7fa6097fa640 1 -- 172.21.15.55:0/618309747 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140d6020 con 0x55dd510c7ee0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.966+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61c00b930 con 0x55dd510c7ee0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.966+0000 7fa6097fa640 1 -- 172.21.15.55:0/618309747 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa6140c1110 con 0x55dd510c7ee0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.966+0000 7fa6097fa640 1 -- 172.21.15.55:0/618309747 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa6140d6020 con 0x55dd510c7ee0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.966+0000 7fa62ab70740 10 client.15909 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.966+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd50d3e3f0 con 0x55dd510c7ee0 2024-04-26T21:55:53.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.966+0000 7fa6097fa640 1 -- 172.21.15.55:0/618309747 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c00b930 con 0x55dd510c7ee0 2024-04-26T21:55:53.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.968+0000 7fa6097fa640 1 -- 172.21.15.55:0/618309747 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa61403b910 con 0x55dd510c7ee0 2024-04-26T21:55:53.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.968+0000 7fa62ab70740 10 client.15909 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:53.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.968+0000 7fa62ab70740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:53.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.968+0000 7fa62ab70740 20 client.15909 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:53.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.968+0000 7fa62ab70740 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dd510ddc00 0x55dd510e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:53.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.968+0000 7fa62ab70740 4 client.15909 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:55:53.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.968+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd5103a990 con 0x55dd510ddc00 2024-04-26T21:55:53.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.969+0000 7fa6253f6640 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dd510ddc00 0x55dd510e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:53.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.970+0000 7fa6253f6640 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dd510ddc00 0x55dd510e0090 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96165896d20882f server_cookie=70a755b8313daafc in_seq=0 out_seq=0 2024-04-26T21:55:53.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.970+0000 7fa6097fa640 10 client.15909 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:53.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.972+0000 7fa6097fa640 1 -- 172.21.15.55:0/618309747 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd5103a990 con 0x55dd510ddc00 2024-04-26T21:55:53.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.972+0000 7fa6097fa640 10 client.15909 handle_command_reply: tid=0 2024-04-26T21:55:53.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.972+0000 7fa62ab70740 1 client.15909 shutdown 2024-04-26T21:55:53.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c7ee0 msgr2=0x55dd510d2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:53.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa62ab70740 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c7ee0 0x55dd510d2190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa6140b94d0 tx=0x7fa6140bcf70 comp rx=0 tx=0).stop 2024-04-26T21:55:53.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 shutdown_connections 2024-04-26T21:55:53.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa62ab70740 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa62ab70740 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 0x55dd510d26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa62ab70740 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dd510ddc00 0x55dd510e0090 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa6097fa640 0 client.15909 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:53.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa62ab70740 1 --2- 172.21.15.55:0/618309747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c7ee0 0x55dd510d2190 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:53.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.973+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 >> 172.21.15.55:0/618309747 conn(0x55dd510c33d0 msgr2=0x55dd510c2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:53.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.974+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 shutdown_connections 2024-04-26T21:55:53.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.974+0000 7fa62ab70740 1 -- 172.21.15.55:0/618309747 wait complete. 2024-04-26T21:55:53.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:53.974+0000 7fa62ab70740 20 client.15909 trim_cache size 0 max 16384 2024-04-26T21:55:54.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.018+0000 7fa62ab70740 1 Processor -- start 2024-04-26T21:55:54.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.018+0000 7fa62ab70740 1 -- start start 2024-04-26T21:55:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.018+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c7ee0 0x55dd510c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.019+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 0x55dd510d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.019+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c9a70 0x55dd510d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:54.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.019+0000 7fa62ab70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201bb970 con 0x55dd510c9a70 2024-04-26T21:55:54.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.019+0000 7fa62ab70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201b9240 con 0x55dd510c90b0 2024-04-26T21:55:54.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.019+0000 7fa62ab70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201a4010 con 0x55dd510c7ee0 2024-04-26T21:55:54.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.019+0000 7fa6263f8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c9a70 0x55dd510d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:54.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.019+0000 7fa6263f8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c9a70 0x55dd510d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60754/0 (socket says 172.21.15.55:60754) 2024-04-26T21:55:54.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.019+0000 7fa6263f8640 1 -- 172.21.15.55:0/958926425 learned_addr learned my addr 172.21.15.55:0/958926425 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:54.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.020+0000 7fa6253f6640 1 --2- 172.21.15.55:0/958926425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 0x55dd510d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:54.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.020+0000 7fa6263f8640 1 -- 172.21.15.55:0/958926425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c7ee0 msgr2=0x55dd510c2550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:55:54.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.020+0000 7fa6263f8640 1 --2- 172.21.15.55:0/958926425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c7ee0 0x55dd510c2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.020+0000 7fa6263f8640 1 -- 172.21.15.55:0/958926425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 msgr2=0x55dd510d96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:54.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.020+0000 7fa6263f8640 1 --2- 172.21.15.55:0/958926425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 0x55dd510d96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.020+0000 7fa6263f8640 1 -- 172.21.15.55:0/958926425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd50cfe0f0 con 0x55dd510c9a70 2024-04-26T21:55:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.022+0000 7fa6263f8640 1 --2- 172.21.15.55:0/958926425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c9a70 0x55dd510d17b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa618009f00 tx=0x7fa61800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b402ae0867d76dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:54.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.029+0000 7fa60b7fe640 1 -- 172.21.15.55:0/958926425 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618014750 con 0x55dd510c9a70 2024-04-26T21:55:54.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.029+0000 7fa60b7fe640 1 -- 172.21.15.55:0/958926425 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa618014da0 con 0x55dd510c9a70 2024-04-26T21:55:54.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.029+0000 7fa60b7fe640 1 -- 172.21.15.55:0/958926425 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6180153e0 con 0x55dd510c9a70 2024-04-26T21:55:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.033+0000 7fa62ab70740 1 -- 172.21.15.55:0/958926425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c9a70 msgr2=0x55dd510d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.033+0000 7fa62ab70740 1 --2- 172.21.15.55:0/958926425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c9a70 0x55dd510d17b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa618009f00 tx=0x7fa61800a7d0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.035+0000 7fa62ab70740 1 -- 172.21.15.55:0/958926425 shutdown_connections 2024-04-26T21:55:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.035+0000 7fa62ab70740 1 --2- 172.21.15.55:0/958926425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c9a70 0x55dd510d17b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.035+0000 7fa62ab70740 1 --2- 172.21.15.55:0/958926425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c90b0 0x55dd510d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.035+0000 7fa62ab70740 1 --2- 172.21.15.55:0/958926425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c7ee0 0x55dd510c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.035+0000 7fa62ab70740 1 -- 172.21.15.55:0/958926425 >> 172.21.15.55:0/958926425 conn(0x55dd510c33d0 msgr2=0x55dd510d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:54.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.056+0000 7fa62ab70740 1 -- 172.21.15.55:0/958926425 shutdown_connections 2024-04-26T21:55:54.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.065+0000 7fa62ab70740 1 -- 172.21.15.55:0/958926425 wait complete. 2024-04-26T21:55:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.065+0000 7fa62ab70740 1 -- start start 2024-04-26T21:55:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.065+0000 7fa62ab70740 1 Processor -- start 2024-04-26T21:55:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.066+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510d6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.066+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510d74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.066+0000 7fa62ab70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.066+0000 7fa62ab70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201a46e0 con 0x55dd510c90b0 2024-04-26T21:55:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.066+0000 7fa62ab70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201b8b70 con 0x55dd510c7ee0 2024-04-26T21:55:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.066+0000 7fa62ab70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa6201a4db0 con 0x55dd510c9a70 2024-04-26T21:55:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.066+0000 7fa62ab70740 10 client.0 _ll_register_callbacks cb 0x7fa614009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:55:54.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.070+0000 7fa625bf7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510d6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:54.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.070+0000 7fa6263f8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:54.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.071+0000 7fa6253f6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510d74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:54.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.072+0000 7fa6253f6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510d74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60764/0 (socket says 172.21.15.55:60764) 2024-04-26T21:55:54.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.072+0000 7fa6253f6640 1 -- 172.21.15.55:0/3318785133 learned_addr learned my addr 172.21.15.55:0/3318785133 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:55:54.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.078+0000 7fa625bf7640 1 -- 172.21.15.55:0/3318785133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 msgr2=0x55dd510d84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:54.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.078+0000 7fa625bf7640 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.078+0000 7fa625bf7640 1 -- 172.21.15.55:0/3318785133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 msgr2=0x55dd510d74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:55:54.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.078+0000 7fa625bf7640 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510d74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.078+0000 7fa625bf7640 1 -- 172.21.15.55:0/3318785133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd50fed4b0 con 0x55dd510c7ee0 2024-04-26T21:55:54.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.079+0000 7fa625bf7640 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510d6f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa61402a9d0 tx=0x7fa61403b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f96e5a0611fae9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:55:54.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.082+0000 7fa6097fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:55:54.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.082+0000 7fa6097fa640 1 -- 172.21.15.55:0/3318785133 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140d6020 con 0x55dd510c7ee0 2024-04-26T21:55:54.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.082+0000 7fa6097fa640 1 -- 172.21.15.55:0/3318785133 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa6140bd4e0 con 0x55dd510c7ee0 2024-04-26T21:55:54.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.083+0000 7fa62ab70740 1 -- 172.21.15.55:0/3318785133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa618014da0 con 0x55dd510c7ee0 2024-04-26T21:55:54.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.087+0000 7fa6097fa640 1 -- 172.21.15.55:0/3318785133 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa614049040 con 0x55dd510c7ee0 2024-04-26T21:55:54.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.088+0000 7fa62ab70740 10 client.25430 fetch_fsmap learned FSMap version 21 2024-04-26T21:55:54.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.088+0000 7fa62ab70740 1 -- 172.21.15.55:0/3318785133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd50fed4b0 con 0x55dd510c7ee0 2024-04-26T21:55:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.092+0000 7fa6097fa640 1 -- 172.21.15.55:0/3318785133 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa618014da0 con 0x55dd510c7ee0 2024-04-26T21:55:54.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.092+0000 7fa6097fa640 1 -- 172.21.15.55:0/3318785133 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa6140c5850 con 0x55dd510c7ee0 2024-04-26T21:55:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.092+0000 7fa62ab70740 10 client.25430 fetch_fsmap finished waiting for FSMap version 21 2024-04-26T21:55:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.092+0000 7fa62ab70740 10 client.25430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:55:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.092+0000 7fa62ab70740 20 client.25430 populate_metadata read hostname 'smithi055' 2024-04-26T21:55:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.092+0000 7fa62ab70740 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dd510ddc00 0x55dd511467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:55:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.093+0000 7fa62ab70740 4 client.25430 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:55:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.093+0000 7fa62ab70740 1 -- 172.21.15.55:0/3318785133 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd51034800 con 0x55dd510ddc00 2024-04-26T21:55:54.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.094+0000 7fa6253f6640 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dd510ddc00 0x55dd511467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:55:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.095+0000 7fa6253f6640 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dd510ddc00 0x55dd511467c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676b76d362e045f7 server_cookie=78e1a0bbe67a6e95 in_seq=0 out_seq=0 2024-04-26T21:55:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.096+0000 7fa6097fa640 10 client.25430 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:54.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.099+0000 7fa6097fa640 1 -- 172.21.15.55:0/3318785133 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55dd51034800 con 0x55dd510ddc00 2024-04-26T21:55:54.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.099+0000 7fa6097fa640 10 client.25430 handle_command_reply: tid=0 2024-04-26T21:55:54.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.099+0000 7fa62ab70740 1 client.25430 shutdown 2024-04-26T21:55:54.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.113+0000 7fa62ab70740 1 -- 172.21.15.55:0/3318785133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 msgr2=0x55dd510d6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:54.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.113+0000 7fa62ab70740 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510d6f70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa61402a9d0 tx=0x7fa61403b840 comp rx=0 tx=0).stop 2024-04-26T21:55:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.119+0000 7fa62ab70740 1 -- 172.21.15.55:0/3318785133 shutdown_connections 2024-04-26T21:55:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.119+0000 7fa62ab70740 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dd510c9a70 0x55dd510d84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.119+0000 7fa62ab70740 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55dd510ddc00 0x55dd511467c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.119+0000 7fa62ab70740 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dd510c90b0 0x55dd510d74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.119+0000 7fa62ab70740 1 --2- 172.21.15.55:0/3318785133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd510c7ee0 0x55dd510d6f70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.119+0000 7fa62ab70740 1 -- 172.21.15.55:0/3318785133 >> 172.21.15.55:0/3318785133 conn(0x55dd510c33d0 msgr2=0x55dd510c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:54.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.119+0000 7fa6097fa640 0 client.25430 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:55:54.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.123+0000 7fa62ab70740 1 -- 172.21.15.55:0/3318785133 shutdown_connections 2024-04-26T21:55:54.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.123+0000 7fa62ab70740 1 -- 172.21.15.55:0/3318785133 wait complete. 2024-04-26T21:55:54.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.127+0000 7fa62ab70740 20 client.25430 trim_cache size 0 max 16384 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (14 inodes in the stack)", 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout: "e04c89dd-e537-4d39-9457-1f51bd5dd736": { 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "e04c89dd-e537-4d39-9457-1f51bd5dd736", 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:55:54.136 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:55:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.145+0000 7fa627e5b640 1 -- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa60c0a3d90 msgr2=0x7fa60c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa60c0a3d90 0x7fa60c0a6250 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa620076400 tx=0x7fa61c006c40 comp rx=0 tx=0).stop 2024-04-26T21:55:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 -- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 msgr2=0x7fa620076690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:55:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 0x7fa620076690 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa614031500 tx=0x7fa614003d50 comp rx=0 tx=0).stop 2024-04-26T21:55:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 -- 172.21.15.55:0/2960396622 shutdown_connections 2024-04-26T21:55:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa60c0a3d90 0x7fa60c0a6250 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa620104530 0x7fa620075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa620103b70 0x7fa620074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 --2- 172.21.15.55:0/2960396622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa620102980 0x7fa620076690 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:55:54.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 -- 172.21.15.55:0/2960396622 >> 172.21.15.55:0/2960396622 conn(0x7fa6200fdf10 msgr2=0x7fa620107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:55:54.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 -- 172.21.15.55:0/2960396622 shutdown_connections 2024-04-26T21:55:54.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:55:54.146+0000 7fa627e5b640 1 -- 172.21.15.55:0/2960396622 wait complete. 2024-04-26T21:55:54.222 INFO:tasks.cephfs.filesystem:scrub status for tag:e04c89dd-e537-4d39-9457-1f51bd5dd736 - {'path': '/', 'tag': 'e04c89dd-e537-4d39-9457-1f51bd5dd736', 'options': 'recursive,force'} 2024-04-26T21:55:54.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:54 smithi165 ceph-mon[33296]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 838 MiB/s wr, 898 op/s 2024-04-26T21:55:54.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:54 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:55:54.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:54 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:54.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:54 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:55:54.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:54 smithi055 ceph-mon[29221]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 838 MiB/s wr, 898 op/s 2024-04-26T21:55:54.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:54 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:55:54.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:54 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:54.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:54 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:55:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:54 smithi064 ceph-mon[32765]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 838 MiB/s wr, 898 op/s 2024-04-26T21:55:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:54 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:55:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:54 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:54.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:54 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:55:56.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:56 smithi165 ceph-mon[33296]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 860 MiB/s wr, 949 op/s 2024-04-26T21:55:56.941 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:56 smithi055 ceph-mon[29221]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 860 MiB/s wr, 949 op/s 2024-04-26T21:55:57.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:56 smithi064 ceph-mon[32765]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 860 MiB/s wr, 949 op/s 2024-04-26T21:55:58.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:58 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:58.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:55:58 smithi165 ceph-mon[33296]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 893 MiB/s wr, 1.02k op/s 2024-04-26T21:55:58.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:58 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:58.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:55:58 smithi055 ceph-mon[29221]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 893 MiB/s wr, 1.02k op/s 2024-04-26T21:55:59.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:58 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:55:59.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:55:58 smithi064 ceph-mon[32765]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 893 MiB/s wr, 1.02k op/s 2024-04-26T21:56:00.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:00 smithi165 ceph-mon[33296]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 866 MiB/s wr, 1.02k op/s 2024-04-26T21:56:00.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:00 smithi055 ceph-mon[29221]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 866 MiB/s wr, 1.02k op/s 2024-04-26T21:56:01.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:00 smithi064 ceph-mon[32765]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 866 MiB/s wr, 1.02k op/s 2024-04-26T21:56:01.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:01 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:56:02.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:01 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:56:02.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:01 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:56:02.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:02 smithi055 ceph-mon[29221]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 799 MiB/s wr, 979 op/s 2024-04-26T21:56:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:02 smithi064 ceph-mon[32765]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 799 MiB/s wr, 979 op/s 2024-04-26T21:56:03.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:02 smithi165 ceph-mon[33296]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 799 MiB/s wr, 979 op/s 2024-04-26T21:56:04.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:04 smithi165 ceph-mon[33296]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 MiB/s wr, 1.05k op/s 2024-04-26T21:56:04.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:04 smithi055 ceph-mon[29221]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 MiB/s wr, 1.05k op/s 2024-04-26T21:56:05.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:04 smithi064 ceph-mon[32765]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 MiB/s wr, 1.05k op/s 2024-04-26T21:56:06.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:06 smithi055 ceph-mon[29221]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 853 MiB/s wr, 1.08k op/s 2024-04-26T21:56:07.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:06 smithi064 ceph-mon[32765]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 853 MiB/s wr, 1.08k op/s 2024-04-26T21:56:07.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:06 smithi165 ceph-mon[33296]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 853 MiB/s wr, 1.08k op/s 2024-04-26T21:56:07.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:07 smithi055 ceph-mon[29221]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:56:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:07 smithi064 ceph-mon[32765]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:56:08.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:07 smithi165 ceph-mon[33296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:56:08.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:08 smithi055 ceph-mon[29221]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.09k op/s 2024-04-26T21:56:09.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:08 smithi064 ceph-mon[32765]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.09k op/s 2024-04-26T21:56:09.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:08 smithi165 ceph-mon[33296]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.09k op/s 2024-04-26T21:56:10.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:10 smithi055 ceph-mon[29221]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 825 MiB/s wr, 1.16k op/s 2024-04-26T21:56:11.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:10 smithi064 ceph-mon[32765]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 825 MiB/s wr, 1.16k op/s 2024-04-26T21:56:11.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:10 smithi165 ceph-mon[33296]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 825 MiB/s wr, 1.16k op/s 2024-04-26T21:56:12.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:12 smithi055 ceph-mon[29221]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 794 MiB/s wr, 1.22k op/s 2024-04-26T21:56:13.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:12 smithi064 ceph-mon[32765]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 794 MiB/s wr, 1.22k op/s 2024-04-26T21:56:13.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:12 smithi165 ceph-mon[33296]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 794 MiB/s wr, 1.22k op/s 2024-04-26T21:56:15.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:14 smithi064 ceph-mon[32765]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 826 MiB/s wr, 1.35k op/s 2024-04-26T21:56:15.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:14 smithi165 ceph-mon[33296]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 826 MiB/s wr, 1.35k op/s 2024-04-26T21:56:15.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:14 smithi055 ceph-mon[29221]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 826 MiB/s wr, 1.35k op/s 2024-04-26T21:56:17.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:16 smithi064 ceph-mon[32765]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 842 MiB/s wr, 1.51k op/s 2024-04-26T21:56:17.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:16 smithi165 ceph-mon[33296]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 842 MiB/s wr, 1.51k op/s 2024-04-26T21:56:17.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:16 smithi055 ceph-mon[29221]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 842 MiB/s wr, 1.51k op/s 2024-04-26T21:56:19.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:18 smithi064 ceph-mon[32765]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 892 MiB/s wr, 1.94k op/s 2024-04-26T21:56:19.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:18 smithi165 ceph-mon[33296]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 892 MiB/s wr, 1.94k op/s 2024-04-26T21:56:19.193 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:18 smithi055 ceph-mon[29221]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 892 MiB/s wr, 1.94k op/s 2024-04-26T21:56:21.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:20 smithi064 ceph-mon[32765]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 885 MiB/s wr, 2.13k op/s 2024-04-26T21:56:21.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:20 smithi165 ceph-mon[33296]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 885 MiB/s wr, 2.13k op/s 2024-04-26T21:56:21.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:20 smithi055 ceph-mon[29221]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 885 MiB/s wr, 2.13k op/s 2024-04-26T21:56:22.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:21 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:22.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:21 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:22.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:21 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:23.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:22 smithi165 ceph-mon[33296]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 850 MiB/s wr, 1.98k op/s 2024-04-26T21:56:23.197 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:22 smithi055 ceph-mon[29221]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 850 MiB/s wr, 1.98k op/s 2024-04-26T21:56:23.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:22 smithi064 ceph-mon[32765]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 850 MiB/s wr, 1.98k op/s 2024-04-26T21:56:24.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:24.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:24.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:23 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:56:24.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:24.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:24.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:23 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:56:24.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:56:24.224 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:56:24.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:24.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:24.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:23 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:56:24.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.599+0000 7fabf935f640 1 -- 172.21.15.55:0/605835923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4105b90 msgr2=0x7fabf4106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.599+0000 7fabf935f640 1 --2- 172.21.15.55:0/605835923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4105b90 0x7fabf4106010 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009900 tx=0x7fabe802a3f0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.599+0000 7fabf935f640 1 -- 172.21.15.55:0/605835923 shutdown_connections 2024-04-26T21:56:24.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.599+0000 7fabf935f640 1 --2- 172.21.15.55:0/605835923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf4106550 0x7fabf410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.599+0000 7fabf935f640 1 --2- 172.21.15.55:0/605835923 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4105b90 0x7fabf4106010 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.599+0000 7fabf935f640 1 --2- 172.21.15.55:0/605835923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf41049a0 0x7fabf4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.599+0000 7fabf935f640 1 -- 172.21.15.55:0/605835923 >> 172.21.15.55:0/605835923 conn(0x7fabf40fff10 msgr2=0x7fabf4102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:24.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.600+0000 7fabf935f640 1 -- 172.21.15.55:0/605835923 shutdown_connections 2024-04-26T21:56:24.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.600+0000 7fabf935f640 1 -- 172.21.15.55:0/605835923 wait complete. 2024-04-26T21:56:24.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.602+0000 7fabf935f640 1 Processor -- start 2024-04-26T21:56:24.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.602+0000 7fabf935f640 1 -- start start 2024-04-26T21:56:24.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf935f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf41049a0 0x7fabf419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf935f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf4105b90 0x7fabf419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf935f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4106550 0x7fabf41a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf935f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4077f70 con 0x7fabf4106550 2024-04-26T21:56:24.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf935f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4077df0 con 0x7fabf41049a0 2024-04-26T21:56:24.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf935f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fabf40780f0 con 0x7fabf4105b90 2024-04-26T21:56:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf8b5e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4106550 0x7fabf41a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf8b5e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4106550 0x7fabf41a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41798/0 (socket says 172.21.15.55:41798) 2024-04-26T21:56:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.603+0000 7fabf8b5e640 1 -- 172.21.15.55:0/2662207826 learned_addr learned my addr 172.21.15.55:0/2662207826 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.604+0000 7fabf8b5e640 1 -- 172.21.15.55:0/2662207826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf4105b90 msgr2=0x7fabf419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.604+0000 7fabf8b5e640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf4105b90 0x7fabf419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.604+0000 7fabf8b5e640 1 -- 172.21.15.55:0/2662207826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf41049a0 msgr2=0x7fabf419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.604+0000 7fabf8b5e640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf41049a0 0x7fabf419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.604+0000 7fabf8b5e640 1 -- 172.21.15.55:0/2662207826 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf41a6c50 con 0x7fabf4106550 2024-04-26T21:56:24.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.605+0000 7fabf8b5e640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4106550 0x7fabf41a65e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b970 tx=0x7fabec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:24.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.605+0000 7fabf17fa640 1 -- 172.21.15.55:0/2662207826 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec019070 con 0x7fabf4106550 2024-04-26T21:56:24.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.605+0000 7fabf17fa640 1 -- 172.21.15.55:0/2662207826 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabec014440 con 0x7fabf4106550 2024-04-26T21:56:24.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.605+0000 7fabf17fa640 1 -- 172.21.15.55:0/2662207826 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec0075d0 con 0x7fabf4106550 2024-04-26T21:56:24.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.605+0000 7fabf935f640 1 -- 172.21.15.55:0/2662207826 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf41a6f40 con 0x7fabf4106550 2024-04-26T21:56:24.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.605+0000 7fabf935f640 1 -- 172.21.15.55:0/2662207826 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf41a7430 con 0x7fabf4106550 2024-04-26T21:56:24.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.609+0000 7fabf17fa640 1 -- 172.21.15.55:0/2662207826 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fabec01c070 con 0x7fabf4106550 2024-04-26T21:56:24.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.609+0000 7fabfd3b3740 1 Processor -- start 2024-04-26T21:56:24.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.609+0000 7fabfd3b3740 1 -- start start 2024-04-26T21:56:24.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee01394d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee013a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee01414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562ee009dd80 con 0x562ee01390b0 2024-04-26T21:56:24.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562ee00578b0 con 0x562ee013ac40 2024-04-26T21:56:24.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562ee0099d40 con 0x562ee013a280 2024-04-26T21:56:24.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabf17fa640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fabdc0a3cc0 0x7fabdc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee01394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee01394d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41804/0 (socket says 172.21.15.55:41804) 2024-04-26T21:56:24.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabf37fe640 1 -- 172.21.15.55:0/4145414281 learned_addr learned my addr 172.21.15.55:0/4145414281 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:24.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabf17fa640 1 -- 172.21.15.55:0/2662207826 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fabec0cfa60 con 0x7fabf4106550 2024-04-26T21:56:24.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabf8b5e640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fabdc0a3cc0 0x7fabdc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.610+0000 7fabf37fe640 1 --2- 172.21.15.55:0/4145414281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee01414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.611+0000 7fabf8b5e640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fabdc0a3cc0 0x7fabdc0a6180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b940 tx=0x7fabec004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.611+0000 7fabf37fe640 1 -- 172.21.15.55:0/4145414281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 msgr2=0x562ee013a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.611+0000 7fabf37fe640 1 --2- 172.21.15.55:0/4145414281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee013a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.611+0000 7fabf37fe640 1 -- 172.21.15.55:0/4145414281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 msgr2=0x562ee01414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.611+0000 7fabf37fe640 1 --2- 172.21.15.55:0/4145414281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee01414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.611+0000 7fabf37fe640 1 -- 172.21.15.55:0/4145414281 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee0028a40 con 0x562ee01390b0 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.611+0000 7fabf37fe640 1 --2- 172.21.15.55:0/4145414281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee01394d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fabe80096c0 tx=0x7fabe8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6435902a6042d92c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.612+0000 7fabda7fc640 1 -- 172.21.15.55:0/4145414281 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8041020 con 0x562ee01390b0 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.612+0000 7fabda7fc640 1 -- 172.21.15.55:0/4145414281 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe8004b70 con 0x562ee01390b0 2024-04-26T21:56:24.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.612+0000 7fabda7fc640 1 -- 172.21.15.55:0/4145414281 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8035040 con 0x562ee01390b0 2024-04-26T21:56:24.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.613+0000 7fabfd3b3740 1 -- 172.21.15.55:0/4145414281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 msgr2=0x562ee01394d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.613+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/4145414281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee01394d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fabe80096c0 tx=0x7fabe8005a20 comp rx=0 tx=0).stop 2024-04-26T21:56:24.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.613+0000 7fabfd3b3740 1 -- 172.21.15.55:0/4145414281 shutdown_connections 2024-04-26T21:56:24.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.613+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/4145414281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee01414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.613+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/4145414281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee013a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.613+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/4145414281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee01394d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.613+0000 7fabfd3b3740 1 -- 172.21.15.55:0/4145414281 >> 172.21.15.55:0/4145414281 conn(0x562ee01345a0 msgr2=0x562ee01349a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:24.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.613+0000 7fabfd3b3740 1 -- 172.21.15.55:0/4145414281 shutdown_connections 2024-04-26T21:56:24.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.614+0000 7fabfd3b3740 1 -- 172.21.15.55:0/4145414281 wait complete. 2024-04-26T21:56:24.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.618+0000 7fabfd3b3740 1 -- start start 2024-04-26T21:56:24.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.619+0000 7fabfd3b3740 1 Processor -- start 2024-04-26T21:56:24.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.619+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0143560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.620+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee0143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.620+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee014aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.620+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562ee0141bf0 con 0x562ee01390b0 2024-04-26T21:56:24.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.620+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562ee0024d10 con 0x562ee013ac40 2024-04-26T21:56:24.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.620+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562ee0141d70 con 0x562ee013a280 2024-04-26T21:56:24.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.620+0000 7fabfd3b3740 10 client.0 _ll_register_callbacks cb 0x562ee0051210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:24.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.622+0000 7fabf3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee0143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.623+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.623+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41808/0 (socket says 172.21.15.55:41808) 2024-04-26T21:56:24.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.623+0000 7fabf37fe640 1 -- 172.21.15.55:0/3061914860 learned_addr learned my addr 172.21.15.55:0/3061914860 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:24.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.623+0000 7fabf37fe640 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee014aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.624+0000 7fabf37fe640 1 -- 172.21.15.55:0/3061914860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 msgr2=0x562ee0143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:24.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.624+0000 7fabf37fe640 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee0143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.624+0000 7fabf37fe640 1 -- 172.21.15.55:0/3061914860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 msgr2=0x562ee014aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:24.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.624+0000 7fabf37fe640 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee014aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.624+0000 7fabf37fe640 1 -- 172.21.15.55:0/3061914860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edfe797d0 con 0x562ee01390b0 2024-04-26T21:56:24.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.625+0000 7fabf37fe640 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0143560 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fabe8004fc0 tx=0x7fabe8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9a9d14d914ee1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.627+0000 7fabd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:56:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.627+0000 7fabd37fe640 1 -- 172.21.15.55:0/3061914860 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe804f020 con 0x562ee01390b0 2024-04-26T21:56:24.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.627+0000 7fabfd3b3740 1 -- 172.21.15.55:0/3061914860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe8004b70 con 0x562ee01390b0 2024-04-26T21:56:24.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.627+0000 7fabd37fe640 1 -- 172.21.15.55:0/3061914860 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fabe8004050 con 0x562ee01390b0 2024-04-26T21:56:24.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.630+0000 7fabd37fe640 1 -- 172.21.15.55:0/3061914860 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fabe8041070 con 0x562ee01390b0 2024-04-26T21:56:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.634+0000 7fabfd3b3740 10 client.15927 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.634+0000 7fabfd3b3740 1 -- 172.21.15.55:0/3061914860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562edfdaf3f0 con 0x562ee01390b0 2024-04-26T21:56:24.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.644+0000 7fabd37fe640 1 -- 172.21.15.55:0/3061914860 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe8004b70 con 0x562ee01390b0 2024-04-26T21:56:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.646+0000 7fabd37fe640 1 -- 172.21.15.55:0/3061914860 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fabe8003c40 con 0x562ee01390b0 2024-04-26T21:56:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.646+0000 7fabfd3b3740 10 client.15927 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.646+0000 7fabfd3b3740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.646+0000 7fabfd3b3740 20 client.15927 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.646+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562ee014f1a0 0x562ee01515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.646+0000 7fabfd3b3740 4 client.15927 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:24.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.646+0000 7fabfd3b3740 1 -- 172.21.15.55:0/3061914860 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ee00ab990 con 0x562ee014f1a0 2024-04-26T21:56:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.649+0000 7fabf3fff640 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562ee014f1a0 0x562ee01515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.667+0000 7fabf3fff640 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562ee014f1a0 0x562ee01515c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e3ab9494a064c8 server_cookie=2c56dd9d44851dd in_seq=0 out_seq=0 2024-04-26T21:56:24.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.668+0000 7fabd37fe640 10 client.15927 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:24.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.673+0000 7fabd37fe640 1 -- 172.21.15.55:0/3061914860 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ee00ab990 con 0x562ee014f1a0 2024-04-26T21:56:24.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.673+0000 7fabd37fe640 10 client.15927 handle_command_reply: tid=0 2024-04-26T21:56:24.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.673+0000 7fabfd3b3740 1 client.15927 shutdown 2024-04-26T21:56:24.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.674+0000 7fabfd3b3740 1 -- 172.21.15.55:0/3061914860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 msgr2=0x562ee0143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.674+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0143560 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fabe8004fc0 tx=0x7fabe8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.675+0000 7fabfd3b3740 1 -- 172.21.15.55:0/3061914860 shutdown_connections 2024-04-26T21:56:24.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.675+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee014aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.675+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee0143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.675+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562ee014f1a0 0x562ee01515c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.675+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/3061914860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0143560 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.675+0000 7fabd37fe640 0 client.15927 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:24.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.675+0000 7fabfd3b3740 1 -- 172.21.15.55:0/3061914860 >> 172.21.15.55:0/3061914860 conn(0x562ee01345a0 msgr2=0x562ee01336b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:24.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.680+0000 7fabfd3b3740 1 -- 172.21.15.55:0/3061914860 shutdown_connections 2024-04-26T21:56:24.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.680+0000 7fabfd3b3740 1 -- 172.21.15.55:0/3061914860 wait complete. 2024-04-26T21:56:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.685+0000 7fabfd3b3740 20 client.15927 trim_cache size 0 max 16384 2024-04-26T21:56:24.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.747+0000 7fabfd3b3740 1 Processor -- start 2024-04-26T21:56:24.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.751+0000 7fabfd3b3740 1 -- start start 2024-04-26T21:56:24.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.751+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0134980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.751+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee014aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.751+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee0143880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.751+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41b2bb0 con 0x562ee01390b0 2024-04-26T21:56:24.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.751+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41b0480 con 0x562ee013ac40 2024-04-26T21:56:24.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.751+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ae380 con 0x562ee013a280 2024-04-26T21:56:24.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.752+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0134980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.753+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee0143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.753+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee0143880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49124/0 (socket says 172.21.15.55:49124) 2024-04-26T21:56:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.753+0000 7fabf37fe640 1 -- 172.21.15.55:0/2125966348 learned_addr learned my addr 172.21.15.55:0/2125966348 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.755+0000 7fabf37fe640 1 -- 172.21.15.55:0/2125966348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 msgr2=0x562ee014aa70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.755+0000 7fabf37fe640 1 --2- 172.21.15.55:0/2125966348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee014aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.755+0000 7fabf37fe640 1 -- 172.21.15.55:0/2125966348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 msgr2=0x562ee0134980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.755+0000 7fabf37fe640 1 --2- 172.21.15.55:0/2125966348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0134980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.755+0000 7fabf37fe640 1 -- 172.21.15.55:0/2125966348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee013e470 con 0x562ee013ac40 2024-04-26T21:56:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.759+0000 7fabf37fe640 1 --2- 172.21.15.55:0/2125966348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee0143880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009900 tx=0x7fabe8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1dc3e99ddb64e45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.760+0000 7fabd9ffb640 1 -- 172.21.15.55:0/2125966348 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe804f070 con 0x562ee013ac40 2024-04-26T21:56:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.761+0000 7fabd9ffb640 1 -- 172.21.15.55:0/2125966348 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe8005850 con 0x562ee013ac40 2024-04-26T21:56:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.762+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2125966348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 msgr2=0x562ee0143880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.762+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2125966348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee0143880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009900 tx=0x7fabe8009260 comp rx=0 tx=0).stop 2024-04-26T21:56:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.762+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2125966348 shutdown_connections 2024-04-26T21:56:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.763+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2125966348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee013ac40 0x562ee0143880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.763+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2125966348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013a280 0x562ee014aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.763+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2125966348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee01390b0 0x562ee0134980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.763+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2125966348 >> 172.21.15.55:0/2125966348 conn(0x562ee01345a0 msgr2=0x562ee0143170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:24.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.766+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2125966348 shutdown_connections 2024-04-26T21:56:24.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.766+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2125966348 wait complete. 2024-04-26T21:56:24.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.767+0000 7fabfd3b3740 1 -- start start 2024-04-26T21:56:24.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.767+0000 7fabfd3b3740 1 Processor -- start 2024-04-26T21:56:24.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.768+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee01390b0 0x562ee0145880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.768+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee013a280 0x562ee0145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.769+0000 7fabfd3b3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013ac40 0x562ee01c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.769+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ae940 con 0x562ee013a280 2024-04-26T21:56:24.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.769+0000 7fabfd3b3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41afdb0 con 0x562ee01390b0 2024-04-26T21:56:24.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.769+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee01390b0 0x562ee0145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.769+0000 7fabf37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee01390b0 0x562ee0145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49130/0 (socket says 172.21.15.55:49130) 2024-04-26T21:56:24.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.769+0000 7fabf37fe640 1 -- 172.21.15.55:0/2291581696 learned_addr learned my addr 172.21.15.55:0/2291581696 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:24.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.769+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41af010 con 0x562ee013ac40 2024-04-26T21:56:24.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.770+0000 7fabfd3b3740 10 client.0 _ll_register_callbacks cb 0x7fabe8005260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:24.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.770+0000 7fabf37fe640 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013ac40 0x562ee01c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.771+0000 7fabf3fff640 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee013a280 0x562ee0145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.773+0000 7fabf37fe640 1 -- 172.21.15.55:0/2291581696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013ac40 msgr2=0x562ee01c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:24.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.773+0000 7fabf37fe640 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013ac40 0x562ee01c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.773+0000 7fabf37fe640 1 -- 172.21.15.55:0/2291581696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee013a280 msgr2=0x562ee0145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:24.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.773+0000 7fabf37fe640 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee013a280 0x562ee0145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.773+0000 7fabf37fe640 1 -- 172.21.15.55:0/2291581696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562edfd6f0f0 con 0x562ee01390b0 2024-04-26T21:56:24.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.773+0000 7fabf3fff640 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee013a280 0x562ee0145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:24.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.779+0000 7fabf37fe640 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee01390b0 0x562ee0145880 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009830 tx=0x7fabe8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb11bceac6cffc12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:24.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.778+0000 7fabd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:56:24.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.780+0000 7fabd37fe640 1 -- 172.21.15.55:0/2291581696 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe804f070 con 0x562ee01390b0 2024-04-26T21:56:24.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.781+0000 7fabd37fe640 1 -- 172.21.15.55:0/2291581696 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fabe80035f0 con 0x562ee01390b0 2024-04-26T21:56:24.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.781+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe8005850 con 0x562ee01390b0 2024-04-26T21:56:24.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.781+0000 7fabd37fe640 1 -- 172.21.15.55:0/2291581696 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fabe8004b70 con 0x562ee01390b0 2024-04-26T21:56:24.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.782+0000 7fabfd3b3740 10 client.25442 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:24.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.782+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562ee005e4b0 con 0x562ee01390b0 2024-04-26T21:56:24.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.782+0000 7fabd37fe640 1 -- 172.21.15.55:0/2291581696 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe8005850 con 0x562ee01390b0 2024-04-26T21:56:24.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.784+0000 7fabd37fe640 1 -- 172.21.15.55:0/2291581696 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fabe805f030 con 0x562ee01390b0 2024-04-26T21:56:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.785+0000 7fabfd3b3740 10 client.25442 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.785+0000 7fabfd3b3740 10 client.25442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.785+0000 7fabfd3b3740 20 client.25442 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.785+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562ee01c60f0 0x562ee01c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.785+0000 7fabfd3b3740 4 client.25442 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:56:24.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.785+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ee00a5800 con 0x562ee01c60f0 2024-04-26T21:56:24.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.790+0000 7fabf3fff640 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562ee01c60f0 0x562ee01c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:24.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.813+0000 7fabf3fff640 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562ee01c60f0 0x562ee01c8510 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2985c8af769188 server_cookie=816a22deb88b53f in_seq=0 out_seq=0 2024-04-26T21:56:24.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.814+0000 7fabd37fe640 10 client.25442 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:24.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.818+0000 7fabd37fe640 1 -- 172.21.15.55:0/2291581696 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562ee00a5800 con 0x562ee01c60f0 2024-04-26T21:56:24.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.818+0000 7fabd37fe640 10 client.25442 handle_command_reply: tid=0 2024-04-26T21:56:24.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.821+0000 7fabfd3b3740 1 client.25442 shutdown 2024-04-26T21:56:24.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.822+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee01390b0 msgr2=0x562ee0145880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.822+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee01390b0 0x562ee0145880 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009830 tx=0x7fabe8000bc0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.823+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 shutdown_connections 2024-04-26T21:56:24.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.823+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562ee013ac40 0x562ee01c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.823+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562ee01c60f0 0x562ee01c8510 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.823+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562ee013a280 0x562ee0145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.823+0000 7fabfd3b3740 1 --2- 172.21.15.55:0/2291581696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562ee01390b0 0x562ee0145880 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.823+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 >> 172.21.15.55:0/2291581696 conn(0x562ee01345a0 msgr2=0x562ee0132e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.823+0000 7fabd37fe640 0 client.25442 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:24.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.826+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 shutdown_connections 2024-04-26T21:56:24.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.826+0000 7fabfd3b3740 1 -- 172.21.15.55:0/2291581696 wait complete. 2024-04-26T21:56:24.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.827+0000 7fabfd3b3740 20 client.25442 trim_cache size 0 max 16384 2024-04-26T21:56:24.832 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:56:24.832 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:56:24.832 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:56:24.832 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:56:24.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.835+0000 7fabf935f640 1 -- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fabdc0a3cc0 msgr2=0x7fabdc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.835+0000 7fabf935f640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fabdc0a3cc0 0x7fabdc0a6180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b940 tx=0x7fabec004b50 comp rx=0 tx=0).stop 2024-04-26T21:56:24.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.835+0000 7fabf935f640 1 -- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4106550 msgr2=0x7fabf41a65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:24.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.835+0000 7fabf935f640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4106550 0x7fabf41a65e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b970 tx=0x7fabec00be30 comp rx=0 tx=0).stop 2024-04-26T21:56:24.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.841+0000 7fabf935f640 1 -- 172.21.15.55:0/2662207826 shutdown_connections 2024-04-26T21:56:24.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.841+0000 7fabf935f640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fabdc0a3cc0 0x7fabdc0a6180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.841+0000 7fabf935f640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fabf4106550 0x7fabf41a65e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.841+0000 7fabf935f640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fabf4105b90 0x7fabf419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.841+0000 7fabf935f640 1 --2- 172.21.15.55:0/2662207826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fabf41049a0 0x7fabf419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:24.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.841+0000 7fabf935f640 1 -- 172.21.15.55:0/2662207826 >> 172.21.15.55:0/2662207826 conn(0x7fabf40fff10 msgr2=0x7fabf407aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:24.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.842+0000 7fabf935f640 1 -- 172.21.15.55:0/2662207826 shutdown_connections 2024-04-26T21:56:24.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:24.842+0000 7fabf935f640 1 -- 172.21.15.55:0/2662207826 wait complete. 2024-04-26T21:56:24.888 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:56:24.888 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:56:25.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:24 smithi165 ceph-mon[33296]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 853 MiB/s wr, 1.86k op/s 2024-04-26T21:56:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.171+0000 7f00ad564640 1 -- 172.21.15.55:0/354477586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a810ea10 msgr2=0x7f00a8119870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.171+0000 7f00ad564640 1 --2- 172.21.15.55:0/354477586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a810ea10 0x7f00a8119870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0098009900 tx=0x7f009802a440 comp rx=0 tx=0).stop 2024-04-26T21:56:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.172+0000 7f00ad564640 1 -- 172.21.15.55:0/354477586 shutdown_connections 2024-04-26T21:56:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.173+0000 7f00ad564640 1 --2- 172.21.15.55:0/354477586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a810ea10 0x7f00a8119870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.173+0000 7f00ad564640 1 --2- 172.21.15.55:0/354477586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a810bf10 0x7f00a810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.173+0000 7f00ad564640 1 --2- 172.21.15.55:0/354477586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a8074760 0x7f00a810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.173+0000 7f00ad564640 1 -- 172.21.15.55:0/354477586 >> 172.21.15.55:0/354477586 conn(0x7f00a806f690 msgr2=0x7f00a8071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:25.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.181+0000 7f00ad564640 1 -- 172.21.15.55:0/354477586 shutdown_connections 2024-04-26T21:56:25.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.182+0000 7f00ad564640 1 -- 172.21.15.55:0/354477586 wait complete. 2024-04-26T21:56:25.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.183+0000 7f00ad564640 1 Processor -- start 2024-04-26T21:56:25.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.183+0000 7f00ad564640 1 -- start start 2024-04-26T21:56:25.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.184+0000 7f00ad564640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8074760 0x7f00a8119130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:25.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.184+0000 7f00ad564640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a810ea10 0x7f00a8119670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:25.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.184+0000 7f00ad564640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a81177b0 0x7f00a8117c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:25.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.184+0000 7f00ad564640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8111960 con 0x7f00a8074760 2024-04-26T21:56:25.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.184+0000 7f00ad564640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00a81117e0 con 0x7f00a810ea10 2024-04-26T21:56:25.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.184+0000 7f00ad564640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f00a8111ae0 con 0x7f00a81177b0 2024-04-26T21:56:25.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.185+0000 7f00a7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8074760 0x7f00a8119130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:25.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.185+0000 7f00a7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8074760 0x7f00a8119130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41860/0 (socket says 172.21.15.55:41860) 2024-04-26T21:56:25.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.185+0000 7f00a7fff640 1 -- 172.21.15.55:0/2724703758 learned_addr learned my addr 172.21.15.55:0/2724703758 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:25.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.185+0000 7f00acd63640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a81177b0 0x7f00a8117c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:25.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:24 smithi055 ceph-mon[29221]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 853 MiB/s wr, 1.86k op/s 2024-04-26T21:56:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.186+0000 7f00a77fe640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a810ea10 0x7f00a8119670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.186+0000 7f00acd63640 1 -- 172.21.15.55:0/2724703758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a810ea10 msgr2=0x7f00a8119670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.186+0000 7f00acd63640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a810ea10 0x7f00a8119670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.187+0000 7f00acd63640 1 -- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8074760 msgr2=0x7f00a8119130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.187+0000 7f00acd63640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8074760 0x7f00a8119130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.187+0000 7f00acd63640 1 -- 172.21.15.55:0/2724703758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a81183e0 con 0x7f00a81177b0 2024-04-26T21:56:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.187+0000 7f00acd63640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a81177b0 0x7f00a8117c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f009802aa30 tx=0x7f0098003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.193+0000 7f00a57fa640 1 -- 172.21.15.55:0/2724703758 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009803f070 con 0x7f00a81177b0 2024-04-26T21:56:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.193+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a807ab90 con 0x7f00a81177b0 2024-04-26T21:56:25.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.193+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a807b100 con 0x7f00a81177b0 2024-04-26T21:56:25.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.193+0000 7f00a57fa640 1 -- 172.21.15.55:0/2724703758 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0098003f20 con 0x7f00a81177b0 2024-04-26T21:56:25.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.193+0000 7f00a57fa640 1 -- 172.21.15.55:0/2724703758 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098035030 con 0x7f00a81177b0 2024-04-26T21:56:25.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.200+0000 7f00a57fa640 1 -- 172.21.15.55:0/2724703758 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0098042070 con 0x7f00a81177b0 2024-04-26T21:56:25.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.202+0000 7f00a57fa640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00900a3e40 0x7f00900a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.203+0000 7f00a57fa640 1 -- 172.21.15.55:0/2724703758 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f00980f0ea0 con 0x7f00a81177b0 2024-04-26T21:56:25.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.209+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00940052c0 con 0x7f00a81177b0 2024-04-26T21:56:25.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.213+0000 7f00a7fff640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00900a3e40 0x7f00900a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:25.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.214+0000 7f00a7fff640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00900a3e40 0x7f00900a6300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f009c006fd0 tx=0x7f009c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:25.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.222+0000 7f00a57fa640 1 -- 172.21.15.55:0/2724703758 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f00980bd680 con 0x7f00a81177b0 2024-04-26T21:56:25.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:24 smithi064 ceph-mon[32765]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 853 MiB/s wr, 1.86k op/s 2024-04-26T21:56:25.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.927+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0094005880 con 0x7f00a81177b0 2024-04-26T21:56:25.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.929+0000 7f00a57fa640 1 -- 172.21.15.55:0/2724703758 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11864 (secure 0 0 0) 0x7f00980bcf80 con 0x7f00a81177b0 2024-04-26T21:56:25.934 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:56:25.934 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:56:06.126250+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24509,24551,24502]},"id":1}]} 2024-04-26T21:56:25.935 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 22 2024-04-26T21:56:25.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.935+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00900a3e40 msgr2=0x7f00900a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.935+0000 7f00ad564640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00900a3e40 0x7f00900a6300 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f009c006fd0 tx=0x7f009c008040 comp rx=0 tx=0).stop 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.935+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a81177b0 msgr2=0x7f00a8117c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.935+0000 7f00ad564640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a81177b0 0x7f00a8117c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f009802aa30 tx=0x7f0098003cf0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.936+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 shutdown_connections 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.936+0000 7f00ad564640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f00900a3e40 0x7f00900a6300 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.936+0000 7f00ad564640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a81177b0 0x7f00a8117c60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.936+0000 7f00ad564640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f00a810ea10 0x7f00a8119670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.936+0000 7f00ad564640 1 --2- 172.21.15.55:0/2724703758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f00a8074760 0x7f00a8119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:25.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.936+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 >> 172.21.15.55:0/2724703758 conn(0x7f00a806f690 msgr2=0x7f00a806fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:25.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.937+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 shutdown_connections 2024-04-26T21:56:25.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:25.937+0000 7f00ad564640 1 -- 172.21.15.55:0/2724703758 wait complete. 2024-04-26T21:56:25.967 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:56:26.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.252+0000 7fa2fae1c640 1 -- 172.21.15.55:0/3227273164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f4073d70 msgr2=0x7fa2f4074190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.252+0000 7fa2fae1c640 1 --2- 172.21.15.55:0/3227273164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f4073d70 0x7fa2f4074190 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009900 tx=0x7fa2e402a440 comp rx=0 tx=0).stop 2024-04-26T21:56:26.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.253+0000 7fa2fae1c640 1 -- 172.21.15.55:0/3227273164 shutdown_connections 2024-04-26T21:56:26.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.253+0000 7fa2fae1c640 1 --2- 172.21.15.55:0/3227273164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f410a9b0 0x7fa2f410f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.253+0000 7fa2fae1c640 1 --2- 172.21.15.55:0/3227273164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f410a010 0x7fa2f410a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.253+0000 7fa2fae1c640 1 --2- 172.21.15.55:0/3227273164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f4073d70 0x7fa2f4074190 secure :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009900 tx=0x7fa2e402a440 comp rx=0 tx=0).stop 2024-04-26T21:56:26.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.253+0000 7fa2fae1c640 1 -- 172.21.15.55:0/3227273164 >> 172.21.15.55:0/3227273164 conn(0x7fa2f406f840 msgr2=0x7fa2f4071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.258+0000 7fa2fae1c640 1 -- 172.21.15.55:0/3227273164 shutdown_connections 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.262+0000 7fa2fae1c640 1 -- 172.21.15.55:0/3227273164 wait complete. 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.263+0000 7fa2fae1c640 1 Processor -- start 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.263+0000 7fa2fae1c640 1 -- start start 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.263+0000 7fa2fae1c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f410a010 0x7fa2f411ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.263+0000 7fa2fae1c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f410a9b0 0x7fa2f411b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.263+0000 7fa2fae1c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f4119410 0x7fa2f41198a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.263+0000 7fa2fae1c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4111960 con 0x7fa2f4119410 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.263+0000 7fa2fae1c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41117e0 con 0x7fa2f410a010 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.263+0000 7fa2fae1c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4111ae0 con 0x7fa2f410a9b0 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.265+0000 7fa2f9e1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f410a010 0x7fa2f411ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.265+0000 7fa2f9e1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f410a010 0x7fa2f411ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49164/0 (socket says 172.21.15.55:49164) 2024-04-26T21:56:26.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.265+0000 7fa2f9e1a640 1 -- 172.21.15.55:0/825479121 learned_addr learned my addr 172.21.15.55:0/825479121 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.267+0000 7fa2f9619640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f410a9b0 0x7fa2f411b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.268+0000 7fa2f9e1a640 1 -- 172.21.15.55:0/825479121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f410a9b0 msgr2=0x7fa2f411b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.268+0000 7fa2f9e1a640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f410a9b0 0x7fa2f411b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.268+0000 7fa2f9e1a640 1 -- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f4119410 msgr2=0x7fa2f41198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.268+0000 7fa2f9e1a640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f4119410 0x7fa2f41198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.268+0000 7fa2f9e1a640 1 -- 172.21.15.55:0/825479121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f411a160 con 0x7fa2f410a010 2024-04-26T21:56:26.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.278+0000 7fa2f9e1a640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f410a010 0x7fa2f411ad40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4033000 tx=0x7fa2e40049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.281+0000 7fa2eaffd640 1 -- 172.21.15.55:0/825479121 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4040020 con 0x7fa2f410a010 2024-04-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.281+0000 7fa2eaffd640 1 -- 172.21.15.55:0/825479121 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e4004dd0 con 0x7fa2f410a010 2024-04-26T21:56:26.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.281+0000 7fa2eaffd640 1 -- 172.21.15.55:0/825479121 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4035020 con 0x7fa2f410a010 2024-04-26T21:56:26.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.284+0000 7fa2fae1c640 1 -- 172.21.15.55:0/825479121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f41229a0 con 0x7fa2f410a010 2024-04-26T21:56:26.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.284+0000 7fa2fae1c640 1 -- 172.21.15.55:0/825479121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f4122e90 con 0x7fa2f410a010 2024-04-26T21:56:26.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.289+0000 7fa2fed97740 1 Processor -- start 2024-04-26T21:56:26.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.294+0000 7fa2fed97740 1 -- start start 2024-04-26T21:56:26.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.298+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d29c0 0x55a9956d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.298+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d3be0 0x55a9956d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.298+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956dae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.298+0000 7fa2fed97740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a995636d80 con 0x55a9956d4580 2024-04-26T21:56:26.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.298+0000 7fa2fed97740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a9955f0a20 con 0x55a9956d3be0 2024-04-26T21:56:26.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.298+0000 7fa2fed97740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a995632d40 con 0x55a9956d29c0 2024-04-26T21:56:26.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.300+0000 7fa2fa61b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d29c0 0x55a9956d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.300+0000 7fa2fa61b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d29c0 0x55a9956d2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:54474/0 (socket says 172.21.15.55:54474) 2024-04-26T21:56:26.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.300+0000 7fa2fa61b640 1 -- 172.21.15.55:0/436260566 learned_addr learned my addr 172.21.15.55:0/436260566 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.300+0000 7fa2eaffd640 1 -- 172.21.15.55:0/825479121 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa2e4043020 con 0x7fa2f410a010 2024-04-26T21:56:26.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.301+0000 7fa2fa61b640 1 -- 172.21.15.55:0/436260566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d3be0 msgr2=0x55a9956d4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.301+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/436260566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d3be0 0x55a9956d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.301+0000 7fa2fa61b640 1 -- 172.21.15.55:0/436260566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 msgr2=0x55a9956dae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.301+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/436260566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956dae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.301+0000 7fa2fa61b640 1 -- 172.21.15.55:0/436260566 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9955fbc30 con 0x55a9956d29c0 2024-04-26T21:56:26.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.302+0000 7fa2eaffd640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2e00a3e10 0x7fa2e00a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.303+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/436260566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d29c0 0x55a9956d2de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009790 tx=0x7fa2f0009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa4bb2e11675977 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.303+0000 7fa2eaffd640 1 -- 172.21.15.55:0/825479121 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa2e40f0d60 con 0x7fa2f410a010 2024-04-26T21:56:26.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.305+0000 7fa2fae1c640 1 -- 172.21.15.55:0/436260566 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0015020 con 0x55a9956d29c0 2024-04-26T21:56:26.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.305+0000 7fa2fae1c640 1 -- 172.21.15.55:0/436260566 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f0011420 con 0x55a9956d29c0 2024-04-26T21:56:26.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.306+0000 7fa2fed97740 1 -- 172.21.15.55:0/436260566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d29c0 msgr2=0x55a9956d2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.306+0000 7fa2fed97740 1 --2- 172.21.15.55:0/436260566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d29c0 0x55a9956d2de0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009790 tx=0x7fa2f0009c50 comp rx=0 tx=0).stop 2024-04-26T21:56:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.308+0000 7fa2f9e1a640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2e00a3e10 0x7fa2e00a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.308+0000 7fa2fed97740 1 -- 172.21.15.55:0/436260566 shutdown_connections 2024-04-26T21:56:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.308+0000 7fa2fed97740 1 --2- 172.21.15.55:0/436260566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956dae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.308+0000 7fa2fed97740 1 --2- 172.21.15.55:0/436260566 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d3be0 0x55a9956d4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.308+0000 7fa2fed97740 1 --2- 172.21.15.55:0/436260566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d29c0 0x55a9956d2de0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.308+0000 7fa2fed97740 1 -- 172.21.15.55:0/436260566 >> 172.21.15.55:0/436260566 conn(0x55a995613910 msgr2=0x55a9956ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.311+0000 7fa2f9e1a640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2e00a3e10 0x7fa2e00a62d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40c5580 tx=0x7fa2e4003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.317+0000 7fa2fed97740 1 -- 172.21.15.55:0/436260566 shutdown_connections 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.320+0000 7fa2fed97740 1 -- 172.21.15.55:0/436260566 wait complete. 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.321+0000 7fa2fed97740 1 -- start start 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.321+0000 7fa2fed97740 1 Processor -- start 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.322+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d29c0 0x55a9956dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.322+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.322+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.322+0000 7fa2fed97740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a9956db600 con 0x55a9956d4580 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.322+0000 7fa2fed97740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a9955bddc0 con 0x55a9956d29c0 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.322+0000 7fa2fed97740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a9956db780 con 0x55a9956d3be0 2024-04-26T21:56:26.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.322+0000 7fa2fed97740 10 client.0 _ll_register_callbacks cb 0x55a9955d9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:26.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.323+0000 7fa2fa61b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.323+0000 7fa2fa61b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41884/0 (socket says 172.21.15.55:41884) 2024-04-26T21:56:26.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.323+0000 7fa2fa61b640 1 -- 172.21.15.55:0/2720006216 learned_addr learned my addr 172.21.15.55:0/2720006216 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.328+0000 7fa2f9619640 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.328+0000 7fa2fa61b640 1 -- 172.21.15.55:0/2720006216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 msgr2=0x55a9956dd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.328+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956dd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.328+0000 7fa2fa61b640 1 -- 172.21.15.55:0/2720006216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d29c0 msgr2=0x55a9956dcd00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:26.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.328+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d29c0 0x55a9956dcd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.328+0000 7fa2fa61b640 1 -- 172.21.15.55:0/2720006216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9953483f0 con 0x55a9956d4580 2024-04-26T21:56:26.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.340+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e4210 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0011800 tx=0x7fa2f0011830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a0d8cd677234d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.348+0000 7fa2dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:56:26.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.348+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/2720006216 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0015030 con 0x55a9956d4580 2024-04-26T21:56:26.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.348+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/2720006216 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa2f000bae0 con 0x55a9956d4580 2024-04-26T21:56:26.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.349+0000 7fa2fed97740 1 -- 172.21.15.55:0/2720006216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f0011420 con 0x55a9956d4580 2024-04-26T21:56:26.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.356+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/2720006216 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa2f00119d0 con 0x55a9956d4580 2024-04-26T21:56:26.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.359+0000 7fa2fed97740 10 client.15942 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:26.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.359+0000 7fa2fed97740 1 -- 172.21.15.55:0/2720006216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a9954127d0 con 0x55a9956d4580 2024-04-26T21:56:26.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.366+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/2720006216 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f0011420 con 0x55a9956d4580 2024-04-26T21:56:26.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.367+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/2720006216 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fa2f001a5a0 con 0x55a9956d4580 2024-04-26T21:56:26.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.368+0000 7fa2fed97740 10 client.15942 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:26.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.368+0000 7fa2fed97740 10 client.15942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:56:26.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.368+0000 7fa2fed97740 20 client.15942 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:26.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.368+0000 7fa2fed97740 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55a9956e8900 0x55a9956ead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.368+0000 7fa2fed97740 4 client.15942 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:26.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.368+0000 7fa2fed97740 1 -- 172.21.15.55:0/2720006216 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a995644990 con 0x55a9956e8900 2024-04-26T21:56:26.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.375+0000 7fa2f9619640 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55a9956e8900 0x55a9956ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.386+0000 7fa2dd7fa640 10 client.15942 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:56:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.386+0000 7fa2f9619640 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55a9956e8900 0x55a9956ead40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98bde595d3628794 server_cookie=2d600a8fc32b2dd2 in_seq=0 out_seq=0 2024-04-26T21:56:26.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.391+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/2720006216 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a995644990 con 0x55a9956e8900 2024-04-26T21:56:26.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.391+0000 7fa2dd7fa640 10 client.15942 handle_command_reply: tid=0 2024-04-26T21:56:26.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.392+0000 7fa2fed97740 1 client.15942 shutdown 2024-04-26T21:56:26.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.396+0000 7fa2fed97740 1 -- 172.21.15.55:0/2720006216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 msgr2=0x55a9956e4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.396+0000 7fa2fed97740 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e4210 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0011800 tx=0x7fa2f0011830 comp rx=0 tx=0).stop 2024-04-26T21:56:26.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.404+0000 7fa2fed97740 1 -- 172.21.15.55:0/2720006216 shutdown_connections 2024-04-26T21:56:26.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.404+0000 7fa2fed97740 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55a9956e8900 0x55a9956ead40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.404+0000 7fa2fed97740 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e4210 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.404+0000 7fa2fed97740 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956dd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.404+0000 7fa2fed97740 1 --2- 172.21.15.55:0/2720006216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d29c0 0x55a9956dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.404+0000 7fa2fed97740 1 -- 172.21.15.55:0/2720006216 >> 172.21.15.55:0/2720006216 conn(0x55a995613910 msgr2=0x55a9956cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.405+0000 7fa2dd7fa640 0 client.15942 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:56:26.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.412+0000 7fa2fed97740 1 -- 172.21.15.55:0/2720006216 shutdown_connections 2024-04-26T21:56:26.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.415+0000 7fa2fed97740 1 -- 172.21.15.55:0/2720006216 wait complete. 2024-04-26T21:56:26.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.416+0000 7fa2fed97740 20 client.15942 trim_cache size 0 max 16384 2024-04-26T21:56:26.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.467+0000 7fa2fed97740 1 Processor -- start 2024-04-26T21:56:26.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.467+0000 7fa2fed97740 1 -- start start 2024-04-26T21:56:26.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d29c0 0x55a9956dad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956e4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d4580 0x55a9956d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fed97740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f41d5350 con 0x55a9956d29c0 2024-04-26T21:56:26.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fed97740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f412c030 con 0x55a9956d4580 2024-04-26T21:56:26.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fa61b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d29c0 0x55a9956dad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fa61b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d29c0 0x55a9956dad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41894/0 (socket says 172.21.15.55:41894) 2024-04-26T21:56:26.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fa61b640 1 -- 172.21.15.55:0/1935987868 learned_addr learned my addr 172.21.15.55:0/1935987868 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fed97740 1 -- 172.21.15.55:0/1935987868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4129ea0 con 0x55a9956d3be0 2024-04-26T21:56:26.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fa61b640 1 -- 172.21.15.55:0/1935987868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 msgr2=0x55a9956e4130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:26.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/1935987868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956e4130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fa61b640 1 -- 172.21.15.55:0/1935987868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d4580 msgr2=0x55a9956d14e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:26.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/1935987868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d4580 0x55a9956d14e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.468+0000 7fa2fa61b640 1 -- 172.21.15.55:0/1935987868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9956bfbb0 con 0x55a9956d29c0 2024-04-26T21:56:26.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.469+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/1935987868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d29c0 0x55a9956dad10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0000c00 tx=0x7fa2f00096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f145928c58211f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.469+0000 7fa2df7fe640 1 -- 172.21.15.55:0/1935987868 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0015040 con 0x55a9956d29c0 2024-04-26T21:56:26.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.470+0000 7fa2df7fe640 1 -- 172.21.15.55:0/1935987868 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f0011420 con 0x55a9956d29c0 2024-04-26T21:56:26.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.470+0000 7fa2df7fe640 1 -- 172.21.15.55:0/1935987868 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0010780 con 0x55a9956d29c0 2024-04-26T21:56:26.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 -- 172.21.15.55:0/1935987868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d29c0 msgr2=0x55a9956dad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 --2- 172.21.15.55:0/1935987868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d29c0 0x55a9956dad10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0000c00 tx=0x7fa2f00096e0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 -- 172.21.15.55:0/1935987868 shutdown_connections 2024-04-26T21:56:26.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 --2- 172.21.15.55:0/1935987868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d4580 0x55a9956d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 --2- 172.21.15.55:0/1935987868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956e4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 --2- 172.21.15.55:0/1935987868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d29c0 0x55a9956dad10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 -- 172.21.15.55:0/1935987868 >> 172.21.15.55:0/1935987868 conn(0x55a995613910 msgr2=0x55a9956d7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 -- 172.21.15.55:0/1935987868 shutdown_connections 2024-04-26T21:56:26.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.471+0000 7fa2fed97740 1 -- 172.21.15.55:0/1935987868 wait complete. 2024-04-26T21:56:26.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.472+0000 7fa2fed97740 1 -- start start 2024-04-26T21:56:26.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.472+0000 7fa2fed97740 1 Processor -- start 2024-04-26T21:56:26.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d29c0 0x55a9956e1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956e1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fed97740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fed97740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f412a4f0 con 0x55a9956d4580 2024-04-26T21:56:26.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fed97740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f412b960 con 0x55a9956d29c0 2024-04-26T21:56:26.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fed97740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f412abc0 con 0x55a9956d3be0 2024-04-26T21:56:26.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fa61b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fed97740 10 client.0 _ll_register_callbacks cb 0x7fa2f00095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:26.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fa61b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41898/0 (socket says 172.21.15.55:41898) 2024-04-26T21:56:26.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.473+0000 7fa2fa61b640 1 -- 172.21.15.55:0/975294548 learned_addr learned my addr 172.21.15.55:0/975294548 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.474+0000 7fa2fa61b640 1 -- 172.21.15.55:0/975294548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 msgr2=0x55a9956e1940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:26.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.474+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956e1940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.474+0000 7fa2fa61b640 1 -- 172.21.15.55:0/975294548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d29c0 msgr2=0x55a9956e1400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:26.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.474+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d29c0 0x55a9956e1400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.474+0000 7fa2fa61b640 1 -- 172.21.15.55:0/975294548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9956e2f40 con 0x55a9956d4580 2024-04-26T21:56:26.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.474+0000 7fa2fa61b640 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e2950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa2f001ab60 tx=0x7fa2f00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401941a573e5b10a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.474+0000 7fa2dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:56:26.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.475+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/975294548 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f001e070 con 0x55a9956d4580 2024-04-26T21:56:26.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.475+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/975294548 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa2f001a630 con 0x55a9956d4580 2024-04-26T21:56:26.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.475+0000 7fa2fed97740 1 -- 172.21.15.55:0/975294548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2f0011420 con 0x55a9956d4580 2024-04-26T21:56:26.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.475+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/975294548 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa2f00059e0 con 0x55a9956d4580 2024-04-26T21:56:26.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.475+0000 7fa2fed97740 10 client.15948 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:26.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.475+0000 7fa2fed97740 1 -- 172.21.15.55:0/975294548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a9956e4240 con 0x55a9956d4580 2024-04-26T21:56:26.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.476+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/975294548 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2f0011420 con 0x55a9956d4580 2024-04-26T21:56:26.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.476+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/975294548 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fa2f0023450 con 0x55a9956d4580 2024-04-26T21:56:26.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.476+0000 7fa2fed97740 10 client.15948 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:26.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.476+0000 7fa2fed97740 10 client.15948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:56:26.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.476+0000 7fa2fed97740 20 client.15948 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:26.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.477+0000 7fa2fed97740 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55a99574f430 0x55a995751850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.477+0000 7fa2fed97740 4 client.15948 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:26.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.477+0000 7fa2fed97740 1 -- 172.21.15.55:0/975294548 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a99563e800 con 0x55a99574f430 2024-04-26T21:56:26.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.479+0000 7fa2f9619640 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55a99574f430 0x55a995751850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.487+0000 7fa2dd7fa640 10 client.15948 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:56:26.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.487+0000 7fa2f9619640 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55a99574f430 0x55a995751850 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f35edb0a63857c4f server_cookie=9d16aef9841aad4c in_seq=0 out_seq=0 2024-04-26T21:56:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.489+0000 7fa2dd7fa640 1 -- 172.21.15.55:0/975294548 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a99563e800 con 0x55a99574f430 2024-04-26T21:56:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.489+0000 7fa2dd7fa640 10 client.15948 handle_command_reply: tid=0 2024-04-26T21:56:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.489+0000 7fa2fed97740 1 client.15948 shutdown 2024-04-26T21:56:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.489+0000 7fa2fed97740 1 -- 172.21.15.55:0/975294548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 msgr2=0x55a9956e2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.489+0000 7fa2fed97740 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e2950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa2f001ab60 tx=0x7fa2f00092c0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 1 -- 172.21.15.55:0/975294548 shutdown_connections 2024-04-26T21:56:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55a99574f430 0x55a995751850 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2dd7fa640 0 client.15948 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:56:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a9956d4580 0x55a9956e2950 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9956d3be0 0x55a9956e1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 1 --2- 172.21.15.55:0/975294548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a9956d29c0 0x55a9956e1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 1 -- 172.21.15.55:0/975294548 >> 172.21.15.55:0/975294548 conn(0x55a995613910 msgr2=0x55a9956ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 1 -- 172.21.15.55:0/975294548 shutdown_connections 2024-04-26T21:56:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 1 -- 172.21.15.55:0/975294548 wait complete. 2024-04-26T21:56:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.490+0000 7fa2fed97740 20 client.15948 trim_cache size 0 max 16384 2024-04-26T21:56:26.493 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:56:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 -- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2e00a3e10 msgr2=0x7fa2e00a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2e00a3e10 0x7fa2e00a62d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40c5580 tx=0x7fa2e4003680 comp rx=0 tx=0).stop 2024-04-26T21:56:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 -- 172.21.15.55:0/825479121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f410a010 msgr2=0x7fa2f411ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f410a010 0x7fa2f411ad40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4033000 tx=0x7fa2e40049d0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 -- 172.21.15.55:0/825479121 shutdown_connections 2024-04-26T21:56:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa2e00a3e10 0x7fa2e00a62d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2f4119410 0x7fa2f41198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2f410a9b0 0x7fa2f411b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 --2- 172.21.15.55:0/825479121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2f410a010 0x7fa2f411ad40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 -- 172.21.15.55:0/825479121 >> 172.21.15.55:0/825479121 conn(0x7fa2f406f840 msgr2=0x7fa2f410d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 -- 172.21.15.55:0/825479121 shutdown_connections 2024-04-26T21:56:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.491+0000 7fa2e8ff9640 1 -- 172.21.15.55:0/825479121 wait complete. 2024-04-26T21:56:26.513 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:56:26.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.778+0000 7f7eb47bf640 1 -- 172.21.15.55:0/12554279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac10f020 msgr2=0x7f7eac1114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.778+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/12554279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac10f020 0x7f7eac1114e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4005fd0 tx=0x7f7ea402b430 comp rx=0 tx=0).stop 2024-04-26T21:56:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.780+0000 7f7eb47bf640 1 -- 172.21.15.55:0/12554279 shutdown_connections 2024-04-26T21:56:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.780+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/12554279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac10f020 0x7f7eac1114e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.780+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/12554279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eac10c560 0x7f7eac10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.780+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/12554279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eac109ba0 0x7f7eac10bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.780+0000 7f7eb47bf640 1 -- 172.21.15.55:0/12554279 >> 172.21.15.55:0/12554279 conn(0x7f7eac06f7c0 msgr2=0x7f7eac071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.785+0000 7f7eb47bf640 1 -- 172.21.15.55:0/12554279 shutdown_connections 2024-04-26T21:56:26.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.786+0000 7f7eb47bf640 1 -- 172.21.15.55:0/12554279 wait complete. 2024-04-26T21:56:26.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.787+0000 7f7eb47bf640 1 Processor -- start 2024-04-26T21:56:26.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.787+0000 7f7eb47bf640 1 -- start start 2024-04-26T21:56:26.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.788+0000 7f7eb47bf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac109ba0 0x7f7eac19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.788+0000 7f7eb47bf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eac10c560 0x7f7eac19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.789+0000 7f7eb47bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eac10f020 0x7f7eac1b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.789+0000 7f7eb255b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac109ba0 0x7f7eac19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.789+0000 7f7eb255b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac109ba0 0x7f7eac19f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41912/0 (socket says 172.21.15.55:41912) 2024-04-26T21:56:26.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.789+0000 7f7eb255b640 1 -- 172.21.15.55:0/1015047056 learned_addr learned my addr 172.21.15.55:0/1015047056 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.789+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac114bb0 con 0x7f7eac109ba0 2024-04-26T21:56:26.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.789+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac114a30 con 0x7f7eac10f020 2024-04-26T21:56:26.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.789+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac114d30 con 0x7f7eac10c560 2024-04-26T21:56:26.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.791+0000 7f7eb255b640 1 -- 172.21.15.55:0/1015047056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eac10c560 msgr2=0x7f7eac19f8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:56:26.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.791+0000 7f7eb255b640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eac10c560 0x7f7eac19f8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.791+0000 7f7eb255b640 1 -- 172.21.15.55:0/1015047056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eac10f020 msgr2=0x7f7eac1b7ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:56:26.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.791+0000 7f7eb255b640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eac10f020 0x7f7eac1b7ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.791+0000 7f7eb255b640 1 -- 172.21.15.55:0/1015047056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac1b8010 con 0x7f7eac109ba0 2024-04-26T21:56:26.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.792+0000 7f7eb255b640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac109ba0 0x7f7eac19f360 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00b450 tx=0x7f7e9c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.793+0000 7f7ea37fe640 1 -- 172.21.15.55:0/1015047056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c015020 con 0x7f7eac109ba0 2024-04-26T21:56:26.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.793+0000 7f7ea37fe640 1 -- 172.21.15.55:0/1015047056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c011d40 con 0x7f7eac109ba0 2024-04-26T21:56:26.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.793+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac1b82a0 con 0x7f7eac109ba0 2024-04-26T21:56:26.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.793+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac1b87e0 con 0x7f7eac109ba0 2024-04-26T21:56:26.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.794+0000 7f7ea37fe640 1 -- 172.21.15.55:0/1015047056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c010e30 con 0x7f7eac109ba0 2024-04-26T21:56:26.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.804+0000 7f7ea37fe640 1 -- 172.21.15.55:0/1015047056 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7e9c018020 con 0x7f7eac109ba0 2024-04-26T21:56:26.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.805+0000 7f7eb75bc740 1 Processor -- start 2024-04-26T21:56:26.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.806+0000 7f7eb75bc740 1 -- start start 2024-04-26T21:56:26.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.807+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d35c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.807+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d36e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.807+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d3dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.807+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x565504c99d80 con 0x565504d35820 2024-04-26T21:56:26.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.808+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565504c53a20 con 0x565504d373b0 2024-04-26T21:56:26.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.808+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x565504c95d40 con 0x565504d369f0 2024-04-26T21:56:26.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.809+0000 7f7ea37fe640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7e980a3e40 0x7f7e980a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.809+0000 7f7eb1d5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d3dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.809+0000 7f7eb1d5a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d3dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49218/0 (socket says 172.21.15.55:49218) 2024-04-26T21:56:26.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.809+0000 7f7eb1d5a640 1 -- 172.21.15.55:0/237021635 learned_addr learned my addr 172.21.15.55:0/237021635 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.810+0000 7f7eb2d5c640 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d35c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.810+0000 7f7eb255b640 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d36e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.810+0000 7f7eb1d5a640 1 -- 172.21.15.55:0/237021635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 msgr2=0x565504d36e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.810+0000 7f7eb1d5a640 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d36e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.810+0000 7f7eb1d5a640 1 -- 172.21.15.55:0/237021635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 msgr2=0x565504d35c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.810+0000 7f7eb1d5a640 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d35c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.810+0000 7f7eb1d5a640 1 -- 172.21.15.55:0/237021635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565504c5ec30 con 0x565504d373b0 2024-04-26T21:56:26.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.811+0000 7f7eb1d5a640 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d3dc40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009740 tx=0x7f7ea8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f030c929650e2b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.811+0000 7f7eb2d5c640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7e980a3e40 0x7f7e980a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.811+0000 7f7ea37fe640 1 -- 172.21.15.55:0/1015047056 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7e9c0cffa0 con 0x7f7eac109ba0 2024-04-26T21:56:26.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.812+0000 7f7ea0ff9640 1 -- 172.21.15.55:0/237021635 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8015020 con 0x565504d373b0 2024-04-26T21:56:26.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.812+0000 7f7ea0ff9640 1 -- 172.21.15.55:0/237021635 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea8009e50 con 0x565504d373b0 2024-04-26T21:56:26.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.812+0000 7f7ea0ff9640 1 -- 172.21.15.55:0/237021635 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8010380 con 0x565504d373b0 2024-04-26T21:56:26.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 -- 172.21.15.55:0/237021635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 msgr2=0x565504d3dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d3dc40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009740 tx=0x7f7ea8009c00 comp rx=0 tx=0).stop 2024-04-26T21:56:26.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 -- 172.21.15.55:0/237021635 shutdown_connections 2024-04-26T21:56:26.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d3dc40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d36e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/237021635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d35c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 -- 172.21.15.55:0/237021635 >> 172.21.15.55:0/237021635 conn(0x565504c76910 msgr2=0x565504d31120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 -- 172.21.15.55:0/237021635 shutdown_connections 2024-04-26T21:56:26.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.814+0000 7f7eb75bc740 1 -- 172.21.15.55:0/237021635 wait complete. 2024-04-26T21:56:26.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.815+0000 7f7eb75bc740 1 -- start start 2024-04-26T21:56:26.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.815+0000 7f7eb75bc740 1 Processor -- start 2024-04-26T21:56:26.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d35820 0x565504d3fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d40020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb2d5c640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7e980a3e40 0x7f7e980a6300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x565504d368f0 tx=0x7f7ea400aec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d47060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x565504d3e410 con 0x565504d373b0 2024-04-26T21:56:26.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565504c20dc0 con 0x565504d35820 2024-04-26T21:56:26.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x565504d3e590 con 0x565504d369f0 2024-04-26T21:56:26.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb75bc740 10 client.0 _ll_register_callbacks cb 0x565504c3c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb255b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d35820 0x565504d3fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb255b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d35820 0x565504d3fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49222/0 (socket says 172.21.15.55:49222) 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.816+0000 7f7eb255b640 1 -- 172.21.15.55:0/2846093035 learned_addr learned my addr 172.21.15.55:0/2846093035 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.817+0000 7f7eb1d5a640 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d40020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.817+0000 7f7eb2d5c640 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d47060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.817+0000 7f7eb1d5a640 1 -- 172.21.15.55:0/2846093035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d35820 msgr2=0x565504d3fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.817+0000 7f7eb1d5a640 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d35820 0x565504d3fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.817+0000 7f7eb1d5a640 1 -- 172.21.15.55:0/2846093035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 msgr2=0x565504d47060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.817+0000 7f7eb1d5a640 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d47060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.817+0000 7f7eb1d5a640 1 -- 172.21.15.55:0/2846093035 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655049ab3f0 con 0x565504d369f0 2024-04-26T21:56:26.819 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:26 smithi055 ceph-mon[29221]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.83k op/s 2024-04-26T21:56:26.819 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:26 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2724703758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:26.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.818+0000 7f7eb1d5a640 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d40020 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8004810 tx=0x7f7ea8011630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4120bb8c137023df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.818+0000 7f7e95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:56:26.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.819+0000 7f7e95ffb640 1 -- 172.21.15.55:0/2846093035 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800bad0 con 0x565504d369f0 2024-04-26T21:56:26.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.820+0000 7f7eb75bc740 1 -- 172.21.15.55:0/2846093035 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea8009e50 con 0x565504d369f0 2024-04-26T21:56:26.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.820+0000 7f7e95ffb640 1 -- 172.21.15.55:0/2846093035 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7ea800bc70 con 0x565504d369f0 2024-04-26T21:56:26.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.822+0000 7f7e95ffb640 1 -- 172.21.15.55:0/2846093035 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7ea8024470 con 0x565504d369f0 2024-04-26T21:56:26.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.822+0000 7f7eb75bc740 10 client.25417 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:26.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.822+0000 7f7eb75bc740 1 -- 172.21.15.55:0/2846093035 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x565504a757d0 con 0x565504d369f0 2024-04-26T21:56:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.823+0000 7f7e95ffb640 1 -- 172.21.15.55:0/2846093035 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea8009e50 con 0x565504d369f0 2024-04-26T21:56:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.824+0000 7f7e95ffb640 1 -- 172.21.15.55:0/2846093035 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f7ea801b4c0 con 0x565504d369f0 2024-04-26T21:56:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.824+0000 7f7eb75bc740 10 client.25417 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.824+0000 7f7eb75bc740 10 client.25417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:56:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.824+0000 7f7eb75bc740 20 client.25417 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.824+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x565504d4b650 0x565504d4da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.824+0000 7f7eb75bc740 4 client.25417 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.824+0000 7f7eb75bc740 1 -- 172.21.15.55:0/2846093035 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565504ca7990 con 0x565504d4b650 2024-04-26T21:56:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.827+0000 7f7eb255b640 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x565504d4b650 0x565504d4da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.829+0000 7f7e95ffb640 10 client.25417 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:56:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.829+0000 7f7eb255b640 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x565504d4b650 0x565504d4da40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ece0ebb311a47fdb server_cookie=84cc664b7b708816 in_seq=0 out_seq=0 2024-04-26T21:56:26.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.830+0000 7f7e95ffb640 1 -- 172.21.15.55:0/2846093035 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565504ca7990 con 0x565504d4b650 2024-04-26T21:56:26.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.830+0000 7f7e95ffb640 10 client.25417 handle_command_reply: tid=0 2024-04-26T21:56:26.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.830+0000 7f7eb75bc740 1 client.25417 shutdown 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 -- 172.21.15.55:0/2846093035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 msgr2=0x565504d40020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d40020 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8004810 tx=0x7f7ea8011630 comp rx=0 tx=0).stop 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 -- 172.21.15.55:0/2846093035 shutdown_connections 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x565504d4b650 0x565504d4da40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d47060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d40020 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/2846093035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d35820 0x565504d3fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 -- 172.21.15.55:0/2846093035 >> 172.21.15.55:0/2846093035 conn(0x565504c76910 msgr2=0x565504d30280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7e95ffb640 0 client.25417 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:56:26.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 -- 172.21.15.55:0/2846093035 shutdown_connections 2024-04-26T21:56:26.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 1 -- 172.21.15.55:0/2846093035 wait complete. 2024-04-26T21:56:26.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.831+0000 7f7eb75bc740 20 client.25417 trim_cache size 0 max 16384 2024-04-26T21:56:26.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.868+0000 7f7eb75bc740 1 Processor -- start 2024-04-26T21:56:26.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.868+0000 7f7eb75bc740 1 -- start start 2024-04-26T21:56:26.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.868+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d33ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.868+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d47020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.868+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d34430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.868+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1c3fc0 con 0x565504d35820 2024-04-26T21:56:26.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.868+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1c1890 con 0x565504d373b0 2024-04-26T21:56:26.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.868+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1bf6e0 con 0x565504d369f0 2024-04-26T21:56:26.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.869+0000 7f7eb255b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d33ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.869+0000 7f7eb255b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d33ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41938/0 (socket says 172.21.15.55:41938) 2024-04-26T21:56:26.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.869+0000 7f7eb255b640 1 -- 172.21.15.55:0/326764891 learned_addr learned my addr 172.21.15.55:0/326764891 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.869+0000 7f7eb1d5a640 1 --2- 172.21.15.55:0/326764891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d47020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.870+0000 7f7eb255b640 1 -- 172.21.15.55:0/326764891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 msgr2=0x565504d47020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.870+0000 7f7eb255b640 1 --2- 172.21.15.55:0/326764891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d47020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.870+0000 7f7eb255b640 1 -- 172.21.15.55:0/326764891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 msgr2=0x565504d34430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:26.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.870+0000 7f7eb255b640 1 --2- 172.21.15.55:0/326764891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d34430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.870+0000 7f7eb255b640 1 -- 172.21.15.55:0/326764891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565504d3d510 con 0x565504d35820 2024-04-26T21:56:26.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.870+0000 7f7eb255b640 1 --2- 172.21.15.55:0/326764891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d33ef0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009fd0 tx=0x7f7e9c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a4281aaffc7d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.871+0000 7f7e97fff640 1 -- 172.21.15.55:0/326764891 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c09c890 con 0x565504d35820 2024-04-26T21:56:26.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.871+0000 7f7e97fff640 1 -- 172.21.15.55:0/326764891 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c09c190 con 0x565504d35820 2024-04-26T21:56:26.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.871+0000 7f7e97fff640 1 -- 172.21.15.55:0/326764891 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c09a520 con 0x565504d35820 2024-04-26T21:56:26.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.873+0000 7f7eb75bc740 1 -- 172.21.15.55:0/326764891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 msgr2=0x565504d33ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.873+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/326764891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d33ef0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009fd0 tx=0x7f7e9c01c040 comp rx=0 tx=0).stop 2024-04-26T21:56:26.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.874+0000 7f7eb75bc740 1 -- 172.21.15.55:0/326764891 shutdown_connections 2024-04-26T21:56:26.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.874+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/326764891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d373b0 0x565504d34430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.874+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/326764891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d369f0 0x565504d47020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.874+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/326764891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d35820 0x565504d33ef0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.874+0000 7f7eb75bc740 1 -- 172.21.15.55:0/326764891 >> 172.21.15.55:0/326764891 conn(0x565504c76910 msgr2=0x565504d3a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.875+0000 7f7eb75bc740 1 -- 172.21.15.55:0/326764891 shutdown_connections 2024-04-26T21:56:26.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.875+0000 7f7eb75bc740 1 -- 172.21.15.55:0/326764891 wait complete. 2024-04-26T21:56:26.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.875+0000 7f7eb75bc740 1 -- start start 2024-04-26T21:56:26.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.875+0000 7f7eb75bc740 1 Processor -- start 2024-04-26T21:56:26.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.876+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d35820 0x565504d44710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.876+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d369f0 0x565504d44c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.876+0000 7f7eb75bc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d45ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.876+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1bfd50 con 0x565504d373b0 2024-04-26T21:56:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.876+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1c11c0 con 0x565504d369f0 2024-04-26T21:56:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.876+0000 7f7eb75bc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac1c0420 con 0x565504d35820 2024-04-26T21:56:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.876+0000 7f7eb75bc740 10 client.0 _ll_register_callbacks cb 0x7f7e9c0d3980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.877+0000 7f7eb2d5c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d45ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.877+0000 7f7eb2d5c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d45ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41950/0 (socket says 172.21.15.55:41950) 2024-04-26T21:56:26.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.877+0000 7f7eb2d5c640 1 -- 172.21.15.55:0/4226362287 learned_addr learned my addr 172.21.15.55:0/4226362287 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:26.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.878+0000 7f7eb2d5c640 1 -- 172.21.15.55:0/4226362287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d35820 msgr2=0x565504d44710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:26.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.878+0000 7f7eb2d5c640 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d35820 0x565504d44710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.878+0000 7f7eb2d5c640 1 -- 172.21.15.55:0/4226362287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d369f0 msgr2=0x565504d44c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:26.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.878+0000 7f7eb2d5c640 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d369f0 0x565504d44c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.878+0000 7f7eb2d5c640 1 -- 172.21.15.55:0/4226362287 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565504d46190 con 0x565504d373b0 2024-04-26T21:56:26.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.878+0000 7f7eb2d5c640 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d45ba0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4005fd0 tx=0x7f7ea402b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1603183df24d1427 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:26.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.878+0000 7f7e95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:56:26.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.879+0000 7f7e95ffb640 1 -- 172.21.15.55:0/4226362287 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea404c020 con 0x565504d373b0 2024-04-26T21:56:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.879+0000 7f7eb75bc740 1 -- 172.21.15.55:0/4226362287 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9c09c190 con 0x565504d373b0 2024-04-26T21:56:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.879+0000 7f7e95ffb640 1 -- 172.21.15.55:0/4226362287 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7ea4004250 con 0x565504d373b0 2024-04-26T21:56:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.880+0000 7f7e95ffb640 1 -- 172.21.15.55:0/4226362287 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7ea4057050 con 0x565504d373b0 2024-04-26T21:56:26.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.880+0000 7f7eb75bc740 10 client.15966 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.880+0000 7f7eb75bc740 1 -- 172.21.15.55:0/4226362287 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x565504db1b80 con 0x565504d373b0 2024-04-26T21:56:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.881+0000 7f7e95ffb640 1 -- 172.21.15.55:0/4226362287 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c09c190 con 0x565504d373b0 2024-04-26T21:56:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.881+0000 7f7e95ffb640 1 -- 172.21.15.55:0/4226362287 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f7ea400a6f0 con 0x565504d373b0 2024-04-26T21:56:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.881+0000 7f7eb75bc740 10 client.15966 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.881+0000 7f7eb75bc740 10 client.15966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:56:26.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.881+0000 7f7eb75bc740 20 client.15966 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.881+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x565504db2580 0x565504db49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.881+0000 7f7eb75bc740 4 client.15966 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:26.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.881+0000 7f7eb75bc740 1 -- 172.21.15.55:0/4226362287 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565504ca1800 con 0x565504db2580 2024-04-26T21:56:26.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.885+0000 7f7eb255b640 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x565504db2580 0x565504db49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.891+0000 7f7eb255b640 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x565504db2580 0x565504db49a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e62f3999b423f11e server_cookie=8cd0a93c7aed3cf9 in_seq=0 out_seq=0 2024-04-26T21:56:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.891+0000 7f7e95ffb640 10 client.15966 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:56:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.891+0000 7f7e95ffb640 1 -- 172.21.15.55:0/4226362287 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565504ca1800 con 0x565504db2580 2024-04-26T21:56:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.891+0000 7f7e95ffb640 10 client.15966 handle_command_reply: tid=0 2024-04-26T21:56:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.891+0000 7f7eb75bc740 1 client.15966 shutdown 2024-04-26T21:56:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7eb75bc740 1 -- 172.21.15.55:0/4226362287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 msgr2=0x565504d45ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d45ba0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4005fd0 tx=0x7f7ea402b430 comp rx=0 tx=0).stop 2024-04-26T21:56:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7eb75bc740 1 -- 172.21.15.55:0/4226362287 shutdown_connections 2024-04-26T21:56:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x565504db2580 0x565504db49a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565504d373b0 0x565504d45ba0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565504d369f0 0x565504d44c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7eb75bc740 1 --2- 172.21.15.55:0/4226362287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565504d35820 0x565504d44710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7eb75bc740 1 -- 172.21.15.55:0/4226362287 >> 172.21.15.55:0/4226362287 conn(0x565504c76910 msgr2=0x565504d2f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.892+0000 7f7e95ffb640 0 client.15966 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:56:26.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.898+0000 7f7eb75bc740 1 -- 172.21.15.55:0/4226362287 shutdown_connections 2024-04-26T21:56:26.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.898+0000 7f7eb75bc740 1 -- 172.21.15.55:0/4226362287 wait complete. 2024-04-26T21:56:26.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.898+0000 7f7eb75bc740 20 client.15966 trim_cache size 0 max 16384 2024-04-26T21:56:26.900 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:56:26.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.913+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7e980a3e40 msgr2=0x7f7e980a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.913+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7e980a3e40 0x7f7e980a6300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x565504d368f0 tx=0x7f7ea400aec0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.913+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac109ba0 msgr2=0x7f7eac19f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:26.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.913+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac109ba0 0x7f7eac19f360 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00b450 tx=0x7f7e9c00b910 comp rx=0 tx=0).stop 2024-04-26T21:56:26.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.914+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 shutdown_connections 2024-04-26T21:56:26.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.914+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7e980a3e40 0x7f7e980a6300 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.914+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eac10f020 0x7f7eac1b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.914+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7eac10c560 0x7f7eac19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.914+0000 7f7eb47bf640 1 --2- 172.21.15.55:0/1015047056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7eac109ba0 0x7f7eac19f360 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:26.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.914+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 >> 172.21.15.55:0/1015047056 conn(0x7f7eac06f7c0 msgr2=0x7f7eac109f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:26.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.914+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 shutdown_connections 2024-04-26T21:56:26.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:26.914+0000 7f7eb47bf640 1 -- 172.21.15.55:0/1015047056 wait complete. 2024-04-26T21:56:26.958 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:56:27.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:26 smithi165 ceph-mon[33296]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.83k op/s 2024-04-26T21:56:27.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:26 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2724703758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:26 smithi064 ceph-mon[32765]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.83k op/s 2024-04-26T21:56:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:26 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2724703758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:56:27.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.343+0000 7f7bd8b16640 1 -- 172.21.15.55:0/3244130111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc0a5d70 msgr2=0x7f7bcc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.343+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/3244130111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc0a5d70 0x7f7bcc0a61d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009900 tx=0x7f7bc002a440 comp rx=0 tx=0).stop 2024-04-26T21:56:27.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.344+0000 7f7bd8b16640 1 -- 172.21.15.55:0/3244130111 shutdown_connections 2024-04-26T21:56:27.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.344+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/3244130111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bcc0a6710 0x7f7bcc0aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.344+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/3244130111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc0a5d70 0x7f7bcc0a61d0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009900 tx=0x7f7bc002a440 comp rx=0 tx=0).stop 2024-04-26T21:56:27.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.344+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/3244130111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bcc0a4c40 0x7f7bcc0a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.344+0000 7f7bd8b16640 1 -- 172.21.15.55:0/3244130111 >> 172.21.15.55:0/3244130111 conn(0x7f7bcc0a00d0 msgr2=0x7f7bcc0a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:27.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.351+0000 7f7bd8b16640 1 -- 172.21.15.55:0/3244130111 shutdown_connections 2024-04-26T21:56:27.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.352+0000 7f7bd8b16640 1 -- 172.21.15.55:0/3244130111 wait complete. 2024-04-26T21:56:27.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.353+0000 7f7bd8b16640 1 Processor -- start 2024-04-26T21:56:27.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.354+0000 7f7bd8b16640 1 -- start start 2024-04-26T21:56:27.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.354+0000 7f7bd8b16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc0a4c40 0x7f7bcc0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.354+0000 7f7bd8b16640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bcc0a6710 0x7f7bcc0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.354+0000 7f7bd8b16640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bcc0b30b0 0x7f7bcc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.354+0000 7f7bd8b16640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc0b8fd0 con 0x7f7bcc0a6710 2024-04-26T21:56:27.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.354+0000 7f7bd8b16640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc0b8e50 con 0x7f7bcc0a4c40 2024-04-26T21:56:27.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.354+0000 7f7bd8b16640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc0b9150 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.357+0000 7f7bd2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bcc0a6710 0x7f7bcc0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.357+0000 7f7bd2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bcc0a6710 0x7f7bcc0b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41970/0 (socket says 172.21.15.55:41970) 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.357+0000 7f7bd2ffd640 1 -- 172.21.15.55:0/2595686437 learned_addr learned my addr 172.21.15.55:0/2595686437 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.358+0000 7f7bd3fff640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bcc0b30b0 0x7f7bcc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.358+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc0a4c40 0x7f7bcc0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.359+0000 7f7bd3fff640 1 -- 172.21.15.55:0/2595686437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc0a4c40 msgr2=0x7f7bcc0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.359+0000 7f7bd3fff640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc0a4c40 0x7f7bcc0b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.359+0000 7f7bd3fff640 1 -- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bcc0a6710 msgr2=0x7f7bcc0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.359+0000 7f7bd3fff640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bcc0a6710 0x7f7bcc0b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.359+0000 7f7bd3fff640 1 -- 172.21.15.55:0/2595686437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc0b3b20 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.359+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bcc0a6710 0x7f7bcc0b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:56:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.360+0000 7f7bd3fff640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bcc0b30b0 0x7f7bcc0b3560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4051fc0 tx=0x7f7bd4050dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:27.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.361+0000 7f7bd0ff9640 1 -- 172.21.15.55:0/2595686437 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd407a020 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.362+0000 7f7bd8b16640 1 -- 172.21.15.55:0/2595686437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc0b3e10 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.362+0000 7f7bd8b16640 1 -- 172.21.15.55:0/2595686437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc0b4360 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.369+0000 7f7bdcb6f740 1 Processor -- start 2024-04-26T21:56:27.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.369+0000 7f7bd0ff9640 1 -- 172.21.15.55:0/2595686437 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd40768d0 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.371+0000 7f7bdcb6f740 1 -- start start 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f226d80 con 0x55ee0f2c4590 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f1e0a20 con 0x55ee0f2c2a00 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f222d40 con 0x55ee0f2c3bd0 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bd2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bd2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41976/0 (socket says 172.21.15.55:41976) 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bd2ffd640 1 -- 172.21.15.55:0/2549694384 learned_addr learned my addr 172.21.15.55:0/2549694384 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.372+0000 7f7bd0ff9640 1 -- 172.21.15.55:0/2595686437 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40756d0 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.373+0000 7f7bd0ff9640 1 -- 172.21.15.55:0/2595686437 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7bd407d020 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.373+0000 7f7bd2ffd640 1 -- 172.21.15.55:0/2549694384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c3bd0 msgr2=0x55ee0f2c4050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.373+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/2549694384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2c4050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.373+0000 7f7bd2ffd640 1 -- 172.21.15.55:0/2549694384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 msgr2=0x55ee0f2c2e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.373+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/2549694384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2c2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.373+0000 7f7bd2ffd640 1 -- 172.21.15.55:0/2549694384 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0f1ebc30 con 0x55ee0f2c4590 2024-04-26T21:56:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.373+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/2549694384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cae20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0000c00 tx=0x7f7bc00315e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83d7ede19b4ffef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.374+0000 7f7bbdffb640 1 -- 172.21.15.55:0/2549694384 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0035050 con 0x55ee0f2c4590 2024-04-26T21:56:27.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.374+0000 7f7bbdffb640 1 -- 172.21.15.55:0/2549694384 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc003f070 con 0x55ee0f2c4590 2024-04-26T21:56:27.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.376+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/2549694384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 msgr2=0x55ee0f2cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.376+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/2549694384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cae20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0000c00 tx=0x7f7bc00315e0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.379+0000 7f7bd0ff9640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7bc40a3ba0 0x7f7bc40a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.379+0000 7f7bd0ff9640 1 -- 172.21.15.55:0/2595686437 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7bd4135300 con 0x7f7bcc0b30b0 2024-04-26T21:56:27.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.381+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7bc40a3ba0 0x7f7bc40a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.381+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/2549694384 shutdown_connections 2024-04-26T21:56:27.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.381+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/2549694384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cae20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.381+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/2549694384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.381+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/2549694384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.381+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/2549694384 >> 172.21.15.55:0/2549694384 conn(0x55ee0f203910 msgr2=0x55ee0f2be2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:27.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.381+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/2549694384 shutdown_connections 2024-04-26T21:56:27.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.381+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/2549694384 wait complete. 2024-04-26T21:56:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.382+0000 7f7bdcb6f740 1 -- start start 2024-04-26T21:56:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.382+0000 7f7bdcb6f740 1 Processor -- start 2024-04-26T21:56:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.383+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.383+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.383+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.383+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f2cb590 con 0x55ee0f2c4590 2024-04-26T21:56:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.383+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f1addc0 con 0x55ee0f2c3bd0 2024-04-26T21:56:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.383+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f2cb710 con 0x55ee0f2c2a00 2024-04-26T21:56:27.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.383+0000 7f7bdcb6f740 10 client.0 _ll_register_callbacks cb 0x55ee0f1c9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:27.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.384+0000 7f7bd2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.384+0000 7f7bd37fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.384+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7bc40a3ba0 0x7f7bc40a6060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55ee0f2bf1b0 tx=0x7f7bc80090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:27.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.384+0000 7f7bd3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.384+0000 7f7bd3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41984/0 (socket says 172.21.15.55:41984) 2024-04-26T21:56:27.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.384+0000 7f7bd3fff640 1 -- 172.21.15.55:0/546257147 learned_addr learned my addr 172.21.15.55:0/546257147 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.386+0000 7f7bd37fe640 1 -- 172.21.15.55:0/546257147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 msgr2=0x55ee0f2cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.386+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.386+0000 7f7bd37fe640 1 -- 172.21.15.55:0/546257147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 msgr2=0x55ee0f2d4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.386+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.386+0000 7f7bd37fe640 1 -- 172.21.15.55:0/546257147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0ef383f0 con 0x55ee0f2c2a00 2024-04-26T21:56:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.386+0000 7f7bd3fff640 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:56:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.386+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:56:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.388+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2ccc00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006fd0 tx=0x7f7bc8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bf2e14abd7f9389 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.391+0000 7f7bb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:56:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.391+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/546257147 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8023020 con 0x55ee0f2c2a00 2024-04-26T21:56:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.391+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/546257147 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7bc8013450 con 0x55ee0f2c2a00 2024-04-26T21:56:27.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.394+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/546257147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc003f070 con 0x55ee0f2c2a00 2024-04-26T21:56:27.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.395+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/546257147 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7bc801b040 con 0x55ee0f2c2a00 2024-04-26T21:56:27.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.397+0000 7f7bdcb6f740 10 client.25426 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:27.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.397+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/546257147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ee0f0027d0 con 0x55ee0f2c2a00 2024-04-26T21:56:27.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.410+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/546257147 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc003f070 con 0x55ee0f2c2a00 2024-04-26T21:56:27.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.410+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/546257147 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f7bc801e030 con 0x55ee0f2c2a00 2024-04-26T21:56:27.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.410+0000 7f7bdcb6f740 10 client.25426 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:27.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.410+0000 7f7bdcb6f740 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:27.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.410+0000 7f7bdcb6f740 20 client.25426 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:27.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.410+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee0f2fab70 0x55ee0f2fd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.410+0000 7f7bdcb6f740 4 client.25426 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:27.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.410+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/546257147 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee0f234990 con 0x55ee0f2fab70 2024-04-26T21:56:27.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.414+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee0f2fab70 0x55ee0f2fd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.422+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee0f2fab70 0x55ee0f2fd010 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=643a0ac02c490a8d server_cookie=b2a343832ec0514b in_seq=0 out_seq=0 2024-04-26T21:56:27.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.423+0000 7f7bb2ffd640 10 client.25426 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:27.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.428+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/546257147 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee0f234990 con 0x55ee0f2fab70 2024-04-26T21:56:27.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.428+0000 7f7bb2ffd640 10 client.25426 handle_command_reply: tid=0 2024-04-26T21:56:27.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.430+0000 7f7bdcb6f740 1 client.25426 shutdown 2024-04-26T21:56:27.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.432+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/546257147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 msgr2=0x55ee0f2ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.432+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2ccc00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006fd0 tx=0x7f7bc8004770 comp rx=0 tx=0).stop 2024-04-26T21:56:27.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.438+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/546257147 shutdown_connections 2024-04-26T21:56:27.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.438+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee0f2fab70 0x55ee0f2fd010 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.438+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.438+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.438+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/546257147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2ccc00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.438+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/546257147 >> 172.21.15.55:0/546257147 conn(0x55ee0f203910 msgr2=0x55ee0f2bd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:27.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.438+0000 7f7bb2ffd640 0 client.25426 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:27.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.439+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/546257147 shutdown_connections 2024-04-26T21:56:27.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.441+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/546257147 wait complete. 2024-04-26T21:56:27.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.441+0000 7f7bdcb6f740 20 client.25426 trim_cache size 0 max 16384 2024-04-26T21:56:27.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.512+0000 7f7bdcb6f740 1 Processor -- start 2024-04-26T21:56:27.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.512+0000 7f7bdcb6f740 1 -- start start 2024-04-26T21:56:27.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.512+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2be440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.513+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.513+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.513+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8010f70 con 0x55ee0f2c3bd0 2024-04-26T21:56:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.513+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb800e840 con 0x55ee0f2c2a00 2024-04-26T21:56:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.513+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb800c690 con 0x55ee0f2c4590 2024-04-26T21:56:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.516+0000 7f7bd3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.516+0000 7f7bd3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:54544/0 (socket says 172.21.15.55:54544) 2024-04-26T21:56:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.516+0000 7f7bd3fff640 1 -- 172.21.15.55:0/105364113 learned_addr learned my addr 172.21.15.55:0/105364113 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.517+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2be440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.521+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.524+0000 7f7bd3fff640 1 -- 172.21.15.55:0/105364113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 msgr2=0x55ee0f2be440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:27.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.524+0000 7f7bd3fff640 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2be440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.524+0000 7f7bd3fff640 1 -- 172.21.15.55:0/105364113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c3bd0 msgr2=0x55ee0f2d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:27.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.524+0000 7f7bd3fff640 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2d4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.524+0000 7f7bd3fff640 1 -- 172.21.15.55:0/105364113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0f2b9cb0 con 0x55ee0f2c4590 2024-04-26T21:56:27.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.529+0000 7f7bd3fff640 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cc810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40684a0 tx=0x7f7bd4068100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a474a7293b57fcc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.533+0000 7f7bbd7fa640 1 -- 172.21.15.55:0/105364113 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4119070 con 0x55ee0f2c4590 2024-04-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.533+0000 7f7bbd7fa640 1 -- 172.21.15.55:0/105364113 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd41058a0 con 0x55ee0f2c4590 2024-04-26T21:56:27.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.533+0000 7f7bbd7fa640 1 -- 172.21.15.55:0/105364113 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40fdcf0 con 0x55ee0f2c4590 2024-04-26T21:56:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.536+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/105364113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c4590 msgr2=0x55ee0f2cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.536+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cc810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40684a0 tx=0x7f7bd4068100 comp rx=0 tx=0).stop 2024-04-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.540+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/105364113 shutdown_connections 2024-04-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.540+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c4590 0x55ee0f2cc810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.540+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2d4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.540+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/105364113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2be440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.540+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/105364113 >> 172.21.15.55:0/105364113 conn(0x55ee0f203910 msgr2=0x55ee0f2bdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:27.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.549+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/105364113 shutdown_connections 2024-04-26T21:56:27.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.552+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/105364113 wait complete. 2024-04-26T21:56:27.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.553+0000 7f7bdcb6f740 1 -- start start 2024-04-26T21:56:27.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.554+0000 7f7bdcb6f740 1 Processor -- start 2024-04-26T21:56:27.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.554+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.555+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.555+0000 7f7bdcb6f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.555+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb800cd00 con 0x55ee0f2c4590 2024-04-26T21:56:27.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.555+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb800e170 con 0x55ee0f2c3bd0 2024-04-26T21:56:27.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.555+0000 7f7bdcb6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb800d3d0 con 0x55ee0f2c2a00 2024-04-26T21:56:27.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.555+0000 7f7bdcb6f740 10 client.0 _ll_register_callbacks cb 0x7f7bc8009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.558+0000 7f7bd3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.558+0000 7f7bd3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41998/0 (socket says 172.21.15.55:41998) 2024-04-26T21:56:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.558+0000 7f7bd3fff640 1 -- 172.21.15.55:0/166905129 learned_addr learned my addr 172.21.15.55:0/166905129 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:27.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.559+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.561+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.564+0000 7f7bd37fe640 1 -- 172.21.15.55:0/166905129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 msgr2=0x55ee0f2d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:27.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.564+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.564+0000 7f7bd37fe640 1 -- 172.21.15.55:0/166905129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 msgr2=0x55ee0f2d2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:27.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.564+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.564+0000 7f7bd37fe640 1 -- 172.21.15.55:0/166905129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0f2d2e00 con 0x55ee0f2c2a00 2024-04-26T21:56:27.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.570+0000 7f7bd37fe640 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2d12c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7bc802a040 tx=0x7f7bc8028d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20075b3e85d1ffb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:27.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.570+0000 7f7bb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:56:27.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.571+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/166905129 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8023040 con 0x55ee0f2c2a00 2024-04-26T21:56:27.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.573+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/166905129 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7bc8004230 con 0x55ee0f2c2a00 2024-04-26T21:56:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.573+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/166905129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bd41058a0 con 0x55ee0f2c2a00 2024-04-26T21:56:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.574+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/166905129 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7bc803f050 con 0x55ee0f2c2a00 2024-04-26T21:56:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.575+0000 7f7bdcb6f740 10 client.25432 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.575+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/166905129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ee0f2d4100 con 0x55ee0f2c2a00 2024-04-26T21:56:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.579+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/166905129 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bd41058a0 con 0x55ee0f2c2a00 2024-04-26T21:56:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.579+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/166905129 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f7bc801e070 con 0x55ee0f2c2a00 2024-04-26T21:56:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.581+0000 7f7bdcb6f740 10 client.25432 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.581+0000 7f7bdcb6f740 10 client.25432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.581+0000 7f7bdcb6f740 20 client.25432 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.581+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee0f3616c0 0x55ee0f363ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.581+0000 7f7bdcb6f740 4 client.25432 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:56:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.581+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/166905129 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee0f22e800 con 0x55ee0f3616c0 2024-04-26T21:56:27.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.585+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee0f3616c0 0x55ee0f363ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:27.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.601+0000 7f7bb2ffd640 10 client.25432 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:27.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.601+0000 7f7bd2ffd640 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee0f3616c0 0x55ee0f363ae0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11cac3e4e6db2319 server_cookie=8a3ebab58faf3250 in_seq=0 out_seq=0 2024-04-26T21:56:27.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.606+0000 7f7bb2ffd640 1 -- 172.21.15.55:0/166905129 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee0f22e800 con 0x55ee0f3616c0 2024-04-26T21:56:27.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.606+0000 7f7bb2ffd640 10 client.25432 handle_command_reply: tid=0 2024-04-26T21:56:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.607+0000 7f7bdcb6f740 1 client.25432 shutdown 2024-04-26T21:56:27.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.617+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/166905129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 msgr2=0x55ee0f2d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.617+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2d12c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7bc802a040 tx=0x7f7bc8028d50 comp rx=0 tx=0).stop 2024-04-26T21:56:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.624+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/166905129 shutdown_connections 2024-04-26T21:56:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.625+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee0f3616c0 0x55ee0f363ae0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.625+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee0f2c4590 0x55ee0f2d2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.625+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee0f2c3bd0 0x55ee0f2d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.625+0000 7f7bdcb6f740 1 --2- 172.21.15.55:0/166905129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee0f2c2a00 0x55ee0f2d12c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.625+0000 7f7bb2ffd640 0 client.25432 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:27.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.625+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/166905129 >> 172.21.15.55:0/166905129 conn(0x55ee0f203910 msgr2=0x55ee0f2cc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:27.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.634+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/166905129 shutdown_connections 2024-04-26T21:56:27.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.639+0000 7f7bdcb6f740 1 -- 172.21.15.55:0/166905129 wait complete. 2024-04-26T21:56:27.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.643+0000 7f7bdcb6f740 20 client.25432 trim_cache size 0 max 16384 2024-04-26T21:56:27.646 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:56:27.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.651+0000 7f7bd8b16640 1 -- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7bc40a3ba0 msgr2=0x7f7bc40a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.651+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7bc40a3ba0 0x7f7bc40a6060 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55ee0f2bf1b0 tx=0x7f7bc80090e0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.651+0000 7f7bd8b16640 1 -- 172.21.15.55:0/2595686437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bcc0b30b0 msgr2=0x7f7bcc0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:27.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.652+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bcc0b30b0 0x7f7bcc0b3560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4051fc0 tx=0x7f7bd4050dc0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.652+0000 7f7bd8b16640 1 -- 172.21.15.55:0/2595686437 shutdown_connections 2024-04-26T21:56:27.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.652+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7bc40a3ba0 0x7f7bc40a6060 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55ee0f2bf1b0 tx=0x7f7bc80090e0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.652+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bcc0b30b0 0x7f7bcc0b3560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.652+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bcc0a6710 0x7f7bcc0b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.652+0000 7f7bd8b16640 1 --2- 172.21.15.55:0/2595686437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bcc0a4c40 0x7f7bcc0b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:27.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.652+0000 7f7bd8b16640 1 -- 172.21.15.55:0/2595686437 >> 172.21.15.55:0/2595686437 conn(0x7f7bcc0a00d0 msgr2=0x7f7bcc0a0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:27.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.657+0000 7f7bd8b16640 1 -- 172.21.15.55:0/2595686437 shutdown_connections 2024-04-26T21:56:27.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:27.660+0000 7f7bd8b16640 1 -- 172.21.15.55:0/2595686437 wait complete. 2024-04-26T21:56:28.712 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:56:28.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:56:28.713 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:56:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.043+0000 7fb53ea20640 1 -- 172.21.15.55:0/3919396436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb53810ab40 msgr2=0x7fb5381113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.043+0000 7fb53ea20640 1 --2- 172.21.15.55:0/3919396436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb53810ab40 0x7fb5381113d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb530009ca0 tx=0x7fb53002a8c0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.043+0000 7fb53ea20640 1 -- 172.21.15.55:0/3919396436 shutdown_connections 2024-04-26T21:56:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.043+0000 7fb53ea20640 1 --2- 172.21.15.55:0/3919396436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb53810ab40 0x7fb5381113d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.043+0000 7fb53ea20640 1 --2- 172.21.15.55:0/3919396436 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb53810a180 0x7fb53810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.043+0000 7fb53ea20640 1 --2- 172.21.15.55:0/3919396436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb538108f90 0x7fb5381093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.043+0000 7fb53ea20640 1 -- 172.21.15.55:0/3919396436 >> 172.21.15.55:0/3919396436 conn(0x7fb538077f30 msgr2=0x7fb53807a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:29.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.044+0000 7fb53ea20640 1 -- 172.21.15.55:0/3919396436 shutdown_connections 2024-04-26T21:56:29.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.044+0000 7fb53ea20640 1 -- 172.21.15.55:0/3919396436 wait complete. 2024-04-26T21:56:29.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.045+0000 7fb53ea20640 1 Processor -- start 2024-04-26T21:56:29.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.046+0000 7fb53ea20640 1 -- start start 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.047+0000 7fb53ea20640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb538108f90 0x7fb53819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.047+0000 7fb53ea20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb53810a180 0x7fb53819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.047+0000 7fb53ea20640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb53810ab40 0x7fb5381a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.047+0000 7fb53ea20640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb538114b20 con 0x7fb538108f90 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.047+0000 7fb53ea20640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5381149a0 con 0x7fb53810a180 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb53ea20640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb538114ca0 con 0x7fb53810ab40 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb537fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb538108f90 0x7fb53819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb537fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb538108f90 0x7fb53819f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42016/0 (socket says 172.21.15.55:42016) 2024-04-26T21:56:29.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb537fff640 1 -- 172.21.15.55:0/1646220295 learned_addr learned my addr 172.21.15.55:0/1646220295 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:29.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb53cfbd640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb53810ab40 0x7fb5381a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb537fff640 1 -- 172.21.15.55:0/1646220295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb53810ab40 msgr2=0x7fb5381a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:29.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb537fff640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb53810ab40 0x7fb5381a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb537fff640 1 -- 172.21.15.55:0/1646220295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb53810a180 msgr2=0x7fb53819f690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:56:29.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.048+0000 7fb537fff640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb53810a180 0x7fb53819f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.049+0000 7fb537fff640 1 -- 172.21.15.55:0/1646220295 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5381a6d70 con 0x7fb538108f90 2024-04-26T21:56:29.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.049+0000 7fb53cfbd640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb53810ab40 0x7fb5381a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:56:29.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.049+0000 7fb537fff640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb538108f90 0x7fb53819f150 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d950 tx=0x7fb52800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:29.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.049+0000 7fb5357fa640 1 -- 172.21.15.55:0/1646220295 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528017020 con 0x7fb538108f90 2024-04-26T21:56:29.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.049+0000 7fb53ea20640 1 -- 172.21.15.55:0/1646220295 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5381a7060 con 0x7fb538108f90 2024-04-26T21:56:29.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.050+0000 7fb53ea20640 1 -- 172.21.15.55:0/1646220295 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5381a7600 con 0x7fb538108f90 2024-04-26T21:56:29.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.050+0000 7fb5357fa640 1 -- 172.21.15.55:0/1646220295 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb528009c30 con 0x7fb538108f90 2024-04-26T21:56:29.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.051+0000 7fb5357fa640 1 -- 172.21.15.55:0/1646220295 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52800f5e0 con 0x7fb538108f90 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.054+0000 7fb541811740 1 Processor -- start 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.054+0000 7fb541811740 1 -- start start 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.054+0000 7fb541811740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6be8b0 0x55613f6becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.054+0000 7fb541811740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6bf2a0 0x55613f6c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.054+0000 7fb541811740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c4410 0x55613f6c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.054+0000 7fb541811740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55613f623b00 con 0x55613f6bf2a0 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.054+0000 7fb541811740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55613f5bf920 con 0x55613f6be8b0 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.054+0000 7fb541811740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55613f61fac0 con 0x55613f6c4410 2024-04-26T21:56:29.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.055+0000 7fb5357fa640 1 -- 172.21.15.55:0/1646220295 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb52801a020 con 0x7fb538108f90 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.055+0000 7fb537fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6bf2a0 0x55613f6c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.055+0000 7fb537fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6bf2a0 0x55613f6c3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42024/0 (socket says 172.21.15.55:42024) 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.055+0000 7fb537fff640 1 -- 172.21.15.55:0/200185885 learned_addr learned my addr 172.21.15.55:0/200185885 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.055+0000 7fb5377fe640 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c4410 0x55613f6c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.055+0000 7fb53cfbd640 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6be8b0 0x55613f6becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb537fff640 1 -- 172.21.15.55:0/200185885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c4410 msgr2=0x55613f6c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb537fff640 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c4410 0x55613f6c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb537fff640 1 -- 172.21.15.55:0/200185885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6be8b0 msgr2=0x55613f6becd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb537fff640 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6be8b0 0x55613f6becd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb537fff640 1 -- 172.21.15.55:0/200185885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613f3fd7e0 con 0x55613f6bf2a0 2024-04-26T21:56:29.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb5357fa640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5200a3cc0 0x7fb5200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb537fff640 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6bf2a0 0x55613f6c3da0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cd620 tx=0x7fb5280d84b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3259475c3d53ad2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:29.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb53cfbd640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5200a3cc0 0x7fb5200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.056+0000 7fb5357fa640 1 -- 172.21.15.55:0/1646220295 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb5280cfd70 con 0x7fb538108f90 2024-04-26T21:56:29.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.057+0000 7fb51e7fc640 1 -- 172.21.15.55:0/200185885 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528098150 con 0x55613f6bf2a0 2024-04-26T21:56:29.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.057+0000 7fb51e7fc640 1 -- 172.21.15.55:0/200185885 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52809c5b0 con 0x55613f6bf2a0 2024-04-26T21:56:29.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.057+0000 7fb51e7fc640 1 -- 172.21.15.55:0/200185885 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528098150 con 0x55613f6bf2a0 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 -- 172.21.15.55:0/200185885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6bf2a0 msgr2=0x55613f6c3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6bf2a0 0x55613f6c3da0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cd620 tx=0x7fb5280d84b0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb53cfbd640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5200a3cc0 0x7fb5200a6180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55613f6bfb00 tx=0x7fb530031f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 -- 172.21.15.55:0/200185885 shutdown_connections 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c4410 0x55613f6c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6bf2a0 0x55613f6c3da0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 --2- 172.21.15.55:0/200185885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6be8b0 0x55613f6becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 -- 172.21.15.55:0/200185885 >> 172.21.15.55:0/200185885 conn(0x55613f6b9f20 msgr2=0x55613f6ba320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 -- 172.21.15.55:0/200185885 shutdown_connections 2024-04-26T21:56:29.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.059+0000 7fb541811740 1 -- 172.21.15.55:0/200185885 wait complete. 2024-04-26T21:56:29.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.060+0000 7fb541811740 1 -- start start 2024-04-26T21:56:29.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.060+0000 7fb541811740 1 Processor -- start 2024-04-26T21:56:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb541811740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6be8b0 0x55613f6c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb541811740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 0x55613f6c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb541811740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 0x55613f6d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb541811740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55613f6c7240 con 0x55613f6be8b0 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb541811740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55613f5bd5d0 con 0x55613f6c4410 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb541811740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55613f6c73c0 con 0x55613f6bf2a0 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb537fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6be8b0 0x55613f6c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb541811740 10 client.0 _ll_register_callbacks cb 0x55613f5ab080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb537fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6be8b0 0x55613f6c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42028/0 (socket says 172.21.15.55:42028) 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.063+0000 7fb537fff640 1 -- 172.21.15.55:0/1193675273 learned_addr learned my addr 172.21.15.55:0/1193675273 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb5377fe640 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 0x55613f6c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb537fff640 1 -- 172.21.15.55:0/1193675273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 msgr2=0x55613f6c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb537fff640 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 0x55613f6c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb537fff640 1 -- 172.21.15.55:0/1193675273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 msgr2=0x55613f6d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb537fff640 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 0x55613f6d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb537fff640 1 -- 172.21.15.55:0/1193675273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613f45df70 con 0x55613f6be8b0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb537fff640 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6be8b0 0x55613f6c8b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cdc60 tx=0x7fb5280cd620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e8abdc4d354888 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb5177fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb5177fe640 1 -- 172.21.15.55:0/1193675273 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528098150 con 0x55613f6be8b0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb5177fe640 1 -- 172.21.15.55:0/1193675273 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb528009de0 con 0x55613f6be8b0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.064+0000 7fb541811740 1 -- 172.21.15.55:0/1193675273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52809c5b0 con 0x55613f6be8b0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb5177fe640 1 -- 172.21.15.55:0/1193675273 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb52809b810 con 0x55613f6be8b0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb541811740 10 client.15993 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb541811740 1 -- 172.21.15.55:0/1193675273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55613f2f5170 con 0x55613f6be8b0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb5177fe640 1 -- 172.21.15.55:0/1193675273 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52809c5b0 con 0x55613f6be8b0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb5177fe640 1 -- 172.21.15.55:0/1193675273 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fb528098520 con 0x55613f6be8b0 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb541811740 10 client.15993 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb541811740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:29.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb541811740 20 client.15993 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:29.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb541811740 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55613f6d4230 0x55613f6d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb541811740 4 client.15993 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:29.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.065+0000 7fb541811740 1 -- 172.21.15.55:0/1193675273 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55613f631710 con 0x55613f6d4230 2024-04-26T21:56:29.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.067+0000 7fb5377fe640 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55613f6d4230 0x55613f6d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.068+0000 7fb5177fe640 10 client.15993 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:29.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.068+0000 7fb5377fe640 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55613f6d4230 0x55613f6d6660 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62eaf43252e1da29 server_cookie=5754ff373d2c8cb9 in_seq=0 out_seq=0 2024-04-26T21:56:29.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.073+0000 7fb5177fe640 1 -- 172.21.15.55:0/1193675273 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55613f631710 con 0x55613f6d4230 2024-04-26T21:56:29.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.073+0000 7fb5177fe640 10 client.15993 handle_command_reply: tid=0 2024-04-26T21:56:29.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:28 smithi165 ceph-mon[33296]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 924 MiB/s wr, 1.84k op/s 2024-04-26T21:56:29.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.077+0000 7fb541811740 1 client.15993 shutdown 2024-04-26T21:56:29.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.078+0000 7fb541811740 1 -- 172.21.15.55:0/1193675273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6be8b0 msgr2=0x55613f6c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:29.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.078+0000 7fb541811740 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6be8b0 0x55613f6c8b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cdc60 tx=0x7fb5280cd620 comp rx=0 tx=0).stop 2024-04-26T21:56:29.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.079+0000 7fb541811740 1 -- 172.21.15.55:0/1193675273 shutdown_connections 2024-04-26T21:56:29.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.079+0000 7fb541811740 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 0x55613f6d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.079+0000 7fb541811740 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 0x55613f6c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.079+0000 7fb541811740 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55613f6d4230 0x55613f6d6660 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.079+0000 7fb541811740 1 --2- 172.21.15.55:0/1193675273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6be8b0 0x55613f6c8b50 secure :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb5280cdc60 tx=0x7fb5280cd620 comp rx=0 tx=0).stop 2024-04-26T21:56:29.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.079+0000 7fb541811740 1 -- 172.21.15.55:0/1193675273 >> 172.21.15.55:0/1193675273 conn(0x55613f6b9f20 msgr2=0x55613f6b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:29.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.079+0000 7fb5177fe640 0 client.15993 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:29.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.083+0000 7fb541811740 1 -- 172.21.15.55:0/1193675273 shutdown_connections 2024-04-26T21:56:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.083+0000 7fb541811740 1 -- 172.21.15.55:0/1193675273 wait complete. 2024-04-26T21:56:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.084+0000 7fb541811740 20 client.15993 trim_cache size 0 max 16384 2024-04-26T21:56:29.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.127+0000 7fb541811740 1 Processor -- start 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.135+0000 7fb541811740 1 -- start start 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.135+0000 7fb541811740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6bf2a0 0x55613f6c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.135+0000 7fb541811740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c4410 0x55613f6d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.135+0000 7fb541811740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c86e0 0x55613f6c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.135+0000 7fb541811740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb5381b2e20 con 0x55613f6c4410 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.135+0000 7fb541811740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5381b06f0 con 0x55613f6bf2a0 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.135+0000 7fb541811740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5381ae540 con 0x55613f6c86e0 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.136+0000 7fb5377fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c4410 0x55613f6d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.136+0000 7fb5377fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c4410 0x55613f6d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42036/0 (socket says 172.21.15.55:42036) 2024-04-26T21:56:29.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.136+0000 7fb5377fe640 1 -- 172.21.15.55:0/920282780 learned_addr learned my addr 172.21.15.55:0/920282780 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:29.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.138+0000 7fb537fff640 1 --2- 172.21.15.55:0/920282780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6bf2a0 0x55613f6c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.139+0000 7fb537fff640 1 -- 172.21.15.55:0/920282780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c86e0 msgr2=0x55613f6c8b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:29.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.139+0000 7fb537fff640 1 --2- 172.21.15.55:0/920282780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c86e0 0x55613f6c8b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.139+0000 7fb537fff640 1 -- 172.21.15.55:0/920282780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c4410 msgr2=0x55613f6d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:29.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.139+0000 7fb537fff640 1 --2- 172.21.15.55:0/920282780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c4410 0x55613f6d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.139+0000 7fb537fff640 1 -- 172.21.15.55:0/920282780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613f6babf0 con 0x55613f6bf2a0 2024-04-26T21:56:29.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.140+0000 7fb537fff640 1 --2- 172.21.15.55:0/920282780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6bf2a0 0x55613f6c81a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5280097f0 tx=0x7fb528005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60d991275456e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:29.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.152+0000 7fb51dffb640 1 -- 172.21.15.55:0/920282780 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52809b9f0 con 0x55613f6bf2a0 2024-04-26T21:56:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.152+0000 7fb51dffb640 1 -- 172.21.15.55:0/920282780 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52809c5b0 con 0x55613f6bf2a0 2024-04-26T21:56:29.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.152+0000 7fb51dffb640 1 -- 172.21.15.55:0/920282780 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52809bc80 con 0x55613f6bf2a0 2024-04-26T21:56:29.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.154+0000 7fb541811740 1 -- 172.21.15.55:0/920282780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6bf2a0 msgr2=0x55613f6c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:29.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.154+0000 7fb541811740 1 --2- 172.21.15.55:0/920282780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6bf2a0 0x55613f6c81a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5280097f0 tx=0x7fb528005bf0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.159+0000 7fb541811740 1 -- 172.21.15.55:0/920282780 shutdown_connections 2024-04-26T21:56:29.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.159+0000 7fb541811740 1 --2- 172.21.15.55:0/920282780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6c86e0 0x55613f6c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.159+0000 7fb541811740 1 --2- 172.21.15.55:0/920282780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c4410 0x55613f6d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.159+0000 7fb541811740 1 --2- 172.21.15.55:0/920282780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6bf2a0 0x55613f6c81a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.159+0000 7fb541811740 1 -- 172.21.15.55:0/920282780 >> 172.21.15.55:0/920282780 conn(0x55613f6b9f20 msgr2=0x55613f6b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:29.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.172+0000 7fb541811740 1 -- 172.21.15.55:0/920282780 shutdown_connections 2024-04-26T21:56:29.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.177+0000 7fb541811740 1 -- 172.21.15.55:0/920282780 wait complete. 2024-04-26T21:56:29.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.177+0000 7fb541811740 1 -- start start 2024-04-26T21:56:29.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.181+0000 7fb541811740 1 Processor -- start 2024-04-26T21:56:29.191 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:28 smithi055 ceph-mon[29221]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 924 MiB/s wr, 1.84k op/s 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.186+0000 7fb541811740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 0x55613f6cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.186+0000 7fb541811740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 0x55613f6cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.186+0000 7fb541811740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c86e0 0x55613f6cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.186+0000 7fb541811740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb5381aebb0 con 0x55613f6c86e0 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.186+0000 7fb541811740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5381b0020 con 0x55613f6c4410 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.186+0000 7fb541811740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5381af280 con 0x55613f6bf2a0 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.186+0000 7fb541811740 10 client.0 _ll_register_callbacks cb 0x7fb528009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.187+0000 7fb53cfbd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c86e0 0x55613f6cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.187+0000 7fb53cfbd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c86e0 0x55613f6cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42050/0 (socket says 172.21.15.55:42050) 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.187+0000 7fb53cfbd640 1 -- 172.21.15.55:0/3585377913 learned_addr learned my addr 172.21.15.55:0/3585377913 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.188+0000 7fb5377fe640 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 0x55613f6cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.192+0000 7fb537fff640 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 0x55613f6cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.194+0000 7fb5377fe640 1 -- 172.21.15.55:0/3585377913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 msgr2=0x55613f6cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:29.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.194+0000 7fb5377fe640 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 0x55613f6cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.194+0000 7fb5377fe640 1 -- 172.21.15.55:0/3585377913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c86e0 msgr2=0x55613f6cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:29.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.194+0000 7fb5377fe640 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c86e0 0x55613f6cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.194+0000 7fb5377fe640 1 -- 172.21.15.55:0/3585377913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613f6ccba0 con 0x55613f6c4410 2024-04-26T21:56:29.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.197+0000 7fb5377fe640 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 0x55613f6cfed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00ea50 tx=0x7fb52c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68672f827b25368f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:29.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.198+0000 7fb5177fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:56:29.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.204+0000 7fb5177fe640 1 -- 172.21.15.55:0/3585377913 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c01a710 con 0x55613f6c4410 2024-04-26T21:56:29.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.204+0000 7fb541811740 1 -- 172.21.15.55:0/3585377913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52809c5b0 con 0x55613f6c4410 2024-04-26T21:56:29.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.204+0000 7fb5177fe640 1 -- 172.21.15.55:0/3585377913 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb52c01a8b0 con 0x55613f6c4410 2024-04-26T21:56:29.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.214+0000 7fb5177fe640 1 -- 172.21.15.55:0/3585377913 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb52c004020 con 0x55613f6c4410 2024-04-26T21:56:29.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.214+0000 7fb541811740 10 client.25472 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:29.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.214+0000 7fb541811740 1 -- 172.21.15.55:0/3585377913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55613f74c570 con 0x55613f6c4410 2024-04-26T21:56:29.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.224+0000 7fb5177fe640 1 -- 172.21.15.55:0/3585377913 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52809c5b0 con 0x55613f6c4410 2024-04-26T21:56:29.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.224+0000 7fb5177fe640 1 -- 172.21.15.55:0/3585377913 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fb52c022400 con 0x55613f6c4410 2024-04-26T21:56:29.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.224+0000 7fb541811740 10 client.25472 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:29.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.224+0000 7fb541811740 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:29.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.224+0000 7fb541811740 20 client.25472 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:29.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.224+0000 7fb541811740 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55613f74cf50 0x55613f74f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:29.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.224+0000 7fb541811740 4 client.25472 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:56:29.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.224+0000 7fb541811740 1 -- 172.21.15.55:0/3585377913 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55613f62b580 con 0x55613f74cf50 2024-04-26T21:56:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.237+0000 7fb537fff640 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55613f74cf50 0x55613f74f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.249+0000 7fb5177fe640 10 client.25472 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:29.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.249+0000 7fb537fff640 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55613f74cf50 0x55613f74f370 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffdfa07f65c97bf4 server_cookie=ea244900bfdc1df3 in_seq=0 out_seq=0 2024-04-26T21:56:29.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:28 smithi064 ceph-mon[32765]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 924 MiB/s wr, 1.84k op/s 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.585+0000 7fb5177fe640 1 -- 172.21.15.55:0/3585377913 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55613f62b580 con 0x55613f74cf50 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.585+0000 7fb5177fe640 10 client.25472 handle_command_reply: tid=0 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.585+0000 7fb541811740 1 client.25472 shutdown 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.585+0000 7fb541811740 1 -- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 msgr2=0x55613f6cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.585+0000 7fb541811740 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 0x55613f6cfed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00ea50 tx=0x7fb52c011810 comp rx=0 tx=0).stop 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 1 -- 172.21.15.55:0/3585377913 shutdown_connections 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55613f74cf50 0x55613f74f370 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55613f6c86e0 0x55613f6cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55613f6c4410 0x55613f6cfed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 1 --2- 172.21.15.55:0/3585377913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55613f6bf2a0 0x55613f6cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 1 -- 172.21.15.55:0/3585377913 >> 172.21.15.55:0/3585377913 conn(0x55613f6b9f20 msgr2=0x55613f6bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb5177fe640 0 client.25472 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 1 -- 172.21.15.55:0/3585377913 shutdown_connections 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 1 -- 172.21.15.55:0/3585377913 wait complete. 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.586+0000 7fb541811740 20 client.25472 trim_cache size 0 max 16384 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "3a7e050c-0a48-4fc2-bea2-56971a9dd66e", 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:56:29.588 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:56:29.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 -- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5200a3cc0 msgr2=0x7fb5200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:29.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5200a3cc0 0x7fb5200a6180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55613f6bfb00 tx=0x7fb530031f90 comp rx=0 tx=0).stop 2024-04-26T21:56:29.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 -- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb538108f90 msgr2=0x7fb53819f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:29.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb538108f90 0x7fb53819f150 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb52800d950 tx=0x7fb52800de10 comp rx=0 tx=0).stop 2024-04-26T21:56:29.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 -- 172.21.15.55:0/1646220295 shutdown_connections 2024-04-26T21:56:29.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5200a3cc0 0x7fb5200a6180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb53810ab40 0x7fb5381a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb53810a180 0x7fb53819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 --2- 172.21.15.55:0/1646220295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb538108f90 0x7fb53819f150 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:29.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.588+0000 7fb53ea20640 1 -- 172.21.15.55:0/1646220295 >> 172.21.15.55:0/1646220295 conn(0x7fb538077f30 msgr2=0x7fb53810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:29.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.591+0000 7fb53ea20640 1 -- 172.21.15.55:0/1646220295 shutdown_connections 2024-04-26T21:56:29.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:29.591+0000 7fb53ea20640 1 -- 172.21.15.55:0/1646220295 wait complete. 2024-04-26T21:56:29.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:56:29.696 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:56:30.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.059+0000 7fa37ffe4640 1 -- 172.21.15.55:0/85430874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781009e0 msgr2=0x7fa378100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.059+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/85430874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781009e0 0x7fa378100e60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa368009900 tx=0x7fa36802a410 comp rx=0 tx=0).stop 2024-04-26T21:56:30.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.060+0000 7fa37ffe4640 1 -- 172.21.15.55:0/85430874 shutdown_connections 2024-04-26T21:56:30.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.060+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/85430874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3781013a0 0x7fa37810af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.060+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/85430874 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781009e0 0x7fa378100e60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.060+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/85430874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa378103f70 0x7fa378104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.060+0000 7fa37ffe4640 1 -- 172.21.15.55:0/85430874 >> 172.21.15.55:0/85430874 conn(0x7fa3780fc560 msgr2=0x7fa3780fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:30.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.060+0000 7fa37ffe4640 1 -- 172.21.15.55:0/85430874 shutdown_connections 2024-04-26T21:56:30.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.060+0000 7fa37ffe4640 1 -- 172.21.15.55:0/85430874 wait complete. 2024-04-26T21:56:30.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.061+0000 7fa37ffe4640 1 Processor -- start 2024-04-26T21:56:30.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.061+0000 7fa37ffe4640 1 -- start start 2024-04-26T21:56:30.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.061+0000 7fa37ffe4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3781009e0 0x7fa378198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.061+0000 7fa37ffe4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781013a0 0x7fa378199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.062+0000 7fa37d57f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781013a0 0x7fa378199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.062+0000 7fa37d57f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781013a0 0x7fa378199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42074/0 (socket says 172.21.15.55:42074) 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.062+0000 7fa37d57f640 1 -- 172.21.15.55:0/2687155095 learned_addr learned my addr 172.21.15.55:0/2687155095 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.062+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa378103f70 0x7fa3781a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.062+0000 7fa37dd80640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3781009e0 0x7fa378198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.062+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa37810e5a0 con 0x7fa3781013a0 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.062+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa37810e420 con 0x7fa3781009e0 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.062+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa37810e720 con 0x7fa378103f70 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.063+0000 7fa37d57f640 1 -- 172.21.15.55:0/2687155095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa378103f70 msgr2=0x7fa3781a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.063+0000 7fa37d57f640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa378103f70 0x7fa3781a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.063+0000 7fa37d57f640 1 -- 172.21.15.55:0/2687155095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3781009e0 msgr2=0x7fa378198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.063+0000 7fa37d57f640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3781009e0 0x7fa378198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.063+0000 7fa37d57f640 1 -- 172.21.15.55:0/2687155095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3781a0970 con 0x7fa3781013a0 2024-04-26T21:56:30.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.063+0000 7fa37d57f640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781013a0 0x7fa378199290 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa368000c00 tx=0x7fa368032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.064+0000 7fa36effd640 1 -- 172.21.15.55:0/2687155095 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36803f030 con 0x7fa3781013a0 2024-04-26T21:56:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.064+0000 7fa36effd640 1 -- 172.21.15.55:0/2687155095 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa368003cb0 con 0x7fa3781013a0 2024-04-26T21:56:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.064+0000 7fa36effd640 1 -- 172.21.15.55:0/2687155095 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368035070 con 0x7fa3781013a0 2024-04-26T21:56:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.065+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa378101af0 con 0x7fa3781013a0 2024-04-26T21:56:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.066+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa378102010 con 0x7fa3781013a0 2024-04-26T21:56:30.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.070+0000 7fa382dd2740 1 Processor -- start 2024-04-26T21:56:30.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.073+0000 7fa36effd640 1 -- 172.21.15.55:0/2687155095 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa368042030 con 0x7fa3781013a0 2024-04-26T21:56:30.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.075+0000 7fa36effd640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa3640a80d0 0x7fa3640aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.076+0000 7fa36effd640 1 -- 172.21.15.55:0/2687155095 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa3680f0820 con 0x7fa3781013a0 2024-04-26T21:56:30.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.079+0000 7fa382dd2740 1 -- start start 2024-04-26T21:56:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.079+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b4ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.079+0000 7fa37e581640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa3640a80d0 0x7fa3640aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.079+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 0x558005b4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.079+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4c280 0x558005b52b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.079+0000 7fa382dd2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558005aa6d80 con 0x558005b4b8c0 2024-04-26T21:56:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.079+0000 7fa382dd2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558005a608b0 con 0x558005b4c280 2024-04-26T21:56:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.079+0000 7fa382dd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558005aa2d40 con 0x558005b4a6f0 2024-04-26T21:56:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.080+0000 7fa37e581640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4c280 0x558005b52b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.080+0000 7fa37e581640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4c280 0x558005b52b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49358/0 (socket says 172.21.15.55:49358) 2024-04-26T21:56:30.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.080+0000 7fa37e581640 1 -- 172.21.15.55:0/2145410376 learned_addr learned my addr 172.21.15.55:0/2145410376 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:30.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.082+0000 7fa37e581640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa3640a80d0 0x7fa3640aa590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa370010560 tx=0x7fa370010960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:30.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.082+0000 7fa37e581640 1 -- 172.21.15.55:0/2145410376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 msgr2=0x558005b4ab10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:56:30.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.082+0000 7fa37e581640 1 --2- 172.21.15.55:0/2145410376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b4ab10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.082+0000 7fa37e581640 1 -- 172.21.15.55:0/2145410376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 msgr2=0x558005b4bd40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:56:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.082+0000 7fa37e581640 1 --2- 172.21.15.55:0/2145410376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 0x558005b4bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.082+0000 7fa37e581640 1 -- 172.21.15.55:0/2145410376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558005a31a40 con 0x558005b4c280 2024-04-26T21:56:30.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.083+0000 7fa37e581640 1 --2- 172.21.15.55:0/2145410376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4c280 0x558005b52b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c8b0 tx=0x7fa370018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd601c5a6c4e57b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.088+0000 7fa363fff640 1 -- 172.21.15.55:0/2145410376 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370010030 con 0x558005b4c280 2024-04-26T21:56:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.088+0000 7fa363fff640 1 -- 172.21.15.55:0/2145410376 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3700101d0 con 0x558005b4c280 2024-04-26T21:56:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.088+0000 7fa363fff640 1 -- 172.21.15.55:0/2145410376 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37000db70 con 0x558005b4c280 2024-04-26T21:56:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.095+0000 7fa382dd2740 1 -- 172.21.15.55:0/2145410376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4c280 msgr2=0x558005b52b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.095+0000 7fa382dd2740 1 --2- 172.21.15.55:0/2145410376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4c280 0x558005b52b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c8b0 tx=0x7fa370018000 comp rx=0 tx=0).stop 2024-04-26T21:56:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.098+0000 7fa382dd2740 1 -- 172.21.15.55:0/2145410376 shutdown_connections 2024-04-26T21:56:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.098+0000 7fa382dd2740 1 --2- 172.21.15.55:0/2145410376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4c280 0x558005b52b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.098+0000 7fa382dd2740 1 --2- 172.21.15.55:0/2145410376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 0x558005b4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.098+0000 7fa382dd2740 1 --2- 172.21.15.55:0/2145410376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b4ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.098+0000 7fa382dd2740 1 -- 172.21.15.55:0/2145410376 >> 172.21.15.55:0/2145410376 conn(0x558005b3d3d0 msgr2=0x558005b3d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:30.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.105+0000 7fa382dd2740 1 -- 172.21.15.55:0/2145410376 shutdown_connections 2024-04-26T21:56:30.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.105+0000 7fa382dd2740 1 -- 172.21.15.55:0/2145410376 wait complete. 2024-04-26T21:56:30.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.106+0000 7fa382dd2740 1 -- start start 2024-04-26T21:56:30.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.106+0000 7fa382dd2740 1 Processor -- start 2024-04-26T21:56:30.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.109+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4a6f0 0x558005b43700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.109+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 0x558005b43c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.109+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4c280 0x558005b48c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.109+0000 7fa382dd2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558005b41c40 con 0x558005b4b8c0 2024-04-26T21:56:30.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.109+0000 7fa382dd2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558005a2dd10 con 0x558005b4a6f0 2024-04-26T21:56:30.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.109+0000 7fa382dd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558005b41dc0 con 0x558005b4c280 2024-04-26T21:56:30.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.109+0000 7fa382dd2740 10 client.0 _ll_register_callbacks cb 0x558005a5a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:30.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.111+0000 7fa37dd80640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4a6f0 0x558005b43700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.111+0000 7fa37dd80640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4a6f0 0x558005b43700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49368/0 (socket says 172.21.15.55:49368) 2024-04-26T21:56:30.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.111+0000 7fa37dd80640 1 -- 172.21.15.55:0/1856318053 learned_addr learned my addr 172.21.15.55:0/1856318053 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:30.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.116+0000 7fa37d57f640 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 0x558005b43c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.122+0000 7fa37e581640 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4c280 0x558005b48c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.124+0000 7fa37d57f640 1 -- 172.21.15.55:0/1856318053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4c280 msgr2=0x558005b48c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.124+0000 7fa37d57f640 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4c280 0x558005b48c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.124+0000 7fa37d57f640 1 -- 172.21.15.55:0/1856318053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4a6f0 msgr2=0x558005b43700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.124+0000 7fa37d57f640 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4a6f0 0x558005b43700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.124+0000 7fa37d57f640 1 -- 172.21.15.55:0/1856318053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580058827d0 con 0x558005b4b8c0 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.129+0000 7fa37d57f640 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 0x558005b43c40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa3680ee190 tx=0x7fa3680bc380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586fc28840db7645 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.129+0000 7fa3617fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.135+0000 7fa3617fa640 1 -- 172.21.15.55:0/1856318053 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368049040 con 0x558005b4b8c0 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.135+0000 7fa3617fa640 1 -- 172.21.15.55:0/1856318053 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa36803bca0 con 0x558005b4b8c0 2024-04-26T21:56:30.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.136+0000 7fa382dd2740 1 -- 172.21.15.55:0/1856318053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3700101d0 con 0x558005b4b8c0 2024-04-26T21:56:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.143+0000 7fa3617fa640 1 -- 172.21.15.55:0/1856318053 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa3680bc960 con 0x558005b4b8c0 2024-04-26T21:56:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.143+0000 7fa382dd2740 10 client.16008 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:30.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.143+0000 7fa382dd2740 1 -- 172.21.15.55:0/1856318053 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5580057b83f0 con 0x558005b4b8c0 2024-04-26T21:56:30.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.147+0000 7fa3617fa640 1 -- 172.21.15.55:0/1856318053 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3700101d0 con 0x558005b4b8c0 2024-04-26T21:56:30.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.147+0000 7fa3617fa640 1 -- 172.21.15.55:0/1856318053 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fa3680bcbc0 con 0x558005b4b8c0 2024-04-26T21:56:30.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.148+0000 7fa382dd2740 10 client.16008 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:30.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.148+0000 7fa382dd2740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:30.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.148+0000 7fa382dd2740 20 client.16008 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:30.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.148+0000 7fa382dd2740 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558005b69050 0x558005b6b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.148+0000 7fa382dd2740 4 client.16008 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:56:30.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.148+0000 7fa382dd2740 1 -- 172.21.15.55:0/1856318053 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558005ab4990 con 0x558005b69050 2024-04-26T21:56:30.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.152+0000 7fa37dd80640 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558005b69050 0x558005b6b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.167+0000 7fa37dd80640 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558005b69050 0x558005b6b4b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411f6b20e6e7e24a server_cookie=fe0720e213e73fae in_seq=0 out_seq=0 2024-04-26T21:56:30.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.167+0000 7fa3617fa640 10 client.16008 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:30.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.176+0000 7fa3617fa640 1 -- 172.21.15.55:0/1856318053 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558005ab4990 con 0x558005b69050 2024-04-26T21:56:30.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.176+0000 7fa3617fa640 10 client.16008 handle_command_reply: tid=0 2024-04-26T21:56:30.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.179+0000 7fa382dd2740 1 client.16008 shutdown 2024-04-26T21:56:30.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.179+0000 7fa382dd2740 1 -- 172.21.15.55:0/1856318053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 msgr2=0x558005b43c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.179+0000 7fa382dd2740 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 0x558005b43c40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa3680ee190 tx=0x7fa3680bc380 comp rx=0 tx=0).stop 2024-04-26T21:56:30.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.179+0000 7fa382dd2740 1 -- 172.21.15.55:0/1856318053 shutdown_connections 2024-04-26T21:56:30.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.179+0000 7fa382dd2740 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4c280 0x558005b48c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.180+0000 7fa382dd2740 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558005b69050 0x558005b6b4b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.180+0000 7fa382dd2740 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4b8c0 0x558005b43c40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.180+0000 7fa382dd2740 1 --2- 172.21.15.55:0/1856318053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4a6f0 0x558005b43700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.180+0000 7fa382dd2740 1 -- 172.21.15.55:0/1856318053 >> 172.21.15.55:0/1856318053 conn(0x558005b3d3d0 msgr2=0x558005b3c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.184+0000 7fa3617fa640 0 client.16008 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.185+0000 7fa382dd2740 1 -- 172.21.15.55:0/1856318053 shutdown_connections 2024-04-26T21:56:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.185+0000 7fa382dd2740 1 -- 172.21.15.55:0/1856318053 wait complete. 2024-04-26T21:56:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.185+0000 7fa382dd2740 20 client.16008 trim_cache size 0 max 16384 2024-04-26T21:56:30.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.268+0000 7fa382dd2740 1 Processor -- start 2024-04-26T21:56:30.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.272+0000 7fa382dd2740 1 -- start start 2024-04-26T21:56:30.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.273+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b3d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.273+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b4e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.273+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b4f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.273+0000 7fa382dd2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781aa690 con 0x558005b4c280 2024-04-26T21:56:30.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.274+0000 7fa382dd2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781a7f60 con 0x558005b4b8c0 2024-04-26T21:56:30.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.274+0000 7fa382dd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781a5db0 con 0x558005b4a6f0 2024-04-26T21:56:30.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.274+0000 7fa37d57f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b4e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.274+0000 7fa37d57f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b4e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49380/0 (socket says 172.21.15.55:49380) 2024-04-26T21:56:30.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.274+0000 7fa37d57f640 1 -- 172.21.15.55:0/851274122 learned_addr learned my addr 172.21.15.55:0/851274122 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:30.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.274+0000 7fa37e581640 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b4f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.278+0000 7fa37dd80640 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b3d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.278+0000 7fa37e581640 1 -- 172.21.15.55:0/851274122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 msgr2=0x558005b3d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.278+0000 7fa37e581640 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b3d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.278+0000 7fa37e581640 1 -- 172.21.15.55:0/851274122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 msgr2=0x558005b4e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:30.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.278+0000 7fa37e581640 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b4e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.278+0000 7fa37e581640 1 -- 172.21.15.55:0/851274122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580057780f0 con 0x558005b4c280 2024-04-26T21:56:30.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.279+0000 7fa37d57f640 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b4e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:56:30.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.280+0000 7fa37e581640 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b4f550 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa37000aa10 tx=0x7fa37000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfa82c731575a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:30.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.285+0000 7fa3637fe640 1 -- 172.21.15.55:0/851274122 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370010040 con 0x558005b4c280 2024-04-26T21:56:30.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.286+0000 7fa3637fe640 1 -- 172.21.15.55:0/851274122 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3700101e0 con 0x558005b4c280 2024-04-26T21:56:30.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.286+0000 7fa3637fe640 1 -- 172.21.15.55:0/851274122 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370027980 con 0x558005b4c280 2024-04-26T21:56:30.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.288+0000 7fa382dd2740 1 -- 172.21.15.55:0/851274122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 msgr2=0x558005b4f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.288+0000 7fa382dd2740 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b4f550 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa37000aa10 tx=0x7fa37000ce30 comp rx=0 tx=0).stop 2024-04-26T21:56:30.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.289+0000 7fa382dd2740 1 -- 172.21.15.55:0/851274122 shutdown_connections 2024-04-26T21:56:30.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.290+0000 7fa382dd2740 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b4f550 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.290+0000 7fa382dd2740 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b4e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.290+0000 7fa382dd2740 1 --2- 172.21.15.55:0/851274122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b3d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.290+0000 7fa382dd2740 1 -- 172.21.15.55:0/851274122 >> 172.21.15.55:0/851274122 conn(0x558005b3d3d0 msgr2=0x558005b38480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:30.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.294+0000 7fa382dd2740 1 -- 172.21.15.55:0/851274122 shutdown_connections 2024-04-26T21:56:30.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.294+0000 7fa382dd2740 1 -- 172.21.15.55:0/851274122 wait complete. 2024-04-26T21:56:30.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.295+0000 7fa382dd2740 1 -- start start 2024-04-26T21:56:30.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.295+0000 7fa382dd2740 1 Processor -- start 2024-04-26T21:56:30.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.296+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b77c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.296+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b78180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.296+0000 7fa382dd2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b7b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.296+0000 7fa382dd2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781a6420 con 0x558005b4c280 2024-04-26T21:56:30.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.296+0000 7fa382dd2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781a7890 con 0x558005b4b8c0 2024-04-26T21:56:30.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.296+0000 7fa382dd2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781a6af0 con 0x558005b4a6f0 2024-04-26T21:56:30.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.296+0000 7fa382dd2740 10 client.0 _ll_register_callbacks cb 0x7fa368032e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:56:30.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.297+0000 7fa37d57f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b78180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.297+0000 7fa37d57f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b78180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:49382/0 (socket says 172.21.15.55:49382) 2024-04-26T21:56:30.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.297+0000 7fa37d57f640 1 -- 172.21.15.55:0/4216129011 learned_addr learned my addr 172.21.15.55:0/4216129011 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:56:30.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.297+0000 7fa37dd80640 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b77c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.297+0000 7fa37e581640 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b7b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.299+0000 7fa37e581640 1 -- 172.21.15.55:0/4216129011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 msgr2=0x558005b77c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:30.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.299+0000 7fa37e581640 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b77c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.299+0000 7fa37e581640 1 -- 172.21.15.55:0/4216129011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 msgr2=0x558005b78180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:56:30.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.299+0000 7fa37e581640 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b78180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.299+0000 7fa37e581640 1 -- 172.21.15.55:0/4216129011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558005a674b0 con 0x558005b4c280 2024-04-26T21:56:30.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.299+0000 7fa3617fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:56:30.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.299+0000 7fa37e581640 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b7b150 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa3700096f0 tx=0x7fa3700100f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad706d02aa6c215 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:56:30.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.300+0000 7fa3617fa640 1 -- 172.21.15.55:0/4216129011 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370034020 con 0x558005b4c280 2024-04-26T21:56:30.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.300+0000 7fa382dd2740 1 -- 172.21.15.55:0/4216129011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3700101e0 con 0x558005b4c280 2024-04-26T21:56:30.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.300+0000 7fa3617fa640 1 -- 172.21.15.55:0/4216129011 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa3700278c0 con 0x558005b4c280 2024-04-26T21:56:30.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.300+0000 7fa3617fa640 1 -- 172.21.15.55:0/4216129011 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa37003f050 con 0x558005b4c280 2024-04-26T21:56:30.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.300+0000 7fa382dd2740 10 client.16014 fetch_fsmap learned FSMap version 22 2024-04-26T21:56:30.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.300+0000 7fa382dd2740 1 -- 172.21.15.55:0/4216129011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558005b7c7f0 con 0x558005b4c280 2024-04-26T21:56:30.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.301+0000 7fa3617fa640 1 -- 172.21.15.55:0/4216129011 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa370027cd0 con 0x558005b4c280 2024-04-26T21:56:30.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.301+0000 7fa3617fa640 1 -- 172.21.15.55:0/4216129011 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fa370024070 con 0x558005b4c280 2024-04-26T21:56:30.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.301+0000 7fa382dd2740 10 client.16014 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:56:30.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.301+0000 7fa382dd2740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:56:30.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.301+0000 7fa382dd2740 20 client.16014 populate_metadata read hostname 'smithi055' 2024-04-26T21:56:30.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.301+0000 7fa382dd2740 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558005bcf1b0 0x558005bd15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:56:30.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.302+0000 7fa382dd2740 4 client.16014 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:56:30.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.302+0000 7fa382dd2740 1 -- 172.21.15.55:0/4216129011 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa3700101e0 con 0x558005bcf1b0 2024-04-26T21:56:30.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.304+0000 7fa37dd80640 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558005bcf1b0 0x558005bd15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:56:30.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.311+0000 7fa37dd80640 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558005bcf1b0 0x558005bd15a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d870282ff6e969 server_cookie=a3f0a28b86bf142d in_seq=0 out_seq=0 2024-04-26T21:56:30.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.311+0000 7fa3617fa640 10 client.16014 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:30.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.316+0000 7fa3617fa640 1 -- 172.21.15.55:0/4216129011 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x7fa3700101e0 con 0x558005bcf1b0 2024-04-26T21:56:30.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.316+0000 7fa3617fa640 10 client.16014 handle_command_reply: tid=0 2024-04-26T21:56:30.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.316+0000 7fa382dd2740 1 client.16014 shutdown 2024-04-26T21:56:30.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.316+0000 7fa382dd2740 1 -- 172.21.15.55:0/4216129011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 msgr2=0x558005b7b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.316+0000 7fa382dd2740 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b7b150 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa3700096f0 tx=0x7fa3700100f0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 1 -- 172.21.15.55:0/4216129011 shutdown_connections 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558005bcf1b0 0x558005bd15a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa3617fa640 0 client.16014 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558005b4c280 0x558005b7b150 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558005b4b8c0 0x558005b78180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 1 --2- 172.21.15.55:0/4216129011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558005b4a6f0 0x558005b77c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 1 -- 172.21.15.55:0/4216129011 >> 172.21.15.55:0/4216129011 conn(0x558005b3d3d0 msgr2=0x558005b436e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 1 -- 172.21.15.55:0/4216129011 shutdown_connections 2024-04-26T21:56:30.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 1 -- 172.21.15.55:0/4216129011 wait complete. 2024-04-26T21:56:30.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.317+0000 7fa382dd2740 20 client.16014 trim_cache size 0 max 16384 2024-04-26T21:56:30.320 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:56:30.320 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-26T21:56:30.320 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:56:30.320 INFO:teuthology.orchestra.run.smithi055.stdout: "3a7e050c-0a48-4fc2-bea2-56971a9dd66e": { 2024-04-26T21:56:30.320 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:56:30.321 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "3a7e050c-0a48-4fc2-bea2-56971a9dd66e", 2024-04-26T21:56:30.321 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:56:30.321 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:56:30.321 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:56:30.321 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:56:30.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa3640a80d0 msgr2=0x7fa3640aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa3640a80d0 0x7fa3640aa590 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa370010560 tx=0x7fa370010960 comp rx=0 tx=0).stop 2024-04-26T21:56:30.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781013a0 msgr2=0x7fa378199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:56:30.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781013a0 0x7fa378199290 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa368000c00 tx=0x7fa368032c80 comp rx=0 tx=0).stop 2024-04-26T21:56:30.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 shutdown_connections 2024-04-26T21:56:30.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa3640a80d0 0x7fa3640aa590 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa378103f70 0x7fa3781a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa3781013a0 0x7fa378199290 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 --2- 172.21.15.55:0/2687155095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3781009e0 0x7fa378198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:56:30.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 >> 172.21.15.55:0/2687155095 conn(0x7fa3780fc560 msgr2=0x7fa3781070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:56:30.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 shutdown_connections 2024-04-26T21:56:30.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:56:30.319+0000 7fa37ffe4640 1 -- 172.21.15.55:0/2687155095 wait complete. 2024-04-26T21:56:30.343 INFO:tasks.cephfs.filesystem:scrub status for tag:3a7e050c-0a48-4fc2-bea2-56971a9dd66e - {'path': '/', 'tag': '3a7e050c-0a48-4fc2-bea2-56971a9dd66e', 'options': 'recursive,force'} 2024-04-26T21:56:31.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:30 smithi055 ceph-mon[29221]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 898 MiB/s wr, 1.75k op/s 2024-04-26T21:56:31.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:30 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:56:31.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:30 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:31.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:30 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:56:31.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:30 smithi064 ceph-mon[32765]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 898 MiB/s wr, 1.75k op/s 2024-04-26T21:56:31.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:30 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:56:31.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:30 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:31.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:30 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:56:31.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:30 smithi165 ceph-mon[33296]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 898 MiB/s wr, 1.75k op/s 2024-04-26T21:56:31.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:30 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:56:31.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:30 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:31.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:30 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:56:32.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:31 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:56:32.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:31 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:32.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:31 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:56:32.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:31 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:32.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:31 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:56:32.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:31 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:33.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:32 smithi055 ceph-mon[29221]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 830 MiB/s wr, 1.61k op/s 2024-04-26T21:56:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:32 smithi064 ceph-mon[32765]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 830 MiB/s wr, 1.61k op/s 2024-04-26T21:56:33.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:32 smithi165 ceph-mon[33296]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 830 MiB/s wr, 1.61k op/s 2024-04-26T21:56:35.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:34 smithi165 ceph-mon[33296]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 857 MiB/s wr, 1.84k op/s 2024-04-26T21:56:35.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:34 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:35.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:34 smithi055 ceph-mon[29221]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 857 MiB/s wr, 1.84k op/s 2024-04-26T21:56:35.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:34 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:35.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:34 smithi064 ceph-mon[32765]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 857 MiB/s wr, 1.84k op/s 2024-04-26T21:56:35.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:34 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:56:37.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:36 smithi055 ceph-mon[29221]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 885 MiB/s wr, 2.27k op/s 2024-04-26T21:56:37.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:36 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:56:37.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:36 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:56:37.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:36 smithi064 ceph-mon[32765]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 885 MiB/s wr, 2.27k op/s 2024-04-26T21:56:37.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:36 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:56:37.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:36 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:56:37.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:36 smithi165 ceph-mon[33296]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 885 MiB/s wr, 2.27k op/s 2024-04-26T21:56:37.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:36 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:56:37.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:36 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:56:39.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:38 smithi055 ceph-mon[29221]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 925 MiB/s wr, 2.70k op/s 2024-04-26T21:56:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:38 smithi064 ceph-mon[32765]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 925 MiB/s wr, 2.70k op/s 2024-04-26T21:56:39.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:38 smithi165 ceph-mon[33296]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 925 MiB/s wr, 2.70k op/s 2024-04-26T21:56:41.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:40 smithi055 ceph-mon[29221]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 859 MiB/s wr, 2.53k op/s 2024-04-26T21:56:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:40 smithi064 ceph-mon[32765]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 859 MiB/s wr, 2.53k op/s 2024-04-26T21:56:41.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:40 smithi165 ceph-mon[33296]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 859 MiB/s wr, 2.53k op/s 2024-04-26T21:56:43.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:42 smithi055 ceph-mon[29221]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 816 MiB/s wr, 2.18k op/s 2024-04-26T21:56:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:42 smithi064 ceph-mon[32765]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 816 MiB/s wr, 2.18k op/s 2024-04-26T21:56:43.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:42 smithi165 ceph-mon[33296]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 816 MiB/s wr, 2.18k op/s 2024-04-26T21:56:45.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:44 smithi055 ceph-mon[29221]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 848 MiB/s wr, 2.12k op/s 2024-04-26T21:56:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:44 smithi064 ceph-mon[32765]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 848 MiB/s wr, 2.12k op/s 2024-04-26T21:56:45.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:44 smithi165 ceph-mon[33296]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 848 MiB/s wr, 2.12k op/s 2024-04-26T21:56:47.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:46 smithi055 ceph-mon[29221]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 1.95k op/s 2024-04-26T21:56:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:46 smithi064 ceph-mon[32765]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 1.95k op/s 2024-04-26T21:56:47.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:46 smithi165 ceph-mon[33296]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 1.95k op/s 2024-04-26T21:56:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:48 smithi064 ceph-mon[32765]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 915 MiB/s wr, 1.59k op/s 2024-04-26T21:56:49.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:48 smithi165 ceph-mon[33296]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 915 MiB/s wr, 1.59k op/s 2024-04-26T21:56:49.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:48 smithi055 ceph-mon[29221]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 915 MiB/s wr, 1.59k op/s 2024-04-26T21:56:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:50 smithi064 ceph-mon[32765]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 881 MiB/s wr, 1.11k op/s 2024-04-26T21:56:51.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:50 smithi165 ceph-mon[33296]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 881 MiB/s wr, 1.11k op/s 2024-04-26T21:56:51.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:50 smithi055 ceph-mon[29221]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 881 MiB/s wr, 1.11k op/s 2024-04-26T21:56:53.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:53 smithi165 ceph-mon[33296]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.04k op/s 2024-04-26T21:56:53.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:53 smithi055 ceph-mon[29221]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.04k op/s 2024-04-26T21:56:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:53 smithi064 ceph-mon[32765]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.04k op/s 2024-04-26T21:56:55.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:55 smithi165 ceph-mon[33296]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 845 MiB/s wr, 997 op/s 2024-04-26T21:56:55.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:55 smithi055 ceph-mon[29221]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 845 MiB/s wr, 997 op/s 2024-04-26T21:56:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:55 smithi064 ceph-mon[32765]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 845 MiB/s wr, 997 op/s 2024-04-26T21:56:57.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:57 smithi165 ceph-mon[33296]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 870 MiB/s wr, 977 op/s 2024-04-26T21:56:57.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:57 smithi055 ceph-mon[29221]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 870 MiB/s wr, 977 op/s 2024-04-26T21:56:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:57 smithi064 ceph-mon[32765]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 870 MiB/s wr, 977 op/s 2024-04-26T21:56:59.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:56:59 smithi165 ceph-mon[33296]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 900 MiB/s wr, 1.00k op/s 2024-04-26T21:56:59.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:56:59 smithi055 ceph-mon[29221]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 900 MiB/s wr, 1.00k op/s 2024-04-26T21:56:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:56:59 smithi064 ceph-mon[32765]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 900 MiB/s wr, 1.00k op/s 2024-04-26T21:57:00.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:57:00.345 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:57:00.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.680+0000 7f10481da640 1 -- 172.21.15.55:0/3576037285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040073be0 msgr2=0x7f1040074000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.680+0000 7f10481da640 1 --2- 172.21.15.55:0/3576037285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040073be0 0x7f1040074000 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f103c009900 tx=0x7f103c02a440 comp rx=0 tx=0).stop 2024-04-26T21:57:00.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.682+0000 7f10481da640 1 -- 172.21.15.55:0/3576037285 shutdown_connections 2024-04-26T21:57:00.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.682+0000 7f10481da640 1 --2- 172.21.15.55:0/3576037285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f104010a640 0x7f104010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.682+0000 7f10481da640 1 --2- 172.21.15.55:0/3576037285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1040074540 0x7f1040109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.682+0000 7f10481da640 1 --2- 172.21.15.55:0/3576037285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040073be0 0x7f1040074000 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.682+0000 7f10481da640 1 -- 172.21.15.55:0/3576037285 >> 172.21.15.55:0/3576037285 conn(0x7f104006f780 msgr2=0x7f1040071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:00.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.683+0000 7f10481da640 1 -- 172.21.15.55:0/3576037285 shutdown_connections 2024-04-26T21:57:00.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.684+0000 7f10481da640 1 -- 172.21.15.55:0/3576037285 wait complete. 2024-04-26T21:57:00.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.685+0000 7f10481da640 1 Processor -- start 2024-04-26T21:57:00.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.685+0000 7f10481da640 1 -- start start 2024-04-26T21:57:00.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.685+0000 7f10481da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040073be0 0x7f10401a7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.685+0000 7f10481da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1040074540 0x7f10401a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.685+0000 7f10481da640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f104010a640 0x7f104007ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.685+0000 7f10481da640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f104011d240 con 0x7f1040073be0 2024-04-26T21:57:00.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.685+0000 7f10481da640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f104011d0c0 con 0x7f1040074540 2024-04-26T21:57:00.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.685+0000 7f10481da640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f104011d3c0 con 0x7f104010a640 2024-04-26T21:57:00.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.687+0000 7f1045775640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1040074540 0x7f10401a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.687+0000 7f1046777640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f104010a640 0x7f104007ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.687+0000 7f1046777640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f104010a640 0x7f104007ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:46244/0 (socket says 172.21.15.55:46244) 2024-04-26T21:57:00.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.687+0000 7f1046777640 1 -- 172.21.15.55:0/1389552694 learned_addr learned my addr 172.21.15.55:0/1389552694 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:00.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.687+0000 7f1045f76640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040073be0 0x7f10401a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.688+0000 7f1046777640 1 -- 172.21.15.55:0/1389552694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1040074540 msgr2=0x7f10401a7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.688+0000 7f1046777640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1040074540 0x7f10401a7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.688+0000 7f1046777640 1 -- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040073be0 msgr2=0x7f10401a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.688+0000 7f1046777640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040073be0 0x7f10401a7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.688+0000 7f1046777640 1 -- 172.21.15.55:0/1389552694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104007b230 con 0x7f104010a640 2024-04-26T21:57:00.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.689+0000 7f1046777640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f104010a640 0x7f104007ab90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f103800b990 tx=0x7f103800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:00.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.695+0000 7f1036ffd640 1 -- 172.21.15.55:0/1389552694 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038019070 con 0x7f104010a640 2024-04-26T21:57:00.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.695+0000 7f1036ffd640 1 -- 172.21.15.55:0/1389552694 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f103800cba0 con 0x7f104010a640 2024-04-26T21:57:00.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.695+0000 7f1036ffd640 1 -- 172.21.15.55:0/1389552694 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10380075d0 con 0x7f104010a640 2024-04-26T21:57:00.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.695+0000 7f10481da640 1 -- 172.21.15.55:0/1389552694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104007b520 con 0x7f104010a640 2024-04-26T21:57:00.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.695+0000 7f10481da640 1 -- 172.21.15.55:0/1389552694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104007ba70 con 0x7f104010a640 2024-04-26T21:57:00.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.703+0000 7f1036ffd640 1 -- 172.21.15.55:0/1389552694 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f103801c070 con 0x7f104010a640 2024-04-26T21:57:00.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.704+0000 7f1036ffd640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f102c0a3e40 0x7f102c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.705+0000 7f1036ffd640 1 -- 172.21.15.55:0/1389552694 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f10380cfa60 con 0x7f104010a640 2024-04-26T21:57:00.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.705+0000 7f104afc9740 1 Processor -- start 2024-04-26T21:57:00.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.706+0000 7f1045f76640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f102c0a3e40 0x7f102c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.706+0000 7f104afc9740 1 -- start start 2024-04-26T21:57:00.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.706+0000 7f104afc9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f50b0 0x563b699f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.706+0000 7f104afc9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f6280 0x563b699f6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.707+0000 7f104afc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b699fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.707+0000 7f104afc9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563b69959d80 con 0x563b699f6280 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.707+0000 7f104afc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563b699138b0 con 0x563b699f6c40 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.707+0000 7f104afc9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563b69955d40 con 0x563b699f50b0 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.707+0000 7f1045775640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f6280 0x563b699f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.707+0000 7f1045775640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f6280 0x563b699f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54542/0 (socket says 172.21.15.55:54542) 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.707+0000 7f1045775640 1 -- 172.21.15.55:0/3710176132 learned_addr learned my addr 172.21.15.55:0/3710176132 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.708+0000 7f1045f76640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f102c0a3e40 0x7f102c0a6300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f103c000c00 tx=0x7f103c0095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.708+0000 7f1045775640 1 -- 172.21.15.55:0/3710176132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f50b0 msgr2=0x563b699f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.708+0000 7f1045775640 1 --2- 172.21.15.55:0/3710176132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f50b0 0x563b699f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.708+0000 7f1045775640 1 -- 172.21.15.55:0/3710176132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 msgr2=0x563b699fd4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.708+0000 7f1045775640 1 --2- 172.21.15.55:0/3710176132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b699fd4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.708+0000 7f1045775640 1 -- 172.21.15.55:0/3710176132 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b698e4a40 con 0x563b699f6280 2024-04-26T21:57:00.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.708+0000 7f1045775640 1 --2- 172.21.15.55:0/3710176132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f6280 0x563b699f6700 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f10300099f0 tx=0x7f1030009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98efcd40eb5c0035 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:00.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.709+0000 7f102bfff640 1 -- 172.21.15.55:0/3710176132 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030017070 con 0x563b699f6280 2024-04-26T21:57:00.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.709+0000 7f102bfff640 1 -- 172.21.15.55:0/3710176132 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1030013470 con 0x563b699f6280 2024-04-26T21:57:00.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.710+0000 7f104afc9740 1 -- 172.21.15.55:0/3710176132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f6280 msgr2=0x563b699f6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.710+0000 7f104afc9740 1 --2- 172.21.15.55:0/3710176132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f6280 0x563b699f6700 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f10300099f0 tx=0x7f1030009eb0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.711+0000 7f104afc9740 1 -- 172.21.15.55:0/3710176132 shutdown_connections 2024-04-26T21:57:00.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.711+0000 7f104afc9740 1 --2- 172.21.15.55:0/3710176132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b699fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.711+0000 7f104afc9740 1 --2- 172.21.15.55:0/3710176132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f6280 0x563b699f6700 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.711+0000 7f104afc9740 1 --2- 172.21.15.55:0/3710176132 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f50b0 0x563b699f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.711+0000 7f104afc9740 1 -- 172.21.15.55:0/3710176132 >> 172.21.15.55:0/3710176132 conn(0x563b699f05a0 msgr2=0x563b699f09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:00.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.711+0000 7f104afc9740 1 -- 172.21.15.55:0/3710176132 shutdown_connections 2024-04-26T21:57:00.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.712+0000 7f104afc9740 1 -- 172.21.15.55:0/3710176132 wait complete. 2024-04-26T21:57:00.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.712+0000 7f104afc9740 1 -- start start 2024-04-26T21:57:00.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.713+0000 7f104afc9740 1 Processor -- start 2024-04-26T21:57:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.717+0000 7f104afc9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a10530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.717+0000 7f104afc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.717+0000 7f104afc9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 0x563b69a17e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.717+0000 7f104afc9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563b69a0ecb0 con 0x563b699f50b0 2024-04-26T21:57:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.717+0000 7f104afc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563b698e0d10 con 0x563b699f6c40 2024-04-26T21:57:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.717+0000 7f104afc9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563b69a0ee30 con 0x563b69a15aa0 2024-04-26T21:57:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.717+0000 7f104afc9740 10 client.0 _ll_register_callbacks cb 0x563b6990d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:00.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.718+0000 7f1045775640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.718+0000 7f1045775640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a10a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45832/0 (socket says 172.21.15.55:45832) 2024-04-26T21:57:00.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.718+0000 7f1045775640 1 -- 172.21.15.55:0/710698705 learned_addr learned my addr 172.21.15.55:0/710698705 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:00.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.718+0000 7f1046777640 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 0x563b69a17e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.719+0000 7f1045775640 1 -- 172.21.15.55:0/710698705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 msgr2=0x563b69a17e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:00.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.719+0000 7f1045775640 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 0x563b69a17e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.719+0000 7f1045775640 1 -- 172.21.15.55:0/710698705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 msgr2=0x563b69a10530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.719+0000 7f1045775640 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a10530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.719+0000 7f1045775640 1 -- 172.21.15.55:0/710698705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b697357d0 con 0x563b699f6c40 2024-04-26T21:57:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.720+0000 7f1045775640 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a10a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1030006b60 tx=0x7f1030004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3c580b6881c4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.720+0000 7f10297fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:57:00.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.722+0000 7f10297fa640 1 -- 172.21.15.55:0/710698705 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030017050 con 0x563b699f6c40 2024-04-26T21:57:00.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.723+0000 7f104afc9740 1 -- 172.21.15.55:0/710698705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b69967990 con 0x563b699f6c40 2024-04-26T21:57:00.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.723+0000 7f10297fa640 1 -- 172.21.15.55:0/710698705 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1030012870 con 0x563b699f6c40 2024-04-26T21:57:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.724+0000 7f10297fa640 1 -- 172.21.15.55:0/710698705 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1030012a70 con 0x563b699f6c40 2024-04-26T21:57:00.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.724+0000 7f104afc9740 10 client.25490 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:00.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.724+0000 7f104afc9740 1 -- 172.21.15.55:0/710698705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563b6966b3f0 con 0x563b699f6c40 2024-04-26T21:57:00.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.727+0000 7f10297fa640 1 -- 172.21.15.55:0/710698705 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563b69967990 con 0x563b699f6c40 2024-04-26T21:57:00.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.728+0000 7f10297fa640 1 -- 172.21.15.55:0/710698705 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f1030013470 con 0x563b699f6c40 2024-04-26T21:57:00.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.728+0000 7f104afc9740 10 client.25490 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:00.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.728+0000 7f104afc9740 10 client.25490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:00.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.728+0000 7f104afc9740 20 client.25490 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:00.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.728+0000 7f104afc9740 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563b69a1c740 0x563b69a1eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.728+0000 7f104afc9740 4 client.25490 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:00.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.728+0000 7f104afc9740 1 -- 172.21.15.55:0/710698705 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b69961800 con 0x563b69a1c740 2024-04-26T21:57:00.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.729+0000 7f1045f76640 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563b69a1c740 0x563b69a1eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.731+0000 7f1045f76640 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563b69a1c740 0x563b69a1eba0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed870a6cc3624ea4 server_cookie=24a66ef8c2e5e639 in_seq=0 out_seq=0 2024-04-26T21:57:00.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.731+0000 7f10297fa640 10 client.25490 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:00.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.735+0000 7f10297fa640 1 -- 172.21.15.55:0/710698705 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b69961800 con 0x563b69a1c740 2024-04-26T21:57:00.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.735+0000 7f10297fa640 10 client.25490 handle_command_reply: tid=0 2024-04-26T21:57:00.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.736+0000 7f104afc9740 1 client.25490 shutdown 2024-04-26T21:57:00.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.737+0000 7f104afc9740 1 -- 172.21.15.55:0/710698705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 msgr2=0x563b69a10a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.737+0000 7f104afc9740 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a10a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1030006b60 tx=0x7f1030004270 comp rx=0 tx=0).stop 2024-04-26T21:57:00.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.737+0000 7f104afc9740 1 -- 172.21.15.55:0/710698705 shutdown_connections 2024-04-26T21:57:00.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.737+0000 7f104afc9740 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 0x563b69a17e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.737+0000 7f104afc9740 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a10a70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.737+0000 7f104afc9740 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563b69a1c740 0x563b69a1eba0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.737+0000 7f104afc9740 1 --2- 172.21.15.55:0/710698705 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a10530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.737+0000 7f104afc9740 1 -- 172.21.15.55:0/710698705 >> 172.21.15.55:0/710698705 conn(0x563b699f05a0 msgr2=0x563b699f6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:00.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.738+0000 7f10297fa640 0 client.25490 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:00.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.738+0000 7f104afc9740 1 -- 172.21.15.55:0/710698705 shutdown_connections 2024-04-26T21:57:00.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.738+0000 7f104afc9740 1 -- 172.21.15.55:0/710698705 wait complete. 2024-04-26T21:57:00.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.738+0000 7f104afc9740 20 client.25490 trim_cache size 0 max 16384 2024-04-26T21:57:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.775+0000 7f104afc9740 1 Processor -- start 2024-04-26T21:57:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.775+0000 7f104afc9740 1 -- start start 2024-04-26T21:57:00.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f104afc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f50b0 0x563b69a10530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f104afc9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f6c40 0x563b69a10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f104afc9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b69a15aa0 0x563b699f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f104afc9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f104007eb30 con 0x563b69a15aa0 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f104afc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f104007c400 con 0x563b699f50b0 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f104afc9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1040117c80 con 0x563b699f6c40 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f1045f76640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f50b0 0x563b69a10530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f1045f76640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f50b0 0x563b69a10530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45840/0 (socket says 172.21.15.55:45840) 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.776+0000 7f1045f76640 1 -- 172.21.15.55:0/996233190 learned_addr learned my addr 172.21.15.55:0/996233190 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.777+0000 7f1045775640 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f6c40 0x563b69a10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.777+0000 7f1046777640 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b69a15aa0 0x563b699f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.777+0000 7f1045f76640 1 -- 172.21.15.55:0/996233190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f6c40 msgr2=0x563b69a10a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.777+0000 7f1045f76640 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f6c40 0x563b69a10a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.777+0000 7f1045f76640 1 -- 172.21.15.55:0/996233190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b69a15aa0 msgr2=0x563b699f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.777+0000 7f1045f76640 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b69a15aa0 0x563b699f9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.777+0000 7f1045f76640 1 -- 172.21.15.55:0/996233190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6962b0f0 con 0x563b699f50b0 2024-04-26T21:57:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.778+0000 7f1045f76640 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f50b0 0x563b69a10530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f103c009850 tx=0x7f103c02a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb137fa38df76370 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.779+0000 7f102b7fe640 1 -- 172.21.15.55:0/996233190 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c054070 con 0x563b699f50b0 2024-04-26T21:57:00.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.780+0000 7f102b7fe640 1 -- 172.21.15.55:0/996233190 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f103c050d50 con 0x563b699f50b0 2024-04-26T21:57:00.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.780+0000 7f102b7fe640 1 -- 172.21.15.55:0/996233190 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c031800 con 0x563b699f50b0 2024-04-26T21:57:00.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.781+0000 7f104afc9740 1 -- 172.21.15.55:0/996233190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f50b0 msgr2=0x563b69a10530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.781+0000 7f104afc9740 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f50b0 0x563b69a10530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f103c009850 tx=0x7f103c02a440 comp rx=0 tx=0).stop 2024-04-26T21:57:00.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.782+0000 7f104afc9740 1 -- 172.21.15.55:0/996233190 shutdown_connections 2024-04-26T21:57:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.782+0000 7f104afc9740 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b69a15aa0 0x563b699f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.782+0000 7f104afc9740 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b699f6c40 0x563b69a10a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.782+0000 7f104afc9740 1 --2- 172.21.15.55:0/996233190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f50b0 0x563b69a10530 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.782+0000 7f104afc9740 1 -- 172.21.15.55:0/996233190 >> 172.21.15.55:0/996233190 conn(0x563b699f05a0 msgr2=0x563b699eb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.782+0000 7f104afc9740 1 -- 172.21.15.55:0/996233190 shutdown_connections 2024-04-26T21:57:00.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.782+0000 7f104afc9740 1 -- 172.21.15.55:0/996233190 wait complete. 2024-04-26T21:57:00.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.783+0000 7f104afc9740 1 -- start start 2024-04-26T21:57:00.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.783+0000 7f104afc9740 1 Processor -- start 2024-04-26T21:57:00.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.784+0000 7f104afc9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a2acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.785+0000 7f104afc9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a2ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.785+0000 7f104afc9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 0x563b69a2b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.785+0000 7f104afc9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f10401182f0 con 0x563b699f50b0 2024-04-26T21:57:00.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.785+0000 7f104afc9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f104007bd30 con 0x563b699f6c40 2024-04-26T21:57:00.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.785+0000 7f104afc9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f10401189c0 con 0x563b69a15aa0 2024-04-26T21:57:00.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.785+0000 7f1045f76640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a2acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.785+0000 7f1045f76640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a2acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54558/0 (socket says 172.21.15.55:54558) 2024-04-26T21:57:00.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.785+0000 7f1045f76640 1 -- 172.21.15.55:0/1574769408 learned_addr learned my addr 172.21.15.55:0/1574769408 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:00.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.786+0000 7f104afc9740 10 client.0 _ll_register_callbacks cb 0x7f1030009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:00.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.786+0000 7f1045f76640 1 -- 172.21.15.55:0/1574769408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 msgr2=0x563b69a2b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:00.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.786+0000 7f1045f76640 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 0x563b69a2b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.786+0000 7f1045f76640 1 -- 172.21.15.55:0/1574769408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 msgr2=0x563b69a2ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:00.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.786+0000 7f1045f76640 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a2ff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.786+0000 7f1045f76640 1 -- 172.21.15.55:0/1574769408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6991a4b0 con 0x563b699f50b0 2024-04-26T21:57:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.786+0000 7f1045f76640 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a2acb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f103c005e00 tx=0x7f103c050930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e15c03873b1b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.786+0000 7f10297fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.787+0000 7f10297fa640 1 -- 172.21.15.55:0/1574769408 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c054050 con 0x563b699f50b0 2024-04-26T21:57:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.787+0000 7f104afc9740 1 -- 172.21.15.55:0/1574769408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f103c050d50 con 0x563b699f50b0 2024-04-26T21:57:00.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.787+0000 7f10297fa640 1 -- 172.21.15.55:0/1574769408 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f103c050450 con 0x563b699f50b0 2024-04-26T21:57:00.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.787+0000 7f10297fa640 1 -- 172.21.15.55:0/1574769408 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f103c050b90 con 0x563b699f50b0 2024-04-26T21:57:00.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.787+0000 7f104afc9740 10 client.16029 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:00.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.787+0000 7f104afc9740 1 -- 172.21.15.55:0/1574769408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x563b69a81990 con 0x563b699f50b0 2024-04-26T21:57:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.788+0000 7f10297fa640 1 -- 172.21.15.55:0/1574769408 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f103c050d50 con 0x563b699f50b0 2024-04-26T21:57:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.788+0000 7f10297fa640 1 -- 172.21.15.55:0/1574769408 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f103c059450 con 0x563b699f50b0 2024-04-26T21:57:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.788+0000 7f104afc9740 10 client.16029 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.788+0000 7f104afc9740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.788+0000 7f104afc9740 20 client.16029 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.789+0000 7f104afc9740 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563b69a822d0 0x563b69a84710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.789+0000 7f104afc9740 4 client.16029 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:57:00.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.789+0000 7f104afc9740 1 -- 172.21.15.55:0/1574769408 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b698dc4a0 con 0x563b69a822d0 2024-04-26T21:57:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.789+0000 7f1045775640 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563b69a822d0 0x563b69a84710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.790+0000 7f10297fa640 10 client.16029 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:00.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.790+0000 7f1045775640 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563b69a822d0 0x563b69a84710 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4c58c5158134d8 server_cookie=63af9e262eb2d77b in_seq=0 out_seq=0 2024-04-26T21:57:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.794+0000 7f10297fa640 1 -- 172.21.15.55:0/1574769408 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b698dc4a0 con 0x563b69a822d0 2024-04-26T21:57:00.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.794+0000 7f10297fa640 10 client.16029 handle_command_reply: tid=0 2024-04-26T21:57:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.794+0000 7f104afc9740 1 client.16029 shutdown 2024-04-26T21:57:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.794+0000 7f104afc9740 1 -- 172.21.15.55:0/1574769408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 msgr2=0x563b69a2acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.794+0000 7f104afc9740 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a2acb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f103c005e00 tx=0x7f103c050930 comp rx=0 tx=0).stop 2024-04-26T21:57:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.794+0000 7f104afc9740 1 -- 172.21.15.55:0/1574769408 shutdown_connections 2024-04-26T21:57:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f104afc9740 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563b69a15aa0 0x563b69a2b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f104afc9740 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563b699f6c40 0x563b69a2ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f104afc9740 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563b69a822d0 0x563b69a84710 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f104afc9740 1 --2- 172.21.15.55:0/1574769408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563b699f50b0 0x563b69a2acb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f10297fa640 0 client.16029 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f104afc9740 1 -- 172.21.15.55:0/1574769408 >> 172.21.15.55:0/1574769408 conn(0x563b699f05a0 msgr2=0x563b699f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f104afc9740 1 -- 172.21.15.55:0/1574769408 shutdown_connections 2024-04-26T21:57:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f104afc9740 1 -- 172.21.15.55:0/1574769408 wait complete. 2024-04-26T21:57:00.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.795+0000 7f104afc9740 20 client.16029 trim_cache size 0 max 16384 2024-04-26T21:57:00.798 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:57:00.798 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:57:00.798 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:57:00.798 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:57:00.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.797+0000 7f10481da640 1 -- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f102c0a3e40 msgr2=0x7f102c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.797+0000 7f10481da640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f102c0a3e40 0x7f102c0a6300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f103c000c00 tx=0x7f103c0095f0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.797+0000 7f10481da640 1 -- 172.21.15.55:0/1389552694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f104010a640 msgr2=0x7f104007ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.797+0000 7f10481da640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f104010a640 0x7f104007ab90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f103800b990 tx=0x7f103800be50 comp rx=0 tx=0).stop 2024-04-26T21:57:00.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.798+0000 7f10481da640 1 -- 172.21.15.55:0/1389552694 shutdown_connections 2024-04-26T21:57:00.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.798+0000 7f10481da640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f102c0a3e40 0x7f102c0a6300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.798+0000 7f10481da640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f104010a640 0x7f104007ab90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.798+0000 7f10481da640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1040074540 0x7f10401a7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.798+0000 7f10481da640 1 --2- 172.21.15.55:0/1389552694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1040073be0 0x7f10401a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.798+0000 7f10481da640 1 -- 172.21.15.55:0/1389552694 >> 172.21.15.55:0/1389552694 conn(0x7f104006f780 msgr2=0x7f10400714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:00.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.798+0000 7f10481da640 1 -- 172.21.15.55:0/1389552694 shutdown_connections 2024-04-26T21:57:00.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.798+0000 7f10481da640 1 -- 172.21.15.55:0/1389552694 wait complete. 2024-04-26T21:57:00.818 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:57:00.818 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:57:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.997+0000 7f800446d640 1 -- 172.21.15.55:0/2150616546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc105d60 msgr2=0x7f7ffc108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.997+0000 7f800446d640 1 --2- 172.21.15.55:0/2150616546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc105d60 0x7f7ffc108150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009900 tx=0x7f7fec02a410 comp rx=0 tx=0).stop 2024-04-26T21:57:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.998+0000 7f800446d640 1 -- 172.21.15.55:0/2150616546 shutdown_connections 2024-04-26T21:57:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.998+0000 7f800446d640 1 --2- 172.21.15.55:0/2150616546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ffc108690 0x7f7ffc10aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.998+0000 7f800446d640 1 --2- 172.21.15.55:0/2150616546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc105d60 0x7f7ffc108150 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.998+0000 7f800446d640 1 --2- 172.21.15.55:0/2150616546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc103430 0x7f7ffc105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.998+0000 7f800446d640 1 -- 172.21.15.55:0/2150616546 >> 172.21.15.55:0/2150616546 conn(0x7f7ffc0fce20 msgr2=0x7f7ffc0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.998+0000 7f800446d640 1 -- 172.21.15.55:0/2150616546 shutdown_connections 2024-04-26T21:57:00.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.998+0000 7f800446d640 1 -- 172.21.15.55:0/2150616546 wait complete. 2024-04-26T21:57:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.999+0000 7f800446d640 1 Processor -- start 2024-04-26T21:57:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.999+0000 7f800446d640 1 -- start start 2024-04-26T21:57:01.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.999+0000 7f800446d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ffc103430 0x7f7ffc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.999+0000 7f800446d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc105d60 0x7f7ffc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.999+0000 7f800446d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc108690 0x7f7ffc1a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:00.999+0000 7f800446d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc10e530 con 0x7f7ffc108690 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f800446d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc10e3b0 con 0x7f7ffc105d60 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f800446d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc10e6b0 con 0x7f7ffc103430 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002a0a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc108690 0x7f7ffc1a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002a0a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc108690 0x7f7ffc1a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54590/0 (socket says 172.21.15.55:54590) 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002a0a640 1 -- 172.21.15.55:0/1389612737 learned_addr learned my addr 172.21.15.55:0/1389612737 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002209640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ffc103430 0x7f7ffc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002a0a640 1 -- 172.21.15.55:0/1389612737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ffc103430 msgr2=0x7f7ffc19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002a0a640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ffc103430 0x7f7ffc19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:01.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002a0a640 1 -- 172.21.15.55:0/1389612737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc105d60 msgr2=0x7f7ffc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002a0a640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc105d60 0x7f7ffc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002a0a640 1 -- 172.21.15.55:0/1389612737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc1a2a30 con 0x7f7ffc108690 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.000+0000 7f8002209640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ffc103430 0x7f7ffc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.001+0000 7f8002a0a640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc108690 0x7f7ffc1a23c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400eb10 tx=0x7f7ff400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.001+0000 7f7ff37fe640 1 -- 172.21.15.55:0/1389612737 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4004b20 con 0x7f7ffc108690 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.001+0000 7f7ff37fe640 1 -- 172.21.15.55:0/1389612737 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ff4004cc0 con 0x7f7ffc108690 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.001+0000 7f7ff37fe640 1 -- 172.21.15.55:0/1389612737 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4010690 con 0x7f7ffc108690 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.001+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc1a2d20 con 0x7f7ffc108690 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.001+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc1a3220 con 0x7f7ffc108690 2024-04-26T21:57:01.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.003+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc10a1e0 con 0x7f7ffc108690 2024-04-26T21:57:01.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.006+0000 7f7ff37fe640 1 -- 172.21.15.55:0/1389612737 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7ff4005020 con 0x7f7ffc108690 2024-04-26T21:57:01.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.008+0000 7f7ff37fe640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7fe80a3cc0 0x7f7fe80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:01.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.008+0000 7f8002209640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7fe80a3cc0 0x7f7fe80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:01.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.008+0000 7f7ff37fe640 1 -- 172.21.15.55:0/1389612737 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7ff40d0ca0 con 0x7f7ffc108690 2024-04-26T21:57:01.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.009+0000 7f8002209640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7fe80a3cc0 0x7f7fe80a6180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800baa0 tx=0x7f7ff8005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:01.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.014+0000 7f7ff37fe640 1 -- 172.21.15.55:0/1389612737 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7ff409d480 con 0x7f7ffc108690 2024-04-26T21:57:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:01 smithi055 ceph-mon[29221]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 827 MiB/s wr, 944 op/s 2024-04-26T21:57:01.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:01 smithi165 ceph-mon[33296]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 827 MiB/s wr, 944 op/s 2024-04-26T21:57:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:01 smithi064 ceph-mon[32765]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 827 MiB/s wr, 944 op/s 2024-04-26T21:57:01.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.572+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ffc02dad0 con 0x7f7ffc108690 2024-04-26T21:57:01.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.586+0000 7f7ff37fe640 1 -- 172.21.15.55:0/1389612737 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11864 (secure 0 0 0) 0x7f7ff409cd80 con 0x7f7ffc108690 2024-04-26T21:57:01.593 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:57:01.593 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:56:06.126250+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24509,24551,24502]},"id":1}]} 2024-04-26T21:57:01.593 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 22 2024-04-26T21:57:01.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.602+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7fe80a3cc0 msgr2=0x7f7fe80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:01.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.602+0000 7f800446d640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7fe80a3cc0 0x7f7fe80a6180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800baa0 tx=0x7f7ff8005d90 comp rx=0 tx=0).stop 2024-04-26T21:57:01.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.603+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc108690 msgr2=0x7f7ffc1a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:01.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.603+0000 7f800446d640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc108690 0x7f7ffc1a23c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400eb10 tx=0x7f7ff400c2d0 comp rx=0 tx=0).stop 2024-04-26T21:57:01.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.604+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 shutdown_connections 2024-04-26T21:57:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.604+0000 7f800446d640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7fe80a3cc0 0x7f7fe80a6180 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.604+0000 7f800446d640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7ffc108690 0x7f7ffc1a23c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.604+0000 7f800446d640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ffc105d60 0x7f7ffc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.604+0000 7f800446d640 1 --2- 172.21.15.55:0/1389612737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ffc103430 0x7f7ffc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.604+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 >> 172.21.15.55:0/1389612737 conn(0x7f7ffc0fce20 msgr2=0x7f7ffc102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.604+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 shutdown_connections 2024-04-26T21:57:01.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:01.605+0000 7f800446d640 1 -- 172.21.15.55:0/1389612737 wait complete. 2024-04-26T21:57:01.667 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 -- 172.21.15.55:0/3021851557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c4079ef0 msgr2=0x7fa4c407c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/3021851557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c4079ef0 0x7fa4c407c3b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009900 tx=0x7fa4b802a410 comp rx=0 tx=0).stop 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 -- 172.21.15.55:0/3021851557 shutdown_connections 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/3021851557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c4079ef0 0x7fa4c407c3b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/3021851557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4c4077520 0x7fa4c40799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/3021851557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c4074640 0x7fa4c4074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 -- 172.21.15.55:0/3021851557 >> 172.21.15.55:0/3021851557 conn(0x7fa4c406f780 msgr2=0x7fa4c4071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 -- 172.21.15.55:0/3021851557 shutdown_connections 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.065+0000 7fa4c8db3640 1 -- 172.21.15.55:0/3021851557 wait complete. 2024-04-26T21:57:02.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.066+0000 7fa4c8db3640 1 Processor -- start 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.066+0000 7fa4c8db3640 1 -- start start 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.067+0000 7fa4c8db3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c4074640 0x7fa4c412db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.067+0000 7fa4c8db3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c4077520 0x7fa4c41300c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.067+0000 7fa4c8db3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4c4079ef0 0x7fa4c4132610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.067+0000 7fa4c8db3640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4083f90 con 0x7fa4c4077520 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.067+0000 7fa4c8db3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4083e10 con 0x7fa4c4074640 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.067+0000 7fa4c8db3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4084110 con 0x7fa4c4079ef0 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.068+0000 7fa4c2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c4077520 0x7fa4c41300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.068+0000 7fa4c2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c4077520 0x7fa4c41300c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54598/0 (socket says 172.21.15.55:54598) 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.068+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/2331591393 learned_addr learned my addr 172.21.15.55:0/2331591393 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.069+0000 7fa4c3fff640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4c4079ef0 0x7fa4c4132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.069+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c4074640 0x7fa4c412db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.070+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/2331591393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4c4079ef0 msgr2=0x7fa4c4132610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.070+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4c4079ef0 0x7fa4c4132610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.070+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/2331591393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c4074640 msgr2=0x7fa4c412db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.070+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c4074640 0x7fa4c412db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.070+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/2331591393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c4132ed0 con 0x7fa4c4077520 2024-04-26T21:57:02.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.071+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c4077520 0x7fa4c41300c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc052a90 tx=0x7fa4bc052f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.091+0000 7fa4c0ff9640 1 -- 172.21.15.55:0/2331591393 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc050e50 con 0x7fa4c4077520 2024-04-26T21:57:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.092+0000 7fa4c0ff9640 1 -- 172.21.15.55:0/2331591393 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4bc005020 con 0x7fa4c4077520 2024-04-26T21:57:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.092+0000 7fa4c0ff9640 1 -- 172.21.15.55:0/2331591393 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0556e0 con 0x7fa4c4077520 2024-04-26T21:57:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.095+0000 7fa4c8db3640 1 -- 172.21.15.55:0/2331591393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c41331c0 con 0x7fa4c4077520 2024-04-26T21:57:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.095+0000 7fa4c8db3640 1 -- 172.21.15.55:0/2331591393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c4133710 con 0x7fa4c4077520 2024-04-26T21:57:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.100+0000 7fa4cce09740 1 Processor -- start 2024-04-26T21:57:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.100+0000 7fa4cce09740 1 -- start start 2024-04-26T21:57:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.100+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac463a30 0x55d7ac463e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.100+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac465080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.100+0000 7fa4c3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac463a30 0x55d7ac463e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.100+0000 7fa4c3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac463a30 0x55d7ac463e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54602/0 (socket says 172.21.15.55:54602) 2024-04-26T21:57:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.100+0000 7fa4c3fff640 1 -- 172.21.15.55:0/1432660045 learned_addr learned my addr 172.21.15.55:0/1432660045 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.100+0000 7fa4cce09740 1 --2- 172.21.15.55:0/1432660045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac4655c0 0x55d7ac46be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.101+0000 7fa4cce09740 1 -- 172.21.15.55:0/1432660045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d7ac3c7d80 con 0x55d7ac463a30 2024-04-26T21:57:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.101+0000 7fa4cce09740 1 -- 172.21.15.55:0/1432660045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d7ac381a20 con 0x55d7ac464c00 2024-04-26T21:57:02.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.101+0000 7fa4cce09740 1 -- 172.21.15.55:0/1432660045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7ac3c3d40 con 0x55d7ac4655c0 2024-04-26T21:57:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.101+0000 7fa4c3fff640 1 -- 172.21.15.55:0/1432660045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac4655c0 msgr2=0x55d7ac46be50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:57:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.101+0000 7fa4c3fff640 1 --2- 172.21.15.55:0/1432660045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac4655c0 0x55d7ac46be50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.101+0000 7fa4c3fff640 1 -- 172.21.15.55:0/1432660045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 msgr2=0x55d7ac465080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.101+0000 7fa4c3fff640 1 --2- 172.21.15.55:0/1432660045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac465080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.101+0000 7fa4c3fff640 1 -- 172.21.15.55:0/1432660045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ac38cc30 con 0x55d7ac463a30 2024-04-26T21:57:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.103+0000 7fa4c3fff640 1 --2- 172.21.15.55:0/1432660045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac463a30 0x55d7ac463e50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009850 tx=0x7fa4b8004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59360a0cf0c4f0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.104+0000 7fa4c0ff9640 1 -- 172.21.15.55:0/2331591393 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa4bc0051d0 con 0x7fa4c4077520 2024-04-26T21:57:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.109+0000 7fa4c0ff9640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa4ac0a3ea0 0x7fa4ac0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.110+0000 7fa4c0ff9640 1 -- 172.21.15.55:0/2331591393 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa4bc114c60 con 0x7fa4c4077520 2024-04-26T21:57:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.112+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa4ac0a3ea0 0x7fa4ac0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.113+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa4ac0a3ea0 0x7fa4ac0a6360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4073cb0 tx=0x7fa4b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.114+0000 7fa4a9ffb640 1 -- 172.21.15.55:0/1432660045 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b803f070 con 0x55d7ac463a30 2024-04-26T21:57:02.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.115+0000 7fa4a9ffb640 1 -- 172.21.15.55:0/1432660045 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b80044b0 con 0x55d7ac463a30 2024-04-26T21:57:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.117+0000 7fa4cce09740 1 -- 172.21.15.55:0/1432660045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac463a30 msgr2=0x55d7ac463e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.117+0000 7fa4cce09740 1 --2- 172.21.15.55:0/1432660045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac463a30 0x55d7ac463e50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009850 tx=0x7fa4b8004110 comp rx=0 tx=0).stop 2024-04-26T21:57:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.118+0000 7fa4cce09740 1 -- 172.21.15.55:0/1432660045 shutdown_connections 2024-04-26T21:57:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.118+0000 7fa4cce09740 1 --2- 172.21.15.55:0/1432660045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac4655c0 0x55d7ac46be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.118+0000 7fa4cce09740 1 --2- 172.21.15.55:0/1432660045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac465080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.118+0000 7fa4cce09740 1 --2- 172.21.15.55:0/1432660045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac463a30 0x55d7ac463e50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.118+0000 7fa4cce09740 1 -- 172.21.15.55:0/1432660045 >> 172.21.15.55:0/1432660045 conn(0x55d7ac3a4910 msgr2=0x55d7ac45f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.121+0000 7fa4cce09740 1 -- 172.21.15.55:0/1432660045 shutdown_connections 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.121+0000 7fa4cce09740 1 -- 172.21.15.55:0/1432660045 wait complete. 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.122+0000 7fa4cce09740 1 -- start start 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.123+0000 7fa4cce09740 1 Processor -- start 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.123+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 0x55d7ac46dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.123+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac464c00 0x55d7ac46e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.123+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac4655c0 0x55d7ac475180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.123+0000 7fa4cce09740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d7ac46c5c0 con 0x55d7ac464c00 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.123+0000 7fa4cce09740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d7ac34edc0 con 0x55d7ac463a30 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.123+0000 7fa4cce09740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7ac46c740 con 0x55d7ac4655c0 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.123+0000 7fa4cce09740 10 client.0 _ll_register_callbacks cb 0x55d7ac36a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.125+0000 7fa4c2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac464c00 0x55d7ac46e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.125+0000 7fa4c2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac464c00 0x55d7ac46e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54616/0 (socket says 172.21.15.55:54616) 2024-04-26T21:57:02.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.125+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/782852007 learned_addr learned my addr 172.21.15.55:0/782852007 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.126+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/782852007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac4655c0 msgr2=0x55d7ac475180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.126+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac4655c0 0x55d7ac475180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.126+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/782852007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 msgr2=0x55d7ac46dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.126+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 0x55d7ac46dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.126+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/782852007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ac0d93f0 con 0x55d7ac464c00 2024-04-26T21:57:02.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.127+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac464c00 0x55d7ac46e140 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0e0cd0 tx=0x7fa4bc0e1580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993ac8ed5fd182a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.127+0000 7fa4a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:02.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.137+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/782852007 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0054f0 con 0x55d7ac464c00 2024-04-26T21:57:02.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.137+0000 7fa4cce09740 1 -- 172.21.15.55:0/782852007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b80044b0 con 0x55d7ac464c00 2024-04-26T21:57:02.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.137+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/782852007 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa4bc112b80 con 0x55d7ac464c00 2024-04-26T21:57:02.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.142+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/782852007 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa4bc005690 con 0x55d7ac464c00 2024-04-26T21:57:02.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.142+0000 7fa4cce09740 10 client.16044 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:02.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.142+0000 7fa4cce09740 1 -- 172.21.15.55:0/782852007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d7ac1a37d0 con 0x55d7ac464c00 2024-04-26T21:57:02.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.144+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/782852007 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b80044b0 con 0x55d7ac464c00 2024-04-26T21:57:02.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.144+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/782852007 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fa4bc1010d0 con 0x55d7ac464c00 2024-04-26T21:57:02.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.145+0000 7fa4cce09740 10 client.16044 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:02.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.145+0000 7fa4cce09740 10 client.16044 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:57:02.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.145+0000 7fa4cce09740 20 client.16044 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:02.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.145+0000 7fa4cce09740 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55d7ac479ea0 0x55d7ac47c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.145+0000 7fa4cce09740 4 client.16044 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:02.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.145+0000 7fa4cce09740 1 -- 172.21.15.55:0/782852007 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7ac3d5990 con 0x55d7ac479ea0 2024-04-26T21:57:02.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.151+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55d7ac479ea0 0x55d7ac47c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.169+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55d7ac479ea0 0x55d7ac47c340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a71a636b4e833686 server_cookie=d359be3c044d1aa0 in_seq=0 out_seq=0 2024-04-26T21:57:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.169+0000 7fa4a2ffd640 10 client.16044 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:57:02.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.171+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/782852007 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7ac3d5990 con 0x55d7ac479ea0 2024-04-26T21:57:02.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.171+0000 7fa4a2ffd640 10 client.16044 handle_command_reply: tid=0 2024-04-26T21:57:02.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.171+0000 7fa4cce09740 1 client.16044 shutdown 2024-04-26T21:57:02.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4cce09740 1 -- 172.21.15.55:0/782852007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac464c00 msgr2=0x55d7ac46e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4cce09740 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac464c00 0x55d7ac46e140 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0e0cd0 tx=0x7fa4bc0e1580 comp rx=0 tx=0).stop 2024-04-26T21:57:02.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4cce09740 1 -- 172.21.15.55:0/782852007 shutdown_connections 2024-04-26T21:57:02.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4cce09740 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55d7ac479ea0 0x55d7ac47c340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4cce09740 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac4655c0 0x55d7ac475180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4cce09740 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac464c00 0x55d7ac46e140 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4cce09740 1 --2- 172.21.15.55:0/782852007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 0x55d7ac46dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4cce09740 1 -- 172.21.15.55:0/782852007 >> 172.21.15.55:0/782852007 conn(0x55d7ac3a4910 msgr2=0x55d7ac45e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.173+0000 7fa4a2ffd640 0 client.16044 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:57:02.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.174+0000 7fa4cce09740 1 -- 172.21.15.55:0/782852007 shutdown_connections 2024-04-26T21:57:02.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.175+0000 7fa4cce09740 1 -- 172.21.15.55:0/782852007 wait complete. 2024-04-26T21:57:02.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.175+0000 7fa4cce09740 20 client.16044 trim_cache size 0 max 16384 2024-04-26T21:57:02.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.239+0000 7fa4cce09740 1 Processor -- start 2024-04-26T21:57:02.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.239+0000 7fa4cce09740 1 -- start start 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.239+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac463a30 0x55d7ac4683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.239+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac475140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.240+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 0x55d7ac468900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.240+0000 7fa4cce09740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c413ef30 con 0x55d7ac4655c0 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.240+0000 7fa4cce09740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c413c800 con 0x55d7ac464c00 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.240+0000 7fa4cce09740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c413a650 con 0x55d7ac463a30 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.241+0000 7fa4c2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac475140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.241+0000 7fa4c2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac475140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45920/0 (socket says 172.21.15.55:45920) 2024-04-26T21:57:02.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.241+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/3058468093 learned_addr learned my addr 172.21.15.55:0/3058468093 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.241+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac463a30 0x55d7ac4683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.241+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/3058468093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac463a30 msgr2=0x55d7ac4683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.241+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac463a30 0x55d7ac4683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.241+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/3058468093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 msgr2=0x55d7ac468900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.241+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 0x55d7ac468900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.242+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/3058468093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ac45acd0 con 0x55d7ac464c00 2024-04-26T21:57:02.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.242+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac463a30 0x55d7ac4683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:02.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.242+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac475140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009270 tx=0x7fa4bc065440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42b740050ebc5b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.249+0000 7fa4a97fa640 1 -- 172.21.15.55:0/3058468093 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0bfbb0 con 0x55d7ac464c00 2024-04-26T21:57:02.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.249+0000 7fa4a97fa640 1 -- 172.21.15.55:0/3058468093 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4bc0fa070 con 0x55d7ac464c00 2024-04-26T21:57:02.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.249+0000 7fa4a97fa640 1 -- 172.21.15.55:0/3058468093 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc112de0 con 0x55d7ac464c00 2024-04-26T21:57:02.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.251+0000 7fa4cce09740 1 -- 172.21.15.55:0/3058468093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 msgr2=0x55d7ac475140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.251+0000 7fa4cce09740 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac475140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009270 tx=0x7fa4bc065440 comp rx=0 tx=0).stop 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.251+0000 7fa4cce09740 1 -- 172.21.15.55:0/3058468093 shutdown_connections 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.251+0000 7fa4cce09740 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 0x55d7ac468900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.251+0000 7fa4cce09740 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac464c00 0x55d7ac475140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.251+0000 7fa4cce09740 1 --2- 172.21.15.55:0/3058468093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac463a30 0x55d7ac4683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.251+0000 7fa4cce09740 1 -- 172.21.15.55:0/3058468093 >> 172.21.15.55:0/3058468093 conn(0x55d7ac3a4910 msgr2=0x55d7ac46dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.253+0000 7fa4cce09740 1 -- 172.21.15.55:0/3058468093 shutdown_connections 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.253+0000 7fa4cce09740 1 -- 172.21.15.55:0/3058468093 wait complete. 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.254+0000 7fa4cce09740 1 -- start start 2024-04-26T21:57:02.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.254+0000 7fa4cce09740 1 Processor -- start 2024-04-26T21:57:02.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.255+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 0x55d7ac471e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.255+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac464c00 0x55d7ac472380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.255+0000 7fa4cce09740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 0x55d7ac4de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.256+0000 7fa4cce09740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c413acc0 con 0x55d7ac4655c0 2024-04-26T21:57:02.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.256+0000 7fa4cce09740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c413c130 con 0x55d7ac463a30 2024-04-26T21:57:02.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.256+0000 7fa4cce09740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c413b390 con 0x55d7ac464c00 2024-04-26T21:57:02.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.256+0000 7fa4cce09740 10 client.0 _ll_register_callbacks cb 0x7fa4bc0e1440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:02.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.256+0000 7fa4c2ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac464c00 0x55d7ac472380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.257+0000 7fa4c37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 0x55d7ac471e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.261+0000 7fa4c2ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac464c00 0x55d7ac472380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:46350/0 (socket says 172.21.15.55:46350) 2024-04-26T21:57:02.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.261+0000 7fa4c2ffd640 1 -- 172.21.15.55:0/629648816 learned_addr learned my addr 172.21.15.55:0/629648816 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.262+0000 7fa4c3fff640 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 0x55d7ac4de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.263+0000 7fa4c37fe640 1 -- 172.21.15.55:0/629648816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac464c00 msgr2=0x55d7ac472380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.263+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac464c00 0x55d7ac472380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.263+0000 7fa4c37fe640 1 -- 172.21.15.55:0/629648816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 msgr2=0x55d7ac4de4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.263+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 0x55d7ac4de4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.263+0000 7fa4c37fe640 1 -- 172.21.15.55:0/629648816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ac472970 con 0x55d7ac463a30 2024-04-26T21:57:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.267+0000 7fa4c37fe640 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 0x55d7ac471e40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55d7ac47cb90 tx=0x7fa4b400c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=627aed505efa3662 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.275+0000 7fa4a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:57:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.275+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/629648816 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4019040 con 0x55d7ac463a30 2024-04-26T21:57:02.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.276+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/629648816 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa4b4013d60 con 0x55d7ac463a30 2024-04-26T21:57:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.279+0000 7fa4cce09740 1 -- 172.21.15.55:0/629648816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4bc0fa070 con 0x55d7ac463a30 2024-04-26T21:57:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.281+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/629648816 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa4b40220e0 con 0x55d7ac463a30 2024-04-26T21:57:02.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.284+0000 7fa4cce09740 10 client.25502 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:02.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.284+0000 7fa4cce09740 1 -- 172.21.15.55:0/629648816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d7ac4dfaa0 con 0x55d7ac463a30 2024-04-26T21:57:02.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.285+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/629648816 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4bc0fa070 con 0x55d7ac463a30 2024-04-26T21:57:02.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.286+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/629648816 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fa4b402b970 con 0x55d7ac463a30 2024-04-26T21:57:02.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.287+0000 7fa4cce09740 10 client.25502 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:02.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.287+0000 7fa4cce09740 10 client.25502 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:57:02.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.287+0000 7fa4cce09740 20 client.25502 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:02.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.287+0000 7fa4cce09740 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55d7ac4e03b0 0x55d7ac4e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.287+0000 7fa4cce09740 4 client.25502 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:02.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.287+0000 7fa4cce09740 1 -- 172.21.15.55:0/629648816 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7ac46c1e0 con 0x55d7ac4e03b0 2024-04-26T21:57:02.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.290+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55d7ac4e03b0 0x55d7ac4e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.296+0000 7fa4c2ffd640 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55d7ac4e03b0 0x55d7ac4e27d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be75a41340fa38b1 server_cookie=330f877e34d39dbe in_seq=0 out_seq=0 2024-04-26T21:57:02.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.296+0000 7fa4a2ffd640 10 client.25502 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:57:02.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.297+0000 7fa4a2ffd640 1 -- 172.21.15.55:0/629648816 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7ac46c1e0 con 0x55d7ac4e03b0 2024-04-26T21:57:02.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.297+0000 7fa4a2ffd640 10 client.25502 handle_command_reply: tid=0 2024-04-26T21:57:02.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.300+0000 7fa4cce09740 1 client.25502 shutdown 2024-04-26T21:57:02.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.300+0000 7fa4cce09740 1 -- 172.21.15.55:0/629648816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 msgr2=0x55d7ac471e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.300+0000 7fa4cce09740 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 0x55d7ac471e40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55d7ac47cb90 tx=0x7fa4b400c7d0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.304+0000 7fa4cce09740 1 -- 172.21.15.55:0/629648816 shutdown_connections 2024-04-26T21:57:02.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.304+0000 7fa4cce09740 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55d7ac4e03b0 0x55d7ac4e27d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.304+0000 7fa4cce09740 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d7ac4655c0 0x55d7ac4de4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.304+0000 7fa4cce09740 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7ac464c00 0x55d7ac472380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.304+0000 7fa4cce09740 1 --2- 172.21.15.55:0/629648816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d7ac463a30 0x55d7ac471e40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.304+0000 7fa4cce09740 1 -- 172.21.15.55:0/629648816 >> 172.21.15.55:0/629648816 conn(0x55d7ac3a4910 msgr2=0x55d7ac46dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.304+0000 7fa4a2ffd640 0 client.25502 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:57:02.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.309+0000 7fa4cce09740 1 -- 172.21.15.55:0/629648816 shutdown_connections 2024-04-26T21:57:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.315+0000 7fa4cce09740 1 -- 172.21.15.55:0/629648816 wait complete. 2024-04-26T21:57:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.315+0000 7fa4cce09740 20 client.25502 trim_cache size 0 max 16384 2024-04-26T21:57:02.318 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:57:02.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.318+0000 7fa4c8db3640 1 -- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa4ac0a3ea0 msgr2=0x7fa4ac0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.318+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa4ac0a3ea0 0x7fa4ac0a6360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4073cb0 tx=0x7fa4b4008040 comp rx=0 tx=0).stop 2024-04-26T21:57:02.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.318+0000 7fa4c8db3640 1 -- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c4077520 msgr2=0x7fa4c41300c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.318+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c4077520 0x7fa4c41300c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc052a90 tx=0x7fa4bc052f50 comp rx=0 tx=0).stop 2024-04-26T21:57:02.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.320+0000 7fa4c8db3640 1 -- 172.21.15.55:0/2331591393 shutdown_connections 2024-04-26T21:57:02.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.320+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa4ac0a3ea0 0x7fa4ac0a6360 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.320+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4c4079ef0 0x7fa4c4132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.320+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c4077520 0x7fa4c41300c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.320+0000 7fa4c8db3640 1 --2- 172.21.15.55:0/2331591393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4c4074640 0x7fa4c412db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.320+0000 7fa4c8db3640 1 -- 172.21.15.55:0/2331591393 >> 172.21.15.55:0/2331591393 conn(0x7fa4c406f780 msgr2=0x7fa4c4077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.327+0000 7fa4c8db3640 1 -- 172.21.15.55:0/2331591393 shutdown_connections 2024-04-26T21:57:02.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.331+0000 7fa4c8db3640 1 -- 172.21.15.55:0/2331591393 wait complete. 2024-04-26T21:57:02.377 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:57:02.377 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:02 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1389612737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:02.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:02 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1389612737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:02.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:02 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1389612737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:02.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.665+0000 7f1c113af640 1 -- 172.21.15.55:0/1078297505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c073e20 msgr2=0x7f1c0c074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.665+0000 7f1c113af640 1 --2- 172.21.15.55:0/1078297505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c073e20 0x7f1c0c074240 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009900 tx=0x7f1c0002a410 comp rx=0 tx=0).stop 2024-04-26T21:57:02.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.671+0000 7f1c113af640 1 -- 172.21.15.55:0/1078297505 shutdown_connections 2024-04-26T21:57:02.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.671+0000 7f1c113af640 1 --2- 172.21.15.55:0/1078297505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c0c10aa10 0x7f1c0c10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.671+0000 7f1c113af640 1 --2- 172.21.15.55:0/1078297505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0c10a070 0x7f1c0c10a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.671+0000 7f1c113af640 1 --2- 172.21.15.55:0/1078297505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c073e20 0x7f1c0c074240 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.671+0000 7f1c113af640 1 -- 172.21.15.55:0/1078297505 >> 172.21.15.55:0/1078297505 conn(0x7f1c0c06f840 msgr2=0x7f1c0c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.676+0000 7f1c113af640 1 -- 172.21.15.55:0/1078297505 shutdown_connections 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.678+0000 7f1c113af640 1 -- 172.21.15.55:0/1078297505 wait complete. 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.680+0000 7f1c113af640 1 Processor -- start 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.680+0000 7f1c113af640 1 -- start start 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.681+0000 7f1c113af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c10a070 0x7f1c0c115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.681+0000 7f1c113af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0c10aa10 0x7f1c0c1163d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.681+0000 7f1c113af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c0c119420 0x7f1c0c1b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.681+0000 7f1c113af640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c11d280 con 0x7f1c0c10a070 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.681+0000 7f1c113af640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c11d100 con 0x7f1c0c119420 2024-04-26T21:57:02.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.681+0000 7f1c113af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c11d400 con 0x7f1c0c10aa10 2024-04-26T21:57:02.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.682+0000 7f1c0affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c10a070 0x7f1c0c115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.682+0000 7f1c0affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c10a070 0x7f1c0c115e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54648/0 (socket says 172.21.15.55:54648) 2024-04-26T21:57:02.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.682+0000 7f1c0affd640 1 -- 172.21.15.55:0/1755174418 learned_addr learned my addr 172.21.15.55:0/1755174418 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.683+0000 7f1c0affd640 1 -- 172.21.15.55:0/1755174418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0c10aa10 msgr2=0x7f1c0c1163d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:57:02.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.683+0000 7f1c0affd640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0c10aa10 0x7f1c0c1163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.683+0000 7f1c0affd640 1 -- 172.21.15.55:0/1755174418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c0c119420 msgr2=0x7f1c0c1b7990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:57:02.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.683+0000 7f1c0affd640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c0c119420 0x7f1c0c1b7990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.683+0000 7f1c0affd640 1 -- 172.21.15.55:0/1755174418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c1b8030 con 0x7f1c0c10a070 2024-04-26T21:57:02.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.683+0000 7f1c0affd640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c10a070 0x7f1c0c115e90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1c00034040 tx=0x7f1c000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.685+0000 7f1bfbfff640 1 -- 172.21.15.55:0/1755174418 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00040070 con 0x7f1c0c10a070 2024-04-26T21:57:02.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.685+0000 7f1bfbfff640 1 -- 172.21.15.55:0/1755174418 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c00003880 con 0x7f1c0c10a070 2024-04-26T21:57:02.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.685+0000 7f1bfbfff640 1 -- 172.21.15.55:0/1755174418 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00035040 con 0x7f1c0c10a070 2024-04-26T21:57:02.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.685+0000 7f1c113af640 1 -- 172.21.15.55:0/1755174418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c1b82c0 con 0x7f1c0c10a070 2024-04-26T21:57:02.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.685+0000 7f1c113af640 1 -- 172.21.15.55:0/1755174418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c1b8760 con 0x7f1c0c10a070 2024-04-26T21:57:02.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.693+0000 7f1bfbfff640 1 -- 172.21.15.55:0/1755174418 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1c00003a30 con 0x7f1c0c10a070 2024-04-26T21:57:02.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.694+0000 7f1c141a4740 1 Processor -- start 2024-04-26T21:57:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.695+0000 7f1bfbfff640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1bf40a3e40 0x7f1bf40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.696+0000 7f1bfbfff640 1 -- 172.21.15.55:0/1755174418 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1c000f0ab0 con 0x7f1c0c10a070 2024-04-26T21:57:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.700+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1bf40a3e40 0x7f1bf40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.700+0000 7f1c141a4740 1 -- start start 2024-04-26T21:57:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.701+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43cceca00 0x55a43ccece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.701+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 0x55a43ccee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.701+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccf4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.701+0000 7f1c141a4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a43cc50d80 con 0x55a43ccee590 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.701+0000 7f1c141a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a43cc0aa20 con 0x55a43ccedbd0 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.701+0000 7f1c141a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a43cc4cd40 con 0x55a43cceca00 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.704+0000 7f1c0b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccf4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.704+0000 7f1c0b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccf4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54650/0 (socket says 172.21.15.55:54650) 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.704+0000 7f1c0b7fe640 1 -- 172.21.15.55:0/3404610206 learned_addr learned my addr 172.21.15.55:0/3404610206 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.704+0000 7f1c0a7fc640 1 --2- 172.21.15.55:0/3404610206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 0x55a43ccee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.705+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1bf40a3e40 0x7f1bf40a6300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00a8e0 tx=0x7f1bfc00ace0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.705+0000 7f1c0b7fe640 1 -- 172.21.15.55:0/3404610206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43cceca00 msgr2=0x55a43ccece20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.705+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/3404610206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43cceca00 0x55a43ccece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.705+0000 7f1c0b7fe640 1 -- 172.21.15.55:0/3404610206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 msgr2=0x55a43ccee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.705+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/3404610206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 0x55a43ccee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.705+0000 7f1c0b7fe640 1 -- 172.21.15.55:0/3404610206 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43cc15c30 con 0x55a43ccee590 2024-04-26T21:57:02.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.706+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/3404610206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccf4e20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00cc70 tx=0x7f1bfc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d401f7164a0559f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.706+0000 7f1bf97fa640 1 -- 172.21.15.55:0/3404610206 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc020070 con 0x55a43ccee590 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.707+0000 7f1bf97fa640 1 -- 172.21.15.55:0/3404610206 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bfc018040 con 0x55a43ccee590 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.707+0000 7f1bf97fa640 1 -- 172.21.15.55:0/3404610206 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc01c7f0 con 0x55a43ccee590 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.708+0000 7f1c141a4740 1 -- 172.21.15.55:0/3404610206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 msgr2=0x55a43ccf4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.708+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3404610206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccf4e20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00cc70 tx=0x7f1bfc015000 comp rx=0 tx=0).stop 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.709+0000 7f1c141a4740 1 -- 172.21.15.55:0/3404610206 shutdown_connections 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.709+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3404610206 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccf4e20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.709+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3404610206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 0x55a43ccee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.709+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3404610206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43cceca00 0x55a43ccece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.709+0000 7f1c141a4740 1 -- 172.21.15.55:0/3404610206 >> 172.21.15.55:0/3404610206 conn(0x55a43cc2d910 msgr2=0x55a43cce82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.709+0000 7f1c141a4740 1 -- 172.21.15.55:0/3404610206 shutdown_connections 2024-04-26T21:57:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.709+0000 7f1c141a4740 1 -- 172.21.15.55:0/3404610206 wait complete. 2024-04-26T21:57:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 1 -- start start 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 1 Processor -- start 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43cceca00 0x55a43ccf6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 0x55a43ccf7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 0x55a43ccfe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a43ccf55a0 con 0x55a43cceca00 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a43cbd7dc0 con 0x55a43ccee590 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a43ccf5720 con 0x55a43ccedbd0 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.710+0000 7f1c141a4740 10 client.0 _ll_register_callbacks cb 0x55a43cbf3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.712+0000 7f1c0a7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 0x55a43ccf7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.712+0000 7f1c0affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43cceca00 0x55a43ccf6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.712+0000 7f1c0a7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 0x55a43ccf7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:46382/0 (socket says 172.21.15.55:46382) 2024-04-26T21:57:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.712+0000 7f1c0a7fc640 1 -- 172.21.15.55:0/3405618259 learned_addr learned my addr 172.21.15.55:0/3405618259 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.712+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 0x55a43ccfe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.714+0000 7f1c0affd640 1 -- 172.21.15.55:0/3405618259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 msgr2=0x55a43ccf7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.714+0000 7f1c0affd640 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 0x55a43ccf7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.714+0000 7f1c0affd640 1 -- 172.21.15.55:0/3405618259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 msgr2=0x55a43ccfe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.714+0000 7f1c0affd640 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 0x55a43ccfe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.714+0000 7f1c0affd640 1 -- 172.21.15.55:0/3405618259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43c9623f0 con 0x55a43cceca00 2024-04-26T21:57:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.715+0000 7f1c0affd640 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43cceca00 0x55a43ccf6d30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1c000ee3c0 tx=0x7f1c000bad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e9f3952f480a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.716+0000 7f1bee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.716+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3405618259 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000c0f40 con 0x55a43cceca00 2024-04-26T21:57:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.717+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3405618259 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1c00031640 con 0x55a43cceca00 2024-04-26T21:57:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.717+0000 7f1c141a4740 1 -- 172.21.15.55:0/3405618259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bfc018040 con 0x55a43cceca00 2024-04-26T21:57:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.717+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3405618259 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1c000c5800 con 0x55a43cceca00 2024-04-26T21:57:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.717+0000 7f1c141a4740 10 client.16059 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.717+0000 7f1c141a4740 1 -- 172.21.15.55:0/3405618259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a43ca2c7d0 con 0x55a43cceca00 2024-04-26T21:57:02.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.718+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3405618259 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bfc018040 con 0x55a43cceca00 2024-04-26T21:57:02.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.719+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3405618259 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f1c0004c060 con 0x55a43cceca00 2024-04-26T21:57:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.720+0000 7f1c141a4740 10 client.16059 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.720+0000 7f1c141a4740 10 client.16059 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:57:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.720+0000 7f1c141a4740 20 client.16059 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:02.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.720+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a43cd02b20 0x55a43cd04f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.720+0000 7f1c141a4740 4 client.16059 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:02.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.720+0000 7f1c141a4740 1 -- 172.21.15.55:0/3405618259 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a43cc5e990 con 0x55a43cd02b20 2024-04-26T21:57:02.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.721+0000 7f1c0a7fc640 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a43cd02b20 0x55a43cd04f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.724+0000 7f1c0a7fc640 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a43cd02b20 0x55a43cd04f50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ac733c88d15e4c2 server_cookie=b5bd06a4430ce0a9 in_seq=0 out_seq=0 2024-04-26T21:57:02.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.724+0000 7f1bee7fc640 10 client.16059 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:57:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.726+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3405618259 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a43cc5e990 con 0x55a43cd02b20 2024-04-26T21:57:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.726+0000 7f1bee7fc640 10 client.16059 handle_command_reply: tid=0 2024-04-26T21:57:02.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.726+0000 7f1c141a4740 1 client.16059 shutdown 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.726+0000 7f1c141a4740 1 -- 172.21.15.55:0/3405618259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43cceca00 msgr2=0x55a43ccf6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.726+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43cceca00 0x55a43ccf6d30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1c000ee3c0 tx=0x7f1c000bad90 comp rx=0 tx=0).stop 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.727+0000 7f1c141a4740 1 -- 172.21.15.55:0/3405618259 shutdown_connections 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.727+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a43cd02b20 0x55a43cd04f50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.727+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 0x55a43ccfe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.727+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 0x55a43ccf7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.727+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3405618259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43cceca00 0x55a43ccf6d30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.727+0000 7f1c141a4740 1 -- 172.21.15.55:0/3405618259 >> 172.21.15.55:0/3405618259 conn(0x55a43cc2d910 msgr2=0x55a43cce7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.727+0000 7f1bee7fc640 0 client.16059 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:57:02.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.728+0000 7f1c141a4740 1 -- 172.21.15.55:0/3405618259 shutdown_connections 2024-04-26T21:57:02.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.728+0000 7f1c141a4740 1 -- 172.21.15.55:0/3405618259 wait complete. 2024-04-26T21:57:02.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.728+0000 7f1c141a4740 20 client.16059 trim_cache size 0 max 16384 2024-04-26T21:57:02.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.750+0000 7f1c141a4740 1 Processor -- start 2024-04-26T21:57:02.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c141a4740 1 -- start start 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 0x55a43cce7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 0x55a43ccfe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccf7050 0x55a43cce6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c141a4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1c3f30 con 0x55a43ccf7050 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c141a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1c1800 con 0x55a43ccee590 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c141a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1bf630 con 0x55a43ccedbd0 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c0b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccf7050 0x55a43cce6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c0b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccf7050 0x55a43cce6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54672/0 (socket says 172.21.15.55:54672) 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.751+0000 7f1c0b7fe640 1 -- 172.21.15.55:0/1809387762 learned_addr learned my addr 172.21.15.55:0/1809387762 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1c0b7fe640 1 -- 172.21.15.55:0/1809387762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 msgr2=0x55a43cce7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/1809387762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 0x55a43cce7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1c0b7fe640 1 -- 172.21.15.55:0/1809387762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 msgr2=0x55a43ccfe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/1809387762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 0x55a43ccfe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1c0b7fe640 1 -- 172.21.15.55:0/1809387762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43ccf1030 con 0x55a43ccf7050 2024-04-26T21:57:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1c0b7fe640 1 --2- 172.21.15.55:0/1809387762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccf7050 0x55a43cce6bc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc022000 tx=0x7f1bfc005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350054bb7d752a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1bf8ff9640 1 -- 172.21.15.55:0/1809387762 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc020040 con 0x55a43ccf7050 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1bf8ff9640 1 -- 172.21.15.55:0/1809387762 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bfc018040 con 0x55a43ccf7050 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.752+0000 7f1bf8ff9640 1 -- 172.21.15.55:0/1809387762 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc003e30 con 0x55a43ccf7050 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 -- 172.21.15.55:0/1809387762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccf7050 msgr2=0x55a43cce6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 --2- 172.21.15.55:0/1809387762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccf7050 0x55a43cce6bc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc022000 tx=0x7f1bfc005de0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 -- 172.21.15.55:0/1809387762 shutdown_connections 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 --2- 172.21.15.55:0/1809387762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccf7050 0x55a43cce6bc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 --2- 172.21.15.55:0/1809387762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccee590 0x55a43ccfe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 --2- 172.21.15.55:0/1809387762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccedbd0 0x55a43cce7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 -- 172.21.15.55:0/1809387762 >> 172.21.15.55:0/1809387762 conn(0x55a43cc2d910 msgr2=0x55a43ccf6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 -- 172.21.15.55:0/1809387762 shutdown_connections 2024-04-26T21:57:02.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.754+0000 7f1c141a4740 1 -- 172.21.15.55:0/1809387762 wait complete. 2024-04-26T21:57:02.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.755+0000 7f1c141a4740 1 -- start start 2024-04-26T21:57:02.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.755+0000 7f1c141a4740 1 Processor -- start 2024-04-26T21:57:02.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.755+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 0x55a43ccfbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.755+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccfc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c141a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccf7050 0x55a43ccfd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c141a4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1bfcc0 con 0x55a43ccee590 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c141a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1c1130 con 0x55a43ccedbd0 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c141a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c1c0390 con 0x55a43ccf7050 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c141a4740 10 client.0 _ll_register_callbacks cb 0x7f1c00005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c0a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccfc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c0a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccfc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54682/0 (socket says 172.21.15.55:54682) 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c0a7fc640 1 -- 172.21.15.55:0/3097000965 learned_addr learned my addr 172.21.15.55:0/3097000965 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c0a7fc640 1 -- 172.21.15.55:0/3097000965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccf7050 msgr2=0x55a43ccfd0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c0a7fc640 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccf7050 0x55a43ccfd0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c0a7fc640 1 -- 172.21.15.55:0/3097000965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 msgr2=0x55a43ccfbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:02.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c0a7fc640 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 0x55a43ccfbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.756+0000 7f1c0a7fc640 1 -- 172.21.15.55:0/3097000965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43ccfd6d0 con 0x55a43ccee590 2024-04-26T21:57:02.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.757+0000 7f1c0a7fc640 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccfc0d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009a80 tx=0x7f1c0400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805fa66c0dd1ae71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:02.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.757+0000 7f1bee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:02.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.757+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3097000965 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04008c30 con 0x55a43ccee590 2024-04-26T21:57:02.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.757+0000 7f1c141a4740 1 -- 172.21.15.55:0/3097000965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bfc018040 con 0x55a43ccee590 2024-04-26T21:57:02.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.757+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3097000965 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1c04008dd0 con 0x55a43ccee590 2024-04-26T21:57:02.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.758+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3097000965 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1c04027050 con 0x55a43ccee590 2024-04-26T21:57:02.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.758+0000 7f1c141a4740 10 client.16065 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:02.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.758+0000 7f1c141a4740 1 -- 172.21.15.55:0/3097000965 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a43cd69380 con 0x55a43ccee590 2024-04-26T21:57:02.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.758+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3097000965 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0401d070 con 0x55a43ccee590 2024-04-26T21:57:02.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.759+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3097000965 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f1c0402a020 con 0x55a43ccee590 2024-04-26T21:57:02.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.759+0000 7f1c141a4740 10 client.16065 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:02.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.759+0000 7f1c141a4740 10 client.16065 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:57:02.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.759+0000 7f1c141a4740 20 client.16065 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:02.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.759+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a43cd69db0 0x55a43cd6c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:02.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.759+0000 7f1c141a4740 4 client.16065 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:02.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.759+0000 7f1c141a4740 1 -- 172.21.15.55:0/3097000965 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1bfc018040 con 0x55a43cd69db0 2024-04-26T21:57:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.760+0000 7f1c0affd640 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a43cd69db0 0x55a43cd6c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:02.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.762+0000 7f1c0affd640 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a43cd69db0 0x55a43cd6c1d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38ffb9d3d603f10c server_cookie=8979e9791fb8c089 in_seq=0 out_seq=0 2024-04-26T21:57:02.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.762+0000 7f1bee7fc640 10 client.16065 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:57:02.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.763+0000 7f1bee7fc640 1 -- 172.21.15.55:0/3097000965 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1bfc018040 con 0x55a43cd69db0 2024-04-26T21:57:02.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.763+0000 7f1bee7fc640 10 client.16065 handle_command_reply: tid=0 2024-04-26T21:57:02.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.763+0000 7f1c141a4740 1 client.16065 shutdown 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.763+0000 7f1c141a4740 1 -- 172.21.15.55:0/3097000965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 msgr2=0x55a43ccfc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.763+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccfc0d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009a80 tx=0x7f1c0400cc90 comp rx=0 tx=0).stop 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1c141a4740 1 -- 172.21.15.55:0/3097000965 shutdown_connections 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a43cd69db0 0x55a43cd6c1d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a43ccf7050 0x55a43ccfd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a43ccee590 0x55a43ccfc0d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1bee7fc640 0 client.16065 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1c141a4740 1 --2- 172.21.15.55:0/3097000965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a43ccedbd0 0x55a43ccfbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1c141a4740 1 -- 172.21.15.55:0/3097000965 >> 172.21.15.55:0/3097000965 conn(0x55a43cc2d910 msgr2=0x55a43cceb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1c141a4740 1 -- 172.21.15.55:0/3097000965 shutdown_connections 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.764+0000 7f1c141a4740 1 -- 172.21.15.55:0/3097000965 wait complete. 2024-04-26T21:57:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.765+0000 7f1c141a4740 20 client.16065 trim_cache size 0 max 16384 2024-04-26T21:57:02.766 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:57:02.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.766+0000 7f1c113af640 1 -- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1bf40a3e40 msgr2=0x7f1bf40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.766+0000 7f1c113af640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1bf40a3e40 0x7f1bf40a6300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00a8e0 tx=0x7f1bfc00ace0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.766+0000 7f1c113af640 1 -- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c10a070 msgr2=0x7f1c0c115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:02.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.766+0000 7f1c113af640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c10a070 0x7f1c0c115e90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1c00034040 tx=0x7f1c000045e0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.767+0000 7f1c113af640 1 -- 172.21.15.55:0/1755174418 shutdown_connections 2024-04-26T21:57:02.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.767+0000 7f1c113af640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1bf40a3e40 0x7f1bf40a6300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.767+0000 7f1c113af640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c0c119420 0x7f1c0c1b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.767+0000 7f1c113af640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0c10aa10 0x7f1c0c1163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.767+0000 7f1c113af640 1 --2- 172.21.15.55:0/1755174418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c0c10a070 0x7f1c0c115e90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:02.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.767+0000 7f1c113af640 1 -- 172.21.15.55:0/1755174418 >> 172.21.15.55:0/1755174418 conn(0x7f1c0c06f840 msgr2=0x7f1c0c10b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:02.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.767+0000 7f1c113af640 1 -- 172.21.15.55:0/1755174418 shutdown_connections 2024-04-26T21:57:02.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:02.767+0000 7f1c113af640 1 -- 172.21.15.55:0/1755174418 wait complete. 2024-04-26T21:57:02.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:57:03.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.141+0000 7f4362a66640 1 -- 172.21.15.55:0/1169241415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c109930 msgr2=0x7f435c10bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:03.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.141+0000 7f4362a66640 1 --2- 172.21.15.55:0/1169241415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c109930 0x7f435c10bd60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4358009860 tx=0x7f435802a340 comp rx=0 tx=0).stop 2024-04-26T21:57:03.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.147+0000 7f4362a66640 1 -- 172.21.15.55:0/1169241415 shutdown_connections 2024-04-26T21:57:03.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.147+0000 7f4362a66640 1 --2- 172.21.15.55:0/1169241415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c10ec50 0x7f435c10f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.147+0000 7f4362a66640 1 --2- 172.21.15.55:0/1169241415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f435c10c2a0 0x7f435c10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.147+0000 7f4362a66640 1 --2- 172.21.15.55:0/1169241415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c109930 0x7f435c10bd60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.147+0000 7f4362a66640 1 -- 172.21.15.55:0/1169241415 >> 172.21.15.55:0/1169241415 conn(0x7f435c0706f0 msgr2=0x7f435c072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:03.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.148+0000 7f4362a66640 1 -- 172.21.15.55:0/1169241415 shutdown_connections 2024-04-26T21:57:03.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.152+0000 7f4362a66640 1 -- 172.21.15.55:0/1169241415 wait complete. 2024-04-26T21:57:03.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.153+0000 7f4362a66640 1 Processor -- start 2024-04-26T21:57:03.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.154+0000 7f4362a66640 1 -- start start 2024-04-26T21:57:03.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.154+0000 7f4362a66640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c109930 0x7f435c1a7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.154+0000 7f4362a66640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f435c10c2a0 0x7f435c1a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.154+0000 7f4362a66640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c10ec50 0x7f435c1aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.154+0000 7f4362a66640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f435c11d1c0 con 0x7f435c10ec50 2024-04-26T21:57:03.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.154+0000 7f4362a66640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f435c11d040 con 0x7f435c109930 2024-04-26T21:57:03.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.154+0000 7f4362a66640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f435c11d340 con 0x7f435c10c2a0 2024-04-26T21:57:03.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.154+0000 7f4362265640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c10ec50 0x7f435c1aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.155+0000 7f4362265640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c10ec50 0x7f435c1aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54706/0 (socket says 172.21.15.55:54706) 2024-04-26T21:57:03.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.155+0000 7f4362265640 1 -- 172.21.15.55:0/1274834160 learned_addr learned my addr 172.21.15.55:0/1274834160 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:03.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.155+0000 7f4362265640 1 -- 172.21.15.55:0/1274834160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f435c10c2a0 msgr2=0x7f435c1a7ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:57:03.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.155+0000 7f4362265640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f435c10c2a0 0x7f435c1a7ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.155+0000 7f4362265640 1 -- 172.21.15.55:0/1274834160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c109930 msgr2=0x7f435c1a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:03.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.155+0000 7f4362265640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c109930 0x7f435c1a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.156+0000 7f4362265640 1 -- 172.21.15.55:0/1274834160 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c1af530 con 0x7f435c10ec50 2024-04-26T21:57:03.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.156+0000 7f4362265640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c10ec50 0x7f435c1aeec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f435400eaa0 tx=0x7f435400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:03.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.157+0000 7f4352ffd640 1 -- 172.21.15.55:0/1274834160 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435400cdb0 con 0x7f435c10ec50 2024-04-26T21:57:03.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.160+0000 7f4362a66640 1 -- 172.21.15.55:0/1274834160 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c1af820 con 0x7f435c10ec50 2024-04-26T21:57:03.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.160+0000 7f4362a66640 1 -- 172.21.15.55:0/1274834160 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1afd70 con 0x7f435c10ec50 2024-04-26T21:57:03.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.161+0000 7f4352ffd640 1 -- 172.21.15.55:0/1274834160 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4354005020 con 0x7f435c10ec50 2024-04-26T21:57:03.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.161+0000 7f4352ffd640 1 -- 172.21.15.55:0/1274834160 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43540116d0 con 0x7f435c10ec50 2024-04-26T21:57:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.170+0000 7f43669e3740 1 Processor -- start 2024-04-26T21:57:03.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.173+0000 7f4352ffd640 1 -- 172.21.15.55:0/1274834160 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4354011870 con 0x7f435c10ec50 2024-04-26T21:57:03.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.173+0000 7f43669e3740 1 -- start start 2024-04-26T21:57:03.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.173+0000 7f43669e3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.174+0000 7f43669e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e6bd0 0x55c5230e7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.174+0000 7f43669e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.174+0000 7f43669e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c523049d80 con 0x55c5230e7590 2024-04-26T21:57:03.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.174+0000 7f43669e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c523003a20 con 0x55c5230e6bd0 2024-04-26T21:57:03.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.174+0000 7f43669e3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c523045d40 con 0x55c5230e5a00 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4361263640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4361263640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230ede20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54722/0 (socket says 172.21.15.55:54722) 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4361263640 1 -- 172.21.15.55:0/2534421839 learned_addr learned my addr 172.21.15.55:0/2534421839 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4352ffd640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f43480a3d00 0x7f43480a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4361263640 1 -- 172.21.15.55:0/2534421839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 msgr2=0x55c5230e5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4361263640 1 --2- 172.21.15.55:0/2534421839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4361263640 1 -- 172.21.15.55:0/2534421839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e6bd0 msgr2=0x55c5230e7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4361263640 1 --2- 172.21.15.55:0/2534421839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e6bd0 0x55c5230e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4361263640 1 -- 172.21.15.55:0/2534421839 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c52300ec30 con 0x55c5230e7590 2024-04-26T21:57:03.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.175+0000 7f4352ffd640 1 -- 172.21.15.55:0/1274834160 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f43540d0710 con 0x7f435c10ec50 2024-04-26T21:57:03.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.176+0000 7f4361263640 1 --2- 172.21.15.55:0/2534421839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230ede20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f434c005fd0 tx=0x7f434c00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc36fdbf8bc22cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:03.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.177+0000 7f4362265640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f43480a3d00 0x7f43480a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.178+0000 7f4347fff640 1 -- 172.21.15.55:0/2534421839 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c016070 con 0x55c5230e7590 2024-04-26T21:57:03.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.178+0000 7f4347fff640 1 -- 172.21.15.55:0/2534421839 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c0054d0 con 0x55c5230e7590 2024-04-26T21:57:03.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.178+0000 7f4347fff640 1 -- 172.21.15.55:0/2534421839 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c004d90 con 0x55c5230e7590 2024-04-26T21:57:03.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.179+0000 7f4362265640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f43480a3d00 0x7f43480a61c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4354099220 tx=0x7f4354098430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:03.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.182+0000 7f43669e3740 1 -- 172.21.15.55:0/2534421839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 msgr2=0x55c5230ede20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:03.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.182+0000 7f43669e3740 1 --2- 172.21.15.55:0/2534421839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230ede20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f434c005fd0 tx=0x7f434c00d2e0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.182+0000 7f43669e3740 1 -- 172.21.15.55:0/2534421839 shutdown_connections 2024-04-26T21:57:03.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.182+0000 7f43669e3740 1 --2- 172.21.15.55:0/2534421839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230ede20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.182+0000 7f43669e3740 1 --2- 172.21.15.55:0/2534421839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e6bd0 0x55c5230e7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.183+0000 7f43669e3740 1 --2- 172.21.15.55:0/2534421839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230e5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.183+0000 7f43669e3740 1 -- 172.21.15.55:0/2534421839 >> 172.21.15.55:0/2534421839 conn(0x55c523026910 msgr2=0x55c5230e12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:03.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.184+0000 7f43669e3740 1 -- 172.21.15.55:0/2534421839 shutdown_connections 2024-04-26T21:57:03.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.184+0000 7f43669e3740 1 -- 172.21.15.55:0/2534421839 wait complete. 2024-04-26T21:57:03.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.185+0000 7f43669e3740 1 -- start start 2024-04-26T21:57:03.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.185+0000 7f43669e3740 1 Processor -- start 2024-04-26T21:57:03.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.186+0000 7f43669e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e5a00 0x55c5230efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.187+0000 7f43669e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.187+0000 7f43669e3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e7590 0x55c5230f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.187+0000 7f43669e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c5230ee5a0 con 0x55c5230e6bd0 2024-04-26T21:57:03.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.187+0000 7f43669e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c522fd0dc0 con 0x55c5230e5a00 2024-04-26T21:57:03.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.187+0000 7f43669e3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5230ee720 con 0x55c5230e7590 2024-04-26T21:57:03.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.187+0000 7f43669e3740 10 client.0 _ll_register_callbacks cb 0x55c522fec600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:03.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.188+0000 7f4361263640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e5a00 0x55c5230efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.188+0000 7f4361263640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e5a00 0x55c5230efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46016/0 (socket says 172.21.15.55:46016) 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.188+0000 7f4361263640 1 -- 172.21.15.55:0/3655214084 learned_addr learned my addr 172.21.15.55:0/3655214084 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.189+0000 7f4361a64640 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.189+0000 7f4361a64640 1 -- 172.21.15.55:0/3655214084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e7590 msgr2=0x55c5230f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.189+0000 7f4361a64640 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e7590 0x55c5230f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.189+0000 7f4361a64640 1 -- 172.21.15.55:0/3655214084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e5a00 msgr2=0x55c5230efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.189+0000 7f4361a64640 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e5a00 0x55c5230efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.189+0000 7f4361a64640 1 -- 172.21.15.55:0/3655214084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c522d5b3f0 con 0x55c5230e6bd0 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.190+0000 7f4361a64640 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f0060 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4358035040 tx=0x7f4358030780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f5fa114c13a4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:03.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.190+0000 7f43457fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:03.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.190+0000 7f43457fa640 1 -- 172.21.15.55:0/3655214084 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358037040 con 0x55c5230e6bd0 2024-04-26T21:57:03.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.191+0000 7f43669e3740 1 -- 172.21.15.55:0/3655214084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c0054d0 con 0x55c5230e6bd0 2024-04-26T21:57:03.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.191+0000 7f43457fa640 1 -- 172.21.15.55:0/3655214084 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4358004240 con 0x55c5230e6bd0 2024-04-26T21:57:03.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.191+0000 7f43457fa640 1 -- 172.21.15.55:0/3655214084 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f43580308f0 con 0x55c5230e6bd0 2024-04-26T21:57:03.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.191+0000 7f43669e3740 10 client.16077 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:03.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.192+0000 7f43669e3740 1 -- 172.21.15.55:0/3655214084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c522e257d0 con 0x55c5230e6bd0 2024-04-26T21:57:03.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.193+0000 7f43457fa640 1 -- 172.21.15.55:0/3655214084 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c0054d0 con 0x55c5230e6bd0 2024-04-26T21:57:03.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.193+0000 7f43457fa640 1 -- 172.21.15.55:0/3655214084 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f4358043030 con 0x55c5230e6bd0 2024-04-26T21:57:03.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.193+0000 7f43669e3740 10 client.16077 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:03.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.193+0000 7f43669e3740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:03.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.193+0000 7f43669e3740 20 client.16077 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:03.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.194+0000 7f43669e3740 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5230fbeb0 0x55c5230fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.194+0000 7f43669e3740 4 client.16077 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:03.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.194+0000 7f43669e3740 1 -- 172.21.15.55:0/3655214084 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c523057990 con 0x55c5230fbeb0 2024-04-26T21:57:03.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.194+0000 7f4361263640 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5230fbeb0 0x55c5230fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.195+0000 7f4361263640 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5230fbeb0 0x55c5230fe2f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddfebb6585a3f34 server_cookie=c866889de58fb5b9 in_seq=0 out_seq=0 2024-04-26T21:57:03.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.195+0000 7f43457fa640 10 client.16077 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:03.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.200+0000 7f43457fa640 1 -- 172.21.15.55:0/3655214084 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c523057990 con 0x55c5230fbeb0 2024-04-26T21:57:03.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.200+0000 7f43457fa640 10 client.16077 handle_command_reply: tid=0 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.200+0000 7f43669e3740 1 client.16077 shutdown 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 -- 172.21.15.55:0/3655214084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 msgr2=0x55c5230f0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f0060 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4358035040 tx=0x7f4358030780 comp rx=0 tx=0).stop 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 -- 172.21.15.55:0/3655214084 shutdown_connections 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e7590 0x55c5230f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5230fbeb0 0x55c5230fe2f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f0060 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 --2- 172.21.15.55:0/3655214084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e5a00 0x55c5230efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 -- 172.21.15.55:0/3655214084 >> 172.21.15.55:0/3655214084 conn(0x55c523026910 msgr2=0x55c5230e0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43457fa640 0 client.16077 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 -- 172.21.15.55:0/3655214084 shutdown_connections 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 1 -- 172.21.15.55:0/3655214084 wait complete. 2024-04-26T21:57:03.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.201+0000 7f43669e3740 20 client.16077 trim_cache size 0 max 16384 2024-04-26T21:57:03.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.229+0000 7f43669e3740 1 Processor -- start 2024-04-26T21:57:03.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f43669e3740 1 -- start start 2024-04-26T21:57:03.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f43669e3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f43669e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e6bd0 0x55c5230f7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f43669e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f43669e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4340010eb0 con 0x55c5230e7590 2024-04-26T21:57:03.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f43669e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f434000e780 con 0x55c5230e6bd0 2024-04-26T21:57:03.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f43669e3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f434000c5d0 con 0x55c5230e5a00 2024-04-26T21:57:03.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f4361263640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f4361263640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230e1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54742/0 (socket says 172.21.15.55:54742) 2024-04-26T21:57:03.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f4361263640 1 -- 172.21.15.55:0/181314056 learned_addr learned my addr 172.21.15.55:0/181314056 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f4361263640 1 -- 172.21.15.55:0/181314056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 msgr2=0x55c5230e0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f4361263640 1 --2- 172.21.15.55:0/181314056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230e0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f4361263640 1 -- 172.21.15.55:0/181314056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e6bd0 msgr2=0x55c5230f7060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f4361263640 1 --2- 172.21.15.55:0/181314056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e6bd0 0x55c5230f7060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.230+0000 7f4361263640 1 -- 172.21.15.55:0/181314056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5230e6650 con 0x55c5230e7590 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.231+0000 7f4361263640 1 --2- 172.21.15.55:0/181314056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230e1270 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f434c0048a0 tx=0x7f434c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a010905fbead4b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.231+0000 7f43477fe640 1 -- 172.21.15.55:0/181314056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c016040 con 0x55c5230e7590 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.231+0000 7f43477fe640 1 -- 172.21.15.55:0/181314056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c004500 con 0x55c5230e7590 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.231+0000 7f43477fe640 1 -- 172.21.15.55:0/181314056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c026810 con 0x55c5230e7590 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.232+0000 7f43669e3740 1 -- 172.21.15.55:0/181314056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 msgr2=0x55c5230e1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:03.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.232+0000 7f43669e3740 1 --2- 172.21.15.55:0/181314056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230e1270 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f434c0048a0 tx=0x7f434c00b810 comp rx=0 tx=0).stop 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.232+0000 7f43669e3740 1 -- 172.21.15.55:0/181314056 shutdown_connections 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.232+0000 7f43669e3740 1 --2- 172.21.15.55:0/181314056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e7590 0x55c5230e1270 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.232+0000 7f43669e3740 1 --2- 172.21.15.55:0/181314056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e6bd0 0x55c5230f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.232+0000 7f43669e3740 1 --2- 172.21.15.55:0/181314056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.232+0000 7f43669e3740 1 -- 172.21.15.55:0/181314056 >> 172.21.15.55:0/181314056 conn(0x55c523026910 msgr2=0x55c5230ea2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.232+0000 7f43669e3740 1 -- 172.21.15.55:0/181314056 shutdown_connections 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.233+0000 7f43669e3740 1 -- 172.21.15.55:0/181314056 wait complete. 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.233+0000 7f43669e3740 1 -- start start 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.233+0000 7f43669e3740 1 Processor -- start 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.233+0000 7f43669e3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f43669e3740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f43669e3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e7590 0x55c5230f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f43669e3740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f434000cc40 con 0x55c5230e6bd0 2024-04-26T21:57:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f43669e3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f434000e0b0 con 0x55c5230e7590 2024-04-26T21:57:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f43669e3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f434000d310 con 0x55c5230e5a00 2024-04-26T21:57:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f43669e3740 10 client.0 _ll_register_callbacks cb 0x7f4358035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361a64640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361a64640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54744/0 (socket says 172.21.15.55:54744) 2024-04-26T21:57:03.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361a64640 1 -- 172.21.15.55:0/2557991587 learned_addr learned my addr 172.21.15.55:0/2557991587 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:03.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361263640 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e7590 0x55c5230f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361a64640 1 -- 172.21.15.55:0/2557991587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 msgr2=0x55c5230f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:03.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361a64640 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361a64640 1 -- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e7590 msgr2=0x55c5230f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361a64640 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e7590 0x55c5230f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361a64640 1 -- 172.21.15.55:0/2557991587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c523160460 con 0x55c5230e6bd0 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.234+0000 7f4361263640 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e7590 0x55c5230f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.235+0000 7f4361a64640 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f3960 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4358004c90 tx=0x7f4358004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cb0bc833ba600f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.235+0000 7f43457fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.236+0000 7f43457fa640 1 -- 172.21.15.55:0/2557991587 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358037040 con 0x55c5230e6bd0 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.236+0000 7f43457fa640 1 -- 172.21.15.55:0/2557991587 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4358005540 con 0x55c5230e6bd0 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.236+0000 7f43669e3740 1 -- 172.21.15.55:0/2557991587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c004500 con 0x55c5230e6bd0 2024-04-26T21:57:03.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.236+0000 7f43457fa640 1 -- 172.21.15.55:0/2557991587 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4358055050 con 0x55c5230e6bd0 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.237+0000 7f43669e3740 10 client.16083 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.237+0000 7f43669e3740 1 -- 172.21.15.55:0/2557991587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55c523161620 con 0x55c5230e6bd0 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.237+0000 7f43457fa640 1 -- 172.21.15.55:0/2557991587 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c004500 con 0x55c5230e6bd0 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.237+0000 7f43457fa640 1 -- 172.21.15.55:0/2557991587 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f435804e070 con 0x55c5230e6bd0 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.238+0000 7f43669e3740 10 client.16083 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.238+0000 7f43669e3740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.238+0000 7f43669e3740 20 client.16083 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.238+0000 7f43669e3740 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c523162020 0x55c523164440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.238+0000 7f43669e3740 4 client.16083 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.238+0000 7f43669e3740 1 -- 172.21.15.55:0/2557991587 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c523051800 con 0x55c523162020 2024-04-26T21:57:03.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.239+0000 7f4361263640 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c523162020 0x55c523164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:03.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.244+0000 7f43457fa640 10 client.16083 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:03.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.244+0000 7f4361263640 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c523162020 0x55c523164440 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1fda70ac001af7 server_cookie=a6a8cdd60e22ba00 in_seq=0 out_seq=0 2024-04-26T21:57:03.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43457fa640 1 -- 172.21.15.55:0/2557991587 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c523051800 con 0x55c523162020 2024-04-26T21:57:03.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43457fa640 10 client.16083 handle_command_reply: tid=0 2024-04-26T21:57:03.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 client.16083 shutdown 2024-04-26T21:57:03.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 -- 172.21.15.55:0/2557991587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 msgr2=0x55c5230f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:03.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f3960 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4358004c90 tx=0x7f4358004f40 comp rx=0 tx=0).stop 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 -- 172.21.15.55:0/2557991587 shutdown_connections 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5230e7590 0x55c5230f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c523162020 0x55c523164440 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43457fa640 0 client.16083 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5230e6bd0 0x55c5230f3960 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 --2- 172.21.15.55:0/2557991587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5230e5a00 0x55c5230f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.249+0000 7f43669e3740 1 -- 172.21.15.55:0/2557991587 >> 172.21.15.55:0/2557991587 conn(0x55c523026910 msgr2=0x55c5230eeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.250+0000 7f43669e3740 1 -- 172.21.15.55:0/2557991587 shutdown_connections 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.250+0000 7f43669e3740 1 -- 172.21.15.55:0/2557991587 wait complete. 2024-04-26T21:57:03.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.250+0000 7f43669e3740 20 client.16083 trim_cache size 0 max 16384 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 -- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f43480a3d00 msgr2=0x7f43480a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f43480a3d00 0x7f43480a61c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4354099220 tx=0x7f4354098430 comp rx=0 tx=0).stop 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 -- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c10ec50 msgr2=0x7f435c1aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c10ec50 0x7f435c1aeec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f435400eaa0 tx=0x7f435400ef60 comp rx=0 tx=0).stop 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 -- 172.21.15.55:0/1274834160 shutdown_connections 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f43480a3d00 0x7f43480a61c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f435c10ec50 0x7f435c1aeec0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f435c10c2a0 0x7f435c1a7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 --2- 172.21.15.55:0/1274834160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f435c109930 0x7f435c1a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:03.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.251+0000 7f4362a66640 1 -- 172.21.15.55:0/1274834160 >> 172.21.15.55:0/1274834160 conn(0x7f435c0706f0 msgr2=0x7f435c10cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:03.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.252+0000 7f4362a66640 1 -- 172.21.15.55:0/1274834160 shutdown_connections 2024-04-26T21:57:03.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:03.252+0000 7f4362a66640 1 -- 172.21.15.55:0/1274834160 wait complete. 2024-04-26T21:57:03.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:03 smithi055 ceph-mon[29221]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 804 MiB/s wr, 929 op/s 2024-04-26T21:57:03.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:03 smithi064 ceph-mon[32765]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 804 MiB/s wr, 929 op/s 2024-04-26T21:57:03.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:03 smithi165 ceph-mon[33296]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 804 MiB/s wr, 929 op/s 2024-04-26T21:57:04.276 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:57:04.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:57:04.277 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:57:04.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.698+0000 7f2575d40640 1 -- 172.21.15.55:0/2689748480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 msgr2=0x7f2570074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:04.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.698+0000 7f2575d40640 1 --2- 172.21.15.55:0/2689748480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 0x7f2570074a60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2568009ca0 tx=0x7f256802a8f0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.699+0000 7f2575d40640 1 -- 172.21.15.55:0/2689748480 shutdown_connections 2024-04-26T21:57:04.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.699+0000 7f2575d40640 1 --2- 172.21.15.55:0/2689748480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2570079ed0 0x7f257007c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.699+0000 7f2575d40640 1 --2- 172.21.15.55:0/2689748480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570077520 0x7f2570079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.699+0000 7f2575d40640 1 --2- 172.21.15.55:0/2689748480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 0x7f2570074a60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.699+0000 7f2575d40640 1 -- 172.21.15.55:0/2689748480 >> 172.21.15.55:0/2689748480 conn(0x7f257006f780 msgr2=0x7f2570071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:04.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.700+0000 7f2575d40640 1 -- 172.21.15.55:0/2689748480 shutdown_connections 2024-04-26T21:57:04.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.701+0000 7f2575d40640 1 -- 172.21.15.55:0/2689748480 wait complete. 2024-04-26T21:57:04.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.702+0000 7f2575d40640 1 Processor -- start 2024-04-26T21:57:04.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.702+0000 7f2575d40640 1 -- start start 2024-04-26T21:57:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.703+0000 7f2575d40640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 0x7f25701b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.703+0000 7f2575d40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570077520 0x7f25701b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.703+0000 7f2575d40640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2570079ed0 0x7f25701bba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.703+0000 7f2575d40640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f257007fd10 con 0x7f2570074640 2024-04-26T21:57:04.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.703+0000 7f2575d40640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f257007fb90 con 0x7f2570077520 2024-04-26T21:57:04.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.703+0000 7f2575d40640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f257007fe90 con 0x7f2570079ed0 2024-04-26T21:57:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 0x7f25701b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570077520 0x7f25701b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570077520 0x7f25701b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46054/0 (socket says 172.21.15.55:46054) 2024-04-26T21:57:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 0x7f25701b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54748/0 (socket says 172.21.15.55:54748) 2024-04-26T21:57:04.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256effd640 1 -- 172.21.15.55:0/736136449 learned_addr learned my addr 172.21.15.55:0/736136449 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256ffff640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2570079ed0 0x7f25701bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256f7fe640 1 -- 172.21.15.55:0/736136449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2570079ed0 msgr2=0x7f25701bba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256f7fe640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2570079ed0 0x7f25701bba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256f7fe640 1 -- 172.21.15.55:0/736136449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570077520 msgr2=0x7f25701b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256f7fe640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570077520 0x7f25701b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.704+0000 7f256f7fe640 1 -- 172.21.15.55:0/736136449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25701bc100 con 0x7f2570074640 2024-04-26T21:57:04.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.705+0000 7f256f7fe640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 0x7f25701b44e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f2568003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:04.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.706+0000 7f256cff9640 1 -- 172.21.15.55:0/736136449 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568004300 con 0x7f2570074640 2024-04-26T21:57:04.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.706+0000 7f256cff9640 1 -- 172.21.15.55:0/736136449 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f256802d070 con 0x7f2570074640 2024-04-26T21:57:04.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.706+0000 7f2575d40640 1 -- 172.21.15.55:0/736136449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25701bc390 con 0x7f2570074640 2024-04-26T21:57:04.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.706+0000 7f256cff9640 1 -- 172.21.15.55:0/736136449 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680305f0 con 0x7f2570074640 2024-04-26T21:57:04.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.706+0000 7f2575d40640 1 -- 172.21.15.55:0/736136449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25701bc960 con 0x7f2570074640 2024-04-26T21:57:04.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.711+0000 7f256cff9640 1 -- 172.21.15.55:0/736136449 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f256802f7f0 con 0x7f2570074640 2024-04-26T21:57:04.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.712+0000 7f2578a5b740 1 Processor -- start 2024-04-26T21:57:04.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.712+0000 7f2578a5b740 1 -- start start 2024-04-26T21:57:04.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.712+0000 7f256cff9640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f25580a3dc0 0x7f25580a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.712+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.712+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 0x5565450c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f2578a5b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556545023b00 con 0x5565450bea80 2024-04-26T21:57:04.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f2578a5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556544fbf920 con 0x5565450c45e0 2024-04-26T21:57:04.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f2578a5b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55654501fac0 con 0x5565450bf470 2024-04-26T21:57:04.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256ffff640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f25580a3dc0 0x7f25580a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450beea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54756/0 (socket says 172.21.15.55:54756) 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256f7fe640 1 -- 172.21.15.55:0/2456692541 learned_addr learned my addr 172.21.15.55:0/2456692541 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256cff9640 1 -- 172.21.15.55:0/736136449 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f25680f1bc0 con 0x7f2570074640 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256f7fe640 1 -- 172.21.15.55:0/2456692541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 msgr2=0x5565450c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256f7fe640 1 --2- 172.21.15.55:0/2456692541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 0x5565450c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256f7fe640 1 -- 172.21.15.55:0/2456692541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 msgr2=0x5565450c6a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.713+0000 7f256f7fe640 1 --2- 172.21.15.55:0/2456692541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450c6a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.714+0000 7f256f7fe640 1 -- 172.21.15.55:0/2456692541 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556544dfd7e0 con 0x5565450bea80 2024-04-26T21:57:04.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.714+0000 7f256ffff640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f25580a3dc0 0x7f25580a6280 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2564006fd0 tx=0x7f2564008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:04.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.714+0000 7f256f7fe640 1 --2- 172.21.15.55:0/2456692541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450beea0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2568031a90 tx=0x7f25680b32f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ab7072761a8361 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:04.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.714+0000 7f255dffb640 1 -- 172.21.15.55:0/2456692541 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680bdbe0 con 0x5565450bea80 2024-04-26T21:57:04.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.714+0000 7f255dffb640 1 -- 172.21.15.55:0/2456692541 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2568036020 con 0x5565450bea80 2024-04-26T21:57:04.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.714+0000 7f255dffb640 1 -- 172.21.15.55:0/2456692541 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680a8b20 con 0x5565450bea80 2024-04-26T21:57:04.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.716+0000 7f2578a5b740 1 -- 172.21.15.55:0/2456692541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 msgr2=0x5565450beea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:04.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.716+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2456692541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450beea0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2568031a90 tx=0x7f25680b32f0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.717+0000 7f2578a5b740 1 -- 172.21.15.55:0/2456692541 shutdown_connections 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.717+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2456692541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.717+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2456692541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 0x5565450c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.717+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2456692541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450beea0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.717+0000 7f2578a5b740 1 -- 172.21.15.55:0/2456692541 >> 172.21.15.55:0/2456692541 conn(0x5565450ba0f0 msgr2=0x5565450ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.717+0000 7f2578a5b740 1 -- 172.21.15.55:0/2456692541 shutdown_connections 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.717+0000 7f2578a5b740 1 -- 172.21.15.55:0/2456692541 wait complete. 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.718+0000 7f2578a5b740 1 -- start start 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.718+0000 7f2578a5b740 1 Processor -- start 2024-04-26T21:57:04.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.718+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bea80 0x5565450d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.718+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 0x5565450d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450c45e0 0x5565450d8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f2578a5b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5565450c7350 con 0x5565450c45e0 2024-04-26T21:57:04.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f2578a5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556544fbd5d0 con 0x5565450bf470 2024-04-26T21:57:04.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f2578a5b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5565450c74d0 con 0x5565450bea80 2024-04-26T21:57:04.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f256ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450c45e0 0x5565450d8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bea80 0x5565450d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f256ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450c45e0 0x5565450d8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54772/0 (socket says 172.21.15.55:54772) 2024-04-26T21:57:04.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f256ffff640 1 -- 172.21.15.55:0/1427607043 learned_addr learned my addr 172.21.15.55:0/1427607043 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:04.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.720+0000 7f256effd640 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 0x5565450d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.719+0000 7f2578a5b740 10 client.0 _ll_register_callbacks cb 0x556544fab080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:04.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.721+0000 7f256ffff640 1 -- 172.21.15.55:0/1427607043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bea80 msgr2=0x5565450d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:04.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.721+0000 7f256ffff640 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bea80 0x5565450d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.721+0000 7f256ffff640 1 -- 172.21.15.55:0/1427607043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 msgr2=0x5565450d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:04.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.721+0000 7f256ffff640 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 0x5565450d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.721+0000 7f256ffff640 1 -- 172.21.15.55:0/1427607043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556544e5df70 con 0x5565450c45e0 2024-04-26T21:57:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.721+0000 7f256ffff640 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450c45e0 0x5565450d8a40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2564006fa0 tx=0x7f2564004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73252e2b4f8e7b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.722+0000 7f2552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.723+0000 7f2552ffd640 1 -- 172.21.15.55:0/1427607043 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564022020 con 0x5565450c45e0 2024-04-26T21:57:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.723+0000 7f2552ffd640 1 -- 172.21.15.55:0/1427607043 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f256401e3f0 con 0x5565450c45e0 2024-04-26T21:57:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.723+0000 7f2578a5b740 1 -- 172.21.15.55:0/1427607043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2568036020 con 0x5565450c45e0 2024-04-26T21:57:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.723+0000 7f2552ffd640 1 -- 172.21.15.55:0/1427607043 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f256401a040 con 0x5565450c45e0 2024-04-26T21:57:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.723+0000 7f2578a5b740 10 client.16095 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.724+0000 7f2578a5b740 1 -- 172.21.15.55:0/1427607043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556544cf5170 con 0x5565450c45e0 2024-04-26T21:57:04.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.724+0000 7f2552ffd640 1 -- 172.21.15.55:0/1427607043 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2564014e20 con 0x5565450c45e0 2024-04-26T21:57:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.726+0000 7f2552ffd640 1 -- 172.21.15.55:0/1427607043 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f256401d030 con 0x5565450c45e0 2024-04-26T21:57:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.726+0000 7f2578a5b740 10 client.16095 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.726+0000 7f2578a5b740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.726+0000 7f2578a5b740 20 client.16095 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.726+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5565450dd0d0 0x5565450df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.726+0000 7f2578a5b740 4 client.16095 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.726+0000 7f2578a5b740 1 -- 172.21.15.55:0/1427607043 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2568036020 con 0x5565450dd0d0 2024-04-26T21:57:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.727+0000 7f256f7fe640 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5565450dd0d0 0x5565450df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.728+0000 7f256f7fe640 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5565450dd0d0 0x5565450df550 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f4d4126b4d2b3c server_cookie=d428b68ba93190ef in_seq=0 out_seq=0 2024-04-26T21:57:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.728+0000 7f2552ffd640 10 client.16095 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:04.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.731+0000 7f2552ffd640 1 -- 172.21.15.55:0/1427607043 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f2568036020 con 0x5565450dd0d0 2024-04-26T21:57:04.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.731+0000 7f2552ffd640 10 client.16095 handle_command_reply: tid=0 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.731+0000 7f2578a5b740 1 client.16095 shutdown 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.731+0000 7f2578a5b740 1 -- 172.21.15.55:0/1427607043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450c45e0 msgr2=0x5565450d8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.731+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450c45e0 0x5565450d8a40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2564006fa0 tx=0x7f2564004750 comp rx=0 tx=0).stop 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.731+0000 7f2578a5b740 1 -- 172.21.15.55:0/1427607043 shutdown_connections 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5565450dd0d0 0x5565450df550 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450c45e0 0x5565450d8a40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2552ffd640 0 client.16095 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 0x5565450d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1427607043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bea80 0x5565450d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2578a5b740 1 -- 172.21.15.55:0/1427607043 >> 172.21.15.55:0/1427607043 conn(0x5565450ba0f0 msgr2=0x5565450b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:04.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2578a5b740 1 -- 172.21.15.55:0/1427607043 shutdown_connections 2024-04-26T21:57:04.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2578a5b740 1 -- 172.21.15.55:0/1427607043 wait complete. 2024-04-26T21:57:04.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.732+0000 7f2578a5b740 20 client.16095 trim_cache size 0 max 16384 2024-04-26T21:57:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.765+0000 7f2578a5b740 1 Processor -- start 2024-04-26T21:57:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.765+0000 7f2578a5b740 1 -- start start 2024-04-26T21:57:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.765+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450bd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.765+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 0x5565450d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450c45e0 0x5565450bd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f2578a5b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f25701c3cf0 con 0x5565450bea80 2024-04-26T21:57:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f2578a5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f25701c15c0 con 0x5565450bf470 2024-04-26T21:57:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f2578a5b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25701bf410 con 0x5565450c45e0 2024-04-26T21:57:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450bd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450bd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54778/0 (socket says 172.21.15.55:54778) 2024-04-26T21:57:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f256f7fe640 1 -- 172.21.15.55:0/1741532288 learned_addr learned my addr 172.21.15.55:0/1741532288 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:04.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f256ffff640 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450c45e0 0x5565450bd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.766+0000 7f256effd640 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 0x5565450d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.767+0000 7f256f7fe640 1 -- 172.21.15.55:0/1741532288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450c45e0 msgr2=0x5565450bd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:04.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.767+0000 7f256f7fe640 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450c45e0 0x5565450bd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.767+0000 7f256f7fe640 1 -- 172.21.15.55:0/1741532288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 msgr2=0x5565450d8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:04.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.767+0000 7f256f7fe640 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 0x5565450d8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.767+0000 7f256f7fe640 1 -- 172.21.15.55:0/1741532288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565450b5ab0 con 0x5565450bea80 2024-04-26T21:57:04.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.768+0000 7f256f7fe640 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450bd000 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f25680ba7e0 tx=0x7f2568000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed72110c81b618cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:04.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.768+0000 7f255d7fa640 1 -- 172.21.15.55:0/1741532288 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680d6070 con 0x5565450bea80 2024-04-26T21:57:04.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.768+0000 7f255d7fa640 1 -- 172.21.15.55:0/1741532288 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2568036040 con 0x5565450bea80 2024-04-26T21:57:04.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.768+0000 7f255d7fa640 1 -- 172.21.15.55:0/1741532288 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25680bd770 con 0x5565450bea80 2024-04-26T21:57:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.769+0000 7f2578a5b740 1 -- 172.21.15.55:0/1741532288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 msgr2=0x5565450bd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.769+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450bd000 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f25680ba7e0 tx=0x7f2568000f40 comp rx=0 tx=0).stop 2024-04-26T21:57:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.770+0000 7f2578a5b740 1 -- 172.21.15.55:0/1741532288 shutdown_connections 2024-04-26T21:57:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.770+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450c45e0 0x5565450bd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.770+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450bf470 0x5565450d8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.770+0000 7f2578a5b740 1 --2- 172.21.15.55:0/1741532288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450bea80 0x5565450bd000 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.770+0000 7f2578a5b740 1 -- 172.21.15.55:0/1741532288 >> 172.21.15.55:0/1741532288 conn(0x5565450ba0f0 msgr2=0x5565450c0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.770+0000 7f2578a5b740 1 -- 172.21.15.55:0/1741532288 shutdown_connections 2024-04-26T21:57:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.770+0000 7f2578a5b740 1 -- 172.21.15.55:0/1741532288 wait complete. 2024-04-26T21:57:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.772+0000 7f2578a5b740 1 -- start start 2024-04-26T21:57:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.772+0000 7f2578a5b740 1 Processor -- start 2024-04-26T21:57:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.773+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 0x5565450d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.773+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450d89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.774+0000 7f2578a5b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450dd0d0 0x5565450d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.774+0000 7f2578a5b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f25701bfa80 con 0x5565450dd0d0 2024-04-26T21:57:04.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.774+0000 7f2578a5b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f25701c0ef0 con 0x5565450c45e0 2024-04-26T21:57:04.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.774+0000 7f2578a5b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25701c0150 con 0x5565450bf470 2024-04-26T21:57:04.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.774+0000 7f256effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450d89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.774+0000 7f256effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450d89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46088/0 (socket says 172.21.15.55:46088) 2024-04-26T21:57:04.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.774+0000 7f256effd640 1 -- 172.21.15.55:0/2166422074 learned_addr learned my addr 172.21.15.55:0/2166422074 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.774+0000 7f256ffff640 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450dd0d0 0x5565450d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.775+0000 7f2578a5b740 10 client.0 _ll_register_callbacks cb 0x7f256400db50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.775+0000 7f256f7fe640 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 0x5565450d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.775+0000 7f256effd640 1 -- 172.21.15.55:0/2166422074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 msgr2=0x5565450d8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.775+0000 7f256effd640 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 0x5565450d8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.775+0000 7f256effd640 1 -- 172.21.15.55:0/2166422074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450dd0d0 msgr2=0x5565450d6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.775+0000 7f256effd640 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450dd0d0 0x5565450d6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.775+0000 7f256effd640 1 -- 172.21.15.55:0/2166422074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565450d7310 con 0x5565450c45e0 2024-04-26T21:57:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.776+0000 7f256effd640 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450d89b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5565450bfee0 tx=0x7f25600136b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d30bedca1c304f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.776+0000 7f2552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:57:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.776+0000 7f2552ffd640 1 -- 172.21.15.55:0/2166422074 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256001a030 con 0x5565450c45e0 2024-04-26T21:57:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.776+0000 7f2552ffd640 1 -- 172.21.15.55:0/2166422074 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2560009ce0 con 0x5565450c45e0 2024-04-26T21:57:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.777+0000 7f2578a5b740 1 -- 172.21.15.55:0/2166422074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2568036040 con 0x5565450c45e0 2024-04-26T21:57:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.777+0000 7f2552ffd640 1 -- 172.21.15.55:0/2166422074 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2560005970 con 0x5565450c45e0 2024-04-26T21:57:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.777+0000 7f2578a5b740 10 client.25526 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.777+0000 7f2578a5b740 1 -- 172.21.15.55:0/2166422074 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556545144320 con 0x5565450c45e0 2024-04-26T21:57:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.779+0000 7f2552ffd640 1 -- 172.21.15.55:0/2166422074 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2560006020 con 0x5565450c45e0 2024-04-26T21:57:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.779+0000 7f2552ffd640 1 -- 172.21.15.55:0/2166422074 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f2560016480 con 0x5565450c45e0 2024-04-26T21:57:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.779+0000 7f2578a5b740 10 client.25526 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:04.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.779+0000 7f2578a5b740 10 client.25526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:04.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.779+0000 7f2578a5b740 20 client.25526 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:04.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.779+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556545144d60 0x556545147180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:04.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.779+0000 7f2578a5b740 4 client.25526 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:57:04.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.779+0000 7f2578a5b740 1 -- 172.21.15.55:0/2166422074 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f2568036040 con 0x556545144d60 2024-04-26T21:57:04.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.780+0000 7f256f7fe640 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556545144d60 0x556545147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.781+0000 7f256f7fe640 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556545144d60 0x556545147180 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9070d6c47e870201 server_cookie=dbde89976ebfbe50 in_seq=0 out_seq=0 2024-04-26T21:57:04.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:04.781+0000 7f2552ffd640 10 client.25526 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2552ffd640 1 -- 172.21.15.55:0/2166422074 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f2568036040 con 0x556545144d60 2024-04-26T21:57:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2552ffd640 10 client.25526 handle_command_reply: tid=0 2024-04-26T21:57:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 client.25526 shutdown 2024-04-26T21:57:05.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 -- 172.21.15.55:0/2166422074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 msgr2=0x5565450d89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450d89b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5565450bfee0 tx=0x7f25600136b0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 -- 172.21.15.55:0/2166422074 shutdown_connections 2024-04-26T21:57:05.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556545144d60 0x556545147180 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5565450dd0d0 0x5565450d6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2552ffd640 0 client.25526 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5565450c45e0 0x5565450d89b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 --2- 172.21.15.55:0/2166422074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5565450bf470 0x5565450d8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 -- 172.21.15.55:0/2166422074 >> 172.21.15.55:0/2166422074 conn(0x5565450ba0f0 msgr2=0x5565450bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 -- 172.21.15.55:0/2166422074 shutdown_connections 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.095+0000 7f2578a5b740 1 -- 172.21.15.55:0/2166422074 wait complete. 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.096+0000 7f2578a5b740 20 client.25526 trim_cache size 0 max 16384 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "7717dbdd-c62b-4490-be7e-c83066aa442f", 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:57:05.097 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:57:05.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 -- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f25580a3dc0 msgr2=0x7f25580a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f25580a3dc0 0x7f25580a6280 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2564006fd0 tx=0x7f2564008040 comp rx=0 tx=0).stop 2024-04-26T21:57:05.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 -- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 msgr2=0x7f25701b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 0x7f25701b44e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f2568003f10 comp rx=0 tx=0).stop 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 -- 172.21.15.55:0/736136449 shutdown_connections 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f25580a3dc0 0x7f25580a6280 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2570079ed0 0x7f25701bba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570077520 0x7f25701b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 --2- 172.21.15.55:0/736136449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2570074640 0x7f25701b44e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 -- 172.21.15.55:0/736136449 >> 172.21.15.55:0/736136449 conn(0x7f257006f780 msgr2=0x7f2570084df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 -- 172.21.15.55:0/736136449 shutdown_connections 2024-04-26T21:57:05.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.097+0000 7f2575d40640 1 -- 172.21.15.55:0/736136449 wait complete. 2024-04-26T21:57:05.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:57:05.141 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:57:05.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:05 smithi055 ceph-mon[29221]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 835 MiB/s wr, 968 op/s 2024-04-26T21:57:05.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 -- 172.21.15.55:0/3341095520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb4108e70 msgr2=0x7fedb4109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 --2- 172.21.15.55:0/3341095520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb4108e70 0x7fedb4109290 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7feda4009900 tx=0x7feda402a410 comp rx=0 tx=0).stop 2024-04-26T21:57:05.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 -- 172.21.15.55:0/3341095520 shutdown_connections 2024-04-26T21:57:05.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 --2- 172.21.15.55:0/3341095520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb410a980 0x7fedb41112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 --2- 172.21.15.55:0/3341095520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedb4109fc0 0x7fedb410a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 --2- 172.21.15.55:0/3341095520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb4108e70 0x7fedb4109290 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 -- 172.21.15.55:0/3341095520 >> 172.21.15.55:0/3341095520 conn(0x7fedb4077e00 msgr2=0x7fedb407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:05.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 -- 172.21.15.55:0/3341095520 shutdown_connections 2024-04-26T21:57:05.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.487+0000 7fedbb000640 1 -- 172.21.15.55:0/3341095520 wait complete. 2024-04-26T21:57:05.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.488+0000 7fedbb000640 1 Processor -- start 2024-04-26T21:57:05.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.488+0000 7fedbb000640 1 -- start start 2024-04-26T21:57:05.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.488+0000 7fedbb000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedb4108e70 0x7fedb419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.488+0000 7fedbb000640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb4109fc0 0x7fedb419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.488+0000 7fedbb000640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb410a980 0x7fedb41a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.488+0000 7fedbb000640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fedb4114a90 con 0x7fedb410a980 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.488+0000 7fedbb000640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fedb4114910 con 0x7fedb4109fc0 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.488+0000 7fedbb000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fedb4114c10 con 0x7fedb4108e70 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedba7ff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb410a980 0x7fedb41a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedba7ff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb410a980 0x7fedb41a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54818/0 (socket says 172.21.15.55:54818) 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedba7ff640 1 -- 172.21.15.55:0/3403213606 learned_addr learned my addr 172.21.15.55:0/3403213606 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedba7ff640 1 -- 172.21.15.55:0/3403213606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedb4108e70 msgr2=0x7fedb419f150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedba7ff640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedb4108e70 0x7fedb419f150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedba7ff640 1 -- 172.21.15.55:0/3403213606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb4109fc0 msgr2=0x7fedb419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedb97fd640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb4109fc0 0x7fedb419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedba7ff640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb4109fc0 0x7fedb419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.489+0000 7fedba7ff640 1 -- 172.21.15.55:0/3403213606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb41a6d70 con 0x7fedb410a980 2024-04-26T21:57:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.490+0000 7fedb97fd640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb4109fc0 0x7fedb419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.490+0000 7fedba7ff640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb410a980 0x7fedb41a6700 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fedac00eaa0 tx=0x7fedac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.490+0000 7fedaaffd640 1 -- 172.21.15.55:0/3403213606 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac00cdb0 con 0x7fedb410a980 2024-04-26T21:57:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.490+0000 7fedaaffd640 1 -- 172.21.15.55:0/3403213606 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedac005020 con 0x7fedb410a980 2024-04-26T21:57:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.490+0000 7fedaaffd640 1 -- 172.21.15.55:0/3403213606 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac0116d0 con 0x7fedb410a980 2024-04-26T21:57:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.490+0000 7fedbb000640 1 -- 172.21.15.55:0/3403213606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb41a7060 con 0x7fedb410a980 2024-04-26T21:57:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.490+0000 7fedbb000640 1 -- 172.21.15.55:0/3403213606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb41a75e0 con 0x7fedb410a980 2024-04-26T21:57:05.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.495+0000 7fedbef7b740 1 Processor -- start 2024-04-26T21:57:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.496+0000 7fedaaffd640 1 -- 172.21.15.55:0/3403213606 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fedac0054d0 con 0x7fedb410a980 2024-04-26T21:57:05.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.496+0000 7fedbef7b740 1 -- start start 2024-04-26T21:57:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.497+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.497+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a5280 0x55b4fd8a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.497+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.497+0000 7fedbef7b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b4fd808d80 con 0x55b4fd8a40b0 2024-04-26T21:57:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.497+0000 7fedbef7b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b4fd7c28b0 con 0x55b4fd8a5c40 2024-04-26T21:57:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.497+0000 7fedbef7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b4fd804d40 con 0x55b4fd8a5280 2024-04-26T21:57:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.497+0000 7fedaaffd640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7feda00a3d40 0x7feda00a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.498+0000 7fedaaffd640 1 -- 172.21.15.55:0/3403213606 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fedac0d08c0 con 0x7fedb410a980 2024-04-26T21:57:05.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.498+0000 7fedb97fd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.498+0000 7fedba7ff640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7feda00a3d40 0x7feda00a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.499+0000 7fedb97fd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54830/0 (socket says 172.21.15.55:54830) 2024-04-26T21:57:05.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.499+0000 7fedb97fd640 1 -- 172.21.15.55:0/1233138978 learned_addr learned my addr 172.21.15.55:0/1233138978 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.499+0000 7fedb97fd640 1 -- 172.21.15.55:0/1233138978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a5280 msgr2=0x55b4fd8a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.499+0000 7fedb97fd640 1 --2- 172.21.15.55:0/1233138978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a5280 0x55b4fd8a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.499+0000 7fedb97fd640 1 -- 172.21.15.55:0/1233138978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5c40 msgr2=0x55b4fd8ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.499+0000 7fedb97fd640 1 --2- 172.21.15.55:0/1233138978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.499+0000 7fedb97fd640 1 -- 172.21.15.55:0/1233138978 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4fd793a40 con 0x55b4fd8a40b0 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.499+0000 7fedba7ff640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7feda00a3d40 0x7feda00a6200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fedac0a50e0 tx=0x7fedac00cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.500+0000 7fedb97fd640 1 --2- 172.21.15.55:0/1233138978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8a44d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fedb000aa30 tx=0x7fedb000aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ce3cc7a0b80136 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.500+0000 7fed9ffff640 1 -- 172.21.15.55:0/1233138978 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb000f040 con 0x55b4fd8a40b0 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.500+0000 7fed9ffff640 1 -- 172.21.15.55:0/1233138978 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb0018070 con 0x55b4fd8a40b0 2024-04-26T21:57:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.500+0000 7fed9ffff640 1 -- 172.21.15.55:0/1233138978 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0013e40 con 0x55b4fd8a40b0 2024-04-26T21:57:05.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 -- 172.21.15.55:0/1233138978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a40b0 msgr2=0x55b4fd8a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1233138978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8a44d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fedb000aa30 tx=0x7fedb000aef0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 -- 172.21.15.55:0/1233138978 shutdown_connections 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1233138978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1233138978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a5280 0x55b4fd8a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1233138978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8a44d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 -- 172.21.15.55:0/1233138978 >> 172.21.15.55:0/1233138978 conn(0x55b4fd89f5a0 msgr2=0x55b4fd89f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 -- 172.21.15.55:0/1233138978 shutdown_connections 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.501+0000 7fedbef7b740 1 -- 172.21.15.55:0/1233138978 wait complete. 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.502+0000 7fedbef7b740 1 -- start start 2024-04-26T21:57:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.502+0000 7fedbef7b740 1 Processor -- start 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5280 0x55b4fd8aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedbef7b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b4fd8acca0 con 0x55b4fd8a5c40 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedbef7b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b4fd78fd10 con 0x55b4fd8a5280 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedbef7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b4fd8ace20 con 0x55b4fd8a40b0 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedbef7b740 10 client.0 _ll_register_callbacks cb 0x55b4fd7bc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedb97fd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedb97fd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54842/0 (socket says 172.21.15.55:54842) 2024-04-26T21:57:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedb97fd640 1 -- 172.21.15.55:0/4257377951 learned_addr learned my addr 172.21.15.55:0/4257377951 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedb97fd640 1 -- 172.21.15.55:0/4257377951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 msgr2=0x55b4fd8ae5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedb97fd640 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8ae5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedb97fd640 1 -- 172.21.15.55:0/4257377951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5280 msgr2=0x55b4fd8aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedb97fd640 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5280 0x55b4fd8aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.503+0000 7fedb97fd640 1 -- 172.21.15.55:0/4257377951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4fd5e47d0 con 0x55b4fd8a5c40 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.504+0000 7fedb97fd640 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b5b60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fedb000a120 tx=0x7fedb0013690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4290222eff293976 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.504+0000 7fed9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.504+0000 7fed9d7fa640 1 -- 172.21.15.55:0/4257377951 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0022070 con 0x55b4fd8a5c40 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.504+0000 7fed9d7fa640 1 -- 172.21.15.55:0/4257377951 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fedb000ece0 con 0x55b4fd8a5c40 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.504+0000 7fedbef7b740 1 -- 172.21.15.55:0/4257377951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedb0018070 con 0x55b4fd8a5c40 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fed9d7fa640 1 -- 172.21.15.55:0/4257377951 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fedb0006c70 con 0x55b4fd8a5c40 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fedbef7b740 10 client.16113 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fedbef7b740 1 -- 172.21.15.55:0/4257377951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b4fd51a3f0 con 0x55b4fd8a5c40 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fed9d7fa640 1 -- 172.21.15.55:0/4257377951 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb0018070 con 0x55b4fd8a5c40 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fed9d7fa640 1 -- 172.21.15.55:0/4257377951 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fedb0026400 con 0x55b4fd8a5c40 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fedbef7b740 10 client.16113 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fedbef7b740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fedbef7b740 20 client.16113 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.505+0000 7fedbef7b740 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b4fd8b9ea0 0x55b4fd8bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.506+0000 7fedbef7b740 4 client.16113 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.506+0000 7fedbef7b740 1 -- 172.21.15.55:0/4257377951 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4fd816990 con 0x55b4fd8b9ea0 2024-04-26T21:57:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.506+0000 7fedb9ffe640 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b4fd8b9ea0 0x55b4fd8bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.511 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:05 smithi064 ceph-mon[32765]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 835 MiB/s wr, 968 op/s 2024-04-26T21:57:05.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.509+0000 7fedb9ffe640 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b4fd8b9ea0 0x55b4fd8bc300 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8682913b32118192 server_cookie=5919b3ece881ab80 in_seq=0 out_seq=0 2024-04-26T21:57:05.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.509+0000 7fed9d7fa640 10 client.16113 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.512+0000 7fed9d7fa640 1 -- 172.21.15.55:0/4257377951 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4fd816990 con 0x55b4fd8b9ea0 2024-04-26T21:57:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.512+0000 7fed9d7fa640 10 client.16113 handle_command_reply: tid=0 2024-04-26T21:57:05.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 client.16113 shutdown 2024-04-26T21:57:05.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 -- 172.21.15.55:0/4257377951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 msgr2=0x55b4fd8b5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b5b60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fedb000a120 tx=0x7fedb0013690 comp rx=0 tx=0).stop 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 -- 172.21.15.55:0/4257377951 shutdown_connections 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b4fd8b9ea0 0x55b4fd8bc300 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fed9d7fa640 0 client.16113 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b5b60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5280 0x55b4fd8aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 --2- 172.21.15.55:0/4257377951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.513+0000 7fedbef7b740 1 -- 172.21.15.55:0/4257377951 >> 172.21.15.55:0/4257377951 conn(0x55b4fd89f5a0 msgr2=0x55b4fd89e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.514+0000 7fedbef7b740 1 -- 172.21.15.55:0/4257377951 shutdown_connections 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.514+0000 7fedbef7b740 1 -- 172.21.15.55:0/4257377951 wait complete. 2024-04-26T21:57:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.515+0000 7fedbef7b740 20 client.16113 trim_cache size 0 max 16384 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedbef7b740 1 Processor -- start 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedbef7b740 1 -- start start 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd89e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedbef7b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41b2cb0 con 0x55b4fd8a5c40 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedbef7b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41b0580 con 0x55b4fd8a5280 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedbef7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41ae3d0 con 0x55b4fd8a40b0 2024-04-26T21:57:05.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedb97fd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd89e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedb97fd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd89e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54844/0 (socket says 172.21.15.55:54844) 2024-04-26T21:57:05.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.541+0000 7fedb97fd640 1 -- 172.21.15.55:0/2752430286 learned_addr learned my addr 172.21.15.55:0/2752430286 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:05.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fedb97fd640 1 -- 172.21.15.55:0/2752430286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 msgr2=0x55b4fd8ad670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:05.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fedb97fd640 1 --2- 172.21.15.55:0/2752430286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8ad670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fedb97fd640 1 -- 172.21.15.55:0/2752430286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5280 msgr2=0x55b4fd8b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:05.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fedb97fd640 1 --2- 172.21.15.55:0/2752430286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fedb97fd640 1 -- 172.21.15.55:0/2752430286 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4fd4da0f0 con 0x55b4fd8a5c40 2024-04-26T21:57:05.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fedb97fd640 1 --2- 172.21.15.55:0/2752430286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd89e530 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fedb000a690 tx=0x7fedb0026590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53a97a01e8a1f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:05.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fed9f7fe640 1 -- 172.21.15.55:0/2752430286 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0022040 con 0x55b4fd8a5c40 2024-04-26T21:57:05.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fed9f7fe640 1 -- 172.21.15.55:0/2752430286 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb0018050 con 0x55b4fd8a5c40 2024-04-26T21:57:05.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.542+0000 7fed9f7fe640 1 -- 172.21.15.55:0/2752430286 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0004020 con 0x55b4fd8a5c40 2024-04-26T21:57:05.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 -- 172.21.15.55:0/2752430286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 msgr2=0x55b4fd89e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 --2- 172.21.15.55:0/2752430286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd89e530 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fedb000a690 tx=0x7fedb0026590 comp rx=0 tx=0).stop 2024-04-26T21:57:05.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 -- 172.21.15.55:0/2752430286 shutdown_connections 2024-04-26T21:57:05.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 --2- 172.21.15.55:0/2752430286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5c40 0x55b4fd89e530 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 --2- 172.21.15.55:0/2752430286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 --2- 172.21.15.55:0/2752430286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 -- 172.21.15.55:0/2752430286 >> 172.21.15.55:0/2752430286 conn(0x55b4fd89f5a0 msgr2=0x55b4fd89de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:05.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 -- 172.21.15.55:0/2752430286 shutdown_connections 2024-04-26T21:57:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.543+0000 7fedbef7b740 1 -- 172.21.15.55:0/2752430286 wait complete. 2024-04-26T21:57:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 1 -- start start 2024-04-26T21:57:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 1 Processor -- start 2024-04-26T21:57:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41aea40 con 0x55b4fd8a5280 2024-04-26T21:57:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41afeb0 con 0x55b4fd8a5c40 2024-04-26T21:57:05.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fedb41af110 con 0x55b4fd8a40b0 2024-04-26T21:57:05.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedbef7b740 10 client.0 _ll_register_callbacks cb 0x7fedb000a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:05.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.544+0000 7fedb9ffe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fedb9ffe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54848/0 (socket says 172.21.15.55:54848) 2024-04-26T21:57:05.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fedb9ffe640 1 -- 172.21.15.55:0/1156990151 learned_addr learned my addr 172.21.15.55:0/1156990151 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fedb9ffe640 1 -- 172.21.15.55:0/1156990151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 msgr2=0x55b4fd8b2e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fedb9ffe640 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8b2e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fedb9ffe640 1 -- 172.21.15.55:0/1156990151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5c40 msgr2=0x55b4fd8b42e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fedb9ffe640 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b42e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fedb9ffe640 1 -- 172.21.15.55:0/1156990151 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4fd7c94b0 con 0x55b4fd8a5280 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fedb9ffe640 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b3340 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7feda402a930 tx=0x7feda4005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609b759e7a218502 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.545+0000 7fed9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fed9d7fa640 1 -- 172.21.15.55:0/1156990151 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4035040 con 0x55b4fd8a5280 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fed9d7fa640 1 -- 172.21.15.55:0/1156990151 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7feda4004450 con 0x55b4fd8a5280 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fedbef7b740 1 -- 172.21.15.55:0/1156990151 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedb0018050 con 0x55b4fd8a5280 2024-04-26T21:57:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fed9d7fa640 1 -- 172.21.15.55:0/1156990151 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feda40401d0 con 0x55b4fd8a5280 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fedbef7b740 10 client.16119 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fedbef7b740 1 -- 172.21.15.55:0/1156990151 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b4fd8b5ab0 con 0x55b4fd8a5280 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fed9d7fa640 1 -- 172.21.15.55:0/1156990151 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb0018050 con 0x55b4fd8a5280 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fed9d7fa640 1 -- 172.21.15.55:0/1156990151 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7feda4004a80 con 0x55b4fd8a5280 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fedbef7b740 10 client.16119 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fedbef7b740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fedbef7b740 20 client.16119 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.546+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b4fd8b9ea0 0x55b4fd9229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.547+0000 7fedbef7b740 4 client.16119 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.547+0000 7fedbef7b740 1 -- 172.21.15.55:0/1156990151 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b4fd810800 con 0x55b4fd8b9ea0 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.548+0000 7fedb97fd640 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b4fd8b9ea0 0x55b4fd9229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.549+0000 7fedb97fd640 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b4fd8b9ea0 0x55b4fd9229e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c49e5248d9e23ca0 server_cookie=4eda3b98f795dcb5 in_seq=0 out_seq=0 2024-04-26T21:57:05.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.549+0000 7fed9d7fa640 10 client.16119 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:05.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.553+0000 7fed9d7fa640 1 -- 172.21.15.55:0/1156990151 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55b4fd810800 con 0x55b4fd8b9ea0 2024-04-26T21:57:05.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.553+0000 7fed9d7fa640 10 client.16119 handle_command_reply: tid=0 2024-04-26T21:57:05.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.553+0000 7fedbef7b740 1 client.16119 shutdown 2024-04-26T21:57:05.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 -- 172.21.15.55:0/1156990151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5280 msgr2=0x55b4fd8b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b3340 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7feda402a930 tx=0x7feda4005440 comp rx=0 tx=0).stop 2024-04-26T21:57:05.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 -- 172.21.15.55:0/1156990151 shutdown_connections 2024-04-26T21:57:05.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b4fd8a5c40 0x55b4fd8b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b4fd8b9ea0 0x55b4fd9229e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b4fd8a5280 0x55b4fd8b3340 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 --2- 172.21.15.55:0/1156990151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b4fd8a40b0 0x55b4fd8b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 -- 172.21.15.55:0/1156990151 >> 172.21.15.55:0/1156990151 conn(0x55b4fd89f5a0 msgr2=0x55b4fd89e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fed9d7fa640 0 client.16119 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 -- 172.21.15.55:0/1156990151 shutdown_connections 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.554+0000 7fedbef7b740 1 -- 172.21.15.55:0/1156990151 wait complete. 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.555+0000 7fedbef7b740 20 client.16119 trim_cache size 0 max 16384 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (2 inodes in the stack)", 2024-04-26T21:57:05.557 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:57:05.558 INFO:teuthology.orchestra.run.smithi055.stdout: "7717dbdd-c62b-4490-be7e-c83066aa442f": { 2024-04-26T21:57:05.558 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:57:05.558 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "7717dbdd-c62b-4490-be7e-c83066aa442f", 2024-04-26T21:57:05.558 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:57:05.558 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:57:05.558 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:57:05.558 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:57:05.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.555+0000 7fedbb000640 1 -- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7feda00a3d40 msgr2=0x7feda00a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.555+0000 7fedbb000640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7feda00a3d40 0x7feda00a6200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fedac0a50e0 tx=0x7fedac00cf50 comp rx=0 tx=0).stop 2024-04-26T21:57:05.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.555+0000 7fedbb000640 1 -- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb410a980 msgr2=0x7fedb41a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:05.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.555+0000 7fedbb000640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb410a980 0x7fedb41a6700 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fedac00eaa0 tx=0x7fedac00ef60 comp rx=0 tx=0).stop 2024-04-26T21:57:05.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.556+0000 7fedbb000640 1 -- 172.21.15.55:0/3403213606 shutdown_connections 2024-04-26T21:57:05.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.556+0000 7fedbb000640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7feda00a3d40 0x7feda00a6200 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.556+0000 7fedbb000640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb410a980 0x7fedb41a6700 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.556+0000 7fedbb000640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fedb4109fc0 0x7fedb419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.556+0000 7fedbb000640 1 --2- 172.21.15.55:0/3403213606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedb4108e70 0x7fedb419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.556+0000 7fedbb000640 1 -- 172.21.15.55:0/3403213606 >> 172.21.15.55:0/3403213606 conn(0x7fedb4077e00 msgr2=0x7fedb410d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.556+0000 7fedbb000640 1 -- 172.21.15.55:0/3403213606 shutdown_connections 2024-04-26T21:57:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:05.556+0000 7fedbb000640 1 -- 172.21.15.55:0/3403213606 wait complete. 2024-04-26T21:57:05.572 INFO:tasks.cephfs.filesystem:scrub status for tag:7717dbdd-c62b-4490-be7e-c83066aa442f - {'path': '/', 'tag': '7717dbdd-c62b-4490-be7e-c83066aa442f', 'options': 'recursive,force'} 2024-04-26T21:57:05.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:05 smithi165 ceph-mon[33296]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 835 MiB/s wr, 968 op/s 2024-04-26T21:57:06.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:06 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:57:06.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:06 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:06.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:06 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:57:06.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:06 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:57:06.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:06 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:06.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:06 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:57:06.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:06 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:57:06.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:06 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:06.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:06 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:57:07.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:07 smithi064 ceph-mon[32765]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 861 MiB/s wr, 1.00k op/s 2024-04-26T21:57:07.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:07 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:57:07.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:07 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:07.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:07 smithi165 ceph-mon[33296]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 861 MiB/s wr, 1.00k op/s 2024-04-26T21:57:07.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:07 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:57:07.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:07 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:07.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:07 smithi055 ceph-mon[29221]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 861 MiB/s wr, 1.00k op/s 2024-04-26T21:57:07.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:07 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:57:07.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:07 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:09.490 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:09 smithi055 ceph-mon[29221]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 884 MiB/s wr, 1.06k op/s 2024-04-26T21:57:09.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:09 smithi064 ceph-mon[32765]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 884 MiB/s wr, 1.06k op/s 2024-04-26T21:57:09.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:09 smithi165 ceph-mon[33296]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 884 MiB/s wr, 1.06k op/s 2024-04-26T21:57:11.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:11 smithi064 ceph-mon[32765]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 783 MiB/s wr, 933 op/s 2024-04-26T21:57:11.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:11 smithi165 ceph-mon[33296]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 783 MiB/s wr, 933 op/s 2024-04-26T21:57:11.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:11 smithi055 ceph-mon[29221]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 783 MiB/s wr, 933 op/s 2024-04-26T21:57:12.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:12 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:12.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:12 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:12.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:12 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:13.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:13 smithi165 ceph-mon[33296]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 795 MiB/s wr, 935 op/s 2024-04-26T21:57:13.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:13 smithi055 ceph-mon[29221]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 795 MiB/s wr, 935 op/s 2024-04-26T21:57:13.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:13 smithi064 ceph-mon[32765]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 795 MiB/s wr, 935 op/s 2024-04-26T21:57:15.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:15 smithi165 ceph-mon[33296]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 822 MiB/s wr, 935 op/s 2024-04-26T21:57:15.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:15 smithi055 ceph-mon[29221]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 822 MiB/s wr, 935 op/s 2024-04-26T21:57:15.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:15 smithi064 ceph-mon[32765]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 822 MiB/s wr, 935 op/s 2024-04-26T21:57:16.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:16 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:57:16.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:16 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:57:16.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:16 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:57:17.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:17 smithi165 ceph-mon[33296]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 951 op/s 2024-04-26T21:57:17.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:17 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:57:17.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:17 smithi055 ceph-mon[29221]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 951 op/s 2024-04-26T21:57:17.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:17 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:57:17.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:17 smithi064 ceph-mon[32765]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 951 op/s 2024-04-26T21:57:17.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:17 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:57:19.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:19 smithi165 ceph-mon[33296]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 883 MiB/s wr, 1.01k op/s 2024-04-26T21:57:19.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:19 smithi055 ceph-mon[29221]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 883 MiB/s wr, 1.01k op/s 2024-04-26T21:57:19.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:19 smithi064 ceph-mon[32765]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 883 MiB/s wr, 1.01k op/s 2024-04-26T21:57:21.578 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:21 smithi165 ceph-mon[33296]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 793 MiB/s wr, 894 op/s 2024-04-26T21:57:21.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:21 smithi055 ceph-mon[29221]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 793 MiB/s wr, 894 op/s 2024-04-26T21:57:21.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:21 smithi064 ceph-mon[32765]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 793 MiB/s wr, 894 op/s 2024-04-26T21:57:23.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:23 smithi165 ceph-mon[33296]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 949 op/s 2024-04-26T21:57:23.702 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:23 smithi055 ceph-mon[29221]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 949 op/s 2024-04-26T21:57:23.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:23 smithi064 ceph-mon[32765]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 949 op/s 2024-04-26T21:57:24.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:24 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:24.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:24 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:24.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:24 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:25.692 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:25 smithi055 ceph-mon[29221]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 873 MiB/s wr, 965 op/s 2024-04-26T21:57:25.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:25 smithi064 ceph-mon[32765]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 873 MiB/s wr, 965 op/s 2024-04-26T21:57:25.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:25 smithi165 ceph-mon[33296]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 873 MiB/s wr, 965 op/s 2024-04-26T21:57:26.690 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:26.690 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:26.690 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:57:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:57:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:26.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:57:27.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:27 smithi055 ceph-mon[29221]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 857 MiB/s wr, 960 op/s 2024-04-26T21:57:27.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:27 smithi064 ceph-mon[32765]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 857 MiB/s wr, 960 op/s 2024-04-26T21:57:27.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:27 smithi165 ceph-mon[33296]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 857 MiB/s wr, 960 op/s 2024-04-26T21:57:28.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:28 smithi055 ceph-mon[29221]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 964 op/s 2024-04-26T21:57:28.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:28 smithi064 ceph-mon[32765]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 964 op/s 2024-04-26T21:57:28.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:28 smithi165 ceph-mon[33296]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 964 op/s 2024-04-26T21:57:30.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:30 smithi055 ceph-mon[29221]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 803 MiB/s wr, 902 op/s 2024-04-26T21:57:31.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:30 smithi064 ceph-mon[32765]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 803 MiB/s wr, 902 op/s 2024-04-26T21:57:31.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:30 smithi165 ceph-mon[33296]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 803 MiB/s wr, 902 op/s 2024-04-26T21:57:32.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:32 smithi055 ceph-mon[29221]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 840 MiB/s wr, 949 op/s 2024-04-26T21:57:33.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:32 smithi064 ceph-mon[32765]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 840 MiB/s wr, 949 op/s 2024-04-26T21:57:33.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:32 smithi165 ceph-mon[33296]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 840 MiB/s wr, 949 op/s 2024-04-26T21:57:34.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:34 smithi055 ceph-mon[29221]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 861 MiB/s wr, 971 op/s 2024-04-26T21:57:35.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:34 smithi064 ceph-mon[32765]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 861 MiB/s wr, 971 op/s 2024-04-26T21:57:35.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:34 smithi165 ceph-mon[33296]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 861 MiB/s wr, 971 op/s 2024-04-26T21:57:35.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:57:35.574 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:57:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.922+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/413549320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2810a010 msgr2=0x7fdd2810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:35.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.922+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/413549320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2810a010 0x7fdd2810a470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009be0 tx=0x7fdd2002a710 comp rx=0 tx=0).stop 2024-04-26T21:57:35.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.922+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/413549320 shutdown_connections 2024-04-26T21:57:35.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.922+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/413549320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd2810a9b0 0x7fdd2810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.922+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/413549320 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2810a010 0x7fdd2810a470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.922+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/413549320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd28073d70 0x7fdd28074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.922+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/413549320 >> 172.21.15.55:0/413549320 conn(0x7fdd2806f840 msgr2=0x7fdd28071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:35.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.926+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/413549320 shutdown_connections 2024-04-26T21:57:35.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.927+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/413549320 wait complete. 2024-04-26T21:57:35.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.928+0000 7fdd2f1dd640 1 Processor -- start 2024-04-26T21:57:35.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.928+0000 7fdd2f1dd640 1 -- start start 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.930+0000 7fdd2f1dd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd28073d70 0x7fdd281a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.930+0000 7fdd2f1dd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd2810a010 0x7fdd281a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.930+0000 7fdd2f1dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2810a9b0 0x7fdd281aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.930+0000 7fdd2f1dd640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2811d200 con 0x7fdd28073d70 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.930+0000 7fdd2f1dd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2811d080 con 0x7fdd2810a9b0 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.930+0000 7fdd2f1dd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2811d380 con 0x7fdd2810a010 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.931+0000 7fdd2e1db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd28073d70 0x7fdd281a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.931+0000 7fdd2e1db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd28073d70 0x7fdd281a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57734/0 (socket says 172.21.15.55:57734) 2024-04-26T21:57:35.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.931+0000 7fdd2e1db640 1 -- 172.21.15.55:0/1344504467 learned_addr learned my addr 172.21.15.55:0/1344504467 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.931+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd2810a010 0x7fdd281a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.932+0000 7fdd2e9dc640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2810a9b0 0x7fdd281aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.932+0000 7fdd2d9da640 1 -- 172.21.15.55:0/1344504467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2810a9b0 msgr2=0x7fdd281aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.932+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2810a9b0 0x7fdd281aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.932+0000 7fdd2d9da640 1 -- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd28073d70 msgr2=0x7fdd281a7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.932+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd28073d70 0x7fdd281a7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.932+0000 7fdd2d9da640 1 -- 172.21.15.55:0/1344504467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd281af530 con 0x7fdd2810a010 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.933+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd2810a010 0x7fdd281a7e50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009b30 tx=0x7fdd20031670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.934+0000 7fdd1f7fe640 1 -- 172.21.15.55:0/1344504467 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20052070 con 0x7fdd2810a010 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.934+0000 7fdd1f7fe640 1 -- 172.21.15.55:0/1344504467 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd2004db30 con 0x7fdd2810a010 2024-04-26T21:57:35.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.934+0000 7fdd1f7fe640 1 -- 172.21.15.55:0/1344504467 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20036040 con 0x7fdd2810a010 2024-04-26T21:57:35.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.936+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/1344504467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd28111060 con 0x7fdd2810a010 2024-04-26T21:57:35.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.936+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/1344504467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd28111490 con 0x7fdd2810a010 2024-04-26T21:57:35.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.948+0000 7fdd1f7fe640 1 -- 172.21.15.55:0/1344504467 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdd20055070 con 0x7fdd2810a010 2024-04-26T21:57:35.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.949+0000 7fdd1f7fe640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd140a3dc0 0x7fdd140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.950+0000 7fdd1f7fe640 1 -- 172.21.15.55:0/1344504467 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fdd20101ff0 con 0x7fdd2810a010 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.952+0000 7fdd33230740 1 Processor -- start 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.952+0000 7fdd33230740 1 -- start start 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.952+0000 7fdd33230740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f74372f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.952+0000 7fdd33230740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f743730280 0x55f743730700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.952+0000 7fdd33230740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730c40 0x55f7437374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.953+0000 7fdd33230740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f743693d80 con 0x55f743730280 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.953+0000 7fdd33230740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f74364d8b0 con 0x55f74372f0b0 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.953+0000 7fdd33230740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f74368fd40 con 0x55f743730c40 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.953+0000 7fdd2e9dc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730c40 0x55f7437374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.953+0000 7fdd2e9dc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730c40 0x55f7437374d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:56432/0 (socket says 172.21.15.55:56432) 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.953+0000 7fdd2e9dc640 1 -- 172.21.15.55:0/3137793594 learned_addr learned my addr 172.21.15.55:0/3137793594 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.954+0000 7fdd2e1db640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd140a3dc0 0x7fdd140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.955+0000 7fdd2e1db640 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f74372f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.955+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f743730280 0x55f743730700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.955+0000 7fdd2e9dc640 1 -- 172.21.15.55:0/3137793594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 msgr2=0x55f74372f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.955+0000 7fdd2e9dc640 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f74372f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.955+0000 7fdd2e9dc640 1 -- 172.21.15.55:0/3137793594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f743730280 msgr2=0x55f743730700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.955+0000 7fdd2e9dc640 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f743730280 0x55f743730700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.955+0000 7fdd2e9dc640 1 -- 172.21.15.55:0/3137793594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74361ea40 con 0x55f743730c40 2024-04-26T21:57:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.957+0000 7fdd2e1db640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd140a3dc0 0x7fdd140a6280 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009fd0 tx=0x7fdd180124d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:35.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.959+0000 7fdd2e9dc640 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730c40 0x55f7437374d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd2402c720 tx=0x7fdd2402cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=268368a7e08aa1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.962+0000 7fdd1cbfe640 1 -- 172.21.15.55:0/3137793594 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24037070 con 0x55f743730c40 2024-04-26T21:57:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.962+0000 7fdd1cbfe640 1 -- 172.21.15.55:0/3137793594 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd2402ce30 con 0x55f743730c40 2024-04-26T21:57:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.963+0000 7fdd33230740 1 -- 172.21.15.55:0/3137793594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730c40 msgr2=0x55f7437374d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.963+0000 7fdd33230740 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730c40 0x55f7437374d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdd2402c720 tx=0x7fdd2402cbe0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.976+0000 7fdd33230740 1 -- 172.21.15.55:0/3137793594 shutdown_connections 2024-04-26T21:57:35.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.976+0000 7fdd33230740 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730c40 0x55f7437374d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.976+0000 7fdd33230740 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f743730280 0x55f743730700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.977+0000 7fdd33230740 1 --2- 172.21.15.55:0/3137793594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f74372f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.977+0000 7fdd33230740 1 -- 172.21.15.55:0/3137793594 >> 172.21.15.55:0/3137793594 conn(0x55f74372a5a0 msgr2=0x55f74372a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:35.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.979+0000 7fdd33230740 1 -- 172.21.15.55:0/3137793594 shutdown_connections 2024-04-26T21:57:35.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.979+0000 7fdd33230740 1 -- 172.21.15.55:0/3137793594 wait complete. 2024-04-26T21:57:35.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.980+0000 7fdd33230740 1 -- start start 2024-04-26T21:57:35.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.980+0000 7fdd33230740 1 Processor -- start 2024-04-26T21:57:35.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.980+0000 7fdd33230740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372f0b0 0x55f7437394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.981+0000 7fdd33230740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743739a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.981+0000 7fdd33230740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730c40 0x55f743740a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:35.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.981+0000 7fdd33230740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f743737ca0 con 0x55f74372f0b0 2024-04-26T21:57:35.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.981+0000 7fdd33230740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f74361ad10 con 0x55f743730c40 2024-04-26T21:57:35.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.981+0000 7fdd33230740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f743737e20 con 0x55f743730280 2024-04-26T21:57:35.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.981+0000 7fdd33230740 10 client.0 _ll_register_callbacks cb 0x55f743647210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:35.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.984+0000 7fdd2e9dc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730c40 0x55f743740a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.984+0000 7fdd2d9da640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743739a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.984+0000 7fdd2e1db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372f0b0 0x55f7437394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:35.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.984+0000 7fdd2e1db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372f0b0 0x55f7437394c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57762/0 (socket says 172.21.15.55:57762) 2024-04-26T21:57:35.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.984+0000 7fdd2e1db640 1 -- 172.21.15.55:0/2714897660 learned_addr learned my addr 172.21.15.55:0/2714897660 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:35.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.986+0000 7fdd2e1db640 1 -- 172.21.15.55:0/2714897660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 msgr2=0x55f743739a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:35.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.986+0000 7fdd2e1db640 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743739a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.986+0000 7fdd2e1db640 1 -- 172.21.15.55:0/2714897660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730c40 msgr2=0x55f743740a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:35.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.986+0000 7fdd2e1db640 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730c40 0x55f743740a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:35.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.986+0000 7fdd2e1db640 1 -- 172.21.15.55:0/2714897660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74346f7d0 con 0x55f74372f0b0 2024-04-26T21:57:35.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.988+0000 7fdd2e1db640 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372f0b0 0x55f7437394c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd18010bd0 tx=0x7fdd18009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd74dffb5269e4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:35.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.988+0000 7fdd11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:35.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.991+0000 7fdd11ffb640 1 -- 172.21.15.55:0/2714897660 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1801ebd0 con 0x55f74372f0b0 2024-04-26T21:57:35.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.991+0000 7fdd11ffb640 1 -- 172.21.15.55:0/2714897660 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fdd1801ed70 con 0x55f74372f0b0 2024-04-26T21:57:35.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.995+0000 7fdd33230740 1 -- 172.21.15.55:0/2714897660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd2402ce30 con 0x55f74372f0b0 2024-04-26T21:57:35.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.996+0000 7fdd11ffb640 1 -- 172.21.15.55:0/2714897660 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdd1802a410 con 0x55f74372f0b0 2024-04-26T21:57:35.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.997+0000 7fdd33230740 10 client.16128 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:35.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.997+0000 7fdd33230740 1 -- 172.21.15.55:0/2714897660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f7433a53f0 con 0x55f74372f0b0 2024-04-26T21:57:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:35.999+0000 7fdd11ffb640 1 -- 172.21.15.55:0/2714897660 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd2402ce30 con 0x55f74372f0b0 2024-04-26T21:57:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.000+0000 7fdd11ffb640 1 -- 172.21.15.55:0/2714897660 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fdd1801f050 con 0x55f74372f0b0 2024-04-26T21:57:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.000+0000 7fdd33230740 10 client.16128 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.000+0000 7fdd33230740 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.000+0000 7fdd33230740 20 client.16128 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.000+0000 7fdd33230740 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f743745030 0x55f7437474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.000+0000 7fdd33230740 4 client.16128 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.000+0000 7fdd33230740 1 -- 172.21.15.55:0/2714897660 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7436a1990 con 0x55f743745030 2024-04-26T21:57:36.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.011+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f743745030 0x55f7437474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.035+0000 7fdd11ffb640 10 client.16128 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:36.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.035+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f743745030 0x55f7437474d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=563dcb100cc741af server_cookie=4a1ba1f50fde826f in_seq=0 out_seq=0 2024-04-26T21:57:36.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.041+0000 7fdd11ffb640 1 -- 172.21.15.55:0/2714897660 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7436a1990 con 0x55f743745030 2024-04-26T21:57:36.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.041+0000 7fdd11ffb640 10 client.16128 handle_command_reply: tid=0 2024-04-26T21:57:36.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.041+0000 7fdd33230740 1 client.16128 shutdown 2024-04-26T21:57:36.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 -- 172.21.15.55:0/2714897660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372f0b0 msgr2=0x55f7437394c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372f0b0 0x55f7437394c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd18010bd0 tx=0x7fdd18009260 comp rx=0 tx=0).stop 2024-04-26T21:57:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 -- 172.21.15.55:0/2714897660 shutdown_connections 2024-04-26T21:57:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730c40 0x55f743740a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743739a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f743745030 0x55f7437474d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 --2- 172.21.15.55:0/2714897660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372f0b0 0x55f7437394c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 -- 172.21.15.55:0/2714897660 >> 172.21.15.55:0/2714897660 conn(0x55f74372a5a0 msgr2=0x55f7437296b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd11ffb640 0 client.16128 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:36.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.045+0000 7fdd33230740 1 -- 172.21.15.55:0/2714897660 shutdown_connections 2024-04-26T21:57:36.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.046+0000 7fdd33230740 1 -- 172.21.15.55:0/2714897660 wait complete. 2024-04-26T21:57:36.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.047+0000 7fdd33230740 20 client.16128 trim_cache size 0 max 16384 2024-04-26T21:57:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.108+0000 7fdd33230740 1 Processor -- start 2024-04-26T21:57:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.108+0000 7fdd33230740 1 -- start start 2024-04-26T21:57:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.108+0000 7fdd33230740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372a5a0 0x55f743739960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.108+0000 7fdd33230740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f743740a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd33230740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743725540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd33230740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28078100 con 0x55f74372a5a0 2024-04-26T21:57:36.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd33230740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd280759d0 con 0x55f74372f0b0 2024-04-26T21:57:36.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd33230740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28118520 con 0x55f743730280 2024-04-26T21:57:36.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd2e9dc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743725540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd2d9da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f743740a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd2e9dc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743725540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:56442/0 (socket says 172.21.15.55:56442) 2024-04-26T21:57:36.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd2e9dc640 1 -- 172.21.15.55:0/1716232924 learned_addr learned my addr 172.21.15.55:0/1716232924 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:36.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.109+0000 7fdd2d9da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f743740a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43370/0 (socket says 172.21.15.55:43370) 2024-04-26T21:57:36.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.110+0000 7fdd2e9dc640 1 -- 172.21.15.55:0/1716232924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 msgr2=0x55f743740a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.110+0000 7fdd2e9dc640 1 --2- 172.21.15.55:0/1716232924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f743740a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.110+0000 7fdd2e9dc640 1 -- 172.21.15.55:0/1716232924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372a5a0 msgr2=0x55f743739960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.110+0000 7fdd2e9dc640 1 --2- 172.21.15.55:0/1716232924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372a5a0 0x55f743739960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.110+0000 7fdd2e9dc640 1 -- 172.21.15.55:0/1716232924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f743725d10 con 0x55f743730280 2024-04-26T21:57:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.110+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/1716232924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f743740a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:57:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.112+0000 7fdd2e9dc640 1 --2- 172.21.15.55:0/1716232924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743725540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd24008010 tx=0x7fdd24033ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82b0f80a563f655 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.126+0000 7fdd13fff640 1 -- 172.21.15.55:0/1716232924 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24042070 con 0x55f743730280 2024-04-26T21:57:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.126+0000 7fdd13fff640 1 -- 172.21.15.55:0/1716232924 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd240338b0 con 0x55f743730280 2024-04-26T21:57:36.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.128+0000 7fdd33230740 1 -- 172.21.15.55:0/1716232924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 msgr2=0x55f743725540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:36.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.128+0000 7fdd33230740 1 --2- 172.21.15.55:0/1716232924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743725540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdd24008010 tx=0x7fdd24033ef0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.131+0000 7fdd33230740 1 -- 172.21.15.55:0/1716232924 shutdown_connections 2024-04-26T21:57:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.131+0000 7fdd33230740 1 --2- 172.21.15.55:0/1716232924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f743730280 0x55f743725540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.131+0000 7fdd33230740 1 --2- 172.21.15.55:0/1716232924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74372f0b0 0x55f743740a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.131+0000 7fdd33230740 1 --2- 172.21.15.55:0/1716232924 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372a5a0 0x55f743739960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.131+0000 7fdd33230740 1 -- 172.21.15.55:0/1716232924 >> 172.21.15.55:0/1716232924 conn(0x55f743737630 msgr2=0x55f743728e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:36.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.139+0000 7fdd33230740 1 -- 172.21.15.55:0/1716232924 shutdown_connections 2024-04-26T21:57:36.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.145+0000 7fdd33230740 1 -- 172.21.15.55:0/1716232924 wait complete. 2024-04-26T21:57:36.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.146+0000 7fdd33230740 1 -- start start 2024-04-26T21:57:36.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.146+0000 7fdd33230740 1 Processor -- start 2024-04-26T21:57:36.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd33230740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372a5a0 0x55f74373e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd33230740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74372f0b0 0x55f74373ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd33230740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730280 0x55f74373fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd33230740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28118b90 con 0x55f74372a5a0 2024-04-26T21:57:36.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd33230740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28075300 con 0x55f743730280 2024-04-26T21:57:36.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd33230740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28119260 con 0x55f74372f0b0 2024-04-26T21:57:36.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd33230740 10 client.0 _ll_register_callbacks cb 0x7fdd18009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:36.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd2d9da640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74372f0b0 0x55f74373ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd2d9da640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74372f0b0 0x55f74373ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:56452/0 (socket says 172.21.15.55:56452) 2024-04-26T21:57:36.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.147+0000 7fdd2d9da640 1 -- 172.21.15.55:0/406057877 learned_addr learned my addr 172.21.15.55:0/406057877 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:36.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.148+0000 7fdd2e9dc640 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730280 0x55f74373fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.148+0000 7fdd2d9da640 1 -- 172.21.15.55:0/406057877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730280 msgr2=0x55f74373fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:36.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.148+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730280 0x55f74373fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.148+0000 7fdd2d9da640 1 -- 172.21.15.55:0/406057877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372a5a0 msgr2=0x55f74373e510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:57:36.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.148+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372a5a0 0x55f74373e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.148+0000 7fdd2d9da640 1 -- 172.21.15.55:0/406057877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7433650f0 con 0x55f74372f0b0 2024-04-26T21:57:36.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.149+0000 7fdd2d9da640 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74372f0b0 0x55f74373ea50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd200cab00 tx=0x7fdd2002ac10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31fd1468f4d5d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:36.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.149+0000 7fdd11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:57:36.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.149+0000 7fdd11ffb640 1 -- 172.21.15.55:0/406057877 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2005c040 con 0x55f74372f0b0 2024-04-26T21:57:36.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.150+0000 7fdd33230740 1 -- 172.21.15.55:0/406057877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd240338b0 con 0x55f74372f0b0 2024-04-26T21:57:36.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.150+0000 7fdd11ffb640 1 -- 172.21.15.55:0/406057877 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fdd200ca7a0 con 0x55f74372f0b0 2024-04-26T21:57:36.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.150+0000 7fdd11ffb640 1 -- 172.21.15.55:0/406057877 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdd200d2120 con 0x55f74372f0b0 2024-04-26T21:57:36.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.150+0000 7fdd33230740 10 client.25501 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:36.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.150+0000 7fdd33230740 1 -- 172.21.15.55:0/406057877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55f7436544b0 con 0x55f74372f0b0 2024-04-26T21:57:36.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.151+0000 7fdd11ffb640 1 -- 172.21.15.55:0/406057877 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd240338b0 con 0x55f74372f0b0 2024-04-26T21:57:36.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.153+0000 7fdd11ffb640 1 -- 172.21.15.55:0/406057877 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fdd200ee040 con 0x55f74372f0b0 2024-04-26T21:57:36.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.155+0000 7fdd33230740 10 client.25501 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:36.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.155+0000 7fdd33230740 10 client.25501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:36.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.155+0000 7fdd33230740 20 client.25501 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:36.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.155+0000 7fdd33230740 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f743730c40 0x55f7437ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.155+0000 7fdd33230740 4 client.25501 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:57:36.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.155+0000 7fdd33230740 1 -- 172.21.15.55:0/406057877 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f74369b800 con 0x55f743730c40 2024-04-26T21:57:36.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.163+0000 7fdd2e1db640 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f743730c40 0x55f7437ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.164+0000 7fdd2e1db640 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f743730c40 0x55f7437ae1c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b643268d8c26558c server_cookie=81ce4f689847be66 in_seq=0 out_seq=0 2024-04-26T21:57:36.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.166+0000 7fdd11ffb640 10 client.25501 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:36.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.167+0000 7fdd11ffb640 1 -- 172.21.15.55:0/406057877 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f74369b800 con 0x55f743730c40 2024-04-26T21:57:36.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.167+0000 7fdd11ffb640 10 client.25501 handle_command_reply: tid=0 2024-04-26T21:57:36.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.167+0000 7fdd33230740 1 client.25501 shutdown 2024-04-26T21:57:36.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 -- 172.21.15.55:0/406057877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74372f0b0 msgr2=0x55f74373ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:36.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74372f0b0 0x55f74373ea50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd200cab00 tx=0x7fdd2002ac10 comp rx=0 tx=0).stop 2024-04-26T21:57:36.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 -- 172.21.15.55:0/406057877 shutdown_connections 2024-04-26T21:57:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f743730280 0x55f74373fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74372f0b0 0x55f74373ea50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f743730c40 0x55f7437ae1c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 --2- 172.21.15.55:0/406057877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74372a5a0 0x55f74373e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 -- 172.21.15.55:0/406057877 >> 172.21.15.55:0/406057877 conn(0x55f743737630 msgr2=0x55f7437296b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd11ffb640 0 client.25501 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 -- 172.21.15.55:0/406057877 shutdown_connections 2024-04-26T21:57:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.172+0000 7fdd33230740 1 -- 172.21.15.55:0/406057877 wait complete. 2024-04-26T21:57:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.173+0000 7fdd33230740 20 client.25501 trim_cache size 0 max 16384 2024-04-26T21:57:36.176 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:57:36.176 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:57:36.176 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:57:36.176 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:57:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.179+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd140a3dc0 msgr2=0x7fdd140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.179+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd140a3dc0 0x7fdd140a6280 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdd18009fd0 tx=0x7fdd180124d0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.179+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/1344504467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd2810a010 msgr2=0x7fdd281a7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.179+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd2810a010 0x7fdd281a7e50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009b30 tx=0x7fdd20031670 comp rx=0 tx=0).stop 2024-04-26T21:57:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.180+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/1344504467 shutdown_connections 2024-04-26T21:57:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.180+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdd140a3dc0 0x7fdd140a6280 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.180+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd2810a9b0 0x7fdd281aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.180+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd2810a010 0x7fdd281a7e50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.180+0000 7fdd2f1dd640 1 --2- 172.21.15.55:0/1344504467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd28073d70 0x7fdd281a7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.180+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/1344504467 >> 172.21.15.55:0/1344504467 conn(0x7fdd2806f840 msgr2=0x7fdd28071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.180+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/1344504467 shutdown_connections 2024-04-26T21:57:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.180+0000 7fdd2f1dd640 1 -- 172.21.15.55:0/1344504467 wait complete. 2024-04-26T21:57:36.219 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:57:36.219 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:57:36.621 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:36 smithi055 ceph-mon[29221]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 841 MiB/s wr, 1.12k op/s 2024-04-26T21:57:36.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.635+0000 7fb708e85640 1 -- 172.21.15.55:0/1194418333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb70407a0b0 msgr2=0x7fb70407c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:36.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.636+0000 7fb708e85640 1 --2- 172.21.15.55:0/1194418333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb70407a0b0 0x7fb70407c540 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009ca0 tx=0x7fb6fc02a8f0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.638+0000 7fb708e85640 1 -- 172.21.15.55:0/1194418333 shutdown_connections 2024-04-26T21:57:36.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.638+0000 7fb708e85640 1 --2- 172.21.15.55:0/1194418333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb70407a0b0 0x7fb70407c540 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.638+0000 7fb708e85640 1 --2- 172.21.15.55:0/1194418333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb704077700 0x7fb704079b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.638+0000 7fb708e85640 1 --2- 172.21.15.55:0/1194418333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704074820 0x7fb704074c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.638+0000 7fb708e85640 1 -- 172.21.15.55:0/1194418333 >> 172.21.15.55:0/1194418333 conn(0x7fb70406f7d0 msgr2=0x7fb704071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:36.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.649+0000 7fb708e85640 1 -- 172.21.15.55:0/1194418333 shutdown_connections 2024-04-26T21:57:36.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.650+0000 7fb708e85640 1 -- 172.21.15.55:0/1194418333 wait complete. 2024-04-26T21:57:36.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.651+0000 7fb708e85640 1 Processor -- start 2024-04-26T21:57:36.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.651+0000 7fb708e85640 1 -- start start 2024-04-26T21:57:36.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.652+0000 7fb708e85640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb704074820 0x7fb704083eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.652+0000 7fb708e85640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704077700 0x7fb7040843f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.652+0000 7fb708e85640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70407a0b0 0x7fb704084930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.653+0000 7fb708e85640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb70412c5f0 con 0x7fb704074820 2024-04-26T21:57:36.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.653+0000 7fb708e85640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb70412c470 con 0x7fb704077700 2024-04-26T21:57:36.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.653+0000 7fb708e85640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb70412c770 con 0x7fb70407a0b0 2024-04-26T21:57:36.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.653+0000 7fb702d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70407a0b0 0x7fb704084930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.653+0000 7fb702d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70407a0b0 0x7fb704084930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:56466/0 (socket says 172.21.15.55:56466) 2024-04-26T21:57:36.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.653+0000 7fb702d9d640 1 -- 172.21.15.55:0/17265041 learned_addr learned my addr 172.21.15.55:0/17265041 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:36.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.653+0000 7fb70259c640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb704074820 0x7fb704083eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.657+0000 7fb702d9d640 1 -- 172.21.15.55:0/17265041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704077700 msgr2=0x7fb7040843f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:57:36.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.657+0000 7fb702d9d640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704077700 0x7fb7040843f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.657+0000 7fb702d9d640 1 -- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb704074820 msgr2=0x7fb704083eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:36.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.657+0000 7fb702d9d640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb704074820 0x7fb704083eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:36.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.657+0000 7fb702d9d640 1 -- 172.21.15.55:0/17265041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb704133120 con 0x7fb70407a0b0 2024-04-26T21:57:36.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.663+0000 7fb702d9d640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70407a0b0 0x7fb704084930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc031a50 tx=0x7fb6fc003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:36.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.668+0000 7fb6f37fe640 1 -- 172.21.15.55:0/17265041 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc004040 con 0x7fb70407a0b0 2024-04-26T21:57:36.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.668+0000 7fb6f37fe640 1 -- 172.21.15.55:0/17265041 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6fc02d070 con 0x7fb70407a0b0 2024-04-26T21:57:36.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.668+0000 7fb6f37fe640 1 -- 172.21.15.55:0/17265041 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc02fc70 con 0x7fb70407a0b0 2024-04-26T21:57:36.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.675+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7041333b0 con 0x7fb70407a0b0 2024-04-26T21:57:36.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.675+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb704133840 con 0x7fb70407a0b0 2024-04-26T21:57:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.677+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7040722d0 con 0x7fb70407a0b0 2024-04-26T21:57:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.682+0000 7fb6f37fe640 1 -- 172.21.15.55:0/17265041 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb6fc004b40 con 0x7fb70407a0b0 2024-04-26T21:57:36.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.684+0000 7fb6f37fe640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb6ec0a3dc0 0x7fb6ec0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:36.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.685+0000 7fb70259c640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb6ec0a3dc0 0x7fb6ec0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:36.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.685+0000 7fb6f37fe640 1 -- 172.21.15.55:0/17265041 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb6fc0f1d40 con 0x7fb70407a0b0 2024-04-26T21:57:36.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.686+0000 7fb70259c640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb6ec0a3dc0 0x7fb6ec0a6280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb704085590 tx=0x7fb6f400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:36.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:36.687+0000 7fb6f37fe640 1 -- 172.21.15.55:0/17265041 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb6fc0be710 con 0x7fb70407a0b0 2024-04-26T21:57:36.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:36 smithi165 ceph-mon[33296]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 841 MiB/s wr, 1.12k op/s 2024-04-26T21:57:37.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:36 smithi064 ceph-mon[32765]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 841 MiB/s wr, 1.12k op/s 2024-04-26T21:57:37.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.255+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb70402dad0 con 0x7fb70407a0b0 2024-04-26T21:57:37.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.262+0000 7fb6f37fe640 1 -- 172.21.15.55:0/17265041 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11864 (secure 0 0 0) 0x7fb6fc0be010 con 0x7fb70407a0b0 2024-04-26T21:57:37.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:57:37.264 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:56:06.126250+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24509,24551,24502]},"id":1}]} 2024-04-26T21:57:37.265 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 22 2024-04-26T21:57:37.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.281+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb6ec0a3dc0 msgr2=0x7fb6ec0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.281+0000 7fb708e85640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb6ec0a3dc0 0x7fb6ec0a6280 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb704085590 tx=0x7fb6f400a2d0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.281+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70407a0b0 msgr2=0x7fb704084930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.281+0000 7fb708e85640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70407a0b0 0x7fb704084930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc031a50 tx=0x7fb6fc003ba0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.284+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 shutdown_connections 2024-04-26T21:57:37.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.284+0000 7fb708e85640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb6ec0a3dc0 0x7fb6ec0a6280 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.284+0000 7fb708e85640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70407a0b0 0x7fb704084930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.284+0000 7fb708e85640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb704077700 0x7fb7040843f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.284+0000 7fb708e85640 1 --2- 172.21.15.55:0/17265041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb704074820 0x7fb704083eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.284+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 >> 172.21.15.55:0/17265041 conn(0x7fb70406f7d0 msgr2=0x7fb704077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:37.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.288+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 shutdown_connections 2024-04-26T21:57:37.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.290+0000 7fb708e85640 1 -- 172.21.15.55:0/17265041 wait complete. 2024-04-26T21:57:37.336 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:57:37.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 -- 172.21.15.55:0/3795004578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10c2a0 msgr2=0x7fc68c10e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 --2- 172.21.15.55:0/3795004578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10c2a0 0x7fc68c10e730 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009900 tx=0x7fc67c02a420 comp rx=0 tx=0).stop 2024-04-26T21:57:37.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 -- 172.21.15.55:0/3795004578 shutdown_connections 2024-04-26T21:57:37.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 --2- 172.21.15.55:0/3795004578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc68c10ec70 0x7fc68c10f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 --2- 172.21.15.55:0/3795004578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10c2a0 0x7fc68c10e730 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 --2- 172.21.15.55:0/3795004578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c109930 0x7fc68c10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 -- 172.21.15.55:0/3795004578 >> 172.21.15.55:0/3795004578 conn(0x7fc68c06f7c0 msgr2=0x7fc68c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:37.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 -- 172.21.15.55:0/3795004578 shutdown_connections 2024-04-26T21:57:37.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.649+0000 7fc693314640 1 -- 172.21.15.55:0/3795004578 wait complete. 2024-04-26T21:57:37.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.650+0000 7fc693314640 1 Processor -- start 2024-04-26T21:57:37.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.650+0000 7fc693314640 1 -- start start 2024-04-26T21:57:37.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.650+0000 7fc693314640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc68c109930 0x7fc68c1a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.651+0000 7fc693314640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c10c2a0 0x7fc68c1a8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.651+0000 7fc693314640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10ec70 0x7fc68c1af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.651+0000 7fc693314640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c11d3f0 con 0x7fc68c10ec70 2024-04-26T21:57:37.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.651+0000 7fc693314640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c11d270 con 0x7fc68c109930 2024-04-26T21:57:37.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.651+0000 7fc693314640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c11d570 con 0x7fc68c10c2a0 2024-04-26T21:57:37.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.651+0000 7fc6918b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10ec70 0x7fc68c1af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.651+0000 7fc6918b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10ec70 0x7fc68c1af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57796/0 (socket says 172.21.15.55:57796) 2024-04-26T21:57:37.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.651+0000 7fc6918b1640 1 -- 172.21.15.55:0/1244805141 learned_addr learned my addr 172.21.15.55:0/1244805141 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:37.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.652+0000 7fc6918b1640 1 -- 172.21.15.55:0/1244805141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c10c2a0 msgr2=0x7fc68c1a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.652+0000 7fc6918b1640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c10c2a0 0x7fc68c1a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.652+0000 7fc6918b1640 1 -- 172.21.15.55:0/1244805141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc68c109930 msgr2=0x7fc68c1a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.652+0000 7fc6918b1640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc68c109930 0x7fc68c1a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.652+0000 7fc6918b1640 1 -- 172.21.15.55:0/1244805141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c1af840 con 0x7fc68c10ec70 2024-04-26T21:57:37.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.652+0000 7fc6918b1640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10ec70 0x7fc68c1af1a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc68400b9e0 tx=0x7fc68400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:37.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.653+0000 7fc6827fc640 1 -- 172.21.15.55:0/1244805141 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684019070 con 0x7fc68c10ec70 2024-04-26T21:57:37.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.653+0000 7fc6827fc640 1 -- 172.21.15.55:0/1244805141 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc684014440 con 0x7fc68c10ec70 2024-04-26T21:57:37.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.653+0000 7fc6827fc640 1 -- 172.21.15.55:0/1244805141 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684019070 con 0x7fc68c10ec70 2024-04-26T21:57:37.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.653+0000 7fc693314640 1 -- 172.21.15.55:0/1244805141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c1afb30 con 0x7fc68c10ec70 2024-04-26T21:57:37.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.653+0000 7fc693314640 1 -- 172.21.15.55:0/1244805141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1b0030 con 0x7fc68c10ec70 2024-04-26T21:57:37.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.659+0000 7fc69602f740 1 Processor -- start 2024-04-26T21:57:37.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.660+0000 7fc6827fc640 1 -- 172.21.15.55:0/1244805141 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc68401c070 con 0x7fc68c10ec70 2024-04-26T21:57:37.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.661+0000 7fc6827fc640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6780a3e40 0x7fc6780a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.662+0000 7fc6827fc640 1 -- 172.21.15.55:0/1244805141 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc684099d00 con 0x7fc68c10ec70 2024-04-26T21:57:37.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.662+0000 7fc6908af640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6780a3e40 0x7fc6780a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc69602f740 1 -- start start 2024-04-26T21:57:37.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc69602f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 0x564db4f7dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc69602f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7e9f0 0x564db4f7ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc69602f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7f3b0 0x564db4f85c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc69602f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564db4ee1d80 con 0x564db4f7f3b0 2024-04-26T21:57:37.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc69602f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564db4e9ba20 con 0x564db4f7d820 2024-04-26T21:57:37.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc69602f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564db4eddd40 con 0x564db4f7e9f0 2024-04-26T21:57:37.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc6908af640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6780a3e40 0x7fc6780a6300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc67c031500 tx=0x7fc67c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:37.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc6918b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7f3b0 0x564db4f85c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc6918b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7f3b0 0x564db4f85c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57806/0 (socket says 172.21.15.55:57806) 2024-04-26T21:57:37.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.663+0000 7fc6918b1640 1 -- 172.21.15.55:0/1700132293 learned_addr learned my addr 172.21.15.55:0/1700132293 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:37.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.664+0000 7fc6910b0640 1 --2- 172.21.15.55:0/1700132293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 0x564db4f7dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.664+0000 7fc6918b1640 1 -- 172.21.15.55:0/1700132293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7e9f0 msgr2=0x564db4f7ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:37.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.664+0000 7fc6918b1640 1 --2- 172.21.15.55:0/1700132293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7e9f0 0x564db4f7ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.664+0000 7fc6918b1640 1 -- 172.21.15.55:0/1700132293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 msgr2=0x564db4f7dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:37.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.664+0000 7fc6918b1640 1 --2- 172.21.15.55:0/1700132293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 0x564db4f7dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.664+0000 7fc6918b1640 1 -- 172.21.15.55:0/1700132293 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db4ea6c30 con 0x564db4f7f3b0 2024-04-26T21:57:37.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.665+0000 7fc6918b1640 1 --2- 172.21.15.55:0/1700132293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7f3b0 0x564db4f85c40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc6840099f0 tx=0x7fc6840a0ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3794d5f83a84bfa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:37.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.666+0000 7fc6777fe640 1 -- 172.21.15.55:0/1700132293 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840983a0 con 0x564db4f7f3b0 2024-04-26T21:57:37.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.666+0000 7fc6777fe640 1 -- 172.21.15.55:0/1700132293 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc684099b40 con 0x564db4f7f3b0 2024-04-26T21:57:37.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.667+0000 7fc6777fe640 1 -- 172.21.15.55:0/1700132293 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684098540 con 0x564db4f7f3b0 2024-04-26T21:57:37.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 -- 172.21.15.55:0/1700132293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7f3b0 msgr2=0x564db4f85c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 --2- 172.21.15.55:0/1700132293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7f3b0 0x564db4f85c40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc6840099f0 tx=0x7fc6840a0ed0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 -- 172.21.15.55:0/1700132293 shutdown_connections 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 --2- 172.21.15.55:0/1700132293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7f3b0 0x564db4f85c40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 --2- 172.21.15.55:0/1700132293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7e9f0 0x564db4f7ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 --2- 172.21.15.55:0/1700132293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 0x564db4f7dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 -- 172.21.15.55:0/1700132293 >> 172.21.15.55:0/1700132293 conn(0x564db4ebe910 msgr2=0x564db4f79120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 -- 172.21.15.55:0/1700132293 shutdown_connections 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 -- 172.21.15.55:0/1700132293 wait complete. 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.668+0000 7fc69602f740 1 -- start start 2024-04-26T21:57:37.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc69602f740 1 Processor -- start 2024-04-26T21:57:37.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc69602f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 0x564db4f87a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc69602f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7e9f0 0x564db4f87fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc69602f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f8efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc69602f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564db4f862d0 con 0x564db4f7e9f0 2024-04-26T21:57:37.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc69602f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564db4e68dc0 con 0x564db4f7d820 2024-04-26T21:57:37.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc6908af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7e9f0 0x564db4f87fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc6908af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7e9f0 0x564db4f87fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57808/0 (socket says 172.21.15.55:57808) 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc6908af640 1 -- 172.21.15.55:0/3205793016 learned_addr learned my addr 172.21.15.55:0/3205793016 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.669+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564db4f86450 con 0x564db4f7f3b0 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc6908af640 1 -- 172.21.15.55:0/3205793016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 msgr2=0x564db4f8efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc6908af640 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f8efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc6908af640 1 -- 172.21.15.55:0/3205793016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 msgr2=0x564db4f87a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc6908af640 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 0x564db4f87a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc6908af640 1 -- 172.21.15.55:0/3205793016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db4bf33f0 con 0x564db4f7e9f0 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc6908af640 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7e9f0 0x564db4f87fa0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0046c0 tx=0x7fc67c0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1d0164021954e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc69602f740 10 client.0 _ll_register_callbacks cb 0x564db4e84600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc674ff9640 1 -- 172.21.15.55:0/3205793016 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c04b020 con 0x564db4f7e9f0 2024-04-26T21:57:37.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.670+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564db4eef990 con 0x564db4f7e9f0 2024-04-26T21:57:37.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.671+0000 7fc674ff9640 1 -- 172.21.15.55:0/3205793016 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc67c0315a0 con 0x564db4f7e9f0 2024-04-26T21:57:37.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.671+0000 7fc674ff9640 1 -- 172.21.15.55:0/3205793016 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc67c056050 con 0x564db4f7e9f0 2024-04-26T21:57:37.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.671+0000 7fc69602f740 10 client.16146 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:37.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.671+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=22}) -- 0x564db4cbd7d0 con 0x564db4f7e9f0 2024-04-26T21:57:37.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.672+0000 7fc674ff9640 1 -- 172.21.15.55:0/3205793016 <== mon.0 v2:172.21.15.55:3300/0 4 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fc67c047440 con 0x564db4f7e9f0 2024-04-26T21:57:37.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.672+0000 7fc69602f740 10 client.16146 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:37.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.672+0000 7fc69602f740 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:57:37.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.672+0000 7fc69602f740 20 client.16146 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:37.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.673+0000 7fc69602f740 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564db4f93720 0x564db4f95bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.673+0000 7fc69602f740 4 client.16146 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:37.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.673+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564db4eef990 con 0x564db4f93720 2024-04-26T21:57:37.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.674+0000 7fc6910b0640 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564db4f93720 0x564db4f95bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.678+0000 7fc674ff9640 10 client.16146 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:57:37.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.678+0000 7fc6910b0640 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564db4f93720 0x564db4f95bb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c6b4468fb0766e server_cookie=89ecefd61d3a2ee1 in_seq=0 out_seq=0 2024-04-26T21:57:37.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc674ff9640 1 -- 172.21.15.55:0/3205793016 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564db4eef990 con 0x564db4f93720 2024-04-26T21:57:37.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc674ff9640 10 client.16146 handle_command_reply: tid=0 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 client.16146 shutdown 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7e9f0 msgr2=0x564db4f87fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7e9f0 0x564db4f87fa0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0046c0 tx=0x7fc67c0046f0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 shutdown_connections 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564db4f93720 0x564db4f95bb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc674ff9640 0 client.16146 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f8efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7e9f0 0x564db4f87fa0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 --2- 172.21.15.55:0/3205793016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7d820 0x564db4f87a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.679+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 >> 172.21.15.55:0/3205793016 conn(0x564db4ebe910 msgr2=0x564db4f78280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:37.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.680+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 shutdown_connections 2024-04-26T21:57:37.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.680+0000 7fc69602f740 1 -- 172.21.15.55:0/3205793016 wait complete. 2024-04-26T21:57:37.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.680+0000 7fc69602f740 20 client.16146 trim_cache size 0 max 16384 2024-04-26T21:57:37.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.712+0000 7fc69602f740 1 Processor -- start 2024-04-26T21:57:37.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.712+0000 7fc69602f740 1 -- start start 2024-04-26T21:57:37.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.713+0000 7fc69602f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4ebe910 0x564db4f779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.713+0000 7fc69602f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7d820 0x564db4f8efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.713+0000 7fc69602f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f77f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.713+0000 7fc69602f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1bb710 con 0x564db4f7d820 2024-04-26T21:57:37.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.713+0000 7fc69602f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b8fe0 con 0x564db4ebe910 2024-04-26T21:57:37.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.713+0000 7fc69602f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b6e30 con 0x564db4f7f3b0 2024-04-26T21:57:37.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.714+0000 7fc6918b1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f77f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.714+0000 7fc6918b1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f77f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:56508/0 (socket says 172.21.15.55:56508) 2024-04-26T21:57:37.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.714+0000 7fc6918b1640 1 -- 172.21.15.55:0/454483523 learned_addr learned my addr 172.21.15.55:0/454483523 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:37.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.715+0000 7fc6908af640 1 --2- 172.21.15.55:0/454483523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7d820 0x564db4f8efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.715+0000 7fc6918b1640 1 -- 172.21.15.55:0/454483523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4ebe910 msgr2=0x564db4f779f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:37.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.715+0000 7fc6918b1640 1 --2- 172.21.15.55:0/454483523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4ebe910 0x564db4f779f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.715+0000 7fc6918b1640 1 -- 172.21.15.55:0/454483523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7d820 msgr2=0x564db4f8efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:37.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.715+0000 7fc6918b1640 1 --2- 172.21.15.55:0/454483523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7d820 0x564db4f8efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.715+0000 7fc6918b1640 1 -- 172.21.15.55:0/454483523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db4f79370 con 0x564db4f7f3b0 2024-04-26T21:57:37.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.715+0000 7fc6918b1640 1 --2- 172.21.15.55:0/454483523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f77f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc684014ce0 tx=0x7fc6840cd1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd65d9837dc28db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:37.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.716+0000 7fc676ffd640 1 -- 172.21.15.55:0/454483523 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840986e0 con 0x564db4f7f3b0 2024-04-26T21:57:37.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.716+0000 7fc676ffd640 1 -- 172.21.15.55:0/454483523 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6840a4670 con 0x564db4f7f3b0 2024-04-26T21:57:37.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.716+0000 7fc676ffd640 1 -- 172.21.15.55:0/454483523 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840988b0 con 0x564db4f7f3b0 2024-04-26T21:57:37.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.720+0000 7fc69602f740 1 -- 172.21.15.55:0/454483523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 msgr2=0x564db4f77f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.720+0000 7fc69602f740 1 --2- 172.21.15.55:0/454483523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f77f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc684014ce0 tx=0x7fc6840cd1d0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.721+0000 7fc69602f740 1 -- 172.21.15.55:0/454483523 shutdown_connections 2024-04-26T21:57:37.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.721+0000 7fc69602f740 1 --2- 172.21.15.55:0/454483523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7f3b0 0x564db4f77f30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.721+0000 7fc69602f740 1 --2- 172.21.15.55:0/454483523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4f7d820 0x564db4f8efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.721+0000 7fc69602f740 1 --2- 172.21.15.55:0/454483523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4ebe910 0x564db4f779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.721+0000 7fc69602f740 1 -- 172.21.15.55:0/454483523 >> 172.21.15.55:0/454483523 conn(0x564db4f85d30 msgr2=0x564db4f82350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:37.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.727+0000 7fc69602f740 1 -- 172.21.15.55:0/454483523 shutdown_connections 2024-04-26T21:57:37.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.727+0000 7fc69602f740 1 -- 172.21.15.55:0/454483523 wait complete. 2024-04-26T21:57:37.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.727+0000 7fc69602f740 1 -- start start 2024-04-26T21:57:37.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.728+0000 7fc69602f740 1 Processor -- start 2024-04-26T21:57:37.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.728+0000 7fc69602f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4ebe910 0x564db4f8cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.728+0000 7fc69602f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7d820 0x564db4f8d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.728+0000 7fc69602f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7f3b0 0x564db4f8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.728+0000 7fc69602f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b74a0 con 0x564db4ebe910 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.728+0000 7fc69602f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b8910 con 0x564db4f7f3b0 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.728+0000 7fc69602f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c1b7b70 con 0x564db4f7d820 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.728+0000 7fc69602f740 10 client.0 _ll_register_callbacks cb 0x7fc67c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.729+0000 7fc6918b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7f3b0 0x564db4f8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.730+0000 7fc6918b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7f3b0 0x564db4f8dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43454/0 (socket says 172.21.15.55:43454) 2024-04-26T21:57:37.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.730+0000 7fc6918b1640 1 -- 172.21.15.55:0/2160110007 learned_addr learned my addr 172.21.15.55:0/2160110007 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:37.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.731+0000 7fc6908af640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7d820 0x564db4f8d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.732+0000 7fc6918b1640 1 -- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7d820 msgr2=0x564db4f8d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:37.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.732+0000 7fc6918b1640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7d820 0x564db4f8d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.732+0000 7fc6910b0640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4ebe910 0x564db4f8cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.732+0000 7fc6918b1640 1 -- 172.21.15.55:0/2160110007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4ebe910 msgr2=0x564db4f8cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:37.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.732+0000 7fc6918b1640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4ebe910 0x564db4f8cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.732+0000 7fc6918b1640 1 -- 172.21.15.55:0/2160110007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db4f8e5b0 con 0x564db4f7f3b0 2024-04-26T21:57:37.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.732+0000 7fc6908af640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7d820 0x564db4f8d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:57:37.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.734+0000 7fc6918b1640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7f3b0 0x564db4f8dfc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc68409bc30 tx=0x7fc684097c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6fc770b75e90ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:37.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.735+0000 7fc6910b0640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4ebe910 0x564db4f8cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:37.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.735+0000 7fc674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:57:37.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.736+0000 7fc674ff9640 1 -- 172.21.15.55:0/2160110007 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684098a50 con 0x564db4f7f3b0 2024-04-26T21:57:37.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.736+0000 7fc69602f740 1 -- 172.21.15.55:0/2160110007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6840a4670 con 0x564db4f7f3b0 2024-04-26T21:57:37.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.736+0000 7fc674ff9640 1 -- 172.21.15.55:0/2160110007 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc684099d00 con 0x564db4f7f3b0 2024-04-26T21:57:37.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.737+0000 7fc674ff9640 1 -- 172.21.15.55:0/2160110007 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc684098c20 con 0x564db4f7f3b0 2024-04-26T21:57:37.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.737+0000 7fc69602f740 10 client.25547 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:37.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.737+0000 7fc69602f740 1 -- 172.21.15.55:0/2160110007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564db4ffa100 con 0x564db4f7f3b0 2024-04-26T21:57:37.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.738+0000 7fc674ff9640 1 -- 172.21.15.55:0/2160110007 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6840a4670 con 0x564db4f7f3b0 2024-04-26T21:57:37.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.738+0000 7fc674ff9640 1 -- 172.21.15.55:0/2160110007 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fc684086110 con 0x564db4f7f3b0 2024-04-26T21:57:37.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.744+0000 7fc69602f740 10 client.25547 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:37.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.744+0000 7fc69602f740 10 client.25547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:57:37.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.744+0000 7fc69602f740 20 client.25547 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:37.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.744+0000 7fc69602f740 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564db4ffa3f0 0x564db4ffc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:37.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.744+0000 7fc69602f740 4 client.25547 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:37.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.744+0000 7fc69602f740 1 -- 172.21.15.55:0/2160110007 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564db4ee9800 con 0x564db4ffa3f0 2024-04-26T21:57:37.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.747+0000 7fc6910b0640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564db4ffa3f0 0x564db4ffc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:37.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.764+0000 7fc674ff9640 10 client.25547 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:57:37.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.764+0000 7fc6910b0640 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564db4ffa3f0 0x564db4ffc7e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfeb65e68f33cde4 server_cookie=c7af903a0f90bc15 in_seq=0 out_seq=0 2024-04-26T21:57:37.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.767+0000 7fc674ff9640 1 -- 172.21.15.55:0/2160110007 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564db4ee9800 con 0x564db4ffa3f0 2024-04-26T21:57:37.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.767+0000 7fc674ff9640 10 client.25547 handle_command_reply: tid=0 2024-04-26T21:57:37.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.768+0000 7fc69602f740 1 client.25547 shutdown 2024-04-26T21:57:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.773+0000 7fc69602f740 1 -- 172.21.15.55:0/2160110007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7f3b0 msgr2=0x564db4f8dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.773+0000 7fc69602f740 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7f3b0 0x564db4f8dfc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc68409bc30 tx=0x7fc684097c70 comp rx=0 tx=0).stop 2024-04-26T21:57:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.774+0000 7fc69602f740 1 -- 172.21.15.55:0/2160110007 shutdown_connections 2024-04-26T21:57:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.774+0000 7fc69602f740 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564db4ffa3f0 0x564db4ffc7e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.774+0000 7fc69602f740 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564db4f7f3b0 0x564db4f8dfc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.774+0000 7fc69602f740 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564db4f7d820 0x564db4f8d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.774+0000 7fc69602f740 1 --2- 172.21.15.55:0/2160110007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564db4ebe910 0x564db4f8cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.774+0000 7fc69602f740 1 -- 172.21.15.55:0/2160110007 >> 172.21.15.55:0/2160110007 conn(0x564db4f85d30 msgr2=0x564db4f7c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:37.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.774+0000 7fc674ff9640 0 client.25547 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:57:37.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.781+0000 7fc69602f740 1 -- 172.21.15.55:0/2160110007 shutdown_connections 2024-04-26T21:57:37.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.784+0000 7fc69602f740 1 -- 172.21.15.55:0/2160110007 wait complete. 2024-04-26T21:57:37.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.784+0000 7fc69602f740 20 client.25547 trim_cache size 0 max 16384 2024-04-26T21:57:37.786 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:57:37.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.796+0000 7fc693314640 1 -- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6780a3e40 msgr2=0x7fc6780a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.796+0000 7fc693314640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6780a3e40 0x7fc6780a6300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc67c031500 tx=0x7fc67c000bc0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.796+0000 7fc693314640 1 -- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10ec70 msgr2=0x7fc68c1af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:37.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.796+0000 7fc693314640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10ec70 0x7fc68c1af1a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc68400b9e0 tx=0x7fc68400bea0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.801+0000 7fc693314640 1 -- 172.21.15.55:0/1244805141 shutdown_connections 2024-04-26T21:57:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.801+0000 7fc693314640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6780a3e40 0x7fc6780a6300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.801+0000 7fc693314640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc68c10ec70 0x7fc68c1af1a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.801+0000 7fc693314640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc68c10c2a0 0x7fc68c1a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.801+0000 7fc693314640 1 --2- 172.21.15.55:0/1244805141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc68c109930 0x7fc68c1a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.801+0000 7fc693314640 1 -- 172.21.15.55:0/1244805141 >> 172.21.15.55:0/1244805141 conn(0x7fc68c06f7c0 msgr2=0x7fc68c10acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:37.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.807+0000 7fc693314640 1 -- 172.21.15.55:0/1244805141 shutdown_connections 2024-04-26T21:57:37.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:37.811+0000 7fc693314640 1 -- 172.21.15.55:0/1244805141 wait complete. 2024-04-26T21:57:37.871 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:57:37.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:37 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/17265041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:38.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:37 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/17265041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:38.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:37 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/17265041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 -- 172.21.15.55:0/96067998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34101870 msgr2=0x7f1e34101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 --2- 172.21.15.55:0/96067998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34101870 0x7f1e34101c90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009900 tx=0x7f1e2802a410 comp rx=0 tx=0).stop 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 -- 172.21.15.55:0/96067998 shutdown_connections 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 --2- 172.21.15.55:0/96067998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e34103420 0x7f1e34109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 --2- 172.21.15.55:0/96067998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e34102a60 0x7f1e34102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 --2- 172.21.15.55:0/96067998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34101870 0x7f1e34101c90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 -- 172.21.15.55:0/96067998 >> 172.21.15.55:0/96067998 conn(0x7f1e340fcde0 msgr2=0x7f1e340ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 -- 172.21.15.55:0/96067998 shutdown_connections 2024-04-26T21:57:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.160+0000 7f1e3af91640 1 -- 172.21.15.55:0/96067998 wait complete. 2024-04-26T21:57:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.161+0000 7f1e3af91640 1 Processor -- start 2024-04-26T21:57:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.161+0000 7f1e3af91640 1 -- start start 2024-04-26T21:57:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.161+0000 7f1e3af91640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e34101870 0x7f1e3410e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e3af91640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e34102a60 0x7f1e3410cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e3af91640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34103420 0x7f1e3410d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e3af91640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3406a510 con 0x7f1e34103420 2024-04-26T21:57:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e3af91640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3406a390 con 0x7f1e34101870 2024-04-26T21:57:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e3af91640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3406a690 con 0x7f1e34102a60 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e33fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e34102a60 0x7f1e3410cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e33fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e34102a60 0x7f1e3410cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:56538/0 (socket says 172.21.15.55:56538) 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e33fff640 1 -- 172.21.15.55:0/2201962284 learned_addr learned my addr 172.21.15.55:0/2201962284 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.162+0000 7f1e3952e640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34103420 0x7f1e3410d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.163+0000 7f1e38d2d640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e34101870 0x7f1e3410e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.163+0000 7f1e3952e640 1 -- 172.21.15.55:0/2201962284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e34102a60 msgr2=0x7f1e3410cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.163+0000 7f1e3952e640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e34102a60 0x7f1e3410cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.163+0000 7f1e3952e640 1 -- 172.21.15.55:0/2201962284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e34101870 msgr2=0x7f1e3410e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.163+0000 7f1e3952e640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e34101870 0x7f1e3410e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.163+0000 7f1e3952e640 1 -- 172.21.15.55:0/2201962284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3410d9d0 con 0x7f1e34103420 2024-04-26T21:57:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.164+0000 7f1e3952e640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34103420 0x7f1e3410d140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00be20 tx=0x7f1e2c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.164+0000 7f1e31ffb640 1 -- 172.21.15.55:0/2201962284 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c019070 con 0x7f1e34103420 2024-04-26T21:57:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.164+0000 7f1e31ffb640 1 -- 172.21.15.55:0/2201962284 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e2c0148f0 con 0x7f1e34103420 2024-04-26T21:57:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.164+0000 7f1e31ffb640 1 -- 172.21.15.55:0/2201962284 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c012860 con 0x7f1e34103420 2024-04-26T21:57:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.165+0000 7f1e3af91640 1 -- 172.21.15.55:0/2201962284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e341a2e70 con 0x7f1e34103420 2024-04-26T21:57:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.165+0000 7f1e3af91640 1 -- 172.21.15.55:0/2201962284 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e341a32a0 con 0x7f1e34103420 2024-04-26T21:57:38.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.172+0000 7f1e3dd82740 1 Processor -- start 2024-04-26T21:57:38.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.173+0000 7f1e3dd82740 1 -- start start 2024-04-26T21:57:38.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.173+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b74fe1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.173+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 0x558b74fe2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.174+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.174+0000 7f1e31ffb640 1 -- 172.21.15.55:0/2201962284 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e2c01c070 con 0x7f1e34103420 2024-04-26T21:57:38.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.175+0000 7f1e33fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b74fe1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.175+0000 7f1e33fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b74fe1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:56546/0 (socket says 172.21.15.55:56546) 2024-04-26T21:57:38.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.175+0000 7f1e33fff640 1 -- 172.21.15.55:0/4186628038 learned_addr learned my addr 172.21.15.55:0/4186628038 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.175+0000 7f1e3dd82740 1 -- 172.21.15.55:0/4186628038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558b74f49d80 con 0x558b74fe2aa0 2024-04-26T21:57:38.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.175+0000 7f1e3dd82740 1 -- 172.21.15.55:0/4186628038 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558b74f03a20 con 0x558b74fe2100 2024-04-26T21:57:38.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.175+0000 7f1e3dd82740 1 -- 172.21.15.55:0/4186628038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558b74f45d40 con 0x558b74fe0ee0 2024-04-26T21:57:38.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.175+0000 7f1e33fff640 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.176+0000 7f1e31ffb640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1e240a3d40 0x7f1e240a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.176+0000 7f1e38d2d640 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 0x558b74fe2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.176+0000 7f1e31ffb640 1 -- 172.21.15.55:0/2201962284 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1e2c0cfdb0 con 0x7f1e34103420 2024-04-26T21:57:38.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.177+0000 7f1e3952e640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1e240a3d40 0x7f1e240a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.177+0000 7f1e33fff640 1 -- 172.21.15.55:0/4186628038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 msgr2=0x558b74fe1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.177+0000 7f1e33fff640 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b74fe1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.177+0000 7f1e33fff640 1 -- 172.21.15.55:0/4186628038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 msgr2=0x558b74fe2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.177+0000 7f1e33fff640 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 0x558b74fe2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.177+0000 7f1e33fff640 1 -- 172.21.15.55:0/4186628038 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b74f0ec30 con 0x558b74fe2aa0 2024-04-26T21:57:38.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.181+0000 7f1e33fff640 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe7360 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1e20015840 tx=0x7f1e200091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99909b8d6368c877 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.182+0000 7f1e1effd640 1 -- 172.21.15.55:0/4186628038 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20015e80 con 0x558b74fe2aa0 2024-04-26T21:57:38.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.182+0000 7f1e1effd640 1 -- 172.21.15.55:0/4186628038 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e2001d070 con 0x558b74fe2aa0 2024-04-26T21:57:38.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.182+0000 7f1e1effd640 1 -- 172.21.15.55:0/4186628038 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20018a40 con 0x558b74fe2aa0 2024-04-26T21:57:38.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.184+0000 7f1e3dd82740 1 -- 172.21.15.55:0/4186628038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 msgr2=0x558b74fe7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.184+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe7360 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1e20015840 tx=0x7f1e200091c0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.191+0000 7f1e3952e640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1e240a3d40 0x7f1e240a6200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0a46e0 tx=0x7f1e2c00ce70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.191+0000 7f1e3dd82740 1 -- 172.21.15.55:0/4186628038 shutdown_connections 2024-04-26T21:57:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.191+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe7360 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.191+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 0x558b74fe2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.191+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/4186628038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b74fe1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.191+0000 7f1e3dd82740 1 -- 172.21.15.55:0/4186628038 >> 172.21.15.55:0/4186628038 conn(0x558b74f26910 msgr2=0x558b74fe9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.192+0000 7f1e3dd82740 1 -- 172.21.15.55:0/4186628038 shutdown_connections 2024-04-26T21:57:38.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.195+0000 7f1e3dd82740 1 -- 172.21.15.55:0/4186628038 wait complete. 2024-04-26T21:57:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.195+0000 7f1e3dd82740 1 -- start start 2024-04-26T21:57:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.195+0000 7f1e3dd82740 1 Processor -- start 2024-04-26T21:57:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.196+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75000ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.196+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe2100 0x558b75001000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.196+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b75008040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.196+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558b74fe7a80 con 0x558b74fe2aa0 2024-04-26T21:57:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.196+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558b74ed0dc0 con 0x558b74fe0ee0 2024-04-26T21:57:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.196+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558b74fe7c00 con 0x558b74fe2100 2024-04-26T21:57:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.196+0000 7f1e3dd82740 10 client.0 _ll_register_callbacks cb 0x558b74eec600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.197+0000 7f1e33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75000ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.197+0000 7f1e33fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75000ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43494/0 (socket says 172.21.15.55:43494) 2024-04-26T21:57:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.197+0000 7f1e33fff640 1 -- 172.21.15.55:0/2640696181 learned_addr learned my addr 172.21.15.55:0/2640696181 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.198+0000 7f1e33fff640 1 -- 172.21.15.55:0/2640696181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe2100 msgr2=0x558b75001000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.198+0000 7f1e33fff640 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe2100 0x558b75001000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.198+0000 7f1e33fff640 1 -- 172.21.15.55:0/2640696181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 msgr2=0x558b75008040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.198+0000 7f1e33fff640 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b75008040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.198+0000 7f1e33fff640 1 -- 172.21.15.55:0/2640696181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b74c5b3f0 con 0x558b74fe0ee0 2024-04-26T21:57:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.199+0000 7f1e33fff640 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75000ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006ea0 tx=0x7f1e20009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125a45ae7fc0c46b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.204+0000 7f1e13fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:57:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.204+0000 7f1e13fff640 1 -- 172.21.15.55:0/2640696181 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20013d00 con 0x558b74fe0ee0 2024-04-26T21:57:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.204+0000 7f1e13fff640 1 -- 172.21.15.55:0/2640696181 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1e200187c0 con 0x558b74fe0ee0 2024-04-26T21:57:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.204+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2640696181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e2001d070 con 0x558b74fe0ee0 2024-04-26T21:57:38.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.205+0000 7f1e13fff640 1 -- 172.21.15.55:0/2640696181 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1e20015e80 con 0x558b74fe0ee0 2024-04-26T21:57:38.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.214+0000 7f1e3dd82740 10 client.25556 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:38.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.214+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2640696181 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558b74d257d0 con 0x558b74fe0ee0 2024-04-26T21:57:38.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.214+0000 7f1e13fff640 1 -- 172.21.15.55:0/2640696181 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e2001d070 con 0x558b74fe0ee0 2024-04-26T21:57:38.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.215+0000 7f1e13fff640 1 -- 172.21.15.55:0/2640696181 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f1e20032020 con 0x558b74fe0ee0 2024-04-26T21:57:38.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.215+0000 7f1e3dd82740 10 client.25556 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:38.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.215+0000 7f1e3dd82740 10 client.25556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:57:38.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.215+0000 7f1e3dd82740 20 client.25556 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:38.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.215+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x558b7500c530 0x558b7500e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.215+0000 7f1e3dd82740 4 client.25556 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:38.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.215+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2640696181 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b74f57990 con 0x558b7500c530 2024-04-26T21:57:38.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.216+0000 7f1e38d2d640 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x558b7500c530 0x558b7500e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.217+0000 7f1e38d2d640 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x558b7500c530 0x558b7500e920 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd8b84872f297be1 server_cookie=7116c593d7b19507 in_seq=0 out_seq=0 2024-04-26T21:57:38.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.217+0000 7f1e13fff640 10 client.25556 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:57:38.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e13fff640 1 -- 172.21.15.55:0/2640696181 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b74f57990 con 0x558b7500c530 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e13fff640 10 client.25556 handle_command_reply: tid=0 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 client.25556 shutdown 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2640696181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 msgr2=0x558b75000ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75000ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e20006ea0 tx=0x7f1e20009bb0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2640696181 shutdown_connections 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x558b7500c530 0x558b7500e920 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b75008040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe2100 0x558b75001000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2640696181 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75000ac0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.225+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2640696181 >> 172.21.15.55:0/2640696181 conn(0x558b74f26910 msgr2=0x558b74fe0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.227+0000 7f1e13fff640 0 client.25556 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.227+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2640696181 shutdown_connections 2024-04-26T21:57:38.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.227+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2640696181 wait complete. 2024-04-26T21:57:38.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.227+0000 7f1e3dd82740 20 client.25556 trim_cache size 0 max 16384 2024-04-26T21:57:38.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.264+0000 7f1e3dd82740 1 Processor -- start 2024-04-26T21:57:38.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.264+0000 7f1e3dd82740 1 -- start start 2024-04-26T21:57:38.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.264+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b75000f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.264+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 0x558b75007fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.264+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.265+0000 7f1e33fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.265+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341aa7b0 con 0x558b74fe2aa0 2024-04-26T21:57:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.265+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341a8080 con 0x558b74fe2100 2024-04-26T21:57:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.265+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341a5ef0 con 0x558b74fe0ee0 2024-04-26T21:57:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.265+0000 7f1e33fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57844/0 (socket says 172.21.15.55:57844) 2024-04-26T21:57:38.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.265+0000 7f1e33fff640 1 -- 172.21.15.55:0/2951633650 learned_addr learned my addr 172.21.15.55:0/2951633650 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.265+0000 7f1e33fff640 1 --2- 172.21.15.55:0/2951633650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b75000f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.266+0000 7f1e33fff640 1 -- 172.21.15.55:0/2951633650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 msgr2=0x558b75000f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.266+0000 7f1e33fff640 1 --2- 172.21.15.55:0/2951633650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b75000f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.266+0000 7f1e33fff640 1 -- 172.21.15.55:0/2951633650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 msgr2=0x558b75007fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.266+0000 7f1e33fff640 1 --2- 172.21.15.55:0/2951633650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 0x558b75007fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.266+0000 7f1e33fff640 1 -- 172.21.15.55:0/2951633650 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b74fe0a00 con 0x558b74fe2aa0 2024-04-26T21:57:38.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.267+0000 7f1e33fff640 1 --2- 172.21.15.55:0/2951633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe3a30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1e20017a10 tx=0x7f1e200046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04c8627feca401 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.267+0000 7f1e1e7fc640 1 -- 172.21.15.55:0/2951633650 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e200098c0 con 0x558b74fe2aa0 2024-04-26T21:57:38.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.267+0000 7f1e1e7fc640 1 -- 172.21.15.55:0/2951633650 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e2001d070 con 0x558b74fe2aa0 2024-04-26T21:57:38.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.269+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2951633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 msgr2=0x558b74fe3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.269+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2951633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe3a30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1e20017a10 tx=0x7f1e200046a0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.269+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2951633650 shutdown_connections 2024-04-26T21:57:38.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.269+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2951633650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2aa0 0x558b74fe3a30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.269+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2951633650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe2100 0x558b75007fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.269+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/2951633650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b74fe0ee0 0x558b75000f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.269+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2951633650 >> 172.21.15.55:0/2951633650 conn(0x558b74f26910 msgr2=0x558b74fe3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.270+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2951633650 shutdown_connections 2024-04-26T21:57:38.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.270+0000 7f1e3dd82740 1 -- 172.21.15.55:0/2951633650 wait complete. 2024-04-26T21:57:38.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.271+0000 7f1e3dd82740 1 -- start start 2024-04-26T21:57:38.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.271+0000 7f1e3dd82740 1 Processor -- start 2024-04-26T21:57:38.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.272+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75002e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.272+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2100 0x558b750033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.272+0000 7f1e3dd82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b7506ea20 0x558b75070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.272+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341a6540 con 0x558b74fe2100 2024-04-26T21:57:38.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.272+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341a79b0 con 0x558b74fe0ee0 2024-04-26T21:57:38.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.272+0000 7f1e3dd82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1e341a6c10 con 0x558b7506ea20 2024-04-26T21:57:38.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.272+0000 7f1e3dd82740 10 client.0 _ll_register_callbacks cb 0x7f1e20009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:38.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.273+0000 7f1e38d2d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2100 0x558b750033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.273+0000 7f1e38d2d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2100 0x558b750033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57852/0 (socket says 172.21.15.55:57852) 2024-04-26T21:57:38.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.273+0000 7f1e38d2d640 1 -- 172.21.15.55:0/1285787136 learned_addr learned my addr 172.21.15.55:0/1285787136 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.276+0000 7f1e33fff640 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b7506ea20 0x558b75070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.276+0000 7f1e33fff640 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75002e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.279+0000 7f1e38d2d640 1 -- 172.21.15.55:0/1285787136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b7506ea20 msgr2=0x558b75070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.279+0000 7f1e38d2d640 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b7506ea20 0x558b75070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.279+0000 7f1e38d2d640 1 -- 172.21.15.55:0/1285787136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 msgr2=0x558b75002e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.279+0000 7f1e38d2d640 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75002e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.279+0000 7f1e38d2d640 1 -- 172.21.15.55:0/1285787136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b75071350 con 0x558b74fe2100 2024-04-26T21:57:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.280+0000 7f1e38d2d640 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2100 0x558b750033b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009900 tx=0x7f1e280054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b1b90bf9124ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.283+0000 7f1e13fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:38.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.283+0000 7f1e13fff640 1 -- 172.21.15.55:0/1285787136 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28035020 con 0x558b74fe2100 2024-04-26T21:57:38.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.283+0000 7f1e13fff640 1 -- 172.21.15.55:0/1285787136 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f1e28005020 con 0x558b74fe2100 2024-04-26T21:57:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.287+0000 7f1e3dd82740 1 -- 172.21.15.55:0/1285787136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558b74f51800 con 0x558b74fe2100 2024-04-26T21:57:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.287+0000 7f1e13fff640 1 -- 172.21.15.55:0/1285787136 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1e28004250 con 0x558b74fe2100 2024-04-26T21:57:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.287+0000 7f1e3dd82740 10 client.16164 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:38.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.287+0000 7f1e3dd82740 1 -- 172.21.15.55:0/1285787136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558b750726b0 con 0x558b74fe2100 2024-04-26T21:57:38.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.289+0000 7f1e13fff640 1 -- 172.21.15.55:0/1285787136 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558b74f51800 con 0x558b74fe2100 2024-04-26T21:57:38.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.290+0000 7f1e13fff640 1 -- 172.21.15.55:0/1285787136 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f1e2803a040 con 0x558b74fe2100 2024-04-26T21:57:38.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.290+0000 7f1e3dd82740 10 client.16164 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:38.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.290+0000 7f1e3dd82740 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:57:38.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.290+0000 7f1e3dd82740 20 client.16164 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:38.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.290+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x558b750730b0 0x558b750754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.290+0000 7f1e3dd82740 4 client.16164 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:38.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.290+0000 7f1e3dd82740 1 -- 172.21.15.55:0/1285787136 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b74ecc610 con 0x558b750730b0 2024-04-26T21:57:38.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.292+0000 7f1e33fff640 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x558b750730b0 0x558b750754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.298+0000 7f1e33fff640 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x558b750730b0 0x558b750754d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9651b392c284873a server_cookie=2d1776078a2ea638 in_seq=0 out_seq=0 2024-04-26T21:57:38.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.298+0000 7f1e13fff640 10 client.16164 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:57:38.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.301+0000 7f1e13fff640 1 -- 172.21.15.55:0/1285787136 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b74ecc610 con 0x558b750730b0 2024-04-26T21:57:38.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.301+0000 7f1e13fff640 10 client.16164 handle_command_reply: tid=0 2024-04-26T21:57:38.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.301+0000 7f1e3dd82740 1 client.16164 shutdown 2024-04-26T21:57:38.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.301+0000 7f1e3dd82740 1 -- 172.21.15.55:0/1285787136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2100 msgr2=0x558b750033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.301+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2100 0x558b750033b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009900 tx=0x7f1e280054a0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 1 -- 172.21.15.55:0/1285787136 shutdown_connections 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x558b750730b0 0x558b750754d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558b7506ea20 0x558b75070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558b74fe2100 0x558b750033b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 1 --2- 172.21.15.55:0/1285787136 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558b74fe0ee0 0x558b75002e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e13fff640 0 client.16164 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 1 -- 172.21.15.55:0/1285787136 >> 172.21.15.55:0/1285787136 conn(0x558b74f26910 msgr2=0x558b74fe0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 1 -- 172.21.15.55:0/1285787136 shutdown_connections 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 1 -- 172.21.15.55:0/1285787136 wait complete. 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.302+0000 7f1e3dd82740 20 client.16164 trim_cache size 0 max 16384 2024-04-26T21:57:38.304 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:57:38.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.306+0000 7f1e3af91640 1 -- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1e240a3d40 msgr2=0x7f1e240a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.306+0000 7f1e3af91640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1e240a3d40 0x7f1e240a6200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0a46e0 tx=0x7f1e2c00ce70 comp rx=0 tx=0).stop 2024-04-26T21:57:38.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.306+0000 7f1e3af91640 1 -- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34103420 msgr2=0x7f1e3410d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.306+0000 7f1e3af91640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34103420 0x7f1e3410d140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00be20 tx=0x7f1e2c00bf10 comp rx=0 tx=0).stop 2024-04-26T21:57:38.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.311+0000 7f1e3af91640 1 -- 172.21.15.55:0/2201962284 shutdown_connections 2024-04-26T21:57:38.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.311+0000 7f1e3af91640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f1e240a3d40 0x7f1e240a6200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.311+0000 7f1e3af91640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e34103420 0x7f1e3410d140 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.311+0000 7f1e3af91640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e34102a60 0x7f1e3410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.311+0000 7f1e3af91640 1 --2- 172.21.15.55:0/2201962284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e34101870 0x7f1e3410e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.311+0000 7f1e3af91640 1 -- 172.21.15.55:0/2201962284 >> 172.21.15.55:0/2201962284 conn(0x7f1e340fcde0 msgr2=0x7f1e34105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.311+0000 7f1e3af91640 1 -- 172.21.15.55:0/2201962284 shutdown_connections 2024-04-26T21:57:38.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.311+0000 7f1e3af91640 1 -- 172.21.15.55:0/2201962284 wait complete. 2024-04-26T21:57:38.361 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:57:38.639 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:38 smithi055 ceph-mon[29221]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 853 MiB/s wr, 1.45k op/s 2024-04-26T21:57:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.685+0000 7fddbf59e640 1 -- 172.21.15.55:0/2396623988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 msgr2=0x7fddc010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.685+0000 7fddbf59e640 1 --2- 172.21.15.55:0/2396623988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 0x7fddc010a470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009900 tx=0x7fddb802a440 comp rx=0 tx=0).stop 2024-04-26T21:57:38.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.694+0000 7fddbf59e640 1 -- 172.21.15.55:0/2396623988 shutdown_connections 2024-04-26T21:57:38.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.694+0000 7fddbf59e640 1 --2- 172.21.15.55:0/2396623988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc010a9b0 0x7fddc010f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.694+0000 7fddbf59e640 1 --2- 172.21.15.55:0/2396623988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 0x7fddc010a470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.694+0000 7fddbf59e640 1 --2- 172.21.15.55:0/2396623988 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fddc0073d70 0x7fddc0074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.694+0000 7fddbf59e640 1 -- 172.21.15.55:0/2396623988 >> 172.21.15.55:0/2396623988 conn(0x7fddc006f840 msgr2=0x7fddc0071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.703+0000 7fddbf59e640 1 -- 172.21.15.55:0/2396623988 shutdown_connections 2024-04-26T21:57:38.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.703+0000 7fddbf59e640 1 -- 172.21.15.55:0/2396623988 wait complete. 2024-04-26T21:57:38.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.704+0000 7fddbf59e640 1 Processor -- start 2024-04-26T21:57:38.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.704+0000 7fddbf59e640 1 -- start start 2024-04-26T21:57:38.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbf59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fddc0073d70 0x7fddc011ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbf59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 0x7fddc011b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc010a9b0 0x7fddc0119150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbf59e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0111920 con 0x7fddc0073d70 2024-04-26T21:57:38.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbf59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fddc01117a0 con 0x7fddc010a9b0 2024-04-26T21:57:38.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbf59e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0111aa0 con 0x7fddc010a010 2024-04-26T21:57:38.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fddc0073d70 0x7fddc011ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fddc0073d70 0x7fddc011ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57878/0 (socket says 172.21.15.55:57878) 2024-04-26T21:57:38.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.705+0000 7fddbe59c640 1 -- 172.21.15.55:0/642109673 learned_addr learned my addr 172.21.15.55:0/642109673 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.706+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 0x7fddc011b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.707+0000 7fddbed9d640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc010a9b0 0x7fddc0119150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.708+0000 7fddbdd9b640 1 -- 172.21.15.55:0/642109673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc010a9b0 msgr2=0x7fddc0119150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.708+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc010a9b0 0x7fddc0119150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.708+0000 7fddbdd9b640 1 -- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fddc0073d70 msgr2=0x7fddc011ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.708+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fddc0073d70 0x7fddc011ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.708+0000 7fddbdd9b640 1 -- 172.21.15.55:0/642109673 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc01197f0 con 0x7fddc010a010 2024-04-26T21:57:38.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.709+0000 7fddbed9d640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc010a9b0 0x7fddc0119150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:38.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.709+0000 7fddbe59c640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fddc0073d70 0x7fddc011ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:57:38.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.711+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 0x7fddc011b050 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009850 tx=0x7fddb80045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.720+0000 7fddaf7fe640 1 -- 172.21.15.55:0/642109673 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb803f070 con 0x7fddc010a010 2024-04-26T21:57:38.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.720+0000 7fddbf59e640 1 -- 172.21.15.55:0/642109673 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc0119a80 con 0x7fddc010a010 2024-04-26T21:57:38.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.720+0000 7fddbf59e640 1 -- 172.21.15.55:0/642109673 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc01b8610 con 0x7fddc010a010 2024-04-26T21:57:38.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.720+0000 7fddaf7fe640 1 -- 172.21.15.55:0/642109673 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb8005510 con 0x7fddc010a010 2024-04-26T21:57:38.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.720+0000 7fddaf7fe640 1 -- 172.21.15.55:0/642109673 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8035030 con 0x7fddc010a010 2024-04-26T21:57:38.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.731+0000 7fddc7be7740 1 Processor -- start 2024-04-26T21:57:38.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.732+0000 7fddc7be7740 1 -- start start 2024-04-26T21:57:38.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.733+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 0x55b6774a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.733+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774a7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.734+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 0x55b6774ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.734+0000 7fddc7be7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b677409d80 con 0x55b6774a6bd0 2024-04-26T21:57:38.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.734+0000 7fddc7be7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b6773c3a20 con 0x55b6774a5a00 2024-04-26T21:57:38.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.734+0000 7fddc7be7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b677405d40 con 0x55b6774a7590 2024-04-26T21:57:38.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.735+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.735+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774a7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57882/0 (socket says 172.21.15.55:57882) 2024-04-26T21:57:38.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.735+0000 7fddbe59c640 1 -- 172.21.15.55:0/1184894131 learned_addr learned my addr 172.21.15.55:0/1184894131 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.736+0000 7fddbed9d640 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 0x55b6774a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.738+0000 7fddaf7fe640 1 -- 172.21.15.55:0/642109673 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fddb8042070 con 0x7fddc010a010 2024-04-26T21:57:38.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.738+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 0x55b6774ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.739+0000 7fddbed9d640 1 -- 172.21.15.55:0/1184894131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 msgr2=0x55b6774ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.739+0000 7fddaf7fe640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdda80a3d00 0x7fdda80a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.744+0000 7fddbed9d640 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 0x55b6774ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.744+0000 7fddbed9d640 1 -- 172.21.15.55:0/1184894131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 msgr2=0x55b6774a7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.744+0000 7fddbed9d640 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774a7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.744+0000 7fddbed9d640 1 -- 172.21.15.55:0/1184894131 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6773cec30 con 0x55b6774a5a00 2024-04-26T21:57:38.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.744+0000 7fddbe59c640 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774a7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:57:38.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.744+0000 7fddaf7fe640 1 -- 172.21.15.55:0/642109673 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fddb80f1030 con 0x7fddc010a010 2024-04-26T21:57:38.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.744+0000 7fddbed9d640 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 0x55b6774a5e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009f90 tx=0x7fddb0010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42b2bdeabf04fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.751+0000 7fddbed9d640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdda80a3d00 0x7fdda80a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.751+0000 7fddacff9640 1 -- 172.21.15.55:0/1184894131 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0011700 con 0x55b6774a5a00 2024-04-26T21:57:38.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.751+0000 7fddacff9640 1 -- 172.21.15.55:0/1184894131 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb0011d50 con 0x55b6774a5a00 2024-04-26T21:57:38.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.751+0000 7fddacff9640 1 -- 172.21.15.55:0/1184894131 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb001adb0 con 0x55b6774a5a00 2024-04-26T21:57:38.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.753+0000 7fddbed9d640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdda80a3d00 0x7fdda80a61c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fddb001afd0 tx=0x7fddb001c5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.753+0000 7fddc7be7740 1 -- 172.21.15.55:0/1184894131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 msgr2=0x55b6774a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.753+0000 7fddc7be7740 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 0x55b6774a5e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009f90 tx=0x7fddb0010860 comp rx=0 tx=0).stop 2024-04-26T21:57:38.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.758+0000 7fddc7be7740 1 -- 172.21.15.55:0/1184894131 shutdown_connections 2024-04-26T21:57:38.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.759+0000 7fddc7be7740 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 0x55b6774ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.759+0000 7fddc7be7740 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774a7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.759+0000 7fddc7be7740 1 --2- 172.21.15.55:0/1184894131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 0x55b6774a5e20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.759+0000 7fddc7be7740 1 -- 172.21.15.55:0/1184894131 >> 172.21.15.55:0/1184894131 conn(0x55b6773e6910 msgr2=0x55b6774a12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.768 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.767+0000 7fddc7be7740 1 -- 172.21.15.55:0/1184894131 shutdown_connections 2024-04-26T21:57:38.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.768+0000 7fddc7be7740 1 -- 172.21.15.55:0/1184894131 wait complete. 2024-04-26T21:57:38.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.770+0000 7fddc7be7740 1 -- start start 2024-04-26T21:57:38.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.770+0000 7fddc7be7740 1 Processor -- start 2024-04-26T21:57:38.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.771+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 0x55b6774afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.771+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a6bd0 0x55b6774b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.772+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a7590 0x55b6774b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.772+0000 7fddc7be7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b6774ae5a0 con 0x55b6774a7590 2024-04-26T21:57:38.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.772+0000 7fddbed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a7590 0x55b6774b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.772+0000 7fddbed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a7590 0x55b6774b70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57888/0 (socket says 172.21.15.55:57888) 2024-04-26T21:57:38.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.772+0000 7fddbed9d640 1 -- 172.21.15.55:0/289752997 learned_addr learned my addr 172.21.15.55:0/289752997 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.772+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b677390dc0 con 0x55b6774a6bd0 2024-04-26T21:57:38.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.773+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6774ae720 con 0x55b6774a5a00 2024-04-26T21:57:38.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.773+0000 7fddc7be7740 10 client.0 _ll_register_callbacks cb 0x55b6773ac600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:38.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.775+0000 7fddbe59c640 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 0x55b6774afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.775+0000 7fddbed9d640 1 -- 172.21.15.55:0/289752997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 msgr2=0x55b6774afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.775+0000 7fddbed9d640 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 0x55b6774afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.775+0000 7fddbed9d640 1 -- 172.21.15.55:0/289752997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a6bd0 msgr2=0x55b6774b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.775+0000 7fddbed9d640 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a6bd0 0x55b6774b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.777 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.775+0000 7fddbed9d640 1 -- 172.21.15.55:0/289752997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b67711b3f0 con 0x55b6774a7590 2024-04-26T21:57:38.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.777+0000 7fddbed9d640 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a7590 0x55b6774b70d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fddb0006010 tx=0x7fddb000e400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39db41856be8ffbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.777+0000 7fdda5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:38.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.784+0000 7fdda5ffb640 1 -- 172.21.15.55:0/289752997 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb001ae30 con 0x55b6774a7590 2024-04-26T21:57:38.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.785+0000 7fdda5ffb640 1 -- 172.21.15.55:0/289752997 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fddb00049c0 con 0x55b6774a7590 2024-04-26T21:57:38.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.786+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb0011d50 con 0x55b6774a7590 2024-04-26T21:57:38.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.794+0000 7fdda5ffb640 1 -- 172.21.15.55:0/289752997 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fddb000ee80 con 0x55b6774a7590 2024-04-26T21:57:38.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.795+0000 7fddc7be7740 10 client.16176 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:38.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.795+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b6771e57d0 con 0x55b6774a7590 2024-04-26T21:57:38.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.798+0000 7fdda5ffb640 1 -- 172.21.15.55:0/289752997 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb0011d50 con 0x55b6774a7590 2024-04-26T21:57:38.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.798+0000 7fdda5ffb640 1 -- 172.21.15.55:0/289752997 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fddb0004bc0 con 0x55b6774a7590 2024-04-26T21:57:38.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.798+0000 7fddc7be7740 10 client.16176 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:38.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.798+0000 7fddc7be7740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:38.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.798+0000 7fddc7be7740 20 client.16176 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:38.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.798+0000 7fddc7be7740 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b6774bb850 0x55b6774bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.798+0000 7fddc7be7740 4 client.16176 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:38.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.798+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b677417990 con 0x55b6774bb850 2024-04-26T21:57:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.806+0000 7fddbe59c640 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b6774bb850 0x55b6774bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.825+0000 7fdda5ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:38.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.825+0000 7fddbe59c640 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b6774bb850 0x55b6774bdcb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74043095ae50d00 server_cookie=cd86e255eed2d22b in_seq=0 out_seq=0 2024-04-26T21:57:38.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.833+0000 7fdda5ffb640 1 -- 172.21.15.55:0/289752997 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b677417990 con 0x55b6774bb850 2024-04-26T21:57:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.833+0000 7fdda5ffb640 10 client.16176 handle_command_reply: tid=0 2024-04-26T21:57:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.833+0000 7fddc7be7740 1 client.16176 shutdown 2024-04-26T21:57:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.833+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a7590 msgr2=0x55b6774b70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.833+0000 7fddc7be7740 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a7590 0x55b6774b70d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fddb0006010 tx=0x7fddb000e400 comp rx=0 tx=0).stop 2024-04-26T21:57:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.833+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 shutdown_connections 2024-04-26T21:57:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.833+0000 7fddc7be7740 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b6774bb850 0x55b6774bdcb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.834+0000 7fddc7be7740 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a7590 0x55b6774b70d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.834+0000 7fddc7be7740 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a6bd0 0x55b6774b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.834+0000 7fddc7be7740 1 --2- 172.21.15.55:0/289752997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 0x55b6774afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.834+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 >> 172.21.15.55:0/289752997 conn(0x55b6773e6910 msgr2=0x55b6774a0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.834+0000 7fdda5ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:38.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.834+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 shutdown_connections 2024-04-26T21:57:38.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.834+0000 7fddc7be7740 1 -- 172.21.15.55:0/289752997 wait complete. 2024-04-26T21:57:38.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.835+0000 7fddc7be7740 20 client.16176 trim_cache size 0 max 16384 2024-04-26T21:57:38.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.854+0000 7fddc7be7740 1 Processor -- start 2024-04-26T21:57:38.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.854+0000 7fddc7be7740 1 -- start start 2024-04-26T21:57:38.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.855+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 0x55b67749c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.855+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.855+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 0x55b6774a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.855+0000 7fddc7be7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0010f00 con 0x55b6774a6bd0 2024-04-26T21:57:38.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.855+0000 7fddc7be7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdda000e7d0 con 0x55b6774a5a00 2024-04-26T21:57:38.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.855+0000 7fddc7be7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdda000c620 con 0x55b6774a7590 2024-04-26T21:57:38.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.855+0000 7fddbdd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.856+0000 7fddbdd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57894/0 (socket says 172.21.15.55:57894) 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.856+0000 7fddbdd9b640 1 -- 172.21.15.55:0/2933299986 learned_addr learned my addr 172.21.15.55:0/2933299986 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.856+0000 7fddbdd9b640 1 -- 172.21.15.55:0/2933299986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 msgr2=0x55b6774a0d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.856+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/2933299986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 0x55b6774a0d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.856+0000 7fddbdd9b640 1 -- 172.21.15.55:0/2933299986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 msgr2=0x55b67749c240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.856+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/2933299986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 0x55b67749c240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.856+0000 7fddbdd9b640 1 -- 172.21.15.55:0/2933299986 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6774a6650 con 0x55b6774a6bd0 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.856+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/2933299986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b7090 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fddb80bcf10 tx=0x7fddb80bcfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fedd478ee7bc591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.857+0000 7fdda7fff640 1 -- 172.21.15.55:0/2933299986 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb80bcb70 con 0x55b6774a6bd0 2024-04-26T21:57:38.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.857+0000 7fdda7fff640 1 -- 172.21.15.55:0/2933299986 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb8039350 con 0x55b6774a6bd0 2024-04-26T21:57:38.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.857+0000 7fdda7fff640 1 -- 172.21.15.55:0/2933299986 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb80b9880 con 0x55b6774a6bd0 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.858+0000 7fddc7be7740 1 -- 172.21.15.55:0/2933299986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 msgr2=0x55b6774b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.858+0000 7fddc7be7740 1 --2- 172.21.15.55:0/2933299986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b7090 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fddb80bcf10 tx=0x7fddb80bcfe0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.858+0000 7fddc7be7740 1 -- 172.21.15.55:0/2933299986 shutdown_connections 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.858+0000 7fddc7be7740 1 --2- 172.21.15.55:0/2933299986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a7590 0x55b6774a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.858+0000 7fddc7be7740 1 --2- 172.21.15.55:0/2933299986 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b7090 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.858+0000 7fddc7be7740 1 --2- 172.21.15.55:0/2933299986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a5a00 0x55b67749c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.858+0000 7fddc7be7740 1 -- 172.21.15.55:0/2933299986 >> 172.21.15.55:0/2933299986 conn(0x55b6773e6910 msgr2=0x55b6774aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.859+0000 7fddc7be7740 1 -- 172.21.15.55:0/2933299986 shutdown_connections 2024-04-26T21:57:38.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.859+0000 7fddc7be7740 1 -- 172.21.15.55:0/2933299986 wait complete. 2024-04-26T21:57:38.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.860+0000 7fddc7be7740 1 -- start start 2024-04-26T21:57:38.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.861+0000 7fddc7be7740 1 Processor -- start 2024-04-26T21:57:38.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.861+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 0x55b6774b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddc7be7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a7590 0x55b67751fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddc7be7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdda000cc90 con 0x55b6774a6bd0 2024-04-26T21:57:38.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddc7be7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdda000e100 con 0x55b6774a7590 2024-04-26T21:57:38.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddc7be7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdda000d360 con 0x55b6774a5a00 2024-04-26T21:57:38.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddc7be7740 10 client.0 _ll_register_callbacks cb 0x7fddb400b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:38.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddbdd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddbdd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57900/0 (socket says 172.21.15.55:57900) 2024-04-26T21:57:38.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.862+0000 7fddbdd9b640 1 -- 172.21.15.55:0/4137194469 learned_addr learned my addr 172.21.15.55:0/4137194469 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:38.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.863+0000 7fddbed9d640 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a7590 0x55b67751fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.863+0000 7fddbdd9b640 1 -- 172.21.15.55:0/4137194469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 msgr2=0x55b6774b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.863+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 0x55b6774b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.863+0000 7fddbdd9b640 1 -- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a7590 msgr2=0x55b67751fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:38.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.863+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a7590 0x55b67751fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.863+0000 7fddbdd9b640 1 -- 172.21.15.55:0/4137194469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b677520360 con 0x55b6774a6bd0 2024-04-26T21:57:38.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.863+0000 7fddbed9d640 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a7590 0x55b67751fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:57:38.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.864+0000 7fddbdd9b640 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b2520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fddb80c5950 tx=0x7fddb80eeec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34af9d3f183b6a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:38.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.864+0000 7fdda5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:38.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.864+0000 7fdda5ffb640 1 -- 172.21.15.55:0/4137194469 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb80c5bf0 con 0x55b6774a6bd0 2024-04-26T21:57:38.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.865+0000 7fdda5ffb640 1 -- 172.21.15.55:0/4137194469 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fddb80c1120 con 0x55b6774a6bd0 2024-04-26T21:57:38.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.866+0000 7fddc7be7740 1 -- 172.21.15.55:0/4137194469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb8039350 con 0x55b6774a6bd0 2024-04-26T21:57:38.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.867+0000 7fdda5ffb640 1 -- 172.21.15.55:0/4137194469 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fddb80bd910 con 0x55b6774a6bd0 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.867+0000 7fddc7be7740 10 client.16182 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.867+0000 7fddc7be7740 1 -- 172.21.15.55:0/4137194469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55b6775215d0 con 0x55b6774a6bd0 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.868+0000 7fdda5ffb640 1 -- 172.21.15.55:0/4137194469 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb8039350 con 0x55b6774a6bd0 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.869+0000 7fdda5ffb640 1 -- 172.21.15.55:0/4137194469 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fddb804c060 con 0x55b6774a6bd0 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.869+0000 7fddc7be7740 10 client.16182 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.869+0000 7fddc7be7740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.869+0000 7fddc7be7740 20 client.16182 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.869+0000 7fddc7be7740 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b677521e20 0x55b677524240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.869+0000 7fddc7be7740 4 client.16182 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:57:38.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.869+0000 7fddc7be7740 1 -- 172.21.15.55:0/4137194469 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b677411800 con 0x55b677521e20 2024-04-26T21:57:38.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.871+0000 7fddbe59c640 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b677521e20 0x55b677524240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:38.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.874+0000 7fddbe59c640 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b677521e20 0x55b677524240 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd12e9be75176f69 server_cookie=61c419469ba00e68 in_seq=0 out_seq=0 2024-04-26T21:57:38.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.874+0000 7fdda5ffb640 10 client.16182 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:38.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.877+0000 7fdda5ffb640 1 -- 172.21.15.55:0/4137194469 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b677411800 con 0x55b677521e20 2024-04-26T21:57:38.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.877+0000 7fdda5ffb640 10 client.16182 handle_command_reply: tid=0 2024-04-26T21:57:38.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.877+0000 7fddc7be7740 1 client.16182 shutdown 2024-04-26T21:57:38.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.877+0000 7fddc7be7740 1 -- 172.21.15.55:0/4137194469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 msgr2=0x55b6774b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.877+0000 7fddc7be7740 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b2520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fddb80c5950 tx=0x7fddb80eeec0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fddc7be7740 1 -- 172.21.15.55:0/4137194469 shutdown_connections 2024-04-26T21:57:38.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fddc7be7740 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b6774a7590 0x55b67751fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fddc7be7740 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55b677521e20 0x55b677524240 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fddc7be7740 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b6774a6bd0 0x55b6774b2520 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fdda5ffb640 0 client.16182 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:38.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fddc7be7740 1 --2- 172.21.15.55:0/4137194469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6774a5a00 0x55b6774b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fddc7be7740 1 -- 172.21.15.55:0/4137194469 >> 172.21.15.55:0/4137194469 conn(0x55b6773e6910 msgr2=0x55b6774a0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fddc7be7740 1 -- 172.21.15.55:0/4137194469 shutdown_connections 2024-04-26T21:57:38.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.878+0000 7fddc7be7740 1 -- 172.21.15.55:0/4137194469 wait complete. 2024-04-26T21:57:38.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.879+0000 7fddc7be7740 20 client.16182 trim_cache size 0 max 16384 2024-04-26T21:57:38.881 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:57:38.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.880+0000 7fddbf59e640 1 -- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdda80a3d00 msgr2=0x7fdda80a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.880+0000 7fddbf59e640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdda80a3d00 0x7fdda80a61c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fddb001afd0 tx=0x7fddb001c5e0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.880+0000 7fddbf59e640 1 -- 172.21.15.55:0/642109673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 msgr2=0x7fddc011b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:38.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.880+0000 7fddbf59e640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 0x7fddc011b050 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009850 tx=0x7fddb80045c0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.881+0000 7fddbf59e640 1 -- 172.21.15.55:0/642109673 shutdown_connections 2024-04-26T21:57:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.881+0000 7fddbf59e640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdda80a3d00 0x7fdda80a61c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.881+0000 7fddbf59e640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fddc010a9b0 0x7fddc0119150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.881+0000 7fddbf59e640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddc010a010 0x7fddc011b050 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.881+0000 7fddbf59e640 1 --2- 172.21.15.55:0/642109673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fddc0073d70 0x7fddc011ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.881+0000 7fddbf59e640 1 -- 172.21.15.55:0/642109673 >> 172.21.15.55:0/642109673 conn(0x7fddc006f840 msgr2=0x7fddc010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.881+0000 7fddbf59e640 1 -- 172.21.15.55:0/642109673 shutdown_connections 2024-04-26T21:57:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:38.881+0000 7fddbf59e640 1 -- 172.21.15.55:0/642109673 wait complete. 2024-04-26T21:57:39.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:38 smithi064 ceph-mon[32765]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 853 MiB/s wr, 1.45k op/s 2024-04-26T21:57:39.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:38 smithi165 ceph-mon[33296]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 853 MiB/s wr, 1.45k op/s 2024-04-26T21:57:39.909 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:57:39.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:57:39.909 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:57:40.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.259+0000 7fa8e0464640 1 -- 172.21.15.55:0/194715840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d8109930 msgr2=0x7fa8d810bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:40.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.259+0000 7fa8e0464640 1 --2- 172.21.15.55:0/194715840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d8109930 0x7fa8d810bd60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009ca0 tx=0x7fa8d002a8e0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.274+0000 7fa8e0464640 1 -- 172.21.15.55:0/194715840 shutdown_connections 2024-04-26T21:57:40.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.274+0000 7fa8e0464640 1 --2- 172.21.15.55:0/194715840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d810ec70 0x7fa8d810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.274+0000 7fa8e0464640 1 --2- 172.21.15.55:0/194715840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8d810c2a0 0x7fa8d810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.274+0000 7fa8e0464640 1 --2- 172.21.15.55:0/194715840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d8109930 0x7fa8d810bd60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.274+0000 7fa8e0464640 1 -- 172.21.15.55:0/194715840 >> 172.21.15.55:0/194715840 conn(0x7fa8d806f7c0 msgr2=0x7fa8d8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:40.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.285+0000 7fa8e0464640 1 -- 172.21.15.55:0/194715840 shutdown_connections 2024-04-26T21:57:40.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.291+0000 7fa8e0464640 1 -- 172.21.15.55:0/194715840 wait complete. 2024-04-26T21:57:40.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.293+0000 7fa8e0464640 1 Processor -- start 2024-04-26T21:57:40.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.296+0000 7fa8e0464640 1 -- start start 2024-04-26T21:57:40.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.296+0000 7fa8e0464640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d8109930 0x7fa8d81a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.296+0000 7fa8e0464640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d810c2a0 0x7fa8d81a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.296+0000 7fa8e0464640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8d810ec70 0x7fa8d81aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.296+0000 7fa8e0464640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d811d2d0 con 0x7fa8d810c2a0 2024-04-26T21:57:40.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.296+0000 7fa8e0464640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d811d150 con 0x7fa8d8109930 2024-04-26T21:57:40.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.296+0000 7fa8e0464640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d811d450 con 0x7fa8d810ec70 2024-04-26T21:57:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.300+0000 7fa8dea01640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8d810ec70 0x7fa8d81aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.300+0000 7fa8dea01640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8d810ec70 0x7fa8d81aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:56670/0 (socket says 172.21.15.55:56670) 2024-04-26T21:57:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.300+0000 7fa8dea01640 1 -- 172.21.15.55:0/3143107154 learned_addr learned my addr 172.21.15.55:0/3143107154 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.300+0000 7fa8de200640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d8109930 0x7fa8d81a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.302+0000 7fa8de200640 1 -- 172.21.15.55:0/3143107154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8d810ec70 msgr2=0x7fa8d81aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.302+0000 7fa8de200640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8d810ec70 0x7fa8d81aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.302+0000 7fa8de200640 1 -- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d810c2a0 msgr2=0x7fa8d81a7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:40.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.302+0000 7fa8de200640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d810c2a0 0x7fa8d81a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.302+0000 7fa8de200640 1 -- 172.21.15.55:0/3143107154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d8111190 con 0x7fa8d8109930 2024-04-26T21:57:40.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.305+0000 7fa8de200640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d8109930 0x7fa8d81a79d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009bf0 tx=0x7fa8d0003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:40.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.305+0000 7fa8cf7fe640 1 -- 172.21.15.55:0/3143107154 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004040 con 0x7fa8d8109930 2024-04-26T21:57:40.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.306+0000 7fa8cf7fe640 1 -- 172.21.15.55:0/3143107154 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d0004690 con 0x7fa8d8109930 2024-04-26T21:57:40.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.306+0000 7fa8cf7fe640 1 -- 172.21.15.55:0/3143107154 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0030de0 con 0x7fa8d8109930 2024-04-26T21:57:40.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.306+0000 7fa8e0464640 1 -- 172.21.15.55:0/3143107154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d8111420 con 0x7fa8d8109930 2024-04-26T21:57:40.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.306+0000 7fa8e0464640 1 -- 172.21.15.55:0/3143107154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d8111920 con 0x7fa8d8109930 2024-04-26T21:57:40.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.313+0000 7fa8e3180740 1 Processor -- start 2024-04-26T21:57:40.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.315+0000 7fa8e3180740 1 -- start start 2024-04-26T21:57:40.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.315+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530cd8b0 0x559a530cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.316+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530ce2a0 0x559a530d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.316+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 0x559a530d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.316+0000 7fa8e3180740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559a53032b00 con 0x559a530ce2a0 2024-04-26T21:57:40.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.316+0000 7fa8e3180740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a52fce920 con 0x559a530cd8b0 2024-04-26T21:57:40.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.316+0000 7fa8e3180740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559a5302eac0 con 0x559a530d3410 2024-04-26T21:57:40.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.318+0000 7fa8dea01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530cd8b0 0x559a530cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.318+0000 7fa8dea01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530cd8b0 0x559a530cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43612/0 (socket says 172.21.15.55:43612) 2024-04-26T21:57:40.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.318+0000 7fa8dea01640 1 -- 172.21.15.55:0/3067708633 learned_addr learned my addr 172.21.15.55:0/3067708633 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:40.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.319+0000 7fa8dd9ff640 1 --2- 172.21.15.55:0/3067708633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 0x559a530d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.320+0000 7fa8cf7fe640 1 -- 172.21.15.55:0/3143107154 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa8d00305e0 con 0x7fa8d8109930 2024-04-26T21:57:40.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.321+0000 7fa8dd9ff640 1 -- 172.21.15.55:0/3067708633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530cd8b0 msgr2=0x559a530cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:40.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.321+0000 7fa8dd9ff640 1 --2- 172.21.15.55:0/3067708633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530cd8b0 0x559a530cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.321+0000 7fa8dd9ff640 1 -- 172.21.15.55:0/3067708633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530ce2a0 msgr2=0x559a530d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:40.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.321+0000 7fa8dd9ff640 1 --2- 172.21.15.55:0/3067708633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530ce2a0 0x559a530d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.321+0000 7fa8dd9ff640 1 -- 172.21.15.55:0/3067708633 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a52e0c7e0 con 0x559a530d3410 2024-04-26T21:57:40.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.321+0000 7fa8dd9ff640 1 --2- 172.21.15.55:0/3067708633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 0x559a530d58a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8074230 tx=0x7fa8c800adb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc9d0645df9c70e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:40.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.322+0000 7fa8cf7fe640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa8c40a3d80 0x7fa8c40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.322+0000 7fa8ccbfe640 1 -- 172.21.15.55:0/3067708633 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800ccc0 con 0x559a530d3410 2024-04-26T21:57:40.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.322+0000 7fa8ccbfe640 1 -- 172.21.15.55:0/3067708633 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c80054d0 con 0x559a530d3410 2024-04-26T21:57:40.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.322+0000 7fa8cf7fe640 1 -- 172.21.15.55:0/3143107154 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa8d00f17c0 con 0x7fa8d8109930 2024-04-26T21:57:40.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.323+0000 7fa8e3180740 1 -- 172.21.15.55:0/3067708633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 msgr2=0x559a530d58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:40.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.323+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3067708633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 0x559a530d58a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8074230 tx=0x7fa8c800adb0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.324+0000 7fa8dea01640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa8c40a3d80 0x7fa8c40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.324+0000 7fa8e3180740 1 -- 172.21.15.55:0/3067708633 shutdown_connections 2024-04-26T21:57:40.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.324+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3067708633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 0x559a530d58a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.324+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3067708633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530ce2a0 0x559a530d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.324+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3067708633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530cd8b0 0x559a530cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.324+0000 7fa8e3180740 1 -- 172.21.15.55:0/3067708633 >> 172.21.15.55:0/3067708633 conn(0x559a530c8f20 msgr2=0x559a530c9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:40.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.325+0000 7fa8dea01640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa8c40a3d80 0x7fa8c40a6240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x559a530ceb00 tx=0x7fa8d400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:40.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.327+0000 7fa8e3180740 1 -- 172.21.15.55:0/3067708633 shutdown_connections 2024-04-26T21:57:40.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.329+0000 7fa8e3180740 1 -- 172.21.15.55:0/3067708633 wait complete. 2024-04-26T21:57:40.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.330+0000 7fa8e3180740 1 -- start start 2024-04-26T21:57:40.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.330+0000 7fa8e3180740 1 Processor -- start 2024-04-26T21:57:40.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.332+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530cd8b0 0x559a530d7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.332+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530ce2a0 0x559a530d8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.332+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530d3410 0x559a530df480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.332+0000 7fa8e3180740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559a530d6460 con 0x559a530d3410 2024-04-26T21:57:40.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.332+0000 7fa8e3180740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a52fcc5d0 con 0x559a530ce2a0 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.332+0000 7fa8e3180740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559a530d65e0 con 0x559a530cd8b0 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.332+0000 7fa8e3180740 10 client.0 _ll_register_callbacks cb 0x559a52fba080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.333+0000 7fa8dea01640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530d3410 0x559a530df480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.333+0000 7fa8dea01640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530d3410 0x559a530df480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57932/0 (socket says 172.21.15.55:57932) 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.334+0000 7fa8dea01640 1 -- 172.21.15.55:0/3708256290 learned_addr learned my addr 172.21.15.55:0/3708256290 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.335+0000 7fa8dea01640 1 -- 172.21.15.55:0/3708256290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530cd8b0 msgr2=0x559a530d7f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.335+0000 7fa8dea01640 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530cd8b0 0x559a530d7f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.335+0000 7fa8dea01640 1 -- 172.21.15.55:0/3708256290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530ce2a0 msgr2=0x559a530d8440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.335+0000 7fa8dea01640 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530ce2a0 0x559a530d8440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.335+0000 7fa8dea01640 1 -- 172.21.15.55:0/3708256290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a52e6cf70 con 0x559a530d3410 2024-04-26T21:57:40.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.335+0000 7fa8dea01640 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530d3410 0x559a530df480 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa8d401f520 tx=0x7fa8d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85adc7c51177e29c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:40.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.336+0000 7fa8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:40.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.336+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3708256290 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d402c060 con 0x559a530d3410 2024-04-26T21:57:40.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.337+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3708256290 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa8d4017030 con 0x559a530d3410 2024-04-26T21:57:40.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.337+0000 7fa8e3180740 1 -- 172.21.15.55:0/3708256290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a53040710 con 0x559a530d3410 2024-04-26T21:57:40.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.338+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3708256290 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa8d4010700 con 0x559a530d3410 2024-04-26T21:57:40.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.339+0000 7fa8e3180740 10 client.16188 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:40.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.339+0000 7fa8e3180740 1 -- 172.21.15.55:0/3708256290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559a52d04170 con 0x559a530d3410 2024-04-26T21:57:40.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.341+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3708256290 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559a53040710 con 0x559a530d3410 2024-04-26T21:57:40.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.341+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3708256290 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fa8d40279e0 con 0x559a530d3410 2024-04-26T21:57:40.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.342+0000 7fa8e3180740 10 client.16188 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:40.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.342+0000 7fa8e3180740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:40.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.342+0000 7fa8e3180740 20 client.16188 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:40.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.342+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a530e3ab0 0x559a530e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.342+0000 7fa8e3180740 4 client.16188 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:40.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.342+0000 7fa8e3180740 1 -- 172.21.15.55:0/3708256290 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a5303a580 con 0x559a530e3ab0 2024-04-26T21:57:40.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.343+0000 7fa8de200640 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a530e3ab0 0x559a530e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.344+0000 7fa8de200640 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a530e3ab0 0x559a530e5f70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49a09d4e7914af4 server_cookie=42b72cba04d766cc in_seq=0 out_seq=0 2024-04-26T21:57:40.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.344+0000 7fa8c1ffb640 10 client.16188 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.347+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3708256290 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a5303a580 con 0x559a530e3ab0 2024-04-26T21:57:40.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.347+0000 7fa8c1ffb640 10 client.16188 handle_command_reply: tid=0 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.347+0000 7fa8e3180740 1 client.16188 shutdown 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 -- 172.21.15.55:0/3708256290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530d3410 msgr2=0x559a530df480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530d3410 0x559a530df480 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa8d401f520 tx=0x7fa8d4004730 comp rx=0 tx=0).stop 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 -- 172.21.15.55:0/3708256290 shutdown_connections 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a530e3ab0 0x559a530e5f70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530d3410 0x559a530df480 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530ce2a0 0x559a530d8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3708256290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530cd8b0 0x559a530d7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 -- 172.21.15.55:0/3708256290 >> 172.21.15.55:0/3708256290 conn(0x559a530c8f20 msgr2=0x559a530c80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8c1ffb640 0 client.16188 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 -- 172.21.15.55:0/3708256290 shutdown_connections 2024-04-26T21:57:40.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 1 -- 172.21.15.55:0/3708256290 wait complete. 2024-04-26T21:57:40.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.348+0000 7fa8e3180740 20 client.16188 trim_cache size 0 max 16384 2024-04-26T21:57:40.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.387+0000 7fa8e3180740 1 Processor -- start 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.387+0000 7fa8e3180740 1 -- start start 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.387+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.387+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530ce2a0 0x559a530df440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.387+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530d3410 0x559a530ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.387+0000 7fa8e3180740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81c3e70 con 0x559a530cd8b0 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.387+0000 7fa8e3180740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81c1740 con 0x559a530d3410 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.387+0000 7fa8e3180740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8118930 con 0x559a530ce2a0 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.388+0000 7fa8de200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.388+0000 7fa8de200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57946/0 (socket says 172.21.15.55:57946) 2024-04-26T21:57:40.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.388+0000 7fa8de200640 1 -- 172.21.15.55:0/2380551962 learned_addr learned my addr 172.21.15.55:0/2380551962 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.388+0000 7fa8de200640 1 -- 172.21.15.55:0/2380551962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530ce2a0 msgr2=0x559a530df440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.388+0000 7fa8de200640 1 --2- 172.21.15.55:0/2380551962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530ce2a0 0x559a530df440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.388+0000 7fa8de200640 1 -- 172.21.15.55:0/2380551962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530d3410 msgr2=0x559a530ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.388+0000 7fa8de200640 1 --2- 172.21.15.55:0/2380551962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530d3410 0x559a530ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.388+0000 7fa8de200640 1 -- 172.21.15.55:0/2380551962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a530d6fe0 con 0x559a530cd8b0 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.389+0000 7fa8de200640 1 --2- 172.21.15.55:0/2380551962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530c7f60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0000f80 tx=0x7fa8d00a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b496e856ec867494 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.389+0000 7fa8c3fff640 1 -- 172.21.15.55:0/2380551962 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d00bd310 con 0x559a530cd8b0 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.389+0000 7fa8c3fff640 1 -- 172.21.15.55:0/2380551962 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d0036040 con 0x559a530cd8b0 2024-04-26T21:57:40.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.389+0000 7fa8c3fff640 1 -- 172.21.15.55:0/2380551962 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d00ba020 con 0x559a530cd8b0 2024-04-26T21:57:40.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.391+0000 7fa8e3180740 1 -- 172.21.15.55:0/2380551962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 msgr2=0x559a530c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:40.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.391+0000 7fa8e3180740 1 --2- 172.21.15.55:0/2380551962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530c7f60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0000f80 tx=0x7fa8d00a8040 comp rx=0 tx=0).stop 2024-04-26T21:57:40.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.391+0000 7fa8e3180740 1 -- 172.21.15.55:0/2380551962 shutdown_connections 2024-04-26T21:57:40.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.391+0000 7fa8e3180740 1 --2- 172.21.15.55:0/2380551962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530d3410 0x559a530ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.391+0000 7fa8e3180740 1 --2- 172.21.15.55:0/2380551962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530ce2a0 0x559a530df440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.391+0000 7fa8e3180740 1 --2- 172.21.15.55:0/2380551962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530c7f60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.391+0000 7fa8e3180740 1 -- 172.21.15.55:0/2380551962 >> 172.21.15.55:0/2380551962 conn(0x559a530c8f20 msgr2=0x559a530c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.392+0000 7fa8e3180740 1 -- 172.21.15.55:0/2380551962 shutdown_connections 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.392+0000 7fa8e3180740 1 -- 172.21.15.55:0/2380551962 wait complete. 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.392+0000 7fa8e3180740 1 -- start start 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.392+0000 7fa8e3180740 1 Processor -- start 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530dd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530ce2a0 0x559a530dd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8e3180740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 0x559a530de830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8e3180740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8118fa0 con 0x559a530cd8b0 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8e3180740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d81c1070 con 0x559a530ce2a0 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8e3180740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8119670 con 0x559a530d3410 2024-04-26T21:57:40.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8e3180740 10 client.0 _ll_register_callbacks cb 0x7fa8d400bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8de200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530dd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8de200640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530dd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57956/0 (socket says 172.21.15.55:57956) 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8de200640 1 -- 172.21.15.55:0/3587487379 learned_addr learned my addr 172.21.15.55:0/3587487379 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.393+0000 7fa8de200640 1 -- 172.21.15.55:0/3587487379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 msgr2=0x559a530de830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8de200640 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 0x559a530de830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8de200640 1 -- 172.21.15.55:0/3587487379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530ce2a0 msgr2=0x559a530dd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8de200640 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530ce2a0 0x559a530dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8de200640 1 -- 172.21.15.55:0/3587487379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a530dee20 con 0x559a530cd8b0 2024-04-26T21:57:40.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8de200640 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530dd2e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00ba380 tx=0x7fa8d0009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e5457b9ed4ee44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:40.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:40.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3587487379 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004840 con 0x559a530cd8b0 2024-04-26T21:57:40.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3587487379 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa8d00ba020 con 0x559a530cd8b0 2024-04-26T21:57:40.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.394+0000 7fa8e3180740 1 -- 172.21.15.55:0/3587487379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d0036040 con 0x559a530cd8b0 2024-04-26T21:57:40.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.395+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3587487379 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa8d00bd9b0 con 0x559a530cd8b0 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.395+0000 7fa8e3180740 10 client.16194 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.395+0000 7fa8e3180740 1 -- 172.21.15.55:0/3587487379 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559a5314a810 con 0x559a530cd8b0 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.395+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3587487379 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8d0036040 con 0x559a530cd8b0 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.396+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3587487379 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7fa8d00bece0 con 0x559a530cd8b0 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.396+0000 7fa8e3180740 10 client.16194 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.396+0000 7fa8e3180740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.396+0000 7fa8e3180740 20 client.16194 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.396+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a5314afd0 0x559a5314d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.396+0000 7fa8e3180740 4 client.16194 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:57:40.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.396+0000 7fa8e3180740 1 -- 172.21.15.55:0/3587487379 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a530d7cf0 con 0x559a5314afd0 2024-04-26T21:57:40.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.397+0000 7fa8dd9ff640 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a5314afd0 0x559a5314d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:40.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.399+0000 7fa8c1ffb640 10 client.16194 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:40.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:40.399+0000 7fa8dd9ff640 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a5314afd0 0x559a5314d3f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebe789efacafb65 server_cookie=b58654fb36540a97 in_seq=0 out_seq=0 2024-04-26T21:57:40.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:40 smithi055 ceph-mon[29221]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 847 MiB/s wr, 1.64k op/s 2024-04-26T21:57:41.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:40 smithi064 ceph-mon[32765]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 847 MiB/s wr, 1.64k op/s 2024-04-26T21:57:41.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:40 smithi165 ceph-mon[33296]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 847 MiB/s wr, 1.64k op/s 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.189+0000 7fa8c1ffb640 1 -- 172.21.15.55:0/3587487379 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a530d7cf0 con 0x559a5314afd0 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.189+0000 7fa8c1ffb640 10 client.16194 handle_command_reply: tid=0 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.189+0000 7fa8e3180740 1 client.16194 shutdown 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 -- 172.21.15.55:0/3587487379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 msgr2=0x559a530dd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530dd2e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00ba380 tx=0x7fa8d0009870 comp rx=0 tx=0).stop 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 -- 172.21.15.55:0/3587487379 shutdown_connections 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559a530d3410 0x559a530de830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a530ce2a0 0x559a530dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559a5314afd0 0x559a5314d3f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 --2- 172.21.15.55:0/3587487379 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559a530cd8b0 0x559a530dd2e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 -- 172.21.15.55:0/3587487379 >> 172.21.15.55:0/3587487379 conn(0x559a530c8f20 msgr2=0x559a530c80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:41.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8c1ffb640 0 client.16194 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:41.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 -- 172.21.15.55:0/3587487379 shutdown_connections 2024-04-26T21:57:41.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 1 -- 172.21.15.55:0/3587487379 wait complete. 2024-04-26T21:57:41.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.190+0000 7fa8e3180740 20 client.16194 trim_cache size 0 max 16384 2024-04-26T21:57:41.192 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:57:41.193 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:57:41.193 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "5fd5d396-128e-4bbf-82d8-441d59e20125", 2024-04-26T21:57:41.193 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:57:41.193 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:57:41.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 -- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa8c40a3d80 msgr2=0x7fa8c40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa8c40a3d80 0x7fa8c40a6240 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x559a530ceb00 tx=0x7fa8d400b390 comp rx=0 tx=0).stop 2024-04-26T21:57:41.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 -- 172.21.15.55:0/3143107154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d8109930 msgr2=0x7fa8d81a79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d8109930 0x7fa8d81a79d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009bf0 tx=0x7fa8d0003ba0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 -- 172.21.15.55:0/3143107154 shutdown_connections 2024-04-26T21:57:41.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa8c40a3d80 0x7fa8c40a6240 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8d810ec70 0x7fa8d81aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8d810c2a0 0x7fa8d81a7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 --2- 172.21.15.55:0/3143107154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8d8109930 0x7fa8d81a79d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 -- 172.21.15.55:0/3143107154 >> 172.21.15.55:0/3143107154 conn(0x7fa8d806f7c0 msgr2=0x7fa8d810c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:41.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 -- 172.21.15.55:0/3143107154 shutdown_connections 2024-04-26T21:57:41.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.191+0000 7fa8e0464640 1 -- 172.21.15.55:0/3143107154 wait complete. 2024-04-26T21:57:41.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:57:41.211 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:57:41.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.487+0000 7f304a5f0640 1 -- 172.21.15.55:0/2339607538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044077520 msgr2=0x7f3044079990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.487+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2339607538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044077520 0x7f3044079990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f303c00afd0 tx=0x7f303c0085b0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.487+0000 7f304a5f0640 1 -- 172.21.15.55:0/2339607538 shutdown_connections 2024-04-26T21:57:41.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.487+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2339607538 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3044079ed0 0x7f304407c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.487+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2339607538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044077520 0x7f3044079990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.487+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2339607538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3044074640 0x7f3044074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.487+0000 7f304a5f0640 1 -- 172.21.15.55:0/2339607538 >> 172.21.15.55:0/2339607538 conn(0x7f304406f780 msgr2=0x7f3044071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:41.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.488+0000 7f304a5f0640 1 -- 172.21.15.55:0/2339607538 shutdown_connections 2024-04-26T21:57:41.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.488+0000 7f304a5f0640 1 -- 172.21.15.55:0/2339607538 wait complete. 2024-04-26T21:57:41.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.489+0000 7f304a5f0640 1 Processor -- start 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.489+0000 7f304a5f0640 1 -- start start 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.489+0000 7f304a5f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3044074640 0x7f30441b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.489+0000 7f304a5f0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044077520 0x7f30441b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.489+0000 7f304a5f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3044079ed0 0x7f30441b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.489+0000 7f304a5f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f304407fbb0 con 0x7f3044079ed0 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.489+0000 7f304a5f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f304407fa30 con 0x7f3044074640 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.489+0000 7f304a5f0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f304407fd30 con 0x7f3044077520 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.490+0000 7f3043fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3044074640 0x7f30441b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.490+0000 7f3043fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3044074640 0x7f30441b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:59440/0 (socket says 172.21.15.55:59440) 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.490+0000 7f3043fff640 1 -- 172.21.15.55:0/2173381276 learned_addr learned my addr 172.21.15.55:0/2173381276 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:41.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.490+0000 7f30437fe640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044077520 0x7f30441b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.491+0000 7f3043fff640 1 -- 172.21.15.55:0/2173381276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044077520 msgr2=0x7f30441b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.491+0000 7f3043fff640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044077520 0x7f30441b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.491+0000 7f3043fff640 1 -- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3044079ed0 msgr2=0x7f30441b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.491+0000 7f3043fff640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3044079ed0 0x7f30441b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.491+0000 7f3043fff640 1 -- 172.21.15.55:0/2173381276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30441b7b00 con 0x7f3044074640 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.492+0000 7f3043fff640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3044074640 0x7f30441b2660 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3034007860 tx=0x7f3034007d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.493+0000 7f30417fa640 1 -- 172.21.15.55:0/2173381276 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034019020 con 0x7f3044074640 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.493+0000 7f304a5f0640 1 -- 172.21.15.55:0/2173381276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30441b7df0 con 0x7f3044074640 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.493+0000 7f304a5f0640 1 -- 172.21.15.55:0/2173381276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30441b8340 con 0x7f3044074640 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.493+0000 7f30417fa640 1 -- 172.21.15.55:0/2173381276 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3034015470 con 0x7f3044074640 2024-04-26T21:57:41.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.494+0000 7f30417fa640 1 -- 172.21.15.55:0/2173381276 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034014630 con 0x7f3044074640 2024-04-26T21:57:41.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.497+0000 7f30417fa640 1 -- 172.21.15.55:0/2173381276 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f303401c020 con 0x7f3044074640 2024-04-26T21:57:41.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.499+0000 7f30417fa640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f302c0a3e40 0x7f302c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.500+0000 7f30417fa640 1 -- 172.21.15.55:0/2173381276 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f30340d17d0 con 0x7f3044074640 2024-04-26T21:57:41.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f30437fe640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f302c0a3e40 0x7f302c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f304d3e4740 1 Processor -- start 2024-04-26T21:57:41.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f304d3e4740 1 -- start start 2024-04-26T21:57:41.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2280 0x5555b90c2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2c40 0x5555b90c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f304d3e4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5555b9025d80 con 0x5555b90c2c40 2024-04-26T21:57:41.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f304d3e4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5555b8fdf8b0 con 0x5555b90c2280 2024-04-26T21:57:41.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.502+0000 7f304d3e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5555b9021d40 con 0x5555b90c10b0 2024-04-26T21:57:41.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.504+0000 7f3048b8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2c40 0x5555b90c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.504+0000 7f3048b8d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2c40 0x5555b90c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48722/0 (socket says 172.21.15.55:48722) 2024-04-26T21:57:41.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.504+0000 7f3048b8d640 1 -- 172.21.15.55:0/2056889673 learned_addr learned my addr 172.21.15.55:0/2056889673 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:41.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.504+0000 7f3043fff640 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.504+0000 7f30437fe640 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2280 0x5555b90c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.505+0000 7f30437fe640 1 -- 172.21.15.55:0/2056889673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 msgr2=0x5555b90c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:41.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.505+0000 7f30437fe640 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90c14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.505+0000 7f30437fe640 1 -- 172.21.15.55:0/2056889673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2c40 msgr2=0x5555b90c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:41.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.505+0000 7f30437fe640 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2c40 0x5555b90c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.505+0000 7f30437fe640 1 -- 172.21.15.55:0/2056889673 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b8fb0a40 con 0x5555b90c2280 2024-04-26T21:57:41.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.506+0000 7f30437fe640 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2280 0x5555b90c2700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f303c02a900 tx=0x7f303c046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc116cd747f83dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:41.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.506+0000 7f30437fe640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f302c0a3e40 0x7f302c0a6300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f303c02c010 tx=0x7f303c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:41.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.510+0000 7f302a7fc640 1 -- 172.21.15.55:0/2056889673 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c045070 con 0x5555b90c2280 2024-04-26T21:57:41.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.510+0000 7f302a7fc640 1 -- 172.21.15.55:0/2056889673 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c003b70 con 0x5555b90c2280 2024-04-26T21:57:41.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.510+0000 7f302a7fc640 1 -- 172.21.15.55:0/2056889673 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c02ae60 con 0x5555b90c2280 2024-04-26T21:57:41.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.511+0000 7f304d3e4740 1 -- 172.21.15.55:0/2056889673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2280 msgr2=0x5555b90c2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.511+0000 7f304d3e4740 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2280 0x5555b90c2700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f303c02a900 tx=0x7f303c046000 comp rx=0 tx=0).stop 2024-04-26T21:57:41.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.512+0000 7f304d3e4740 1 -- 172.21.15.55:0/2056889673 shutdown_connections 2024-04-26T21:57:41.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.512+0000 7f304d3e4740 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2c40 0x5555b90c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.512+0000 7f304d3e4740 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2280 0x5555b90c2700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.512+0000 7f304d3e4740 1 --2- 172.21.15.55:0/2056889673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.512+0000 7f304d3e4740 1 -- 172.21.15.55:0/2056889673 >> 172.21.15.55:0/2056889673 conn(0x5555b90bc5a0 msgr2=0x5555b90bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:41.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.512+0000 7f304d3e4740 1 -- 172.21.15.55:0/2056889673 shutdown_connections 2024-04-26T21:57:41.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.513+0000 7f304d3e4740 1 -- 172.21.15.55:0/2056889673 wait complete. 2024-04-26T21:57:41.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.513+0000 7f304d3e4740 1 -- start start 2024-04-26T21:57:41.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.514+0000 7f304d3e4740 1 Processor -- start 2024-04-26T21:57:41.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.514+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.514+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.514+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.514+0000 7f304d3e4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5555b90c9bf0 con 0x5555b90c2280 2024-04-26T21:57:41.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.514+0000 7f304d3e4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5555b8facd10 con 0x5555b90c2c40 2024-04-26T21:57:41.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.514+0000 7f304d3e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5555b90c9d70 con 0x5555b90c10b0 2024-04-26T21:57:41.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.514+0000 7f304d3e4740 10 client.0 _ll_register_callbacks cb 0x5555b8fd9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:41.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.516+0000 7f3043fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.516+0000 7f3043fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42506/0 (socket says 172.21.15.55:42506) 2024-04-26T21:57:41.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.516+0000 7f3043fff640 1 -- 172.21.15.55:0/3763941208 learned_addr learned my addr 172.21.15.55:0/3763941208 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:41.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.517+0000 7f3048b8d640 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.517+0000 7f30437fe640 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.517+0000 7f3048b8d640 1 -- 172.21.15.55:0/3763941208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 msgr2=0x5555b90cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:41.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.517+0000 7f3048b8d640 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.517+0000 7f3048b8d640 1 -- 172.21.15.55:0/3763941208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 msgr2=0x5555b90cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:41.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.517+0000 7f3048b8d640 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.517+0000 7f3048b8d640 1 -- 172.21.15.55:0/3763941208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b8e017d0 con 0x5555b90c2c40 2024-04-26T21:57:41.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.519+0000 7f3048b8d640 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90d2ab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f303800c550 tx=0x7f303800ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7d10a205aa14ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:41.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.520+0000 7f30237fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:57:41.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.520+0000 7f30237fe640 1 -- 172.21.15.55:0/3763941208 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303801c060 con 0x5555b90c2c40 2024-04-26T21:57:41.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.520+0000 7f304d3e4740 1 -- 172.21.15.55:0/3763941208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303c003b70 con 0x5555b90c2c40 2024-04-26T21:57:41.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.520+0000 7f30237fe640 1 -- 172.21.15.55:0/3763941208 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3038004320 con 0x5555b90c2c40 2024-04-26T21:57:41.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.521+0000 7f30237fe640 1 -- 172.21.15.55:0/3763941208 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f30380035f0 con 0x5555b90c2c40 2024-04-26T21:57:41.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.521+0000 7f304d3e4740 10 client.25589 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:41.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.521+0000 7f304d3e4740 1 -- 172.21.15.55:0/3763941208 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5555b8d373f0 con 0x5555b90c2c40 2024-04-26T21:57:41.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.522+0000 7f30237fe640 1 -- 172.21.15.55:0/3763941208 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c003b70 con 0x5555b90c2c40 2024-04-26T21:57:41.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.522+0000 7f30237fe640 1 -- 172.21.15.55:0/3763941208 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f3038016030 con 0x5555b90c2c40 2024-04-26T21:57:41.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.522+0000 7f304d3e4740 10 client.25589 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:41.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.522+0000 7f304d3e4740 10 client.25589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:41.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.522+0000 7f304d3e4740 20 client.25589 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:41.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.522+0000 7f304d3e4740 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5555b90d74f0 0x5555b90d9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.522+0000 7f304d3e4740 4 client.25589 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:57:41.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.522+0000 7f304d3e4740 1 -- 172.21.15.55:0/3763941208 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555b9033990 con 0x5555b90d74f0 2024-04-26T21:57:41.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.526+0000 7f3043fff640 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5555b90d74f0 0x5555b90d9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.544+0000 7f3043fff640 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5555b90d74f0 0x5555b90d9970 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=460780bd150d3c10 server_cookie=fbc7717a56bc514f in_seq=0 out_seq=0 2024-04-26T21:57:41.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.544+0000 7f30237fe640 10 client.25589 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:41.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.553+0000 7f30237fe640 1 -- 172.21.15.55:0/3763941208 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555b9033990 con 0x5555b90d74f0 2024-04-26T21:57:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.553+0000 7f30237fe640 10 client.25589 handle_command_reply: tid=0 2024-04-26T21:57:41.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.554+0000 7f304d3e4740 1 client.25589 shutdown 2024-04-26T21:57:41.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.557+0000 7f304d3e4740 1 -- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 msgr2=0x5555b90d2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.557+0000 7f304d3e4740 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90d2ab0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f303800c550 tx=0x7f303800ca10 comp rx=0 tx=0).stop 2024-04-26T21:57:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.557+0000 7f304d3e4740 1 -- 172.21.15.55:0/3763941208 shutdown_connections 2024-04-26T21:57:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.557+0000 7f304d3e4740 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90d2ab0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.557+0000 7f304d3e4740 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5555b90d74f0 0x5555b90d9970 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.557+0000 7f304d3e4740 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.557+0000 7f304d3e4740 1 --2- 172.21.15.55:0/3763941208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.557+0000 7f304d3e4740 1 -- 172.21.15.55:0/3763941208 >> 172.21.15.55:0/3763941208 conn(0x5555b90bc5a0 msgr2=0x5555b90bb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.558+0000 7f30237fe640 0 client.25589 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:41.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.558+0000 7f304d3e4740 1 -- 172.21.15.55:0/3763941208 shutdown_connections 2024-04-26T21:57:41.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.563+0000 7f304d3e4740 1 -- 172.21.15.55:0/3763941208 wait complete. 2024-04-26T21:57:41.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.569+0000 7f304d3e4740 20 client.25589 trim_cache size 0 max 16384 2024-04-26T21:57:41.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.619+0000 7f304d3e4740 1 Processor -- start 2024-04-26T21:57:41.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.620+0000 7f304d3e4740 1 -- start start 2024-04-26T21:57:41.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.620+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c10b0 0x5555b90c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.620+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.621+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c2c40 0x5555b90c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.621+0000 7f304d3e4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3030010f00 con 0x5555b90c2280 2024-04-26T21:57:41.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.621+0000 7f304d3e4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f303000e7d0 con 0x5555b90c10b0 2024-04-26T21:57:41.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.621+0000 7f304d3e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f303000c620 con 0x5555b90c2c40 2024-04-26T21:57:41.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.622+0000 7f3048b8d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c2c40 0x5555b90c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.622+0000 7f3048b8d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c2c40 0x5555b90c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42514/0 (socket says 172.21.15.55:42514) 2024-04-26T21:57:41.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.622+0000 7f30437fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.622+0000 7f3048b8d640 1 -- 172.21.15.55:0/4042303032 learned_addr learned my addr 172.21.15.55:0/4042303032 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:41.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.623+0000 7f30437fe640 1 -- 172.21.15.55:0/4042303032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c2c40 msgr2=0x5555b90c62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:41.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.623+0000 7f30437fe640 1 --2- 172.21.15.55:0/4042303032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c2c40 0x5555b90c62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.623+0000 7f30437fe640 1 -- 172.21.15.55:0/4042303032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c10b0 msgr2=0x5555b90c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:41.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.623+0000 7f30437fe640 1 --2- 172.21.15.55:0/4042303032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c10b0 0x5555b90c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.623+0000 7f30437fe640 1 -- 172.21.15.55:0/4042303032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b8cf70f0 con 0x5555b90c2280 2024-04-26T21:57:41.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.624+0000 7f30437fe640 1 --2- 172.21.15.55:0/4042303032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90d2a70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x5555b90cca10 tx=0x7f303c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecda5fd1b62bac25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:41.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.628+0000 7f3029ffb640 1 -- 172.21.15.55:0/4042303032 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c056070 con 0x5555b90c2280 2024-04-26T21:57:41.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.628+0000 7f3029ffb640 1 -- 172.21.15.55:0/4042303032 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c003b70 con 0x5555b90c2280 2024-04-26T21:57:41.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.629+0000 7f3029ffb640 1 -- 172.21.15.55:0/4042303032 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c045040 con 0x5555b90c2280 2024-04-26T21:57:41.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 -- 172.21.15.55:0/4042303032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 msgr2=0x5555b90d2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 --2- 172.21.15.55:0/4042303032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90d2a70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x5555b90cca10 tx=0x7f303c00c000 comp rx=0 tx=0).stop 2024-04-26T21:57:41.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 -- 172.21.15.55:0/4042303032 shutdown_connections 2024-04-26T21:57:41.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 --2- 172.21.15.55:0/4042303032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c2c40 0x5555b90c62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 --2- 172.21.15.55:0/4042303032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90d2a70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 --2- 172.21.15.55:0/4042303032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c10b0 0x5555b90c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 -- 172.21.15.55:0/4042303032 >> 172.21.15.55:0/4042303032 conn(0x5555b90bc5a0 msgr2=0x5555b90d9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:41.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 -- 172.21.15.55:0/4042303032 shutdown_connections 2024-04-26T21:57:41.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.630+0000 7f304d3e4740 1 -- 172.21.15.55:0/4042303032 wait complete. 2024-04-26T21:57:41.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.631+0000 7f304d3e4740 1 -- start start 2024-04-26T21:57:41.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.631+0000 7f304d3e4740 1 Processor -- start 2024-04-26T21:57:41.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.632+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.632+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.632+0000 7f304d3e4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.632+0000 7f304d3e4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f303000cc90 con 0x5555b90c2280 2024-04-26T21:57:41.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.632+0000 7f304d3e4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f303000e100 con 0x5555b90c2c40 2024-04-26T21:57:41.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.632+0000 7f304d3e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f303000d360 con 0x5555b90c10b0 2024-04-26T21:57:41.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.632+0000 7f304d3e4740 10 client.0 _ll_register_callbacks cb 0x7f303800d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:57:41.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.633+0000 7f30437fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.633+0000 7f30437fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48760/0 (socket says 172.21.15.55:48760) 2024-04-26T21:57:41.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.633+0000 7f30437fe640 1 -- 172.21.15.55:0/1893862444 learned_addr learned my addr 172.21.15.55:0/1893862444 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:57:41.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.633+0000 7f3043fff640 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.633+0000 7f3048b8d640 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.633+0000 7f30437fe640 1 -- 172.21.15.55:0/1893862444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 msgr2=0x5555b90d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:41.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.633+0000 7f30437fe640 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.633+0000 7f30437fe640 1 -- 172.21.15.55:0/1893862444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 msgr2=0x5555b90cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:57:41.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.634+0000 7f30437fe640 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.634+0000 7f30437fe640 1 -- 172.21.15.55:0/1893862444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b8fe64b0 con 0x5555b90c2280 2024-04-26T21:57:41.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.634+0000 7f30437fe640 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cf390 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f303c005ba0 tx=0x7f303c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771e69fe7f09a17f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:57:41.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.634+0000 7f30237fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:57:41.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.635+0000 7f30237fe640 1 -- 172.21.15.55:0/1893862444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c056070 con 0x5555b90c2280 2024-04-26T21:57:41.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.635+0000 7f30237fe640 1 -- 172.21.15.55:0/1893862444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f303c03f7c0 con 0x5555b90c2280 2024-04-26T21:57:41.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.635+0000 7f304d3e4740 1 -- 172.21.15.55:0/1893862444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303c003b70 con 0x5555b90c2280 2024-04-26T21:57:41.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.637+0000 7f30237fe640 1 -- 172.21.15.55:0/1893862444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f303c045040 con 0x5555b90c2280 2024-04-26T21:57:41.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.637+0000 7f304d3e4740 10 client.16209 fetch_fsmap learned FSMap version 22 2024-04-26T21:57:41.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.637+0000 7f304d3e4740 1 -- 172.21.15.55:0/1893862444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5555b913cb40 con 0x5555b90c2280 2024-04-26T21:57:41.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.638+0000 7f30237fe640 1 -- 172.21.15.55:0/1893862444 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f303c049460 con 0x5555b90c2280 2024-04-26T21:57:41.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.638+0000 7f30237fe640 1 -- 172.21.15.55:0/1893862444 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 22) ==== 8342+0+0 (secure 0 0 0) 0x7f303c03d400 con 0x5555b90c2280 2024-04-26T21:57:41.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.638+0000 7f304d3e4740 10 client.16209 fetch_fsmap finished waiting for FSMap version 22 2024-04-26T21:57:41.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.638+0000 7f304d3e4740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:57:41.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.638+0000 7f304d3e4740 20 client.16209 populate_metadata read hostname 'smithi055' 2024-04-26T21:57:41.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.638+0000 7f304d3e4740 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5555b913cf80 0x5555b913f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:57:41.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.638+0000 7f304d3e4740 4 client.16209 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:57:41.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.638+0000 7f304d3e4740 1 -- 172.21.15.55:0/1893862444 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f303c003b70 con 0x5555b913cf80 2024-04-26T21:57:41.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.640+0000 7f3043fff640 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5555b913cf80 0x5555b913f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:57:41.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.643+0000 7f3043fff640 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5555b913cf80 0x5555b913f3a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b619a757c4d83c server_cookie=ad0de29820fa06a3 in_seq=0 out_seq=0 2024-04-26T21:57:41.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.643+0000 7f30237fe640 10 client.16209 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:41.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f30237fe640 1 -- 172.21.15.55:0/1893862444 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x7f303c003b70 con 0x5555b913cf80 2024-04-26T21:57:41.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f30237fe640 10 client.16209 handle_command_reply: tid=0 2024-04-26T21:57:41.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 client.16209 shutdown 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 -- 172.21.15.55:0/1893862444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 msgr2=0x5555b90cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cf390 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f303c005ba0 tx=0x7f303c003f50 comp rx=0 tx=0).stop 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 -- 172.21.15.55:0/1893862444 shutdown_connections 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5555b90c2c40 0x5555b90cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5555b913cf80 0x5555b913f3a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5555b90c2280 0x5555b90cf390 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 --2- 172.21.15.55:0/1893862444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5555b90c10b0 0x5555b90d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f304d3e4740 1 -- 172.21.15.55:0/1893862444 >> 172.21.15.55:0/1893862444 conn(0x5555b90bc5a0 msgr2=0x5555b90c5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.648+0000 7f30237fe640 0 client.16209 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:57:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.649+0000 7f304d3e4740 1 -- 172.21.15.55:0/1893862444 shutdown_connections 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.649+0000 7f304d3e4740 1 -- 172.21.15.55:0/1893862444 wait complete. 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.649+0000 7f304d3e4740 20 client.16209 trim_cache size 0 max 16384 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stdout: "5fd5d396-128e-4bbf-82d8-441d59e20125": { 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "5fd5d396-128e-4bbf-82d8-441d59e20125", 2024-04-26T21:57:41.650 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:57:41.651 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:57:41.651 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:57:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:57:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 -- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f302c0a3e40 msgr2=0x7f302c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f302c0a3e40 0x7f302c0a6300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f303c02c010 tx=0x7f303c000f40 comp rx=0 tx=0).stop 2024-04-26T21:57:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 -- 172.21.15.55:0/2173381276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3044074640 msgr2=0x7f30441b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:57:41.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3044074640 0x7f30441b2660 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3034007860 tx=0x7f3034007d20 comp rx=0 tx=0).stop 2024-04-26T21:57:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 -- 172.21.15.55:0/2173381276 shutdown_connections 2024-04-26T21:57:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f302c0a3e40 0x7f302c0a6300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3044079ed0 0x7f30441b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044077520 0x7f30441b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.650+0000 7f304a5f0640 1 --2- 172.21.15.55:0/2173381276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3044074640 0x7f30441b2660 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:57:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.651+0000 7f304a5f0640 1 -- 172.21.15.55:0/2173381276 >> 172.21.15.55:0/2173381276 conn(0x7f304406f780 msgr2=0x7f304407ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:57:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.651+0000 7f304a5f0640 1 -- 172.21.15.55:0/2173381276 shutdown_connections 2024-04-26T21:57:41.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:57:41.651+0000 7f304a5f0640 1 -- 172.21.15.55:0/2173381276 wait complete. 2024-04-26T21:57:41.677 INFO:tasks.cephfs.filesystem:scrub status for tag:5fd5d396-128e-4bbf-82d8-441d59e20125 - {'path': '/', 'tag': '5fd5d396-128e-4bbf-82d8-441d59e20125', 'options': 'recursive,force'} 2024-04-26T21:57:41.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:41 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:57:41.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:41 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:41.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:41 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:57:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:41 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:57:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:41 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:41 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:57:42.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:41 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:57:42.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:41 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:42.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:41 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:57:42.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:42 smithi055 ceph-mon[29221]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 829 MiB/s wr, 1.71k op/s 2024-04-26T21:57:42.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:42 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:57:42.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:42 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:42 smithi064 ceph-mon[32765]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 829 MiB/s wr, 1.71k op/s 2024-04-26T21:57:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:42 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:57:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:42 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:43.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:42 smithi165 ceph-mon[33296]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 829 MiB/s wr, 1.71k op/s 2024-04-26T21:57:43.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:42 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:57:43.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:42 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:45.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:44 smithi064 ceph-mon[32765]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 MiB/s wr, 1.73k op/s 2024-04-26T21:57:45.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:44 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:45.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:44 smithi165 ceph-mon[33296]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 MiB/s wr, 1.73k op/s 2024-04-26T21:57:45.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:44 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:45.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:44 smithi055 ceph-mon[29221]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 866 MiB/s wr, 1.73k op/s 2024-04-26T21:57:45.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:44 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:57:47.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:46 smithi064 ceph-mon[32765]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 857 MiB/s wr, 1.77k op/s 2024-04-26T21:57:47.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:46 smithi165 ceph-mon[33296]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 857 MiB/s wr, 1.77k op/s 2024-04-26T21:57:47.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:46 smithi055 ceph-mon[29221]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 857 MiB/s wr, 1.77k op/s 2024-04-26T21:57:49.009 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:48 smithi064 ceph-mon[32765]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 862 MiB/s wr, 1.67k op/s 2024-04-26T21:57:49.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:48 smithi165 ceph-mon[33296]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 862 MiB/s wr, 1.67k op/s 2024-04-26T21:57:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:48 smithi055 ceph-mon[29221]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 862 MiB/s wr, 1.67k op/s 2024-04-26T21:57:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:50 smithi064 ceph-mon[32765]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 839 MiB/s wr, 1.36k op/s 2024-04-26T21:57:51.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:50 smithi165 ceph-mon[33296]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 839 MiB/s wr, 1.36k op/s 2024-04-26T21:57:51.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:50 smithi055 ceph-mon[29221]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 839 MiB/s wr, 1.36k op/s 2024-04-26T21:57:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:51 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:57:52.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:51 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:57:52.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:51 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:57:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:52 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:57:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:52 smithi064 ceph-mon[32765]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 842 MiB/s wr, 1.17k op/s 2024-04-26T21:57:53.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:52 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:57:53.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:52 smithi165 ceph-mon[33296]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 842 MiB/s wr, 1.17k op/s 2024-04-26T21:57:53.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:52 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:57:53.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:52 smithi055 ceph-mon[29221]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 842 MiB/s wr, 1.17k op/s 2024-04-26T21:57:55.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:54 smithi064 ceph-mon[32765]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 878 MiB/s wr, 1.12k op/s 2024-04-26T21:57:55.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:54 smithi165 ceph-mon[33296]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 878 MiB/s wr, 1.12k op/s 2024-04-26T21:57:55.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:54 smithi055 ceph-mon[29221]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 878 MiB/s wr, 1.12k op/s 2024-04-26T21:57:57.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:56 smithi165 ceph-mon[33296]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 821 MiB/s wr, 1.06k op/s 2024-04-26T21:57:57.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:56 smithi055 ceph-mon[29221]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 821 MiB/s wr, 1.06k op/s 2024-04-26T21:57:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:56 smithi064 ceph-mon[32765]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 821 MiB/s wr, 1.06k op/s 2024-04-26T21:57:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:57:58 smithi165 ceph-mon[33296]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 859 MiB/s wr, 1.04k op/s 2024-04-26T21:57:59.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:57:58 smithi055 ceph-mon[29221]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 859 MiB/s wr, 1.04k op/s 2024-04-26T21:57:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:57:58 smithi064 ceph-mon[32765]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 859 MiB/s wr, 1.04k op/s 2024-04-26T21:58:01.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:00 smithi165 ceph-mon[33296]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 845 MiB/s wr, 983 op/s 2024-04-26T21:58:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:00 smithi055 ceph-mon[29221]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 845 MiB/s wr, 983 op/s 2024-04-26T21:58:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:00 smithi064 ceph-mon[32765]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 845 MiB/s wr, 983 op/s 2024-04-26T21:58:03.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:02 smithi165 ceph-mon[33296]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 839 MiB/s wr, 974 op/s 2024-04-26T21:58:03.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:02 smithi055 ceph-mon[29221]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 839 MiB/s wr, 974 op/s 2024-04-26T21:58:03.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:02 smithi064 ceph-mon[32765]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 839 MiB/s wr, 974 op/s 2024-04-26T21:58:05.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:04 smithi165 ceph-mon[33296]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 857 MiB/s wr, 1.14k op/s 2024-04-26T21:58:05.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:04 smithi055 ceph-mon[29221]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 857 MiB/s wr, 1.14k op/s 2024-04-26T21:58:05.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:04 smithi064 ceph-mon[32765]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 857 MiB/s wr, 1.14k op/s 2024-04-26T21:58:07.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:06 smithi165 ceph-mon[33296]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 804 MiB/s wr, 1.17k op/s 2024-04-26T21:58:07.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:06 smithi055 ceph-mon[29221]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 804 MiB/s wr, 1.17k op/s 2024-04-26T21:58:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:06 smithi064 ceph-mon[32765]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 804 MiB/s wr, 1.17k op/s 2024-04-26T21:58:09.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:08 smithi055 ceph-mon[29221]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 889 MiB/s wr, 1.32k op/s 2024-04-26T21:58:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:08 smithi064 ceph-mon[32765]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 889 MiB/s wr, 1.32k op/s 2024-04-26T21:58:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:08 smithi165 ceph-mon[33296]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 889 MiB/s wr, 1.32k op/s 2024-04-26T21:58:11.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:10 smithi055 ceph-mon[29221]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 841 MiB/s wr, 1.27k op/s 2024-04-26T21:58:11.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:10 smithi064 ceph-mon[32765]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 841 MiB/s wr, 1.27k op/s 2024-04-26T21:58:11.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:10 smithi165 ceph-mon[33296]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 841 MiB/s wr, 1.27k op/s 2024-04-26T21:58:11.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:58:11.678 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:58:11.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.861+0000 7fc2d0912640 1 -- 172.21.15.55:0/2417784999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc103420 msgr2=0x7fc2cc109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:11.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.861+0000 7fc2d0912640 1 --2- 172.21.15.55:0/2417784999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc103420 0x7fc2cc109cb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009ca0 tx=0x7fc2c402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.862+0000 7fc2d0912640 1 -- 172.21.15.55:0/2417784999 shutdown_connections 2024-04-26T21:58:11.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.862+0000 7fc2d0912640 1 --2- 172.21.15.55:0/2417784999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc103420 0x7fc2cc109cb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.862+0000 7fc2d0912640 1 --2- 172.21.15.55:0/2417784999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2cc102a60 0x7fc2cc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.862+0000 7fc2d0912640 1 --2- 172.21.15.55:0/2417784999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc101870 0x7fc2cc101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.862+0000 7fc2d0912640 1 -- 172.21.15.55:0/2417784999 >> 172.21.15.55:0/2417784999 conn(0x7fc2cc0fce20 msgr2=0x7fc2cc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:11.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.862+0000 7fc2d0912640 1 -- 172.21.15.55:0/2417784999 shutdown_connections 2024-04-26T21:58:11.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.863+0000 7fc2d0912640 1 -- 172.21.15.55:0/2417784999 wait complete. 2024-04-26T21:58:11.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.865+0000 7fc2d0912640 1 Processor -- start 2024-04-26T21:58:11.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.865+0000 7fc2d0912640 1 -- start start 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2d0912640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc101870 0x7fc2cc198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2d0912640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc102a60 0x7fc2cc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2d0912640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2cc103420 0x7fc2cc1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2d0912640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10e590 con 0x7fc2cc102a60 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2d0912640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10e410 con 0x7fc2cc103420 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2d0912640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc10e710 con 0x7fc2cc101870 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2c9d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc102a60 0x7fc2cc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2c9d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc102a60 0x7fc2cc199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49600/0 (socket says 172.21.15.55:49600) 2024-04-26T21:58:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.866+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/4082145493 learned_addr learned my addr 172.21.15.55:0/4082145493 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.867+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/4082145493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc101870 msgr2=0x7fc2cc198d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:58:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.867+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc101870 0x7fc2cc198d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.867+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/4082145493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2cc103420 msgr2=0x7fc2cc1a0260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:58:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.867+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2cc103420 0x7fc2cc1a0260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.867+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/4082145493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc1a0900 con 0x7fc2cc102a60 2024-04-26T21:58:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.867+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc102a60 0x7fc2cc199280 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000b4a0 tx=0x7fc2c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.868+0000 7fc2bb7fe640 1 -- 172.21.15.55:0/4082145493 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011830 con 0x7fc2cc102a60 2024-04-26T21:58:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.868+0000 7fc2bb7fe640 1 -- 172.21.15.55:0/4082145493 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c0016070 con 0x7fc2cc102a60 2024-04-26T21:58:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.868+0000 7fc2d0912640 1 -- 172.21.15.55:0/4082145493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1a0bf0 con 0x7fc2cc102a60 2024-04-26T21:58:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.868+0000 7fc2bb7fe640 1 -- 172.21.15.55:0/4082145493 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0010e10 con 0x7fc2cc102a60 2024-04-26T21:58:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.868+0000 7fc2d0912640 1 -- 172.21.15.55:0/4082145493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1a1190 con 0x7fc2cc102a60 2024-04-26T21:58:11.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.872+0000 7fc2d361f740 1 Processor -- start 2024-04-26T21:58:11.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.872+0000 7fc2d361f740 1 -- start start 2024-04-26T21:58:11.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.873+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2eb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.873+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2ec0b0 0x55624a2ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.874+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2eca70 0x55624a2f3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.874+0000 7fc2d361f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55624a24fd80 con 0x55624a2eca70 2024-04-26T21:58:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.874+0000 7fc2d361f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55624a2098b0 con 0x55624a2ec0b0 2024-04-26T21:58:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.874+0000 7fc2d361f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55624a24bd40 con 0x55624a2eaee0 2024-04-26T21:58:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.874+0000 7fc2bb7fe640 1 -- 172.21.15.55:0/4082145493 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc2c00119d0 con 0x7fc2cc102a60 2024-04-26T21:58:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.874+0000 7fc2c9d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2eca70 0x55624a2f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.875+0000 7fc2c9d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2eca70 0x55624a2f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49610/0 (socket says 172.21.15.55:49610) 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.875+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/60318570 learned_addr learned my addr 172.21.15.55:0/60318570 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.875+0000 7fc2ca59c640 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2ec0b0 0x55624a2ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.875+0000 7fc2bb7fe640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2b40a3d80 0x7fc2b40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.876+0000 7fc2cad9d640 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2eb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.876+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/60318570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 msgr2=0x55624a2eb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.876+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2eb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.876+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/60318570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2ec0b0 msgr2=0x55624a2ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.876+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2ec0b0 0x55624a2ec530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.876+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/60318570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55624a1daa40 con 0x55624a2eca70 2024-04-26T21:58:11.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.876+0000 7fc2cad9d640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2b40a3d80 0x7fc2b40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:11.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.876+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2eca70 0x55624a2f3300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc2c004daa0 tx=0x7fc2c001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31a9bdea37752d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:11.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.877+0000 7fc2b8ff9640 1 -- 172.21.15.55:0/60318570 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c009bae0 con 0x55624a2eca70 2024-04-26T21:58:11.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.877+0000 7fc2b8ff9640 1 -- 172.21.15.55:0/60318570 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c0011cc0 con 0x55624a2eca70 2024-04-26T21:58:11.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.877+0000 7fc2bb7fe640 1 -- 172.21.15.55:0/4082145493 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2c00cfb20 con 0x7fc2cc102a60 2024-04-26T21:58:11.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.877+0000 7fc2b8ff9640 1 -- 172.21.15.55:0/60318570 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c004dad0 con 0x55624a2eca70 2024-04-26T21:58:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.878+0000 7fc2d361f740 1 -- 172.21.15.55:0/60318570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2eca70 msgr2=0x55624a2f3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.878+0000 7fc2cad9d640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2b40a3d80 0x7fc2b40a6240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4031b50 tx=0x7fc2c402a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.878+0000 7fc2d361f740 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2eca70 0x55624a2f3300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc2c004daa0 tx=0x7fc2c001f000 comp rx=0 tx=0).stop 2024-04-26T21:58:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.878+0000 7fc2d361f740 1 -- 172.21.15.55:0/60318570 shutdown_connections 2024-04-26T21:58:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.878+0000 7fc2d361f740 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2eca70 0x55624a2f3300 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.878+0000 7fc2d361f740 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2ec0b0 0x55624a2ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.878+0000 7fc2d361f740 1 --2- 172.21.15.55:0/60318570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2eb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.878+0000 7fc2d361f740 1 -- 172.21.15.55:0/60318570 >> 172.21.15.55:0/60318570 conn(0x55624a2e63d0 msgr2=0x55624a2e67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:11.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.879+0000 7fc2d361f740 1 -- 172.21.15.55:0/60318570 shutdown_connections 2024-04-26T21:58:11.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.879+0000 7fc2d361f740 1 -- 172.21.15.55:0/60318570 wait complete. 2024-04-26T21:58:11.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.879+0000 7fc2d361f740 1 -- start start 2024-04-26T21:58:11.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.879+0000 7fc2d361f740 1 Processor -- start 2024-04-26T21:58:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2f5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2d361f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55624a2f38a0 con 0x55624a2ec0b0 2024-04-26T21:58:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2d361f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55624a1d6d10 con 0x55624a2eca70 2024-04-26T21:58:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2d361f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55624a2f3a20 con 0x55624a2eaee0 2024-04-26T21:58:11.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2d361f740 10 client.0 _ll_register_callbacks cb 0x55624a203210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2ca59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2ca59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33500/0 (socket says 172.21.15.55:33500) 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.880+0000 7fc2ca59c640 1 -- 172.21.15.55:0/514786971 learned_addr learned my addr 172.21.15.55:0/514786971 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.881+0000 7fc2ca59c640 1 -- 172.21.15.55:0/514786971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 msgr2=0x55624a2fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.881+0000 7fc2ca59c640 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.881+0000 7fc2ca59c640 1 -- 172.21.15.55:0/514786971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 msgr2=0x55624a2f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.881+0000 7fc2ca59c640 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.881+0000 7fc2ca59c640 1 -- 172.21.15.55:0/514786971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55624a02b7d0 con 0x55624a2eaee0 2024-04-26T21:58:11.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.882+0000 7fc2ca59c640 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2f5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00aed0 tx=0x7fc2bc00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154cd65579e40cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.882+0000 7fc2b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.882+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/514786971 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc015070 con 0x55624a2eaee0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.882+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/514786971 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2bc0115f0 con 0x55624a2eaee0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.882+0000 7fc2d361f740 1 -- 172.21.15.55:0/514786971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55624a25d990 con 0x55624a2eaee0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.883+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/514786971 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2bc01f050 con 0x55624a2eaee0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.883+0000 7fc2d361f740 10 client.25564 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.883+0000 7fc2d361f740 1 -- 172.21.15.55:0/514786971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556249f613f0 con 0x55624a2eaee0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.883+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/514786971 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc011ca0 con 0x55624a2eaee0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.883+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/514786971 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc2bc01ab30 con 0x55624a2eaee0 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.883+0000 7fc2d361f740 10 client.25564 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.883+0000 7fc2d361f740 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.884+0000 7fc2d361f740 20 client.25564 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.884+0000 7fc2d361f740 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55624a300d80 0x55624a303200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:11.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.884+0000 7fc2d361f740 4 client.25564 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:11.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.884+0000 7fc2d361f740 1 -- 172.21.15.55:0/514786971 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55624a25d990 con 0x55624a300d80 2024-04-26T21:58:11.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.921+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55624a300d80 0x55624a303200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.926+0000 7fc2b1ffb640 10 client.25564 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.926+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55624a300d80 0x55624a303200 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f16e5fd844f747de server_cookie=454ca3b9bff08dcd in_seq=0 out_seq=0 2024-04-26T21:58:11.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.929+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/514786971 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55624a25d990 con 0x55624a300d80 2024-04-26T21:58:11.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.929+0000 7fc2b1ffb640 10 client.25564 handle_command_reply: tid=0 2024-04-26T21:58:11.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.929+0000 7fc2d361f740 1 client.25564 shutdown 2024-04-26T21:58:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.944+0000 7fc2d361f740 1 -- 172.21.15.55:0/514786971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 msgr2=0x55624a2f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.944+0000 7fc2d361f740 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2f5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00aed0 tx=0x7fc2bc00c520 comp rx=0 tx=0).stop 2024-04-26T21:58:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.945+0000 7fc2d361f740 1 -- 172.21.15.55:0/514786971 shutdown_connections 2024-04-26T21:58:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.945+0000 7fc2d361f740 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.945+0000 7fc2d361f740 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55624a300d80 0x55624a303200 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.945+0000 7fc2d361f740 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.945+0000 7fc2d361f740 1 --2- 172.21.15.55:0/514786971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2eaee0 0x55624a2f5160 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00aed0 tx=0x7fc2bc00c520 comp rx=0 tx=0).stop 2024-04-26T21:58:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.945+0000 7fc2d361f740 1 -- 172.21.15.55:0/514786971 >> 172.21.15.55:0/514786971 conn(0x55624a2e63d0 msgr2=0x55624a2e5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.945+0000 7fc2b1ffb640 0 client.25564 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.950+0000 7fc2d361f740 1 -- 172.21.15.55:0/514786971 shutdown_connections 2024-04-26T21:58:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.950+0000 7fc2d361f740 1 -- 172.21.15.55:0/514786971 wait complete. 2024-04-26T21:58:11.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:11.956+0000 7fc2d361f740 20 client.25564 trim_cache size 0 max 16384 2024-04-26T21:58:12.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.022+0000 7fc2d361f740 1 Processor -- start 2024-04-26T21:58:12.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.022+0000 7fc2d361f740 1 -- start start 2024-04-26T21:58:12.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.023+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2ec0b0 0x55624a2e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.023+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.023+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2e9fd0 0x55624a2f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.023+0000 7fc2d361f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1ac930 con 0x55624a2e9fd0 2024-04-26T21:58:12.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.023+0000 7fc2d361f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1aa200 con 0x55624a2eca70 2024-04-26T21:58:12.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.023+0000 7fc2d361f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1a8050 con 0x55624a2ec0b0 2024-04-26T21:58:12.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.025+0000 7fc2cad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2e9fd0 0x55624a2f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.025+0000 7fc2cad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2e9fd0 0x55624a2f47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49636/0 (socket says 172.21.15.55:49636) 2024-04-26T21:58:12.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.026+0000 7fc2cad9d640 1 -- 172.21.15.55:0/1182465258 learned_addr learned my addr 172.21.15.55:0/1182465258 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:12.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.026+0000 7fc2ca59c640 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2ec0b0 0x55624a2e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.030+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.031+0000 7fc2cad9d640 1 -- 172.21.15.55:0/1182465258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2ec0b0 msgr2=0x55624a2e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:12.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.031+0000 7fc2cad9d640 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2ec0b0 0x55624a2e9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.031+0000 7fc2cad9d640 1 -- 172.21.15.55:0/1182465258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 msgr2=0x55624a2fc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:12.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.031+0000 7fc2cad9d640 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.031+0000 7fc2cad9d640 1 -- 172.21.15.55:0/1182465258 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556249f210f0 con 0x55624a2e9fd0 2024-04-26T21:58:12.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.045+0000 7fc2cad9d640 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2e9fd0 0x55624a2f47b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009a60 tx=0x7fc2c403e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa22084c181805ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:12.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.050+0000 7fc2b3fff640 1 -- 172.21.15.55:0/1182465258 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4004040 con 0x55624a2e9fd0 2024-04-26T21:58:12.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.050+0000 7fc2b3fff640 1 -- 172.21.15.55:0/1182465258 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c404c030 con 0x55624a2e9fd0 2024-04-26T21:58:12.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.051+0000 7fc2d361f740 1 -- 172.21.15.55:0/1182465258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2e9fd0 msgr2=0x55624a2f47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:12.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.051+0000 7fc2d361f740 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2e9fd0 0x55624a2f47b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009a60 tx=0x7fc2c403e070 comp rx=0 tx=0).stop 2024-04-26T21:58:12.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.055+0000 7fc2d361f740 1 -- 172.21.15.55:0/1182465258 shutdown_connections 2024-04-26T21:58:12.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.055+0000 7fc2d361f740 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2e9fd0 0x55624a2f47b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.055+0000 7fc2d361f740 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.055+0000 7fc2d361f740 1 --2- 172.21.15.55:0/1182465258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2ec0b0 0x55624a2e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.055+0000 7fc2d361f740 1 -- 172.21.15.55:0/1182465258 >> 172.21.15.55:0/1182465258 conn(0x55624a2e63d0 msgr2=0x55624a2e9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:12.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.063+0000 7fc2d361f740 1 -- 172.21.15.55:0/1182465258 shutdown_connections 2024-04-26T21:58:12.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.066+0000 7fc2d361f740 1 -- 172.21.15.55:0/1182465258 wait complete. 2024-04-26T21:58:12.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.066+0000 7fc2d361f740 1 -- start start 2024-04-26T21:58:12.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.067+0000 7fc2d361f740 1 Processor -- start 2024-04-26T21:58:12.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.067+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2e9fd0 0x55624a2fa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.067+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2fa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.067+0000 7fc2d361f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.067+0000 7fc2d361f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1a86c0 con 0x55624a2ec0b0 2024-04-26T21:58:12.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.067+0000 7fc2d361f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1a9b30 con 0x55624a2eca70 2024-04-26T21:58:12.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.067+0000 7fc2d361f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1a8d90 con 0x55624a2e9fd0 2024-04-26T21:58:12.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.067+0000 7fc2d361f740 10 client.0 _ll_register_callbacks cb 0x7fc2bc006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.072+0000 7fc2c9d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.072+0000 7fc2ca59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2e9fd0 0x55624a2fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.072+0000 7fc2cad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.072+0000 7fc2c9d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2fa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49646/0 (socket says 172.21.15.55:49646) 2024-04-26T21:58:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.072+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/4270716032 learned_addr learned my addr 172.21.15.55:0/4270716032 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.073+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/4270716032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2e9fd0 msgr2=0x55624a2fa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.073+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2e9fd0 0x55624a2fa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.073+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/4270716032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 msgr2=0x55624a2fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.073+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.073+0000 7fc2c9d9b640 1 -- 172.21.15.55:0/4270716032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55624a2104b0 con 0x55624a2ec0b0 2024-04-26T21:58:12.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.074+0000 7fc2c9d9b640 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2fa950 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00980e0 tx=0x7fc2c000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab37b22dfe49c81d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:12.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.082+0000 7fc2b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:12.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.082+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/4270716032 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0099e50 con 0x55624a2ec0b0 2024-04-26T21:58:12.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.082+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/4270716032 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2c000ed40 con 0x55624a2ec0b0 2024-04-26T21:58:12.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.082+0000 7fc2d361f740 1 -- 172.21.15.55:0/4270716032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c404c030 con 0x55624a2ec0b0 2024-04-26T21:58:12.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.083+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/4270716032 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2c009b670 con 0x55624a2ec0b0 2024-04-26T21:58:12.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.083+0000 7fc2d361f740 10 client.16224 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:12.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.083+0000 7fc2d361f740 1 -- 172.21.15.55:0/4270716032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55624a380f20 con 0x55624a2ec0b0 2024-04-26T21:58:12.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.083+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/4270716032 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c404c030 con 0x55624a2ec0b0 2024-04-26T21:58:12.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.084+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/4270716032 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc2c00cd5f0 con 0x55624a2ec0b0 2024-04-26T21:58:12.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.084+0000 7fc2d361f740 10 client.16224 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:12.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.084+0000 7fc2d361f740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:12.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.084+0000 7fc2d361f740 20 client.16224 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:12.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.084+0000 7fc2d361f740 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55624a381520 0x55624a383940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.084+0000 7fc2d361f740 4 client.16224 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:58:12.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.084+0000 7fc2d361f740 1 -- 172.21.15.55:0/4270716032 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55624a257800 con 0x55624a381520 2024-04-26T21:58:12.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.088+0000 7fc2ca59c640 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55624a381520 0x55624a383940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.101+0000 7fc2ca59c640 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55624a381520 0x55624a383940 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53aa20e83e39e7d server_cookie=43c54f610ea8e127 in_seq=0 out_seq=0 2024-04-26T21:58:12.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.101+0000 7fc2b1ffb640 10 client.16224 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:12.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.104+0000 7fc2b1ffb640 1 -- 172.21.15.55:0/4270716032 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55624a257800 con 0x55624a381520 2024-04-26T21:58:12.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.104+0000 7fc2b1ffb640 10 client.16224 handle_command_reply: tid=0 2024-04-26T21:58:12.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.104+0000 7fc2d361f740 1 client.16224 shutdown 2024-04-26T21:58:12.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.111+0000 7fc2d361f740 1 -- 172.21.15.55:0/4270716032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 msgr2=0x55624a2fa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:12.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.111+0000 7fc2d361f740 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2fa950 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00980e0 tx=0x7fc2c000bf90 comp rx=0 tx=0).stop 2024-04-26T21:58:12.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.113+0000 7fc2d361f740 1 -- 172.21.15.55:0/4270716032 shutdown_connections 2024-04-26T21:58:12.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.113+0000 7fc2d361f740 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55624a2eca70 0x55624a2fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.113+0000 7fc2d361f740 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55624a381520 0x55624a383940 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.113+0000 7fc2d361f740 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55624a2ec0b0 0x55624a2fa950 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.113+0000 7fc2d361f740 1 --2- 172.21.15.55:0/4270716032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55624a2e9fd0 0x55624a2fa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.113+0000 7fc2d361f740 1 -- 172.21.15.55:0/4270716032 >> 172.21.15.55:0/4270716032 conn(0x55624a2e63d0 msgr2=0x55624a2e4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:12.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.113+0000 7fc2b1ffb640 0 client.16224 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:12.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.114+0000 7fc2d361f740 1 -- 172.21.15.55:0/4270716032 shutdown_connections 2024-04-26T21:58:12.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.114+0000 7fc2d361f740 1 -- 172.21.15.55:0/4270716032 wait complete. 2024-04-26T21:58:12.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.118+0000 7fc2d361f740 20 client.16224 trim_cache size 0 max 16384 2024-04-26T21:58:12.126 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:58:12.126 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:58:12.126 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:58:12.126 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:58:12.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.129+0000 7fc2d0912640 1 -- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2b40a3d80 msgr2=0x7fc2b40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:12.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.129+0000 7fc2d0912640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2b40a3d80 0x7fc2b40a6240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4031b50 tx=0x7fc2c402a8c0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.129+0000 7fc2d0912640 1 -- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc102a60 msgr2=0x7fc2cc199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:12.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.129+0000 7fc2d0912640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc102a60 0x7fc2cc199280 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000b4a0 tx=0x7fc2c000b960 comp rx=0 tx=0).stop 2024-04-26T21:58:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.131+0000 7fc2d0912640 1 -- 172.21.15.55:0/4082145493 shutdown_connections 2024-04-26T21:58:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.131+0000 7fc2d0912640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2b40a3d80 0x7fc2b40a6240 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.131+0000 7fc2d0912640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2cc103420 0x7fc2cc1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.131+0000 7fc2d0912640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2cc102a60 0x7fc2cc199280 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.131+0000 7fc2d0912640 1 --2- 172.21.15.55:0/4082145493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc101870 0x7fc2cc198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.131+0000 7fc2d0912640 1 -- 172.21.15.55:0/4082145493 >> 172.21.15.55:0/4082145493 conn(0x7fc2cc0fce20 msgr2=0x7fc2cc105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.135+0000 7fc2d0912640 1 -- 172.21.15.55:0/4082145493 shutdown_connections 2024-04-26T21:58:12.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.138+0000 7fc2d0912640 1 -- 172.21.15.55:0/4082145493 wait complete. 2024-04-26T21:58:12.188 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:58:12.188 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:58:12.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 -- 172.21.15.55:0/2364662001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0106760 msgr2=0x7f68d010cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 --2- 172.21.15.55:0/2364662001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0106760 0x7f68d010cff0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009900 tx=0x7f68c002a3f0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 -- 172.21.15.55:0/2364662001 shutdown_connections 2024-04-26T21:58:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 --2- 172.21.15.55:0/2364662001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0106760 0x7f68d010cff0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 --2- 172.21.15.55:0/2364662001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d0105da0 0x7f68d0106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 --2- 172.21.15.55:0/2364662001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d0104bb0 0x7f68d0104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 -- 172.21.15.55:0/2364662001 >> 172.21.15.55:0/2364662001 conn(0x7f68d0100120 msgr2=0x7f68d0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 -- 172.21.15.55:0/2364662001 shutdown_connections 2024-04-26T21:58:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.398+0000 7f68d57f7640 1 -- 172.21.15.55:0/2364662001 wait complete. 2024-04-26T21:58:12.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.399+0000 7f68d57f7640 1 Processor -- start 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.399+0000 7f68d57f7640 1 -- start start 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68d57f7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0104bb0 0x7f68d019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68d57f7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d0105da0 0x7f68d019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68d57f7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d0106760 0x7f68d01a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68d57f7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f68d01107d0 con 0x7f68d0104bb0 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68d57f7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0110650 con 0x7f68d0105da0 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68d57f7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f68d0110950 con 0x7f68d0106760 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68ceffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0104bb0 0x7f68d019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68ceffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0104bb0 0x7f68d019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49654/0 (socket says 172.21.15.55:49654) 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68ceffd640 1 -- 172.21.15.55:0/3370437948 learned_addr learned my addr 172.21.15.55:0/3370437948 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68cf7fe640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d0106760 0x7f68d01a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68ce7fc640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d0105da0 0x7f68d019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68ceffd640 1 -- 172.21.15.55:0/3370437948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d0106760 msgr2=0x7f68d01a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.400+0000 7f68ceffd640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d0106760 0x7f68d01a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68ceffd640 1 -- 172.21.15.55:0/3370437948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d0105da0 msgr2=0x7f68d019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68ceffd640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d0105da0 0x7f68d019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68ceffd640 1 -- 172.21.15.55:0/3370437948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d01a2a60 con 0x7f68d0104bb0 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68ce7fc640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d0105da0 0x7f68d019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68cf7fe640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d0106760 0x7f68d01a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68ceffd640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0104bb0 0x7f68d019ae40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b450 tx=0x7f68c800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68c7fff640 1 -- 172.21.15.55:0/3370437948 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8015020 con 0x7f68d0104bb0 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68c7fff640 1 -- 172.21.15.55:0/3370437948 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c8011d40 con 0x7f68d0104bb0 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d01a2d50 con 0x7f68d0104bb0 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68c7fff640 1 -- 172.21.15.55:0/3370437948 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c800f340 con 0x7f68d0104bb0 2024-04-26T21:58:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.401+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d01a3290 con 0x7f68d0104bb0 2024-04-26T21:58:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.403+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d010bed0 con 0x7f68d0104bb0 2024-04-26T21:58:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.404+0000 7f68c7fff640 1 -- 172.21.15.55:0/3370437948 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f68c8018020 con 0x7f68d0104bb0 2024-04-26T21:58:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.404+0000 7f68c7fff640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f68bc0a3cc0 0x7f68bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.405+0000 7f68c7fff640 1 -- 172.21.15.55:0/3370437948 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f68c80cff70 con 0x7f68d0104bb0 2024-04-26T21:58:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.405+0000 7f68ce7fc640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f68bc0a3cc0 0x7f68bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.405+0000 7f68ce7fc640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f68bc0a3cc0 0x7f68bc0a6180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f68d019c350 tx=0x7f68b80093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:12.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:12.411+0000 7f68c7fff640 1 -- 172.21.15.55:0/3370437948 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f68c801f020 con 0x7f68d0104bb0 2024-04-26T21:58:13.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.028+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68d002dc80 con 0x7f68d0104bb0 2024-04-26T21:58:13.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.034+0000 7f68c7fff640 1 -- 172.21.15.55:0/3370437948 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f68c800d040 con 0x7f68d0104bb0 2024-04-26T21:58:13.042 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:58:13.042 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:58:13.042 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T21:58:13.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f68bc0a3cc0 msgr2=0x7f68bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f68bc0a3cc0 0x7f68bc0a6180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f68d019c350 tx=0x7f68b80093e0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0104bb0 msgr2=0x7f68d019ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0104bb0 0x7f68d019ae40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b450 tx=0x7f68c800b910 comp rx=0 tx=0).stop 2024-04-26T21:58:13.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 shutdown_connections 2024-04-26T21:58:13.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f68bc0a3cc0 0x7f68bc0a6180 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f68d019c350 tx=0x7f68b80093e0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68d0106760 0x7f68d01a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68d0105da0 0x7f68d019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 --2- 172.21.15.55:0/3370437948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d0104bb0 0x7f68d019ae40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.055+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 >> 172.21.15.55:0/3370437948 conn(0x7f68d0100120 msgr2=0x7f68d01090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:13.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.068+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 shutdown_connections 2024-04-26T21:58:13.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.068+0000 7f68d57f7640 1 -- 172.21.15.55:0/3370437948 wait complete. 2024-04-26T21:58:13.137 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:58:13.140 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:12 smithi055 ceph-mon[29221]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.42k op/s 2024-04-26T21:58:13.140 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:12 smithi055 ceph-mon[29221]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:58:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:12 smithi064 ceph-mon[32765]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.42k op/s 2024-04-26T21:58:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:12 smithi064 ceph-mon[32765]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:58:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:12 smithi165 ceph-mon[33296]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.42k op/s 2024-04-26T21:58:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:12 smithi165 ceph-mon[33296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.413+0000 7f23a7037640 1 -- 172.21.15.55:0/3074855511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 msgr2=0x7f23a010adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.413+0000 7f23a7037640 1 --2- 172.21.15.55:0/3074855511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 0x7f23a010adc0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2398009ca0 tx=0x7f239802a8e0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 -- 172.21.15.55:0/3074855511 shutdown_connections 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 --2- 172.21.15.55:0/3074855511 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 0x7f23a010adc0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 --2- 172.21.15.55:0/3074855511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23a0103b70 0x7f23a0103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 --2- 172.21.15.55:0/3074855511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23a0102980 0x7f23a0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 -- 172.21.15.55:0/3074855511 >> 172.21.15.55:0/3074855511 conn(0x7f23a00fdf10 msgr2=0x7f23a0100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 -- 172.21.15.55:0/3074855511 shutdown_connections 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 -- 172.21.15.55:0/3074855511 wait complete. 2024-04-26T21:58:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 Processor -- start 2024-04-26T21:58:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.414+0000 7f23a7037640 1 -- start start 2024-04-26T21:58:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a7037640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23a0102980 0x7f23a019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a7037640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23a0103b70 0x7f23a019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a7037640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 0x7f23a01a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a7037640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f23a010e530 con 0x7f23a0104530 2024-04-26T21:58:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a7037640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23a010e3b0 con 0x7f23a0102980 2024-04-26T21:58:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a7037640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23a010e6b0 con 0x7f23a0103b70 2024-04-26T21:58:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a55d4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 0x7f23a01a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a55d4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 0x7f23a01a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49666/0 (socket says 172.21.15.55:49666) 2024-04-26T21:58:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a55d4640 1 -- 172.21.15.55:0/150124683 learned_addr learned my addr 172.21.15.55:0/150124683 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f239ffff640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23a0103b70 0x7f23a019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a55d4640 1 -- 172.21.15.55:0/150124683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23a0103b70 msgr2=0x7f23a019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a55d4640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23a0103b70 0x7f23a019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a55d4640 1 -- 172.21.15.55:0/150124683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23a0102980 msgr2=0x7f23a019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a55d4640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23a0102980 0x7f23a019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.415+0000 7f23a55d4640 1 -- 172.21.15.55:0/150124683 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a01a2a10 con 0x7f23a0104530 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.416+0000 7f23a55d4640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 0x7f23a01a23a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2398005e30 tx=0x7f2398003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.416+0000 7f239dffb640 1 -- 172.21.15.55:0/150124683 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398004240 con 0x7f23a0104530 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.416+0000 7f239dffb640 1 -- 172.21.15.55:0/150124683 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2398003e10 con 0x7f23a0104530 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.416+0000 7f23a7037640 1 -- 172.21.15.55:0/150124683 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a01a2ca0 con 0x7f23a0104530 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.416+0000 7f239dffb640 1 -- 172.21.15.55:0/150124683 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239803a400 con 0x7f23a0104530 2024-04-26T21:58:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.417+0000 7f23a7037640 1 -- 172.21.15.55:0/150124683 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a01a31a0 con 0x7f23a0104530 2024-04-26T21:58:13.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.422+0000 7f23a9e2b740 1 Processor -- start 2024-04-26T21:58:13.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.423+0000 7f239dffb640 1 -- 172.21.15.55:0/150124683 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f239803a5a0 con 0x7f23a0104530 2024-04-26T21:58:13.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.423+0000 7f23a9e2b740 1 -- start start 2024-04-26T21:58:13.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.423+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad0820 0x564315ad0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.423+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad19f0 0x564315ad1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f23a4dd3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad19f0 0x564315ad1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f239dffb640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23900a3c40 0x7f23900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f23a4dd3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad19f0 0x564315ad1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49678/0 (socket says 172.21.15.55:49678) 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f23a4dd3640 1 -- 172.21.15.55:0/2994974374 learned_addr learned my addr 172.21.15.55:0/2994974374 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad23b0 0x564315ad8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f239ffff640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23900a3c40 0x7f23900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f239dffb640 1 -- 172.21.15.55:0/150124683 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f23980f1a60 con 0x7f23a0104530 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2994974374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564315a34d80 con 0x564315ad19f0 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2994974374 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643159eea20 con 0x564315ad0820 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2994974374 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564315a30d40 con 0x564315ad23b0 2024-04-26T21:58:13.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.424+0000 7f23a55d4640 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad23b0 0x564315ad8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.425+0000 7f23a4dd3640 1 -- 172.21.15.55:0/2994974374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad23b0 msgr2=0x564315ad8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.425+0000 7f23a4dd3640 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad23b0 0x564315ad8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.425+0000 7f23a4dd3640 1 -- 172.21.15.55:0/2994974374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad0820 msgr2=0x564315ad0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.425+0000 7f23a4dd3640 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad0820 0x564315ad0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.425+0000 7f23a4dd3640 1 -- 172.21.15.55:0/2994974374 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643159f9c30 con 0x564315ad19f0 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.425+0000 7f239ffff640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23900a3c40 0x7f23900a6100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f238c006dc0 tx=0x7f238c006d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.425+0000 7f23a4dd3640 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad19f0 0x564315ad1e70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f239400bb70 tx=0x7f239400d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20e167b2d96f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.426+0000 7f238affd640 1 -- 172.21.15.55:0/2994974374 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2394016020 con 0x564315ad19f0 2024-04-26T21:58:13.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.426+0000 7f238affd640 1 -- 172.21.15.55:0/2994974374 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2394010910 con 0x564315ad19f0 2024-04-26T21:58:13.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.427+0000 7f238affd640 1 -- 172.21.15.55:0/2994974374 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239400f6e0 con 0x564315ad19f0 2024-04-26T21:58:13.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.427+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2994974374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad19f0 msgr2=0x564315ad1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.427+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad19f0 0x564315ad1e70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f239400bb70 tx=0x7f239400d6b0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.427+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2994974374 shutdown_connections 2024-04-26T21:58:13.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.428+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad23b0 0x564315ad8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.428+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad19f0 0x564315ad1e70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.428+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2994974374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad0820 0x564315ad0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.428+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2994974374 >> 172.21.15.55:0/2994974374 conn(0x564315a11910 msgr2=0x564315acc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:13.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.428+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2994974374 shutdown_connections 2024-04-26T21:58:13.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.428+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2994974374 wait complete. 2024-04-26T21:58:13.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.429+0000 7f23a9e2b740 1 -- start start 2024-04-26T21:58:13.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.429+0000 7f23a9e2b740 1 Processor -- start 2024-04-26T21:58:13.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.430+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad0820 0x564315adaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.430+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315adb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.430+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad23b0 0x564315ae2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564315ad9460 con 0x564315ad0820 2024-04-26T21:58:13.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643159bbdc0 con 0x564315ad23b0 2024-04-26T21:58:13.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564315ad95e0 con 0x564315ad19f0 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f23a9e2b740 10 client.0 _ll_register_callbacks cb 0x5643159d7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f239ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315adb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f239ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315adb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33566/0 (socket says 172.21.15.55:33566) 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f239ffff640 1 -- 172.21.15.55:0/3886336934 learned_addr learned my addr 172.21.15.55:0/3886336934 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f23a55d4640 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad23b0 0x564315ae2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f239ffff640 1 -- 172.21.15.55:0/3886336934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad23b0 msgr2=0x564315ae2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f239ffff640 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad23b0 0x564315ae2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f239ffff640 1 -- 172.21.15.55:0/3886336934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad0820 msgr2=0x564315adaac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f239ffff640 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad0820 0x564315adaac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.431+0000 7f239ffff640 1 -- 172.21.15.55:0/3886336934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643157463f0 con 0x564315ad19f0 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.432+0000 7f239ffff640 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315adb000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f238c014760 tx=0x7f238c0046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f1812536e6e51f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:13.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.432+0000 7f237ffff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:58:13.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.432+0000 7f237ffff640 1 -- 172.21.15.55:0/3886336934 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c022070 con 0x564315ad19f0 2024-04-26T21:58:13.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.432+0000 7f237ffff640 1 -- 172.21.15.55:0/3886336934 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f238c0137a0 con 0x564315ad19f0 2024-04-26T21:58:13.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.432+0000 7f23a9e2b740 1 -- 172.21.15.55:0/3886336934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2394010910 con 0x564315ad19f0 2024-04-26T21:58:13.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.432+0000 7f237ffff640 1 -- 172.21.15.55:0/3886336934 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f238c02e050 con 0x564315ad19f0 2024-04-26T21:58:13.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.432+0000 7f23a9e2b740 10 client.25582 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:13.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.432+0000 7f23a9e2b740 1 -- 172.21.15.55:0/3886336934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5643158107d0 con 0x564315ad19f0 2024-04-26T21:58:13.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.433+0000 7f237ffff640 1 -- 172.21.15.55:0/3886336934 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2394010910 con 0x564315ad19f0 2024-04-26T21:58:13.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.433+0000 7f237ffff640 1 -- 172.21.15.55:0/3886336934 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f238c01d070 con 0x564315ad19f0 2024-04-26T21:58:13.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.433+0000 7f23a9e2b740 10 client.25582 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:13.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.433+0000 7f23a9e2b740 10 client.25582 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:58:13.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.433+0000 7f23a9e2b740 20 client.25582 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:13.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.433+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564315ae69e0 0x564315ae8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.433+0000 7f23a9e2b740 4 client.25582 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:13.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.433+0000 7f23a9e2b740 1 -- 172.21.15.55:0/3886336934 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564315a42990 con 0x564315ae69e0 2024-04-26T21:58:13.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.434+0000 7f23a4dd3640 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564315ae69e0 0x564315ae8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.434+0000 7f237ffff640 10 client.25582 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:58:13.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.434+0000 7f23a4dd3640 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564315ae69e0 0x564315ae8e70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97174fa19a7f0fb5 server_cookie=8ca6d03984c7f8f0 in_seq=0 out_seq=0 2024-04-26T21:58:13.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.435+0000 7f237ffff640 1 -- 172.21.15.55:0/3886336934 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564315a42990 con 0x564315ae69e0 2024-04-26T21:58:13.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.435+0000 7f237ffff640 10 client.25582 handle_command_reply: tid=0 2024-04-26T21:58:13.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.435+0000 7f23a9e2b740 1 client.25582 shutdown 2024-04-26T21:58:13.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.435+0000 7f23a9e2b740 1 -- 172.21.15.55:0/3886336934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 msgr2=0x564315adb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.435+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315adb000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f238c014760 tx=0x7f238c0046e0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f23a9e2b740 1 -- 172.21.15.55:0/3886336934 shutdown_connections 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564315ae69e0 0x564315ae8e70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad23b0 0x564315ae2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315adb000 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/3886336934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad0820 0x564315adaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f23a9e2b740 1 -- 172.21.15.55:0/3886336934 >> 172.21.15.55:0/3886336934 conn(0x564315a11910 msgr2=0x564315acb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f237ffff640 0 client.25582 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f23a9e2b740 1 -- 172.21.15.55:0/3886336934 shutdown_connections 2024-04-26T21:58:13.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.436+0000 7f23a9e2b740 1 -- 172.21.15.55:0/3886336934 wait complete. 2024-04-26T21:58:13.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.437+0000 7f23a9e2b740 20 client.25582 trim_cache size 0 max 16384 2024-04-26T21:58:13.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.494+0000 7f23a9e2b740 1 Processor -- start 2024-04-26T21:58:13.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.505+0000 7f23a9e2b740 1 -- start start 2024-04-26T21:58:13.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.506+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad0820 0x564315ad5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.506+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315ad56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.506+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315ae2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.506+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01ae9e0 con 0x564315ad23b0 2024-04-26T21:58:13.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.506+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01ac2b0 con 0x564315ad0820 2024-04-26T21:58:13.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.506+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01aa100 con 0x564315ad19f0 2024-04-26T21:58:13.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.508+0000 7f23a55d4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315ae2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.508+0000 7f23a55d4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315ae2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49688/0 (socket says 172.21.15.55:49688) 2024-04-26T21:58:13.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.508+0000 7f23a55d4640 1 -- 172.21.15.55:0/2423832600 learned_addr learned my addr 172.21.15.55:0/2423832600 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:13.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.510+0000 7f239ffff640 1 --2- 172.21.15.55:0/2423832600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315ad56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.515+0000 7f23a55d4640 1 -- 172.21.15.55:0/2423832600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 msgr2=0x564315ad56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.515+0000 7f23a55d4640 1 --2- 172.21.15.55:0/2423832600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315ad56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.515+0000 7f23a55d4640 1 -- 172.21.15.55:0/2423832600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad0820 msgr2=0x564315ad5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.515+0000 7f23a55d4640 1 --2- 172.21.15.55:0/2423832600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad0820 0x564315ad5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.515+0000 7f23a55d4640 1 -- 172.21.15.55:0/2423832600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564315ac7ad0 con 0x564315ad23b0 2024-04-26T21:58:13.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.520+0000 7f23a55d4640 1 --2- 172.21.15.55:0/2423832600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315ae2000 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2398009430 tx=0x7f23980de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793fd720378b9583 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:13.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.524+0000 7f238a7fc640 1 -- 172.21.15.55:0/2423832600 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980098b0 con 0x564315ad23b0 2024-04-26T21:58:13.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.524+0000 7f238a7fc640 1 -- 172.21.15.55:0/2423832600 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2398036040 con 0x564315ad23b0 2024-04-26T21:58:13.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.524+0000 7f238a7fc640 1 -- 172.21.15.55:0/2423832600 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980ef5c0 con 0x564315ad23b0 2024-04-26T21:58:13.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.526+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2423832600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 msgr2=0x564315ae2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.526+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2423832600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315ae2000 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2398009430 tx=0x7f23980de040 comp rx=0 tx=0).stop 2024-04-26T21:58:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.528+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2423832600 shutdown_connections 2024-04-26T21:58:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.528+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2423832600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315ae2000 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.528+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2423832600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad19f0 0x564315ad56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.528+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/2423832600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad0820 0x564315ad5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.528+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2423832600 >> 172.21.15.55:0/2423832600 conn(0x564315a11910 msgr2=0x564315adaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:13.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.535+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2423832600 shutdown_connections 2024-04-26T21:58:13.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.537+0000 7f23a9e2b740 1 -- 172.21.15.55:0/2423832600 wait complete. 2024-04-26T21:58:13.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.538+0000 7f23a9e2b740 1 -- start start 2024-04-26T21:58:13.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.539+0000 7f23a9e2b740 1 Processor -- start 2024-04-26T21:58:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.543+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad0820 0x564315aded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.543+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad19f0 0x564315adf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.543+0000 7f23a9e2b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315b537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.543+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01aa770 con 0x564315ad23b0 2024-04-26T21:58:13.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.543+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01abbe0 con 0x564315ad19f0 2024-04-26T21:58:13.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.543+0000 7f23a9e2b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f23a01aae40 con 0x564315ad0820 2024-04-26T21:58:13.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.543+0000 7f23a9e2b740 10 client.0 _ll_register_callbacks cb 0x7f238c014430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:13.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.545+0000 7f23a55d4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315b537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.545+0000 7f23a55d4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315b537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49698/0 (socket says 172.21.15.55:49698) 2024-04-26T21:58:13.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.545+0000 7f23a55d4640 1 -- 172.21.15.55:0/244323245 learned_addr learned my addr 172.21.15.55:0/244323245 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:13.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.545+0000 7f239ffff640 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad19f0 0x564315adf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.545+0000 7f23a4dd3640 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad0820 0x564315aded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.549+0000 7f23a55d4640 1 -- 172.21.15.55:0/244323245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad0820 msgr2=0x564315aded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:13.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.549+0000 7f23a55d4640 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad0820 0x564315aded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.549+0000 7f23a55d4640 1 -- 172.21.15.55:0/244323245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad19f0 msgr2=0x564315adf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:13.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.549+0000 7f23a55d4640 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad19f0 0x564315adf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.549+0000 7f23a55d4640 1 -- 172.21.15.55:0/244323245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564315adf910 con 0x564315ad23b0 2024-04-26T21:58:13.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.549+0000 7f239ffff640 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad19f0 0x564315adf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:13.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.554+0000 7f23a55d4640 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315b537b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f23980bde10 tx=0x7f23980ba3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d072569d43630855 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:13.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.554+0000 7f237ffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:13.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.557+0000 7f237ffff640 1 -- 172.21.15.55:0/244323245 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980c68c0 con 0x564315ad23b0 2024-04-26T21:58:13.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.557+0000 7f237ffff640 1 -- 172.21.15.55:0/244323245 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f23980ba780 con 0x564315ad23b0 2024-04-26T21:58:13.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.559+0000 7f23a9e2b740 1 -- 172.21.15.55:0/244323245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2398036040 con 0x564315ad23b0 2024-04-26T21:58:13.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.565+0000 7f237ffff640 1 -- 172.21.15.55:0/244323245 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f23980098b0 con 0x564315ad23b0 2024-04-26T21:58:13.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.569+0000 7f23a9e2b740 10 client.16245 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:13.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.570+0000 7f23a9e2b740 1 -- 172.21.15.55:0/244323245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564315b48070 con 0x564315ad23b0 2024-04-26T21:58:13.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.576+0000 7f237ffff640 1 -- 172.21.15.55:0/244323245 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2398036040 con 0x564315ad23b0 2024-04-26T21:58:13.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.576+0000 7f237ffff640 1 -- 172.21.15.55:0/244323245 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f23980bd560 con 0x564315ad23b0 2024-04-26T21:58:13.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.579+0000 7f23a9e2b740 10 client.16245 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:13.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.579+0000 7f23a9e2b740 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:58:13.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.579+0000 7f23a9e2b740 20 client.16245 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:13.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.579+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564315b48740 0x564315b4ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:13.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.579+0000 7f23a9e2b740 4 client.16245 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:13.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.579+0000 7f23a9e2b740 1 -- 172.21.15.55:0/244323245 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564315ada8b0 con 0x564315b48740 2024-04-26T21:58:13.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.584+0000 7f23a4dd3640 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564315b48740 0x564315b4ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.609+0000 7f23a4dd3640 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564315b48740 0x564315b4ab60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46d976a14291fccf server_cookie=f331704898c35a0c in_seq=0 out_seq=0 2024-04-26T21:58:13.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.609+0000 7f237ffff640 10 client.16245 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:58:13.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.612+0000 7f237ffff640 1 -- 172.21.15.55:0/244323245 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564315ada8b0 con 0x564315b48740 2024-04-26T21:58:13.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.612+0000 7f237ffff640 10 client.16245 handle_command_reply: tid=0 2024-04-26T21:58:13.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.617+0000 7f23a9e2b740 1 client.16245 shutdown 2024-04-26T21:58:13.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.626+0000 7f23a9e2b740 1 -- 172.21.15.55:0/244323245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 msgr2=0x564315b537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.626+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315b537b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f23980bde10 tx=0x7f23980ba3f0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.630+0000 7f23a9e2b740 1 -- 172.21.15.55:0/244323245 shutdown_connections 2024-04-26T21:58:13.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.630+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x564315b48740 0x564315b4ab60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.630+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564315ad23b0 0x564315b537b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.630+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564315ad19f0 0x564315adf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.630+0000 7f23a9e2b740 1 --2- 172.21.15.55:0/244323245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564315ad0820 0x564315aded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.630+0000 7f23a9e2b740 1 -- 172.21.15.55:0/244323245 >> 172.21.15.55:0/244323245 conn(0x564315a11910 msgr2=0x564315adaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:13.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.630+0000 7f237ffff640 0 client.16245 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:58:13.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.635+0000 7f23a9e2b740 1 -- 172.21.15.55:0/244323245 shutdown_connections 2024-04-26T21:58:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.638+0000 7f23a9e2b740 1 -- 172.21.15.55:0/244323245 wait complete. 2024-04-26T21:58:13.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.641+0000 7f23a9e2b740 20 client.16245 trim_cache size 0 max 16384 2024-04-26T21:58:13.646 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:58:13.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.653+0000 7f23a7037640 1 -- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23900a3c40 msgr2=0x7f23900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.653+0000 7f23a7037640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23900a3c40 0x7f23900a6100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f238c006dc0 tx=0x7f238c006d50 comp rx=0 tx=0).stop 2024-04-26T21:58:13.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.653+0000 7f23a7037640 1 -- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 msgr2=0x7f23a01a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:13.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.653+0000 7f23a7037640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 0x7f23a01a23a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2398005e30 tx=0x7f2398003c20 comp rx=0 tx=0).stop 2024-04-26T21:58:13.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.658+0000 7f23a7037640 1 -- 172.21.15.55:0/150124683 shutdown_connections 2024-04-26T21:58:13.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.658+0000 7f23a7037640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f23900a3c40 0x7f23900a6100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.659+0000 7f23a7037640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23a0104530 0x7f23a01a23a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.659+0000 7f23a7037640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23a0103b70 0x7f23a019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.659+0000 7f23a7037640 1 --2- 172.21.15.55:0/150124683 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23a0102980 0x7f23a019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:13.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.659+0000 7f23a7037640 1 -- 172.21.15.55:0/150124683 >> 172.21.15.55:0/150124683 conn(0x7f23a00fdf10 msgr2=0x7f23a0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:13.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.665+0000 7f23a7037640 1 -- 172.21.15.55:0/150124683 shutdown_connections 2024-04-26T21:58:13.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:13.668+0000 7f23a7037640 1 -- 172.21.15.55:0/150124683 wait complete. 2024-04-26T21:58:13.707 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:58:14.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.038+0000 7fc74e54d640 1 -- 172.21.15.55:0/119700559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc74810a010 msgr2=0x7fc74810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.038+0000 7fc74e54d640 1 --2- 172.21.15.55:0/119700559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc74810a010 0x7fc74810a470 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc74401a910 tx=0x7fc74403b450 comp rx=0 tx=0).stop 2024-04-26T21:58:14.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.046+0000 7fc74e54d640 1 -- 172.21.15.55:0/119700559 shutdown_connections 2024-04-26T21:58:14.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.046+0000 7fc74e54d640 1 --2- 172.21.15.55:0/119700559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc74810a9b0 0x7fc74810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.046+0000 7fc74e54d640 1 --2- 172.21.15.55:0/119700559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc74810a010 0x7fc74810a470 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.046+0000 7fc74e54d640 1 --2- 172.21.15.55:0/119700559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc748073d70 0x7fc748074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.046+0000 7fc74e54d640 1 -- 172.21.15.55:0/119700559 >> 172.21.15.55:0/119700559 conn(0x7fc74806f840 msgr2=0x7fc748071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.053+0000 7fc74e54d640 1 -- 172.21.15.55:0/119700559 shutdown_connections 2024-04-26T21:58:14.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.055+0000 7fc74e54d640 1 -- 172.21.15.55:0/119700559 wait complete. 2024-04-26T21:58:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.056+0000 7fc74e54d640 1 Processor -- start 2024-04-26T21:58:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.057+0000 7fc74e54d640 1 -- start start 2024-04-26T21:58:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.057+0000 7fc74e54d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc748073d70 0x7fc74819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.057+0000 7fc74e54d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc74810a010 0x7fc74819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.057+0000 7fc74e54d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc74810a9b0 0x7fc7481a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.057+0000 7fc74e54d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc748112b20 con 0x7fc74810a010 2024-04-26T21:58:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.057+0000 7fc74e54d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7481129a0 con 0x7fc74810a9b0 2024-04-26T21:58:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.057+0000 7fc74e54d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc748112ca0 con 0x7fc748073d70 2024-04-26T21:58:14.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.058+0000 7fc74dd4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc74810a9b0 0x7fc7481a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.058+0000 7fc74dd4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc74810a9b0 0x7fc7481a66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:55348/0 (socket says 172.21.15.55:55348) 2024-04-26T21:58:14.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.058+0000 7fc74dd4c640 1 -- 172.21.15.55:0/4067478698 learned_addr learned my addr 172.21.15.55:0/4067478698 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.063+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc74810a010 0x7fc74819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.065+0000 7fc74d54b640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc748073d70 0x7fc74819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.069+0000 7fc74dd4c640 1 -- 172.21.15.55:0/4067478698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc748073d70 msgr2=0x7fc74819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.069+0000 7fc74dd4c640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc748073d70 0x7fc74819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.069+0000 7fc74dd4c640 1 -- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc74810a010 msgr2=0x7fc74819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.069+0000 7fc74dd4c640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc74810a010 0x7fc74819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.069+0000 7fc74dd4c640 1 -- 172.21.15.55:0/4067478698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7481a6d10 con 0x7fc74810a9b0 2024-04-26T21:58:14.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.074+0000 7fc74dd4c640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc74810a9b0 0x7fc7481a66a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc74000cad0 tx=0x7fc74000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.076+0000 7fc73e7fc640 1 -- 172.21.15.55:0/4067478698 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74000ae20 con 0x7fc74810a9b0 2024-04-26T21:58:14.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.076+0000 7fc73e7fc640 1 -- 172.21.15.55:0/4067478698 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc740005020 con 0x7fc74810a9b0 2024-04-26T21:58:14.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.076+0000 7fc73e7fc640 1 -- 172.21.15.55:0/4067478698 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74000f6b0 con 0x7fc74810a9b0 2024-04-26T21:58:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.077+0000 7fc74e54d640 1 -- 172.21.15.55:0/4067478698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7481a6fa0 con 0x7fc74810a9b0 2024-04-26T21:58:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.077+0000 7fc74e54d640 1 -- 172.21.15.55:0/4067478698 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7481a74f0 con 0x7fc74810a9b0 2024-04-26T21:58:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.089+0000 7fc73e7fc640 1 -- 172.21.15.55:0/4067478698 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc74000f850 con 0x7fc74810a9b0 2024-04-26T21:58:14.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.091+0000 7fc73e7fc640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7340a3e10 0x7fc7340a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.091+0000 7fc73e7fc640 1 -- 172.21.15.55:0/4067478698 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc7400d03f0 con 0x7fc74810a9b0 2024-04-26T21:58:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.092+0000 7fc7525a6740 1 Processor -- start 2024-04-26T21:58:14.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.094+0000 7fc7525a6740 1 -- start start 2024-04-26T21:58:14.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.094+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216ea00 0x55b43216ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.094+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b432170050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.094+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 0x55b432176e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.094+0000 7fc7525a6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b4320d2d80 con 0x55b43216fbd0 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.094+0000 7fc7525a6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b43208ca20 con 0x55b43216ea00 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.094+0000 7fc7525a6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b4320ced40 con 0x55b432170590 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.096+0000 7fc74d54b640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7340a3e10 0x7fc7340a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.097+0000 7fc74cd4a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b432170050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.098+0000 7fc74cd4a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b432170050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49730/0 (socket says 172.21.15.55:49730) 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.098+0000 7fc74cd4a640 1 -- 172.21.15.55:0/1047796774 learned_addr learned my addr 172.21.15.55:0/1047796774 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.098+0000 7fc74dd4c640 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 0x55b432176e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.098+0000 7fc74d54b640 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216ea00 0x55b43216ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.102+0000 7fc74cd4a640 1 -- 172.21.15.55:0/1047796774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 msgr2=0x55b432176e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.102+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 0x55b432176e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.102+0000 7fc74cd4a640 1 -- 172.21.15.55:0/1047796774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216ea00 msgr2=0x55b43216ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.102+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216ea00 0x55b43216ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.103+0000 7fc74cd4a640 1 -- 172.21.15.55:0/1047796774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b432097c30 con 0x55b43216fbd0 2024-04-26T21:58:14.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.105+0000 7fc74d54b640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7340a3e10 0x7fc7340a62d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc738006fd0 tx=0x7fc738009f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.108+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b432170050 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc74401a910 tx=0x7fc7440047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324879019edf75cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.114+0000 7fc7337fe640 1 -- 172.21.15.55:0/1047796774 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744046030 con 0x55b43216fbd0 2024-04-26T21:58:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.114+0000 7fc7337fe640 1 -- 172.21.15.55:0/1047796774 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc744004aa0 con 0x55b43216fbd0 2024-04-26T21:58:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.115+0000 7fc7525a6740 1 -- 172.21.15.55:0/1047796774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 msgr2=0x55b432170050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.115+0000 7fc7525a6740 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b432170050 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc74401a910 tx=0x7fc7440047f0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.117+0000 7fc7525a6740 1 -- 172.21.15.55:0/1047796774 shutdown_connections 2024-04-26T21:58:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.117+0000 7fc7525a6740 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 0x55b432176e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.117+0000 7fc7525a6740 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b432170050 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.117+0000 7fc7525a6740 1 --2- 172.21.15.55:0/1047796774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216ea00 0x55b43216ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.117+0000 7fc7525a6740 1 -- 172.21.15.55:0/1047796774 >> 172.21.15.55:0/1047796774 conn(0x55b4320af910 msgr2=0x55b43216a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.129+0000 7fc7525a6740 1 -- 172.21.15.55:0/1047796774 shutdown_connections 2024-04-26T21:58:14.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.133+0000 7fc7525a6740 1 -- 172.21.15.55:0/1047796774 wait complete. 2024-04-26T21:58:14.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.133+0000 7fc7525a6740 1 -- start start 2024-04-26T21:58:14.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.133+0000 7fc7525a6740 1 Processor -- start 2024-04-26T21:58:14.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.134+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b432178e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.134+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b4321793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.134+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b432170590 0x55b432180410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.134+0000 7fc7525a6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55b432177590 con 0x55b43216fbd0 2024-04-26T21:58:14.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.134+0000 7fc7525a6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b432059dc0 con 0x55b432170590 2024-04-26T21:58:14.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.134+0000 7fc7525a6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b432177710 con 0x55b43216ea00 2024-04-26T21:58:14.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.134+0000 7fc7525a6740 10 client.0 _ll_register_callbacks cb 0x55b432075600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:14.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.135+0000 7fc74cd4a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b4321793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.135+0000 7fc74cd4a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b4321793d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49736/0 (socket says 172.21.15.55:49736) 2024-04-26T21:58:14.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.135+0000 7fc74cd4a640 1 -- 172.21.15.55:0/933536706 learned_addr learned my addr 172.21.15.55:0/933536706 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.136+0000 7fc74dd4c640 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b432170590 0x55b432180410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.138+0000 7fc74cd4a640 1 -- 172.21.15.55:0/933536706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 msgr2=0x55b432178e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:14.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.138+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b432178e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.138+0000 7fc74cd4a640 1 -- 172.21.15.55:0/933536706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b432170590 msgr2=0x55b432180410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.138+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b432170590 0x55b432180410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.138+0000 7fc74cd4a640 1 -- 172.21.15.55:0/933536706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b431de43f0 con 0x55b43216fbd0 2024-04-26T21:58:14.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.139+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b4321793d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc74403ba40 tx=0x7fc744005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bc36c3026e4866 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.150+0000 7fc730ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:14.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.150+0000 7fc730ff9640 1 -- 172.21.15.55:0/933536706 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74405c020 con 0x55b43216fbd0 2024-04-26T21:58:14.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.151+0000 7fc7525a6740 1 -- 172.21.15.55:0/933536706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b4320e0990 con 0x55b43216fbd0 2024-04-26T21:58:14.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.151+0000 7fc730ff9640 1 -- 172.21.15.55:0/933536706 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc744040510 con 0x55b43216fbd0 2024-04-26T21:58:14.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.151+0000 7fc730ff9640 1 -- 172.21.15.55:0/933536706 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc744046070 con 0x55b43216fbd0 2024-04-26T21:58:14.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.156+0000 7fc7525a6740 10 client.16257 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:14.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.156+0000 7fc7525a6740 1 -- 172.21.15.55:0/933536706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b431eae7d0 con 0x55b43216fbd0 2024-04-26T21:58:14.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.157+0000 7fc730ff9640 1 -- 172.21.15.55:0/933536706 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b4320e0990 con 0x55b43216fbd0 2024-04-26T21:58:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.158+0000 7fc730ff9640 1 -- 172.21.15.55:0/933536706 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc74404d6e0 con 0x55b43216fbd0 2024-04-26T21:58:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.158+0000 7fc7525a6740 10 client.16257 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.158+0000 7fc7525a6740 10 client.16257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:58:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.158+0000 7fc7525a6740 20 client.16257 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.158+0000 7fc7525a6740 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55b432184f20 0x55b4321873c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.158+0000 7fc7525a6740 4 client.16257 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.158+0000 7fc7525a6740 1 -- 172.21.15.55:0/933536706 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4320da800 con 0x55b432184f20 2024-04-26T21:58:14.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.159+0000 7fc74d54b640 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55b432184f20 0x55b4321873c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.161+0000 7fc74d54b640 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55b432184f20 0x55b4321873c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3dd4a000ad578603 server_cookie=d28ddcea751143c0 in_seq=0 out_seq=0 2024-04-26T21:58:14.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.162+0000 7fc730ff9640 10 client.16257 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:58:14.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.164+0000 7fc730ff9640 1 -- 172.21.15.55:0/933536706 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4320da800 con 0x55b432184f20 2024-04-26T21:58:14.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.164+0000 7fc730ff9640 10 client.16257 handle_command_reply: tid=0 2024-04-26T21:58:14.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.164+0000 7fc7525a6740 1 client.16257 shutdown 2024-04-26T21:58:14.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.168+0000 7fc7525a6740 1 -- 172.21.15.55:0/933536706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 msgr2=0x55b4321793d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.168+0000 7fc7525a6740 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b4321793d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc74403ba40 tx=0x7fc744005970 comp rx=0 tx=0).stop 2024-04-26T21:58:14.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.179+0000 7fc7525a6740 1 -- 172.21.15.55:0/933536706 shutdown_connections 2024-04-26T21:58:14.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.179+0000 7fc7525a6740 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55b432184f20 0x55b4321873c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.179+0000 7fc7525a6740 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b432170590 0x55b432180410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.179+0000 7fc7525a6740 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216fbd0 0x55b4321793d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.179+0000 7fc7525a6740 1 --2- 172.21.15.55:0/933536706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b432178e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.179+0000 7fc7525a6740 1 -- 172.21.15.55:0/933536706 >> 172.21.15.55:0/933536706 conn(0x55b4320af910 msgr2=0x55b432169450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.181+0000 7fc730ff9640 0 client.16257 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:58:14.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.185+0000 7fc7525a6740 1 -- 172.21.15.55:0/933536706 shutdown_connections 2024-04-26T21:58:14.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:13 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3370437948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:14.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.187+0000 7fc7525a6740 1 -- 172.21.15.55:0/933536706 wait complete. 2024-04-26T21:58:14.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.189+0000 7fc7525a6740 20 client.16257 trim_cache size 0 max 16384 2024-04-26T21:58:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.240+0000 7fc7525a6740 1 Processor -- start 2024-04-26T21:58:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.240+0000 7fc7525a6740 1 -- start start 2024-04-26T21:58:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.240+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b4321692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.240+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b4321803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.240+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b432170590 0x55b43216cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.241+0000 7fc74d54b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b4321692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.242+0000 7fc74cd4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b4321803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.242+0000 7fc74d54b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b4321692d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33632/0 (socket says 172.21.15.55:33632) 2024-04-26T21:58:14.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.242+0000 7fc74dd4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b432170590 0x55b43216cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.242+0000 7fc74dd4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b432170590 0x55b43216cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49746/0 (socket says 172.21.15.55:49746) 2024-04-26T21:58:14.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.242+0000 7fc7525a6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c010f70 con 0x55b432170590 2024-04-26T21:58:14.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.242+0000 7fc7525a6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c00e840 con 0x55b43216fbd0 2024-04-26T21:58:14.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.242+0000 7fc7525a6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c00c690 con 0x55b43216ea00 2024-04-26T21:58:14.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.242+0000 7fc74d54b640 1 -- 172.21.15.55:0/14622999 learned_addr learned my addr 172.21.15.55:0/14622999 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.246+0000 7fc74d54b640 1 -- 172.21.15.55:0/14622999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 msgr2=0x55b4321803d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.246+0000 7fc74d54b640 1 --2- 172.21.15.55:0/14622999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b4321803d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.246+0000 7fc74d54b640 1 -- 172.21.15.55:0/14622999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b432170590 msgr2=0x55b43216cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.246+0000 7fc74d54b640 1 --2- 172.21.15.55:0/14622999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b432170590 0x55b43216cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.246+0000 7fc74d54b640 1 -- 172.21.15.55:0/14622999 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b432165cb0 con 0x55b43216ea00 2024-04-26T21:58:14.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.248+0000 7fc74d54b640 1 --2- 172.21.15.55:0/14622999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b4321692d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55b43216fad0 tx=0x7fc738019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3f4882f86636831 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.252+0000 7fc732ffd640 1 -- 172.21.15.55:0/14622999 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7380261f0 con 0x55b43216ea00 2024-04-26T21:58:14.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.252+0000 7fc732ffd640 1 -- 172.21.15.55:0/14622999 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc738026840 con 0x55b43216ea00 2024-04-26T21:58:14.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:13 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3370437948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.253+0000 7fc7525a6740 1 -- 172.21.15.55:0/14622999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 msgr2=0x55b4321692d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.253+0000 7fc7525a6740 1 --2- 172.21.15.55:0/14622999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b4321692d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55b43216fad0 tx=0x7fc738019000 comp rx=0 tx=0).stop 2024-04-26T21:58:14.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.258+0000 7fc7525a6740 1 -- 172.21.15.55:0/14622999 shutdown_connections 2024-04-26T21:58:14.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.258+0000 7fc7525a6740 1 --2- 172.21.15.55:0/14622999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b432170590 0x55b43216cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.258+0000 7fc7525a6740 1 --2- 172.21.15.55:0/14622999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b4321803d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.258+0000 7fc7525a6740 1 --2- 172.21.15.55:0/14622999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b43216ea00 0x55b4321692d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.258+0000 7fc7525a6740 1 -- 172.21.15.55:0/14622999 >> 172.21.15.55:0/14622999 conn(0x55b4320af910 msgr2=0x55b432168bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.262+0000 7fc7525a6740 1 -- 172.21.15.55:0/14622999 shutdown_connections 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.264+0000 7fc7525a6740 1 -- 172.21.15.55:0/14622999 wait complete. 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.264+0000 7fc7525a6740 1 -- start start 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.264+0000 7fc7525a6740 1 Processor -- start 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.265+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216ea00 0x55b43217d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.265+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b43217d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.265+0000 7fc7525a6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 0x55b4321e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.265+0000 7fc7525a6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c00cd00 con 0x55b43216ea00 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.265+0000 7fc7525a6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c00e170 con 0x55b43216fbd0 2024-04-26T21:58:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.265+0000 7fc7525a6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c00d3d0 con 0x55b432170590 2024-04-26T21:58:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.266+0000 7fc7525a6740 10 client.0 _ll_register_callbacks cb 0x7fc74009c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.270+0000 7fc74cd4a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b43217d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.270+0000 7fc74dd4c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 0x55b4321e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.270+0000 7fc74d54b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216ea00 0x55b43217d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.271+0000 7fc74d54b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216ea00 0x55b43217d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49758/0 (socket says 172.21.15.55:49758) 2024-04-26T21:58:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.271+0000 7fc74d54b640 1 -- 172.21.15.55:0/247491750 learned_addr learned my addr 172.21.15.55:0/247491750 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.279+0000 7fc74cd4a640 1 -- 172.21.15.55:0/247491750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 msgr2=0x55b4321e94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.279+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 0x55b4321e94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.279+0000 7fc74cd4a640 1 -- 172.21.15.55:0/247491750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216ea00 msgr2=0x55b43217d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.279+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216ea00 0x55b43217d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.279+0000 7fc74cd4a640 1 -- 172.21.15.55:0/247491750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4321e9a30 con 0x55b43216fbd0 2024-04-26T21:58:14.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.280+0000 7fc74cd4a640 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b43217d810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc74403b970 tx=0x7fc74403bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7f7c7d798e66866 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.280+0000 7fc730ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:58:14.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.281+0000 7fc730ff9640 1 -- 172.21.15.55:0/247491750 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74405c040 con 0x55b43216fbd0 2024-04-26T21:58:14.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.281+0000 7fc730ff9640 1 -- 172.21.15.55:0/247491750 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc74404d850 con 0x55b43216fbd0 2024-04-26T21:58:14.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.282+0000 7fc7525a6740 1 -- 172.21.15.55:0/247491750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc738026840 con 0x55b43216fbd0 2024-04-26T21:58:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.288+0000 7fc730ff9640 1 -- 172.21.15.55:0/247491750 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc744004030 con 0x55b43216fbd0 2024-04-26T21:58:14.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.290+0000 7fc7525a6740 10 client.25619 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:14.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.290+0000 7fc7525a6740 1 -- 172.21.15.55:0/247491750 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b43220cc20 con 0x55b43216fbd0 2024-04-26T21:58:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.292+0000 7fc730ff9640 1 -- 172.21.15.55:0/247491750 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc738026840 con 0x55b43216fbd0 2024-04-26T21:58:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.292+0000 7fc730ff9640 1 -- 172.21.15.55:0/247491750 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc74404ece0 con 0x55b43216fbd0 2024-04-26T21:58:14.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.294+0000 7fc7525a6740 10 client.25619 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:14.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.294+0000 7fc7525a6740 10 client.25619 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:58:14.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.294+0000 7fc7525a6740 20 client.25619 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:14.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.294+0000 7fc7525a6740 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55b43220d540 0x55b43220f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.294+0000 7fc7525a6740 4 client.25619 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:14.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.294+0000 7fc7525a6740 1 -- 172.21.15.55:0/247491750 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b432055610 con 0x55b43220d540 2024-04-26T21:58:14.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.295+0000 7fc74d54b640 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55b43220d540 0x55b43220f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.304+0000 7fc74d54b640 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55b43220d540 0x55b43220f960 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c22f3e44ce8864f9 server_cookie=a8fe39d871193c9e in_seq=0 out_seq=0 2024-04-26T21:58:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.308+0000 7fc730ff9640 10 client.25619 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:58:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.308+0000 7fc730ff9640 1 -- 172.21.15.55:0/247491750 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b432055610 con 0x55b43220d540 2024-04-26T21:58:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.308+0000 7fc730ff9640 10 client.25619 handle_command_reply: tid=0 2024-04-26T21:58:14.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.309+0000 7fc7525a6740 1 client.25619 shutdown 2024-04-26T21:58:14.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.312+0000 7fc7525a6740 1 -- 172.21.15.55:0/247491750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 msgr2=0x55b43217d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.312+0000 7fc7525a6740 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b43217d810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc74403b970 tx=0x7fc74403bbd0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.316+0000 7fc7525a6740 1 -- 172.21.15.55:0/247491750 shutdown_connections 2024-04-26T21:58:14.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.316+0000 7fc7525a6740 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55b43220d540 0x55b43220f960 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.316+0000 7fc7525a6740 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b432170590 0x55b4321e94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.316+0000 7fc7525a6740 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b43216fbd0 0x55b43217d810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.316+0000 7fc7525a6740 1 --2- 172.21.15.55:0/247491750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55b43216ea00 0x55b43217d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.316+0000 7fc730ff9640 0 client.25619 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:58:14.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.316+0000 7fc7525a6740 1 -- 172.21.15.55:0/247491750 >> 172.21.15.55:0/247491750 conn(0x55b4320af910 msgr2=0x55b432169450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.322+0000 7fc7525a6740 1 -- 172.21.15.55:0/247491750 shutdown_connections 2024-04-26T21:58:14.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.324+0000 7fc7525a6740 1 -- 172.21.15.55:0/247491750 wait complete. 2024-04-26T21:58:14.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.324+0000 7fc7525a6740 20 client.25619 trim_cache size 0 max 16384 2024-04-26T21:58:14.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:13 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3370437948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:14.327 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:58:14.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.327+0000 7fc74e54d640 1 -- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7340a3e10 msgr2=0x7fc7340a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7340a3e10 0x7fc7340a62d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc738006fd0 tx=0x7fc738009f30 comp rx=0 tx=0).stop 2024-04-26T21:58:14.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 -- 172.21.15.55:0/4067478698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc74810a9b0 msgr2=0x7fc7481a66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc74810a9b0 0x7fc7481a66a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc74000cad0 tx=0x7fc74000cf90 comp rx=0 tx=0).stop 2024-04-26T21:58:14.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 -- 172.21.15.55:0/4067478698 shutdown_connections 2024-04-26T21:58:14.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7340a3e10 0x7fc7340a62d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc74810a9b0 0x7fc7481a66a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc74810a010 0x7fc74819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 --2- 172.21.15.55:0/4067478698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc748073d70 0x7fc74819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.328+0000 7fc74e54d640 1 -- 172.21.15.55:0/4067478698 >> 172.21.15.55:0/4067478698 conn(0x7fc74806f840 msgr2=0x7fc74810d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.331+0000 7fc74e54d640 1 -- 172.21.15.55:0/4067478698 shutdown_connections 2024-04-26T21:58:14.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.331+0000 7fc74e54d640 1 -- 172.21.15.55:0/4067478698 wait complete. 2024-04-26T21:58:14.373 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:58:14.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.812+0000 7f934ddf0640 1 -- 172.21.15.55:0/3625941489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9348100a70 msgr2=0x7f9348100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.812+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3625941489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9348100a70 0x7f9348100e90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f933c009900 tx=0x7f933c02a420 comp rx=0 tx=0).stop 2024-04-26T21:58:14.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.814+0000 7f934ddf0640 1 -- 172.21.15.55:0/3625941489 shutdown_connections 2024-04-26T21:58:14.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.814+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3625941489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348102190 0x7f9348106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.814+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3625941489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93481017d0 0x7f9348101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.814+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3625941489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9348100a70 0x7f9348100e90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.814+0000 7f934ddf0640 1 -- 172.21.15.55:0/3625941489 >> 172.21.15.55:0/3625941489 conn(0x7f93480fc3a0 msgr2=0x7f93480fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.819+0000 7f934ddf0640 1 -- 172.21.15.55:0/3625941489 shutdown_connections 2024-04-26T21:58:14.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.820+0000 7f934ddf0640 1 -- 172.21.15.55:0/3625941489 wait complete. 2024-04-26T21:58:14.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.820+0000 7f934ddf0640 1 Processor -- start 2024-04-26T21:58:14.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.821+0000 7f934ddf0640 1 -- start start 2024-04-26T21:58:14.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.821+0000 7f934ddf0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348100a70 0x7f934819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.821+0000 7f934ddf0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93481017d0 0x7f934819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.821+0000 7f934ddf0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9348102190 0x7f93481a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.821+0000 7f934ddf0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9348114a60 con 0x7f93481017d0 2024-04-26T21:58:14.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.821+0000 7f934ddf0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f93481148e0 con 0x7f9348100a70 2024-04-26T21:58:14.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.821+0000 7f934ddf0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9348114be0 con 0x7f9348102190 2024-04-26T21:58:14.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.824+0000 7f9346ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93481017d0 0x7f934819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.824+0000 7f9346ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93481017d0 0x7f934819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49782/0 (socket says 172.21.15.55:49782) 2024-04-26T21:58:14.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.824+0000 7f9346ffd640 1 -- 172.21.15.55:0/3794488230 learned_addr learned my addr 172.21.15.55:0/3794488230 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.826+0000 7f9346ffd640 1 -- 172.21.15.55:0/3794488230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9348102190 msgr2=0x7f93481a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.826+0000 7f93477fe640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348100a70 0x7f934819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.826+0000 7f9346ffd640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9348102190 0x7f93481a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.826+0000 7f9346ffd640 1 -- 172.21.15.55:0/3794488230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348100a70 msgr2=0x7f934819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.826+0000 7f9346ffd640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348100a70 0x7f934819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.827+0000 7f9346ffd640 1 -- 172.21.15.55:0/3794488230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93481a6dc0 con 0x7f93481017d0 2024-04-26T21:58:14.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.827+0000 7f93477fe640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348100a70 0x7f934819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:14.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.827+0000 7f9346ffd640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93481017d0 0x7f934819f730 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f934000b9c0 tx=0x7f934000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.831+0000 7f9344ff9640 1 -- 172.21.15.55:0/3794488230 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340019070 con 0x7f93481017d0 2024-04-26T21:58:14.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.831+0000 7f934ddf0640 1 -- 172.21.15.55:0/3794488230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93481a70b0 con 0x7f93481017d0 2024-04-26T21:58:14.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.831+0000 7f934ddf0640 1 -- 172.21.15.55:0/3794488230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93481a76d0 con 0x7f93481017d0 2024-04-26T21:58:14.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.832+0000 7f9344ff9640 1 -- 172.21.15.55:0/3794488230 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934000cba0 con 0x7f93481017d0 2024-04-26T21:58:14.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.832+0000 7f9344ff9640 1 -- 172.21.15.55:0/3794488230 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340019070 con 0x7f93481017d0 2024-04-26T21:58:14.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.851+0000 7f9350be8740 1 Processor -- start 2024-04-26T21:58:14.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.852+0000 7f9350be8740 1 -- start start 2024-04-26T21:58:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.852+0000 7f9350be8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e22a00 0x561019e22e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.852+0000 7f9350be8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e23bd0 0x561019e24050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.853+0000 7f9350be8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e24590 0x561019e2ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.853+0000 7f9350be8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561019d86d80 con 0x561019e23bd0 2024-04-26T21:58:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.853+0000 7f9350be8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561019d40a20 con 0x561019e22a00 2024-04-26T21:58:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.853+0000 7f9350be8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561019d82d40 con 0x561019e24590 2024-04-26T21:58:14.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.855+0000 7f93477fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e23bd0 0x561019e24050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.855+0000 7f93477fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e23bd0 0x561019e24050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49784/0 (socket says 172.21.15.55:49784) 2024-04-26T21:58:14.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.855+0000 7f93477fe640 1 -- 172.21.15.55:0/2490443755 learned_addr learned my addr 172.21.15.55:0/2490443755 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.858+0000 7f9347fff640 1 --2- 172.21.15.55:0/2490443755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e22a00 0x561019e22e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.859+0000 7f93477fe640 1 -- 172.21.15.55:0/2490443755 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e24590 msgr2=0x561019e2ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:58:14.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.859+0000 7f93477fe640 1 --2- 172.21.15.55:0/2490443755 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e24590 0x561019e2ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.859+0000 7f93477fe640 1 -- 172.21.15.55:0/2490443755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e22a00 msgr2=0x561019e22e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.859+0000 7f93477fe640 1 --2- 172.21.15.55:0/2490443755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e22a00 0x561019e22e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.859+0000 7f93477fe640 1 -- 172.21.15.55:0/2490443755 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561019d4bc30 con 0x561019e23bd0 2024-04-26T21:58:14.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.859+0000 7f9344ff9640 1 -- 172.21.15.55:0/3794488230 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f934001c070 con 0x7f93481017d0 2024-04-26T21:58:14.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.861+0000 7f9344ff9640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93300a4070 0x7f93300a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.861+0000 7f9344ff9640 1 -- 172.21.15.55:0/3794488230 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f934000c9a0 con 0x7f93481017d0 2024-04-26T21:58:14.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.861+0000 7f93477fe640 1 --2- 172.21.15.55:0/2490443755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e23bd0 0x561019e24050 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f933c006010 tx=0x7f933c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6152e4b28cff1f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.863+0000 7f9347fff640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93300a4070 0x7f93300a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.864+0000 7f932dffb640 1 -- 172.21.15.55:0/2490443755 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c03f070 con 0x561019e23bd0 2024-04-26T21:58:14.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.864+0000 7f932dffb640 1 -- 172.21.15.55:0/2490443755 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f933c0054f0 con 0x561019e23bd0 2024-04-26T21:58:14.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.866+0000 7f9350be8740 1 -- 172.21.15.55:0/2490443755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e23bd0 msgr2=0x561019e24050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.866+0000 7f9350be8740 1 --2- 172.21.15.55:0/2490443755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e23bd0 0x561019e24050 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f933c006010 tx=0x7f933c0040f0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.866+0000 7f9347fff640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93300a4070 0x7f93300a6530 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x561019e23ad0 tx=0x7f9338008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.867+0000 7f9350be8740 1 -- 172.21.15.55:0/2490443755 shutdown_connections 2024-04-26T21:58:14.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.867+0000 7f9350be8740 1 --2- 172.21.15.55:0/2490443755 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e24590 0x561019e2ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.867+0000 7f9350be8740 1 --2- 172.21.15.55:0/2490443755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e23bd0 0x561019e24050 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.867+0000 7f9350be8740 1 --2- 172.21.15.55:0/2490443755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e22a00 0x561019e22e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.867+0000 7f9350be8740 1 -- 172.21.15.55:0/2490443755 >> 172.21.15.55:0/2490443755 conn(0x561019d63910 msgr2=0x561019e1e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.871+0000 7f9350be8740 1 -- 172.21.15.55:0/2490443755 shutdown_connections 2024-04-26T21:58:14.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.872+0000 7f9350be8740 1 -- 172.21.15.55:0/2490443755 wait complete. 2024-04-26T21:58:14.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.873+0000 7f9350be8740 1 -- start start 2024-04-26T21:58:14.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.874+0000 7f9350be8740 1 Processor -- start 2024-04-26T21:58:14.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.878+0000 7f9350be8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e2cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.878+0000 7f9350be8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e24590 0x561019e2d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.878+0000 7f9350be8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e321c0 0x561019e345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.878+0000 7f9350be8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561019e2b590 con 0x561019e24590 2024-04-26T21:58:14.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.878+0000 7f9350be8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561019d0ddc0 con 0x561019e321c0 2024-04-26T21:58:14.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.878+0000 7f9350be8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561019e2b710 con 0x561019e22a00 2024-04-26T21:58:14.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.878+0000 7f9350be8740 10 client.0 _ll_register_callbacks cb 0x561019d29600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.881+0000 7f9346ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e24590 0x561019e2d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.881+0000 7f9346ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e24590 0x561019e2d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49786/0 (socket says 172.21.15.55:49786) 2024-04-26T21:58:14.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.881+0000 7f9347fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e321c0 0x561019e345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.881+0000 7f9346ffd640 1 -- 172.21.15.55:0/2945813837 learned_addr learned my addr 172.21.15.55:0/2945813837 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.881+0000 7f93477fe640 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e2cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.885+0000 7f93477fe640 1 -- 172.21.15.55:0/2945813837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e321c0 msgr2=0x561019e345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.885+0000 7f93477fe640 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e321c0 0x561019e345b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.885+0000 7f93477fe640 1 -- 172.21.15.55:0/2945813837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e24590 msgr2=0x561019e2d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.885+0000 7f93477fe640 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e24590 0x561019e2d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.885+0000 7f93477fe640 1 -- 172.21.15.55:0/2945813837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561019a983f0 con 0x561019e22a00 2024-04-26T21:58:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.886+0000 7f93477fe640 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e2cc50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f933c0096c0 tx=0x7f933c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604420233024de1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.887+0000 7f9326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:58:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.887+0000 7f9326ffd640 1 -- 172.21.15.55:0/2945813837 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c03f040 con 0x561019e22a00 2024-04-26T21:58:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.887+0000 7f9326ffd640 1 -- 172.21.15.55:0/2945813837 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f933c0049c0 con 0x561019e22a00 2024-04-26T21:58:14.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.887+0000 7f9350be8740 1 -- 172.21.15.55:0/2945813837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561019d94990 con 0x561019e22a00 2024-04-26T21:58:14.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.895+0000 7f9326ffd640 1 -- 172.21.15.55:0/2945813837 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f933c004bc0 con 0x561019e22a00 2024-04-26T21:58:14.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.895+0000 7f9350be8740 10 client.25606 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:14.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.895+0000 7f9350be8740 1 -- 172.21.15.55:0/2945813837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561019b627d0 con 0x561019e22a00 2024-04-26T21:58:14.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.904+0000 7f9326ffd640 1 -- 172.21.15.55:0/2945813837 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561019d94990 con 0x561019e22a00 2024-04-26T21:58:14.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.904+0000 7f9326ffd640 1 -- 172.21.15.55:0/2945813837 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f933c0315a0 con 0x561019e22a00 2024-04-26T21:58:14.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.905+0000 7f9350be8740 10 client.25606 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:14.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.905+0000 7f9350be8740 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:14.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.905+0000 7f9350be8740 20 client.25606 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:14.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.905+0000 7f9350be8740 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561019e38b30 0x561019e3af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.905+0000 7f9350be8740 4 client.25606 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:14.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.905+0000 7f9350be8740 1 -- 172.21.15.55:0/2945813837 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561019d8e800 con 0x561019e38b30 2024-04-26T21:58:14.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.906+0000 7f9346ffd640 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561019e38b30 0x561019e3af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.916+0000 7f9346ffd640 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561019e38b30 0x561019e3af20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f75b2be727dda8c server_cookie=56910a0e3e13b316 in_seq=0 out_seq=0 2024-04-26T21:58:14.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.916+0000 7f9326ffd640 10 client.25606 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:14.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.920+0000 7f9326ffd640 1 -- 172.21.15.55:0/2945813837 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561019d8e800 con 0x561019e38b30 2024-04-26T21:58:14.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.920+0000 7f9326ffd640 10 client.25606 handle_command_reply: tid=0 2024-04-26T21:58:14.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.920+0000 7f9350be8740 1 client.25606 shutdown 2024-04-26T21:58:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.923+0000 7f9350be8740 1 -- 172.21.15.55:0/2945813837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 msgr2=0x561019e2cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.923+0000 7f9350be8740 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e2cc50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f933c0096c0 tx=0x7f933c0046c0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.925+0000 7f9350be8740 1 -- 172.21.15.55:0/2945813837 shutdown_connections 2024-04-26T21:58:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.925+0000 7f9350be8740 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e321c0 0x561019e345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.925+0000 7f9350be8740 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561019e38b30 0x561019e3af20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.925+0000 7f9350be8740 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e24590 0x561019e2d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.925+0000 7f9350be8740 1 --2- 172.21.15.55:0/2945813837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e2cc50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.925+0000 7f9350be8740 1 -- 172.21.15.55:0/2945813837 >> 172.21.15.55:0/2945813837 conn(0x561019d63910 msgr2=0x561019e213a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.925+0000 7f9326ffd640 0 client.25606 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:14.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.926+0000 7f9350be8740 1 -- 172.21.15.55:0/2945813837 shutdown_connections 2024-04-26T21:58:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.928+0000 7f9350be8740 1 -- 172.21.15.55:0/2945813837 wait complete. 2024-04-26T21:58:14.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.929+0000 7f9350be8740 20 client.25606 trim_cache size 0 max 16384 2024-04-26T21:58:14.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.990+0000 7f9350be8740 1 Processor -- start 2024-04-26T21:58:14.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.993+0000 7f9350be8740 1 -- start start 2024-04-26T21:58:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.993+0000 7f9350be8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e1e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.993+0000 7f9350be8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e20fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.993+0000 7f9350be8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e321c0 0x561019e21500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.993+0000 7f9350be8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9334010f80 con 0x561019e321c0 2024-04-26T21:58:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.993+0000 7f9350be8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f933400e850 con 0x561019e24590 2024-04-26T21:58:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.993+0000 7f9350be8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f933400c6a0 con 0x561019e22a00 2024-04-26T21:58:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.994+0000 7f93477fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e1e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.994+0000 7f93477fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e1e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33686/0 (socket says 172.21.15.55:33686) 2024-04-26T21:58:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.994+0000 7f93477fe640 1 -- 172.21.15.55:0/4030652591 learned_addr learned my addr 172.21.15.55:0/4030652591 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.994+0000 7f9346ffd640 1 --2- 172.21.15.55:0/4030652591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e20fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.995+0000 7f93477fe640 1 -- 172.21.15.55:0/4030652591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 msgr2=0x561019e20fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.995+0000 7f93477fe640 1 --2- 172.21.15.55:0/4030652591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e20fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.995+0000 7f93477fe640 1 -- 172.21.15.55:0/4030652591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e321c0 msgr2=0x561019e21500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.995+0000 7f93477fe640 1 --2- 172.21.15.55:0/4030652591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e321c0 0x561019e21500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:14.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.995+0000 7f93477fe640 1 -- 172.21.15.55:0/4030652591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561019e34920 con 0x561019e22a00 2024-04-26T21:58:14.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:14.995+0000 7f93477fe640 1 --2- 172.21.15.55:0/4030652591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e1e480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f933c031b30 tx=0x7f933c005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a88e1b0af0ed2c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.000+0000 7f932d7fa640 1 -- 172.21.15.55:0/4030652591 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c03f070 con 0x561019e22a00 2024-04-26T21:58:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.001+0000 7f932d7fa640 1 -- 172.21.15.55:0/4030652591 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f933c005480 con 0x561019e22a00 2024-04-26T21:58:15.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.001+0000 7f932d7fa640 1 -- 172.21.15.55:0/4030652591 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c003850 con 0x561019e22a00 2024-04-26T21:58:15.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.002+0000 7f9350be8740 1 -- 172.21.15.55:0/4030652591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 msgr2=0x561019e1e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:15.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.002+0000 7f9350be8740 1 --2- 172.21.15.55:0/4030652591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e1e480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f933c031b30 tx=0x7f933c005c90 comp rx=0 tx=0).stop 2024-04-26T21:58:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.013+0000 7f9350be8740 1 -- 172.21.15.55:0/4030652591 shutdown_connections 2024-04-26T21:58:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.013+0000 7f9350be8740 1 --2- 172.21.15.55:0/4030652591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e321c0 0x561019e21500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.013+0000 7f9350be8740 1 --2- 172.21.15.55:0/4030652591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e20fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.013+0000 7f9350be8740 1 --2- 172.21.15.55:0/4030652591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e1e480 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.013+0000 7f9350be8740 1 -- 172.21.15.55:0/4030652591 >> 172.21.15.55:0/4030652591 conn(0x561019d63910 msgr2=0x561019e1dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.016+0000 7f9350be8740 1 -- 172.21.15.55:0/4030652591 shutdown_connections 2024-04-26T21:58:15.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.016+0000 7f9350be8740 1 -- 172.21.15.55:0/4030652591 wait complete. 2024-04-26T21:58:15.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.016+0000 7f9350be8740 1 -- start start 2024-04-26T21:58:15.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.018+0000 7f9350be8740 1 Processor -- start 2024-04-26T21:58:15.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.019+0000 7f9350be8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e46ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:15.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.019+0000 7f9350be8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e47420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:15.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.019+0000 7f9350be8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e321c0 0x561019e9ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:15.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.019+0000 7f9350be8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f933400cd10 con 0x561019e321c0 2024-04-26T21:58:15.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.019+0000 7f9350be8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f933400e180 con 0x561019e24590 2024-04-26T21:58:15.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.019+0000 7f9350be8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f933400d3e0 con 0x561019e22a00 2024-04-26T21:58:15.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.019+0000 7f9350be8740 10 client.0 _ll_register_callbacks cb 0x7f933c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:15.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.020+0000 7f93477fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e46ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:15.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.020+0000 7f93477fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e46ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33702/0 (socket says 172.21.15.55:33702) 2024-04-26T21:58:15.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.020+0000 7f93477fe640 1 -- 172.21.15.55:0/3508897893 learned_addr learned my addr 172.21.15.55:0/3508897893 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:15.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.020+0000 7f9346ffd640 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e47420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:15.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.030+0000 7f9346ffd640 1 -- 172.21.15.55:0/3508897893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 msgr2=0x561019e46ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:15.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.030+0000 7f9346ffd640 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e46ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.030+0000 7f9346ffd640 1 -- 172.21.15.55:0/3508897893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e321c0 msgr2=0x561019e9ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:15.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.030+0000 7f9346ffd640 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e321c0 0x561019e9ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.030+0000 7f9346ffd640 1 -- 172.21.15.55:0/3508897893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561019e4c500 con 0x561019e24590 2024-04-26T21:58:15.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.039+0000 7f9346ffd640 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e47420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f93400a3df0 tx=0x7f9340000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0c025136ac33b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:15.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.040+0000 7f9326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:58:15.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.042+0000 7f9326ffd640 1 -- 172.21.15.55:0/3508897893 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340019040 con 0x561019e24590 2024-04-26T21:58:15.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.042+0000 7f9326ffd640 1 -- 172.21.15.55:0/3508897893 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f93400999b0 con 0x561019e24590 2024-04-26T21:58:15.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.043+0000 7f9350be8740 1 -- 172.21.15.55:0/3508897893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f933c005480 con 0x561019e24590 2024-04-26T21:58:15.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.043+0000 7f9326ffd640 1 -- 172.21.15.55:0/3508897893 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f93400a4320 con 0x561019e24590 2024-04-26T21:58:15.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.044+0000 7f9350be8740 10 client.25631 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:15.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.044+0000 7f9350be8740 1 -- 172.21.15.55:0/3508897893 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561019eaf070 con 0x561019e24590 2024-04-26T21:58:15.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.047+0000 7f9326ffd640 1 -- 172.21.15.55:0/3508897893 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f933c005480 con 0x561019e24590 2024-04-26T21:58:15.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.047+0000 7f9326ffd640 1 -- 172.21.15.55:0/3508897893 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f93400a3e90 con 0x561019e24590 2024-04-26T21:58:15.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.047+0000 7f9350be8740 10 client.25631 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:15.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.047+0000 7f9350be8740 10 client.25631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:15.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.048+0000 7f9350be8740 20 client.25631 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:15.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.048+0000 7f9350be8740 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561019eaf930 0x561019eb1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:15.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.048+0000 7f9350be8740 4 client.25631 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:15.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.048+0000 7f9350be8740 1 -- 172.21.15.55:0/3508897893 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561019d09610 con 0x561019eaf930 2024-04-26T21:58:15.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.048+0000 7f93477fe640 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561019eaf930 0x561019eb1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:15.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.066+0000 7f93477fe640 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561019eaf930 0x561019eb1d50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd52065f72fc299d server_cookie=76e91dc7a9a1b608 in_seq=0 out_seq=0 2024-04-26T21:58:15.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.067+0000 7f9326ffd640 10 client.25631 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:15.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.075+0000 7f9326ffd640 1 -- 172.21.15.55:0/3508897893 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561019d09610 con 0x561019eaf930 2024-04-26T21:58:15.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.075+0000 7f9326ffd640 10 client.25631 handle_command_reply: tid=0 2024-04-26T21:58:15.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.075+0000 7f9350be8740 1 client.25631 shutdown 2024-04-26T21:58:15.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 -- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 msgr2=0x561019e47420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:15.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e47420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f93400a3df0 tx=0x7f9340000f40 comp rx=0 tx=0).stop 2024-04-26T21:58:15.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 -- 172.21.15.55:0/3508897893 shutdown_connections 2024-04-26T21:58:15.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561019eaf930 0x561019eb1d50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561019e321c0 0x561019e9ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561019e24590 0x561019e47420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 --2- 172.21.15.55:0/3508897893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561019e22a00 0x561019e46ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 -- 172.21.15.55:0/3508897893 >> 172.21.15.55:0/3508897893 conn(0x561019d63910 msgr2=0x561019e1cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9326ffd640 0 client.25631 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 -- 172.21.15.55:0/3508897893 shutdown_connections 2024-04-26T21:58:15.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 1 -- 172.21.15.55:0/3508897893 wait complete. 2024-04-26T21:58:15.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.079+0000 7f9350be8740 20 client.25631 trim_cache size 0 max 16384 2024-04-26T21:58:15.086 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:58:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.086+0000 7f934ddf0640 1 -- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93300a4070 msgr2=0x7f93300a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.086+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93300a4070 0x7f93300a6530 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x561019e23ad0 tx=0x7f9338008040 comp rx=0 tx=0).stop 2024-04-26T21:58:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.086+0000 7f934ddf0640 1 -- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93481017d0 msgr2=0x7f934819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:15.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.086+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93481017d0 0x7f934819f730 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f934000b9c0 tx=0x7f934000be80 comp rx=0 tx=0).stop 2024-04-26T21:58:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.095+0000 7f934ddf0640 1 -- 172.21.15.55:0/3794488230 shutdown_connections 2024-04-26T21:58:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.095+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93300a4070 0x7f93300a6530 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.095+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9348102190 0x7f93481a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.095+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93481017d0 0x7f934819f730 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.095+0000 7f934ddf0640 1 --2- 172.21.15.55:0/3794488230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9348100a70 0x7f934819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.095+0000 7f934ddf0640 1 -- 172.21.15.55:0/3794488230 >> 172.21.15.55:0/3794488230 conn(0x7f93480fc3a0 msgr2=0x7f9348102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:15.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.098+0000 7f934ddf0640 1 -- 172.21.15.55:0/3794488230 shutdown_connections 2024-04-26T21:58:15.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:15.099+0000 7f934ddf0640 1 -- 172.21.15.55:0/3794488230 wait complete. 2024-04-26T21:58:15.169 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:14 smithi055 ceph-mon[29221]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 869 MiB/s wr, 1.73k op/s 2024-04-26T21:58:15.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:14 smithi064 ceph-mon[32765]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 869 MiB/s wr, 1.73k op/s 2024-04-26T21:58:15.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:14 smithi165 ceph-mon[33296]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 869 MiB/s wr, 1.73k op/s 2024-04-26T21:58:16.172 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:58:16.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:58:16.172 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:58:16.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.475+0000 7f5fbc924640 1 -- 172.21.15.55:0/4047339362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb8077520 msgr2=0x7f5fb8079990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.475+0000 7f5fbc924640 1 --2- 172.21.15.55:0/4047339362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb8077520 0x7f5fb8079990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009cb0 tx=0x7f5fa802ac10 comp rx=0 tx=0).stop 2024-04-26T21:58:16.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.480+0000 7f5fbc924640 1 -- 172.21.15.55:0/4047339362 shutdown_connections 2024-04-26T21:58:16.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.480+0000 7f5fbc924640 1 --2- 172.21.15.55:0/4047339362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fb8079ed0 0x7f5fb807c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.480+0000 7f5fbc924640 1 --2- 172.21.15.55:0/4047339362 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb8077520 0x7f5fb8079990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.480+0000 7f5fbc924640 1 --2- 172.21.15.55:0/4047339362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fb8074640 0x7f5fb8074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.480+0000 7f5fbc924640 1 -- 172.21.15.55:0/4047339362 >> 172.21.15.55:0/4047339362 conn(0x7f5fb806f780 msgr2=0x7f5fb8071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.481+0000 7f5fbc924640 1 -- 172.21.15.55:0/4047339362 shutdown_connections 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.481+0000 7f5fbc924640 1 -- 172.21.15.55:0/4047339362 wait complete. 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.483+0000 7f5fbc924640 1 Processor -- start 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.483+0000 7f5fbc924640 1 -- start start 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.483+0000 7f5fbc924640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb8074640 0x7f5fb8082310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.483+0000 7f5fbc924640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fb8077520 0x7f5fb80809a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.484+0000 7f5fbc924640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fb8079ed0 0x7f5fb8080ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.484+0000 7f5fbc924640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb807eeb0 con 0x7f5fb8079ed0 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.484+0000 7f5fbc924640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb807ed30 con 0x7f5fb8074640 2024-04-26T21:58:16.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.484+0000 7f5fbc924640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb807f030 con 0x7f5fb8077520 2024-04-26T21:58:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.484+0000 7f5fb5d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fb8077520 0x7f5fb80809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.484+0000 7f5fb5d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fb8077520 0x7f5fb80809a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33714/0 (socket says 172.21.15.55:33714) 2024-04-26T21:58:16.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.484+0000 7f5fb5d9b640 1 -- 172.21.15.55:0/2936496989 learned_addr learned my addr 172.21.15.55:0/2936496989 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:16.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.485+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fb8079ed0 0x7f5fb8080ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.486+0000 7f5fb6d9d640 1 -- 172.21.15.55:0/2936496989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fb8077520 msgr2=0x7f5fb80809a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.486+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fb8077520 0x7f5fb80809a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.486+0000 7f5fb6d9d640 1 -- 172.21.15.55:0/2936496989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb8074640 msgr2=0x7f5fb8082310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.486+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb8074640 0x7f5fb8082310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.486+0000 7f5fb6d9d640 1 -- 172.21.15.55:0/2936496989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb8081420 con 0x7f5fb8079ed0 2024-04-26T21:58:16.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.487+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fb8079ed0 0x7f5fb8080ee0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000e940 tx=0x7f5fb000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.495+0000 7f5fa77fe640 1 -- 172.21.15.55:0/2936496989 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0004d20 con 0x7f5fb8079ed0 2024-04-26T21:58:16.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.495+0000 7f5fbc924640 1 -- 172.21.15.55:0/2936496989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb8081710 con 0x7f5fb8079ed0 2024-04-26T21:58:16.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.495+0000 7f5fbc924640 1 -- 172.21.15.55:0/2936496989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb81c0cb0 con 0x7f5fb8079ed0 2024-04-26T21:58:16.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.495+0000 7f5fa77fe640 1 -- 172.21.15.55:0/2936496989 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fb0005020 con 0x7f5fb8079ed0 2024-04-26T21:58:16.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.495+0000 7f5fa77fe640 1 -- 172.21.15.55:0/2936496989 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00116d0 con 0x7f5fb8079ed0 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.505+0000 7f5fbf643740 1 Processor -- start 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.506+0000 7f5fbf643740 1 -- start start 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.506+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d50beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.506+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d510f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.506+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d513a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.506+0000 7f5fbf643740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55909d45fb00 con 0x55909d5115f0 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.506+0000 7f5fbf643740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55909d3fb920 con 0x55909d50c480 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.506+0000 7f5fbf643740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55909d45bac0 con 0x55909d50ba90 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.511+0000 7f5fb5d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d50beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.511+0000 7f5fb5d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d50beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33728/0 (socket says 172.21.15.55:33728) 2024-04-26T21:58:16.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.511+0000 7f5fb5d9b640 1 -- 172.21.15.55:0/2914194556 learned_addr learned my addr 172.21.15.55:0/2914194556 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:16.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.511+0000 7f5fb5d9b640 1 --2- 172.21.15.55:0/2914194556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d513a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.511+0000 7f5fb659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d510f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.519+0000 7f5fb659c640 1 -- 172.21.15.55:0/2914194556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 msgr2=0x55909d50beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.519+0000 7f5fb659c640 1 --2- 172.21.15.55:0/2914194556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d50beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.519+0000 7f5fb659c640 1 -- 172.21.15.55:0/2914194556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 msgr2=0x55909d513a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.519+0000 7f5fb659c640 1 --2- 172.21.15.55:0/2914194556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d513a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.519+0000 7f5fb659c640 1 -- 172.21.15.55:0/2914194556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909d2397e0 con 0x55909d50c480 2024-04-26T21:58:16.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.520+0000 7f5fa77fe640 1 -- 172.21.15.55:0/2936496989 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5fb000cb20 con 0x7f5fb8079ed0 2024-04-26T21:58:16.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.522+0000 7f5fa77fe640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5fa00a3e40 0x7f5fa00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.522+0000 7f5fa77fe640 1 -- 172.21.15.55:0/2936496989 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5fb00d0670 con 0x7f5fb8079ed0 2024-04-26T21:58:16.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.523+0000 7f5fb5d9b640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5fa00a3e40 0x7f5fa00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.525+0000 7f5fb659c640 1 --2- 172.21.15.55:0/2914194556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d510f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009af0 tx=0x7f5fac00e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc190aec2fd6fbff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.527+0000 7f5fa4ff9640 1 -- 172.21.15.55:0/2914194556 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00ed00 con 0x55909d50c480 2024-04-26T21:58:16.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.527+0000 7f5fa4ff9640 1 -- 172.21.15.55:0/2914194556 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fac005020 con 0x55909d50c480 2024-04-26T21:58:16.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.529+0000 7f5fbf643740 1 -- 172.21.15.55:0/2914194556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 msgr2=0x55909d510f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.529+0000 7f5fbf643740 1 --2- 172.21.15.55:0/2914194556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d510f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009af0 tx=0x7f5fac00e350 comp rx=0 tx=0).stop 2024-04-26T21:58:16.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.530+0000 7f5fbf643740 1 -- 172.21.15.55:0/2914194556 shutdown_connections 2024-04-26T21:58:16.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.530+0000 7f5fbf643740 1 --2- 172.21.15.55:0/2914194556 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d513a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.530+0000 7f5fbf643740 1 --2- 172.21.15.55:0/2914194556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d510f80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.530+0000 7f5fbf643740 1 --2- 172.21.15.55:0/2914194556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d50beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.530+0000 7f5fbf643740 1 -- 172.21.15.55:0/2914194556 >> 172.21.15.55:0/2914194556 conn(0x55909d4f60f0 msgr2=0x55909d4f64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.536+0000 7f5fbf643740 1 -- 172.21.15.55:0/2914194556 shutdown_connections 2024-04-26T21:58:16.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.536+0000 7f5fbf643740 1 -- 172.21.15.55:0/2914194556 wait complete. 2024-04-26T21:58:16.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.537+0000 7f5fbf643740 1 -- start start 2024-04-26T21:58:16.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.538+0000 7f5fbf643740 1 Processor -- start 2024-04-26T21:58:16.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.538+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d515e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.538+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d516350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.538+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d5115f0 0x55909d51d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.538+0000 7f5fbf643740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55909d5145f0 con 0x55909d50ba90 2024-04-26T21:58:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.538+0000 7f5fbf643740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55909d3f95d0 con 0x55909d50c480 2024-04-26T21:58:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.538+0000 7f5fbf643740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55909d514770 con 0x55909d5115f0 2024-04-26T21:58:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.538+0000 7f5fbf643740 10 client.0 _ll_register_callbacks cb 0x55909d3e7080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.541+0000 7f5fb6d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d5115f0 0x55909d51d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.542+0000 7f5fb6d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d5115f0 0x55909d51d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33732/0 (socket says 172.21.15.55:33732) 2024-04-26T21:58:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.542+0000 7f5fb6d9d640 1 -- 172.21.15.55:0/1915039743 learned_addr learned my addr 172.21.15.55:0/1915039743 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.543+0000 7f5fb5d9b640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5fa00a3e40 0x7f5fa00a6300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8033a30 tx=0x7f5fa802ac10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.543+0000 7f5fb659c640 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d515e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.543+0000 7f5fb6d9d640 1 -- 172.21.15.55:0/1915039743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 msgr2=0x55909d516350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:16.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.543+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d516350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.543+0000 7f5fb6d9d640 1 -- 172.21.15.55:0/1915039743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 msgr2=0x55909d515e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.543+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d515e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.544+0000 7f5fb6d9d640 1 -- 172.21.15.55:0/1915039743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909d299f70 con 0x55909d5115f0 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.562+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d5115f0 0x55909d51d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00a4e90 tx=0x7f5fb009c1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7bd70692d4b96b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.562+0000 7f5f9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.566+0000 7f5f9dffb640 1 -- 172.21.15.55:0/1915039743 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb001f040 con 0x55909d5115f0 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.566+0000 7f5fbf643740 1 -- 172.21.15.55:0/1915039743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fac005020 con 0x55909d5115f0 2024-04-26T21:58:16.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.566+0000 7f5f9dffb640 1 -- 172.21.15.55:0/1915039743 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5fb009bda0 con 0x55909d5115f0 2024-04-26T21:58:16.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.572+0000 7f5f9dffb640 1 -- 172.21.15.55:0/1915039743 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5fb00a53c0 con 0x55909d5115f0 2024-04-26T21:58:16.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.574+0000 7f5fbf643740 10 client.25624 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:16.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.574+0000 7f5fbf643740 1 -- 172.21.15.55:0/1915039743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55909d131170 con 0x55909d5115f0 2024-04-26T21:58:16.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.580+0000 7f5f9dffb640 1 -- 172.21.15.55:0/1915039743 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fac005020 con 0x55909d5115f0 2024-04-26T21:58:16.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.580+0000 7f5f9dffb640 1 -- 172.21.15.55:0/1915039743 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f5fb000f040 con 0x55909d5115f0 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.584+0000 7f5fbf643740 10 client.25624 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.584+0000 7f5fbf643740 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.584+0000 7f5fbf643740 20 client.25624 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.584+0000 7f5fbf643740 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55909d521950 0x55909d523dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.584+0000 7f5fbf643740 4 client.25624 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:16.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.584+0000 7f5fbf643740 1 -- 172.21.15.55:0/1915039743 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55909d46d710 con 0x55909d521950 2024-04-26T21:58:16.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.589+0000 7f5fb659c640 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55909d521950 0x55909d523dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.608+0000 7f5fb659c640 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55909d521950 0x55909d523dd0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a00ddf5d5db7ea server_cookie=813946d629d04bf4 in_seq=0 out_seq=0 2024-04-26T21:58:16.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.608+0000 7f5f9dffb640 10 client.25624 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:16.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.615+0000 7f5f9dffb640 1 -- 172.21.15.55:0/1915039743 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55909d46d710 con 0x55909d521950 2024-04-26T21:58:16.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.615+0000 7f5f9dffb640 10 client.25624 handle_command_reply: tid=0 2024-04-26T21:58:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.622+0000 7f5fbf643740 1 client.25624 shutdown 2024-04-26T21:58:16.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.626+0000 7f5fbf643740 1 -- 172.21.15.55:0/1915039743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d5115f0 msgr2=0x55909d51d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.626+0000 7f5fbf643740 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d5115f0 0x55909d51d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00a4e90 tx=0x7f5fb009c1a0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.635+0000 7f5fbf643740 1 -- 172.21.15.55:0/1915039743 shutdown_connections 2024-04-26T21:58:16.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.635+0000 7f5fbf643740 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d5115f0 0x55909d51d390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.635+0000 7f5fbf643740 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d516350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.635+0000 7f5fbf643740 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55909d521950 0x55909d523dd0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.635+0000 7f5fbf643740 1 --2- 172.21.15.55:0/1915039743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d515e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.635+0000 7f5fbf643740 1 -- 172.21.15.55:0/1915039743 >> 172.21.15.55:0/1915039743 conn(0x55909d4f60f0 msgr2=0x55909d4f52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.640+0000 7f5f9dffb640 0 client.25624 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:16.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.640+0000 7f5fbf643740 1 -- 172.21.15.55:0/1915039743 shutdown_connections 2024-04-26T21:58:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.640+0000 7f5fbf643740 1 -- 172.21.15.55:0/1915039743 wait complete. 2024-04-26T21:58:16.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.642+0000 7f5fbf643740 20 client.25624 trim_cache size 0 max 16384 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.704+0000 7f5fbf643740 1 Processor -- start 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.704+0000 7f5fbf643740 1 -- start start 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d4f5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50c480 0x55909d51d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d5115f0 0x55909d5286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fb659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d4f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fb659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d4f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49848/0 (socket says 172.21.15.55:49848) 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fb659c640 1 -- 172.21.15.55:0/3156275714 learned_addr learned my addr 172.21.15.55:0/3156275714 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fbf643740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98010ee0 con 0x55909d50ba90 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fbf643740 1 -- 172.21.15.55:0/3156275714 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800e7b0 con 0x55909d5115f0 2024-04-26T21:58:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fb5d9b640 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50c480 0x55909d51d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d5115f0 0x55909d5286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.705+0000 7f5fbf643740 1 -- 172.21.15.55:0/3156275714 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800c600 con 0x55909d50c480 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5fb659c640 1 -- 172.21.15.55:0/3156275714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50c480 msgr2=0x55909d51d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5fb659c640 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50c480 0x55909d51d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5fb659c640 1 -- 172.21.15.55:0/3156275714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d5115f0 msgr2=0x55909d5286d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5fb659c640 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d5115f0 0x55909d5286d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5fb659c640 1 -- 172.21.15.55:0/3156275714 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909d5164f0 con 0x55909d50ba90 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5fb659c640 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d4f5170 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009ac0 tx=0x7f5fac009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409fb4c867ade2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5f9ffff640 1 -- 172.21.15.55:0/3156275714 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac02f070 con 0x55909d50ba90 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5f9ffff640 1 -- 172.21.15.55:0/3156275714 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fac0044e0 con 0x55909d50ba90 2024-04-26T21:58:16.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.706+0000 7f5f9ffff640 1 -- 172.21.15.55:0/3156275714 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac029760 con 0x55909d50ba90 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 -- 172.21.15.55:0/3156275714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 msgr2=0x55909d4f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d4f5170 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009ac0 tx=0x7f5fac009fd0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 -- 172.21.15.55:0/3156275714 shutdown_connections 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d5115f0 0x55909d5286d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50c480 0x55909d51d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3156275714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d50ba90 0x55909d4f5170 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 -- 172.21.15.55:0/3156275714 >> 172.21.15.55:0/3156275714 conn(0x55909d4f60f0 msgr2=0x55909d4f4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 -- 172.21.15.55:0/3156275714 shutdown_connections 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.707+0000 7f5fbf643740 1 -- 172.21.15.55:0/3156275714 wait complete. 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 1 -- start start 2024-04-26T21:58:16.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 1 Processor -- start 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d51d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d51b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d51bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800cc70 con 0x55909d5115f0 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800e0e0 con 0x55909d50c480 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9800d340 con 0x55909d50ba90 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fbf643740 10 client.0 _ll_register_callbacks cb 0x7f5fb009c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.708+0000 7f5fb659c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d51d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb6d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d51bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb659c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d51d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33742/0 (socket says 172.21.15.55:33742) 2024-04-26T21:58:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb659c640 1 -- 172.21.15.55:0/3701496011 learned_addr learned my addr 172.21.15.55:0/3701496011 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb659c640 1 -- 172.21.15.55:0/3701496011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 msgr2=0x55909d51b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb659c640 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d51b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb659c640 1 -- 172.21.15.55:0/3701496011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 msgr2=0x55909d51bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb659c640 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d51bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb659c640 1 -- 172.21.15.55:0/3701496011 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909d51c590 con 0x55909d50ba90 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb6d9d640 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d51bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5fb659c640 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d51d210 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00e850 tx=0x7f5fac00e900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9ba748c99b255d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.709+0000 7f5f9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.710+0000 7f5f9dffb640 1 -- 172.21.15.55:0/3701496011 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac02f040 con 0x55909d50ba90 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.710+0000 7f5f9dffb640 1 -- 172.21.15.55:0/3701496011 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5fac00ed00 con 0x55909d50ba90 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.710+0000 7f5fbf643740 1 -- 172.21.15.55:0/3701496011 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fac0044e0 con 0x55909d50ba90 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.710+0000 7f5f9dffb640 1 -- 172.21.15.55:0/3701496011 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5fac0294e0 con 0x55909d50ba90 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.710+0000 7f5fbf643740 10 client.25630 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.710+0000 7f5fbf643740 1 -- 172.21.15.55:0/3701496011 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55909d588fa0 con 0x55909d50ba90 2024-04-26T21:58:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.711+0000 7f5f9dffb640 1 -- 172.21.15.55:0/3701496011 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fac029680 con 0x55909d50ba90 2024-04-26T21:58:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.711+0000 7f5f9dffb640 1 -- 172.21.15.55:0/3701496011 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f5fac03c030 con 0x55909d50ba90 2024-04-26T21:58:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.711+0000 7f5fbf643740 10 client.25630 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.711+0000 7f5fbf643740 10 client.25630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.711+0000 7f5fbf643740 20 client.25630 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.711+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55909d589840 0x55909d58bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.711+0000 7f5fbf643740 4 client.25630 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:58:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.711+0000 7f5fbf643740 1 -- 172.21.15.55:0/3701496011 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f5fac0044e0 con 0x55909d589840 2024-04-26T21:58:16.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.713+0000 7f5fb5d9b640 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55909d589840 0x55909d58bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.715+0000 7f5fb5d9b640 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55909d589840 0x55909d58bc60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee361521887606aa server_cookie=1c4613e4086fca8f in_seq=0 out_seq=0 2024-04-26T21:58:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:16.715+0000 7f5f9dffb640 10 client.25630 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:17.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.098+0000 7f5f9dffb640 1 -- 172.21.15.55:0/3701496011 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f5fac0044e0 con 0x55909d589840 2024-04-26T21:58:17.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.098+0000 7f5f9dffb640 10 client.25630 handle_command_reply: tid=0 2024-04-26T21:58:17.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.098+0000 7f5fbf643740 1 client.25630 shutdown 2024-04-26T21:58:17.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 -- 172.21.15.55:0/3701496011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 msgr2=0x55909d51d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d51d210 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00e850 tx=0x7f5fac00e900 comp rx=0 tx=0).stop 2024-04-26T21:58:17.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 -- 172.21.15.55:0/3701496011 shutdown_connections 2024-04-26T21:58:17.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55909d589840 0x55909d58bc60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55909d5115f0 0x55909d51bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5f9dffb640 0 client.25630 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55909d50c480 0x55909d51b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 --2- 172.21.15.55:0/3701496011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55909d50ba90 0x55909d51d210 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 -- 172.21.15.55:0/3701496011 >> 172.21.15.55:0/3701496011 conn(0x55909d4f60f0 msgr2=0x55909d4f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 -- 172.21.15.55:0/3701496011 shutdown_connections 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 1 -- 172.21.15.55:0/3701496011 wait complete. 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.099+0000 7f5fbf643740 20 client.25630 trim_cache size 0 max 16384 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "a34e9f6e-bf3b-4368-b638-e7407d9d28d6", 2024-04-26T21:58:17.102 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:58:17.103 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:58:17.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.100+0000 7f5fbc924640 1 -- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5fa00a3e40 msgr2=0x7f5fa00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5fa00a3e40 0x7f5fa00a6300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8033a30 tx=0x7f5fa802ac10 comp rx=0 tx=0).stop 2024-04-26T21:58:17.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 -- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fb8079ed0 msgr2=0x7f5fb8080ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fb8079ed0 0x7f5fb8080ee0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000e940 tx=0x7f5fb000ee00 comp rx=0 tx=0).stop 2024-04-26T21:58:17.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 -- 172.21.15.55:0/2936496989 shutdown_connections 2024-04-26T21:58:17.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5fa00a3e40 0x7f5fa00a6300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fb8079ed0 0x7f5fb8080ee0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fb8077520 0x7f5fb80809a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 --2- 172.21.15.55:0/2936496989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fb8074640 0x7f5fb8082310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 -- 172.21.15.55:0/2936496989 >> 172.21.15.55:0/2936496989 conn(0x7f5fb806f780 msgr2=0x7f5fb807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 -- 172.21.15.55:0/2936496989 shutdown_connections 2024-04-26T21:58:17.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.101+0000 7f5fbc924640 1 -- 172.21.15.55:0/2936496989 wait complete. 2024-04-26T21:58:17.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:58:17.125 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:58:17.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:16 smithi055 ceph-mon[29221]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 828 MiB/s wr, 1.69k op/s 2024-04-26T21:58:17.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:16 smithi064 ceph-mon[32765]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 828 MiB/s wr, 1.69k op/s 2024-04-26T21:58:17.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:16 smithi165 ceph-mon[33296]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 828 MiB/s wr, 1.69k op/s 2024-04-26T21:58:17.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.424+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1021499678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc1011f0 msgr2=0x7f0dcc103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.424+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1021499678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc1011f0 0x7f0dcc103620 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009900 tx=0x7f0dc002a440 comp rx=0 tx=0).stop 2024-04-26T21:58:17.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.428+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1021499678 shutdown_connections 2024-04-26T21:58:17.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.428+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1021499678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dcc106510 0x7f0dcc106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.428+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1021499678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc103b60 0x7f0dcc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.428+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1021499678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc1011f0 0x7f0dcc103620 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.428+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1021499678 >> 172.21.15.55:0/1021499678 conn(0x7f0dcc0fcdf0 msgr2=0x7f0dcc0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.435+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1021499678 shutdown_connections 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.437+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1021499678 wait complete. 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.439+0000 7f0dd0ac5640 1 Processor -- start 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.439+0000 7f0dd0ac5640 1 -- start start 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.439+0000 7f0dd0ac5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc1011f0 0x7f0dcc073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.439+0000 7f0dd0ac5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dcc103b60 0x7f0dcc073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.439+0000 7f0dd0ac5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc106510 0x7f0dcc10e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.439+0000 7f0dd0ac5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc107cc0 con 0x7f0dcc1011f0 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.439+0000 7f0dd0ac5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc107b40 con 0x7f0dcc106510 2024-04-26T21:58:17.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.439+0000 7f0dd0ac5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc107e40 con 0x7f0dcc103b60 2024-04-26T21:58:17.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.441+0000 7f0dca59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc1011f0 0x7f0dcc073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.441+0000 7f0dca59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc1011f0 0x7f0dcc073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49862/0 (socket says 172.21.15.55:49862) 2024-04-26T21:58:17.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.441+0000 7f0dca59c640 1 -- 172.21.15.55:0/1586642933 learned_addr learned my addr 172.21.15.55:0/1586642933 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:17.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.442+0000 7f0dca59c640 1 -- 172.21.15.55:0/1586642933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dcc103b60 msgr2=0x7f0dcc073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.442+0000 7f0dca59c640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dcc103b60 0x7f0dcc073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.442+0000 7f0dca59c640 1 -- 172.21.15.55:0/1586642933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc106510 msgr2=0x7f0dcc10e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.442+0000 7f0dcad9d640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc106510 0x7f0dcc10e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.443+0000 7f0dca59c640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc106510 0x7f0dcc10e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.443+0000 7f0dca59c640 1 -- 172.21.15.55:0/1586642933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc10ec00 con 0x7f0dcc1011f0 2024-04-26T21:58:17.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.443+0000 7f0dcad9d640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc106510 0x7f0dcc10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:17.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.447+0000 7f0dca59c640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc1011f0 0x7f0dcc073130 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009230 tx=0x7f0dc0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.452+0000 7f0dbb7fe640 1 -- 172.21.15.55:0/1586642933 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc003f030 con 0x7f0dcc1011f0 2024-04-26T21:58:17.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.452+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1586642933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc10ee90 con 0x7f0dcc1011f0 2024-04-26T21:58:17.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.452+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1586642933 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc10f400 con 0x7f0dcc1011f0 2024-04-26T21:58:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.453+0000 7f0dbb7fe640 1 -- 172.21.15.55:0/1586642933 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc0003d10 con 0x7f0dcc1011f0 2024-04-26T21:58:17.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.453+0000 7f0dbb7fe640 1 -- 172.21.15.55:0/1586642933 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0035070 con 0x7f0dcc1011f0 2024-04-26T21:58:17.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.460+0000 7f0dbb7fe640 1 -- 172.21.15.55:0/1586642933 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0dc0042030 con 0x7f0dcc1011f0 2024-04-26T21:58:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.461+0000 7f0dbb7fe640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0db40a3e40 0x7f0db40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.462+0000 7f0dc9d9b640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0db40a3e40 0x7f0db40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.463+0000 7f0dbb7fe640 1 -- 172.21.15.55:0/1586642933 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0dc00f1ef0 con 0x7f0dcc1011f0 2024-04-26T21:58:17.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.463+0000 7f0dc9d9b640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0db40a3e40 0x7f0db40a6300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0743e0 tx=0x7f0dbc006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.464+0000 7f0dd37d7740 1 Processor -- start 2024-04-26T21:58:17.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.464+0000 7f0dd37d7740 1 -- start start 2024-04-26T21:58:17.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.465+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e7428ee0 0x55c6e7429300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.465+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742a0b0 0x55c6e742a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.465+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 0x55c6e7431300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.465+0000 7f0dd37d7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c6e738dd80 con 0x55c6e742a0b0 2024-04-26T21:58:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.466+0000 7f0dca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e7428ee0 0x55c6e7429300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.466+0000 7f0dca59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e7428ee0 0x55c6e7429300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:55566/0 (socket says 172.21.15.55:55566) 2024-04-26T21:58:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.466+0000 7f0dca59c640 1 -- 172.21.15.55:0/1849568800 learned_addr learned my addr 172.21.15.55:0/1849568800 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.466+0000 7f0dd37d7740 1 -- 172.21.15.55:0/1849568800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6e73478b0 con 0x55c6e7428ee0 2024-04-26T21:58:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.466+0000 7f0dcad9d640 1 --2- 172.21.15.55:0/1849568800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 0x55c6e7431300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.466+0000 7f0dd37d7740 1 -- 172.21.15.55:0/1849568800 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c6e7389d40 con 0x55c6e742aa70 2024-04-26T21:58:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.467+0000 7f0dca59c640 1 -- 172.21.15.55:0/1849568800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 msgr2=0x55c6e7431300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.467+0000 7f0dca59c640 1 --2- 172.21.15.55:0/1849568800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 0x55c6e7431300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.467+0000 7f0dca59c640 1 -- 172.21.15.55:0/1849568800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742a0b0 msgr2=0x55c6e742a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.467+0000 7f0dca59c640 1 --2- 172.21.15.55:0/1849568800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742a0b0 0x55c6e742a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.467+0000 7f0dca59c640 1 -- 172.21.15.55:0/1849568800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e7318a40 con 0x55c6e7428ee0 2024-04-26T21:58:17.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.468+0000 7f0dca59c640 1 --2- 172.21.15.55:0/1849568800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e7428ee0 0x55c6e7429300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00096c0 tx=0x7f0dc002a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a758bf4ecc1c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.468+0000 7f0db8ff9640 1 -- 172.21.15.55:0/1849568800 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00c6c40 con 0x55c6e7428ee0 2024-04-26T21:58:17.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.468+0000 7f0db8ff9640 1 -- 172.21.15.55:0/1849568800 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc00bc2d0 con 0x55c6e7428ee0 2024-04-26T21:58:17.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.469+0000 7f0db8ff9640 1 -- 172.21.15.55:0/1849568800 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc003b9a0 con 0x55c6e7428ee0 2024-04-26T21:58:17.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.470+0000 7f0dd37d7740 1 -- 172.21.15.55:0/1849568800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e7428ee0 msgr2=0x55c6e7429300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.470+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/1849568800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e7428ee0 0x55c6e7429300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00096c0 tx=0x7f0dc002a440 comp rx=0 tx=0).stop 2024-04-26T21:58:17.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.470+0000 7f0dd37d7740 1 -- 172.21.15.55:0/1849568800 shutdown_connections 2024-04-26T21:58:17.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.470+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/1849568800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 0x55c6e7431300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.470+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/1849568800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742a0b0 0x55c6e742a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.470+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/1849568800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e7428ee0 0x55c6e7429300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.470+0000 7f0dd37d7740 1 -- 172.21.15.55:0/1849568800 >> 172.21.15.55:0/1849568800 conn(0x55c6e74243d0 msgr2=0x55c6e74247d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.476+0000 7f0dd37d7740 1 -- 172.21.15.55:0/1849568800 shutdown_connections 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.476+0000 7f0dd37d7740 1 -- 172.21.15.55:0/1849568800 wait complete. 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.477+0000 7f0dd37d7740 1 -- start start 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.477+0000 7f0dd37d7740 1 Processor -- start 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.477+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.477+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7433640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.477+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e743a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.477+0000 7f0dd37d7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c6e7431840 con 0x55c6e742aa70 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.477+0000 7f0dd37d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6e7314d10 con 0x55c6e742a0b0 2024-04-26T21:58:17.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.477+0000 7f0dd37d7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c6e74319c0 con 0x55c6e7428ee0 2024-04-26T21:58:17.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.478+0000 7f0dd37d7740 10 client.0 _ll_register_callbacks cb 0x55c6e7341210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:17.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.478+0000 7f0dca59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.478+0000 7f0dca59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:33768/0 (socket says 172.21.15.55:33768) 2024-04-26T21:58:17.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.478+0000 7f0dca59c640 1 -- 172.21.15.55:0/3199380732 learned_addr learned my addr 172.21.15.55:0/3199380732 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:17.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.480+0000 7f0dc9d9b640 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7433640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.481+0000 7f0dca59c640 1 -- 172.21.15.55:0/3199380732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 msgr2=0x55c6e7433640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.481+0000 7f0dca59c640 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7433640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.481+0000 7f0dca59c640 1 -- 172.21.15.55:0/3199380732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 msgr2=0x55c6e743a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.481+0000 7f0dca59c640 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e743a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.481+0000 7f0dca59c640 1 -- 172.21.15.55:0/3199380732 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e71697d0 con 0x55c6e7428ee0 2024-04-26T21:58:17.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.487+0000 7f0dca59c640 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00ba9a0 tx=0x7f0dc00ef7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8162ee70069e8ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.487+0000 7f0db1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T21:58:17.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.489+0000 7f0db1ffb640 1 -- 172.21.15.55:0/3199380732 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00db070 con 0x55c6e7428ee0 2024-04-26T21:58:17.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.490+0000 7f0dd37d7740 1 -- 172.21.15.55:0/3199380732 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc00bc2d0 con 0x55c6e7428ee0 2024-04-26T21:58:17.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.490+0000 7f0db1ffb640 1 -- 172.21.15.55:0/3199380732 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0dc003b9a0 con 0x55c6e7428ee0 2024-04-26T21:58:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.495+0000 7f0db1ffb640 1 -- 172.21.15.55:0/3199380732 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0dc00c6c40 con 0x55c6e7428ee0 2024-04-26T21:58:17.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.500+0000 7f0dd37d7740 10 client.25636 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:17.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.500+0000 7f0dd37d7740 1 -- 172.21.15.55:0/3199380732 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c6e709f3f0 con 0x55c6e7428ee0 2024-04-26T21:58:17.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.505+0000 7f0db1ffb640 1 -- 172.21.15.55:0/3199380732 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc00bc2d0 con 0x55c6e7428ee0 2024-04-26T21:58:17.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.505+0000 7f0db1ffb640 1 -- 172.21.15.55:0/3199380732 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0dc00c67b0 con 0x55c6e7428ee0 2024-04-26T21:58:17.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.509+0000 7f0dd37d7740 10 client.25636 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:17.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.509+0000 7f0dd37d7740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:17.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.509+0000 7f0dd37d7740 20 client.25636 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:17.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.509+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c6e743e8a0 0x55c6e7440ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.509+0000 7f0dd37d7740 4 client.25636 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:17.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.509+0000 7f0dd37d7740 1 -- 172.21.15.55:0/3199380732 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6e739b990 con 0x55c6e743e8a0 2024-04-26T21:58:17.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.514+0000 7f0dc9d9b640 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c6e743e8a0 0x55c6e7440ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.536+0000 7f0db1ffb640 10 client.25636 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:17.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.536+0000 7f0dc9d9b640 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c6e743e8a0 0x55c6e7440ce0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f7234efe855abb server_cookie=ca7903f3f40460b5 in_seq=0 out_seq=0 2024-04-26T21:58:17.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.540+0000 7f0db1ffb640 1 -- 172.21.15.55:0/3199380732 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6e739b990 con 0x55c6e743e8a0 2024-04-26T21:58:17.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.540+0000 7f0db1ffb640 10 client.25636 handle_command_reply: tid=0 2024-04-26T21:58:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.542+0000 7f0dd37d7740 1 client.25636 shutdown 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.547+0000 7f0dd37d7740 1 -- 172.21.15.55:0/3199380732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 msgr2=0x55c6e7433100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.547+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00ba9a0 tx=0x7f0dc00ef7d0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.547+0000 7f0dd37d7740 1 -- 172.21.15.55:0/3199380732 shutdown_connections 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.547+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c6e743e8a0 0x55c6e7440ce0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.548+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e743a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.548+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7433640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.548+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/3199380732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.548+0000 7f0db1ffb640 0 client.25636 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.548+0000 7f0dd37d7740 1 -- 172.21.15.55:0/3199380732 >> 172.21.15.55:0/3199380732 conn(0x55c6e74243d0 msgr2=0x55c6e7423550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.548+0000 7f0dd37d7740 1 -- 172.21.15.55:0/3199380732 shutdown_connections 2024-04-26T21:58:17.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.548+0000 7f0dd37d7740 1 -- 172.21.15.55:0/3199380732 wait complete. 2024-04-26T21:58:17.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.549+0000 7f0dd37d7740 20 client.25636 trim_cache size 0 max 16384 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dd37d7740 1 Processor -- start 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dd37d7740 1 -- start start 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7431df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e7432330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dd37d7740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac010f30 con 0x55c6e742aa70 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dd37d7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac00e800 con 0x55c6e742a0b0 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dd37d7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac00c650 con 0x55c6e7428ee0 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dcad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e7432330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dcad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e7432330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49890/0 (socket says 172.21.15.55:49890) 2024-04-26T21:58:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.584+0000 7f0dcad9d640 1 -- 172.21.15.55:0/2826583719 learned_addr learned my addr 172.21.15.55:0/2826583719 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.585+0000 7f0dcad9d640 1 -- 172.21.15.55:0/2826583719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 msgr2=0x55c6e7433420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.585+0000 7f0dcad9d640 1 --2- 172.21.15.55:0/2826583719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.585+0000 7f0dcad9d640 1 -- 172.21.15.55:0/2826583719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 msgr2=0x55c6e7431df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:17.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.585+0000 7f0dcad9d640 1 --2- 172.21.15.55:0/2826583719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7431df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.585+0000 7f0dcad9d640 1 -- 172.21.15.55:0/2826583719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e705f0f0 con 0x55c6e742aa70 2024-04-26T21:58:17.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.585+0000 7f0dcad9d640 1 --2- 172.21.15.55:0/2826583719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e7432330 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007f10 tx=0x7f0dc400f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1442f1701d11442 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.585+0000 7f0db3fff640 1 -- 172.21.15.55:0/2826583719 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4010830 con 0x55c6e742aa70 2024-04-26T21:58:17.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.586+0000 7f0db3fff640 1 -- 172.21.15.55:0/2826583719 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc40183f0 con 0x55c6e742aa70 2024-04-26T21:58:17.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.586+0000 7f0db3fff640 1 -- 172.21.15.55:0/2826583719 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4017600 con 0x55c6e742aa70 2024-04-26T21:58:17.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 -- 172.21.15.55:0/2826583719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 msgr2=0x55c6e7432330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/2826583719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e7432330 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007f10 tx=0x7f0dc400f780 comp rx=0 tx=0).stop 2024-04-26T21:58:17.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 -- 172.21.15.55:0/2826583719 shutdown_connections 2024-04-26T21:58:17.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/2826583719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e742aa70 0x55c6e7432330 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/2826583719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7431df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/2826583719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e7428ee0 0x55c6e7433420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 -- 172.21.15.55:0/2826583719 >> 172.21.15.55:0/2826583719 conn(0x55c6e74243d0 msgr2=0x55c6e7432d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 -- 172.21.15.55:0/2826583719 shutdown_connections 2024-04-26T21:58:17.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.587+0000 7f0dd37d7740 1 -- 172.21.15.55:0/2826583719 wait complete. 2024-04-26T21:58:17.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.588+0000 7f0dd37d7740 1 -- start start 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.588+0000 7f0dd37d7740 1 Processor -- start 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.589+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e7428ee0 0x55c6e7436df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.589+0000 7f0dd37d7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7437330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.589+0000 7f0dca59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e7428ee0 0x55c6e7436df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.589+0000 7f0dca59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e7428ee0 0x55c6e7436df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49902/0 (socket says 172.21.15.55:49902) 2024-04-26T21:58:17.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.589+0000 7f0dca59c640 1 -- 172.21.15.55:0/847336327 learned_addr learned my addr 172.21.15.55:0/847336327 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.589+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 0x55c6e7438230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac00ccc0 con 0x55c6e7428ee0 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac00e130 con 0x55c6e742a0b0 2024-04-26T21:58:17.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac00d390 con 0x55c6e742aa70 2024-04-26T21:58:17.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dd37d7740 10 client.0 _ll_register_callbacks cb 0x7f0dc40075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:17.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dca59c640 1 -- 172.21.15.55:0/847336327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 msgr2=0x55c6e7438230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:17.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dca59c640 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 0x55c6e7438230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dca59c640 1 -- 172.21.15.55:0/847336327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 msgr2=0x55c6e7437330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:17.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dca59c640 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7437330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.590+0000 7f0dca59c640 1 -- 172.21.15.55:0/847336327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e734e4b0 con 0x55c6e7428ee0 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.591+0000 7f0dca59c640 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e7428ee0 0x55c6e7436df0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00ba430 tx=0x7f0dc00efcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2962372e38a12b0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:17.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.591+0000 7f0db1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.592+0000 7f0db1ffb640 1 -- 172.21.15.55:0/847336327 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00db030 con 0x55c6e7428ee0 2024-04-26T21:58:17.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.592+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc40183f0 con 0x55c6e7428ee0 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.592+0000 7f0db1ffb640 1 -- 172.21.15.55:0/847336327 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0dc00d7040 con 0x55c6e7428ee0 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.592+0000 7f0db1ffb640 1 -- 172.21.15.55:0/847336327 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0dc00bdda0 con 0x55c6e7428ee0 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.592+0000 7f0dd37d7740 10 client.16299 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.592+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c6e734e4b0 con 0x55c6e7428ee0 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.593+0000 7f0db1ffb640 1 -- 172.21.15.55:0/847336327 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc00a0020 con 0x55c6e7428ee0 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.593+0000 7f0db1ffb640 1 -- 172.21.15.55:0/847336327 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0dc00a8ce0 con 0x55c6e7428ee0 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.593+0000 7f0dd37d7740 10 client.16299 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.593+0000 7f0dd37d7740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.593+0000 7f0dd37d7740 20 client.16299 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.593+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c6e74a4e60 0x55c6e74a7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:17.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.593+0000 7f0dd37d7740 4 client.16299 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:58:17.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.593+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0dc40183f0 con 0x55c6e74a4e60 2024-04-26T21:58:17.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.597+0000 7f0dc9d9b640 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c6e74a4e60 0x55c6e74a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.602+0000 7f0db1ffb640 10 client.16299 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:17.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.602+0000 7f0dc9d9b640 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c6e74a4e60 0x55c6e74a7250 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c8844f9dab4473 server_cookie=3f3bf778f6f93823 in_seq=0 out_seq=0 2024-04-26T21:58:17.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.606+0000 7f0db1ffb640 1 -- 172.21.15.55:0/847336327 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x7f0dc40183f0 con 0x55c6e74a4e60 2024-04-26T21:58:17.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.606+0000 7f0db1ffb640 10 client.16299 handle_command_reply: tid=0 2024-04-26T21:58:17.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.606+0000 7f0dd37d7740 1 client.16299 shutdown 2024-04-26T21:58:17.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e7428ee0 msgr2=0x55c6e7436df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e7428ee0 0x55c6e7436df0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00ba430 tx=0x7f0dc00efcd0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 shutdown_connections 2024-04-26T21:58:17.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e742aa70 0x55c6e7438230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6e742a0b0 0x55c6e7437330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c6e74a4e60 0x55c6e74a7250 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 --2- 172.21.15.55:0/847336327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c6e7428ee0 0x55c6e7436df0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 >> 172.21.15.55:0/847336327 conn(0x55c6e74243d0 msgr2=0x55c6e7432d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0db1ffb640 0 client.16299 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:17.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 shutdown_connections 2024-04-26T21:58:17.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.607+0000 7f0dd37d7740 1 -- 172.21.15.55:0/847336327 wait complete. 2024-04-26T21:58:17.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.608+0000 7f0dd37d7740 20 client.16299 trim_cache size 0 max 16384 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (0 inodes in the stack)", 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout: "a34e9f6e-bf3b-4368-b638-e7407d9d28d6": { 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "a34e9f6e-bf3b-4368-b638-e7407d9d28d6", 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:58:17.612 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.608+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0db40a3e40 msgr2=0x7f0db40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.608+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0db40a3e40 0x7f0db40a6300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0743e0 tx=0x7f0dbc006c90 comp rx=0 tx=0).stop 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.608+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc1011f0 msgr2=0x7f0dcc073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.608+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc1011f0 0x7f0dcc073130 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009230 tx=0x7f0dc0003910 comp rx=0 tx=0).stop 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.609+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1586642933 shutdown_connections 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.609+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0db40a3e40 0x7f0db40a6300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.609+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dcc106510 0x7f0dcc10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.609+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0dcc103b60 0x7f0dcc073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.609+0000 7f0dd0ac5640 1 --2- 172.21.15.55:0/1586642933 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0dcc1011f0 0x7f0dcc073130 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:17.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.609+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1586642933 >> 172.21.15.55:0/1586642933 conn(0x7f0dcc0fcdf0 msgr2=0x7f0dcc0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:17.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.609+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1586642933 shutdown_connections 2024-04-26T21:58:17.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:17.609+0000 7f0dd0ac5640 1 -- 172.21.15.55:0/1586642933 wait complete. 2024-04-26T21:58:17.637 INFO:tasks.cephfs.filesystem:scrub status for tag:a34e9f6e-bf3b-4368-b638-e7407d9d28d6 - {'path': '/', 'tag': 'a34e9f6e-bf3b-4368-b638-e7407d9d28d6', 'options': 'recursive,force'} 2024-04-26T21:58:18.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:17 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:58:18.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:17 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:18.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:17 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:58:18.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:17 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:58:18.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:17 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:18.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:17 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:58:18.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:17 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:18.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:17 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:58:18.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:17 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:58:18.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:17 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:18.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:17 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:58:18.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:17 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:18.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:17 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:58:18.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:17 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:58:18.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:17 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:18 smithi064 ceph-mon[32765]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 913 MiB/s wr, 1.90k op/s 2024-04-26T21:58:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:18 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:19.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:18 smithi165 ceph-mon[33296]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 913 MiB/s wr, 1.90k op/s 2024-04-26T21:58:19.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:18 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:19.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:18 smithi055 ceph-mon[29221]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 913 MiB/s wr, 1.90k op/s 2024-04-26T21:58:19.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:18 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:21.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:20 smithi064 ceph-mon[32765]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.97k op/s 2024-04-26T21:58:21.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:20 smithi165 ceph-mon[33296]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.97k op/s 2024-04-26T21:58:21.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:20 smithi055 ceph-mon[29221]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.97k op/s 2024-04-26T21:58:23.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:22 smithi064 ceph-mon[32765]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 806 MiB/s wr, 1.95k op/s 2024-04-26T21:58:23.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:22 smithi165 ceph-mon[33296]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 806 MiB/s wr, 1.95k op/s 2024-04-26T21:58:23.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:22 smithi055 ceph-mon[29221]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 806 MiB/s wr, 1.95k op/s 2024-04-26T21:58:25.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:24 smithi064 ceph-mon[32765]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 891 MiB/s wr, 2.13k op/s 2024-04-26T21:58:25.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:24 smithi165 ceph-mon[33296]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 891 MiB/s wr, 2.13k op/s 2024-04-26T21:58:25.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:24 smithi055 ceph-mon[29221]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 891 MiB/s wr, 2.13k op/s 2024-04-26T21:58:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:26 smithi064 ceph-mon[32765]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 833 MiB/s wr, 1.96k op/s 2024-04-26T21:58:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:26 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:58:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:26 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:58:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:26 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:27.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:26 smithi165 ceph-mon[33296]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 833 MiB/s wr, 1.96k op/s 2024-04-26T21:58:27.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:26 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:58:27.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:26 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:58:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:26 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:27.447 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:26 smithi055 ceph-mon[29221]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 833 MiB/s wr, 1.96k op/s 2024-04-26T21:58:27.447 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:26 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:58:27.447 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:26 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:58:27.447 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:26 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:29.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:29 smithi064 ceph-mon[32765]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 904 MiB/s wr, 2.03k op/s 2024-04-26T21:58:29.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:29 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:29.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:29 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:29.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:29 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:58:29.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:29 smithi165 ceph-mon[33296]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 904 MiB/s wr, 2.03k op/s 2024-04-26T21:58:29.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:29 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:29.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:29 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:29.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:29 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:58:29.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:29 smithi055 ceph-mon[29221]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 904 MiB/s wr, 2.03k op/s 2024-04-26T21:58:29.460 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:29 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:29.460 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:29 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:29.460 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:29 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:58:31.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:31 smithi165 ceph-mon[33296]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 MiB/s wr, 1.76k op/s 2024-04-26T21:58:31.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:31 smithi055 ceph-mon[29221]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 MiB/s wr, 1.76k op/s 2024-04-26T21:58:31.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:31 smithi064 ceph-mon[32765]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 854 MiB/s wr, 1.76k op/s 2024-04-26T21:58:33.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:33 smithi165 ceph-mon[33296]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.59k op/s 2024-04-26T21:58:33.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:33 smithi055 ceph-mon[29221]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.59k op/s 2024-04-26T21:58:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:33 smithi064 ceph-mon[32765]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.59k op/s 2024-04-26T21:58:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:35 smithi165 ceph-mon[33296]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 922 MiB/s wr, 1.75k op/s 2024-04-26T21:58:35.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:35 smithi055 ceph-mon[29221]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 922 MiB/s wr, 1.75k op/s 2024-04-26T21:58:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:35 smithi064 ceph-mon[32765]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 922 MiB/s wr, 1.75k op/s 2024-04-26T21:58:37.308 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:37 smithi064 ceph-mon[32765]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.51k op/s 2024-04-26T21:58:37.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:37 smithi165 ceph-mon[33296]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.51k op/s 2024-04-26T21:58:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:37 smithi055 ceph-mon[29221]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 822 MiB/s wr, 1.51k op/s 2024-04-26T21:58:39.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:39 smithi165 ceph-mon[33296]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 900 MiB/s wr, 1.80k op/s 2024-04-26T21:58:39.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:39 smithi055 ceph-mon[29221]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 900 MiB/s wr, 1.80k op/s 2024-04-26T21:58:39.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:39 smithi064 ceph-mon[32765]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 900 MiB/s wr, 1.80k op/s 2024-04-26T21:58:41.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:41 smithi165 ceph-mon[33296]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.86k op/s 2024-04-26T21:58:41.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:41 smithi055 ceph-mon[29221]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.86k op/s 2024-04-26T21:58:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:41 smithi064 ceph-mon[32765]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.86k op/s 2024-04-26T21:58:43.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:43 smithi165 ceph-mon[33296]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 843 MiB/s wr, 1.98k op/s 2024-04-26T21:58:43.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:43 smithi055 ceph-mon[29221]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 843 MiB/s wr, 1.98k op/s 2024-04-26T21:58:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:43 smithi064 ceph-mon[32765]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 843 MiB/s wr, 1.98k op/s 2024-04-26T21:58:45.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:45 smithi055 ceph-mon[29221]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.92k op/s 2024-04-26T21:58:45.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:45 smithi064 ceph-mon[32765]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.92k op/s 2024-04-26T21:58:45.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:45 smithi165 ceph-mon[33296]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 1.92k op/s 2024-04-26T21:58:47.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:47 smithi055 ceph-mon[29221]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 826 MiB/s wr, 1.76k op/s 2024-04-26T21:58:47.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:47 smithi064 ceph-mon[32765]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 826 MiB/s wr, 1.76k op/s 2024-04-26T21:58:47.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:47 smithi165 ceph-mon[33296]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 826 MiB/s wr, 1.76k op/s 2024-04-26T21:58:47.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:58:47.638 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:58:48.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.058+0000 7f7d0fe47640 1 -- 172.21.15.55:0/1832680946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810ab40 msgr2=0x7f7d081113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.058+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/1832680946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810ab40 0x7f7d081113d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009860 tx=0x7f7cf802a380 comp rx=0 tx=0).stop 2024-04-26T21:58:48.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.062+0000 7f7d0fe47640 1 -- 172.21.15.55:0/1832680946 shutdown_connections 2024-04-26T21:58:48.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.062+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/1832680946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810ab40 0x7f7d081113d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.062+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/1832680946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d0810a180 0x7f7d0810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.062+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/1832680946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d08108f90 0x7f7d081093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.062+0000 7f7d0fe47640 1 -- 172.21.15.55:0/1832680946 >> 172.21.15.55:0/1832680946 conn(0x7f7d08077f30 msgr2=0x7f7d0807a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:48.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.069+0000 7f7d0fe47640 1 -- 172.21.15.55:0/1832680946 shutdown_connections 2024-04-26T21:58:48.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.072+0000 7f7d0fe47640 1 -- 172.21.15.55:0/1832680946 wait complete. 2024-04-26T21:58:48.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.073+0000 7f7d0fe47640 1 Processor -- start 2024-04-26T21:58:48.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.073+0000 7f7d0fe47640 1 -- start start 2024-04-26T21:58:48.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.073+0000 7f7d0fe47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d08108f90 0x7f7d080800d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.073+0000 7f7d0fe47640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810a180 0x7f7d08080610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.073+0000 7f7d0fe47640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d0810ab40 0x7f7d0819e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.073+0000 7f7d0fe47640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08114ae0 con 0x7f7d0810a180 2024-04-26T21:58:48.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.073+0000 7f7d0fe47640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08114960 con 0x7f7d08108f90 2024-04-26T21:58:48.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.073+0000 7f7d0fe47640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08114c60 con 0x7f7d0810ab40 2024-04-26T21:58:48.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.074+0000 7f7d0d3e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810a180 0x7f7d08080610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.075+0000 7f7d0d3e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810a180 0x7f7d08080610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34830/0 (socket says 172.21.15.55:34830) 2024-04-26T21:58:48.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.075+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/2420160543 learned_addr learned my addr 172.21.15.55:0/2420160543 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:48.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.075+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/2420160543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d0810ab40 msgr2=0x7f7d0819e1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:58:48.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.075+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d0810ab40 0x7f7d0819e1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.075+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/2420160543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d08108f90 msgr2=0x7f7d080800d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:58:48.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.075+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d08108f90 0x7f7d080800d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.075+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/2420160543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0819e720 con 0x7f7d0810a180 2024-04-26T21:58:48.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.078+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810a180 0x7f7d08080610 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9c0 tx=0x7f7d0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:48.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.084+0000 7f7cfeffd640 1 -- 172.21.15.55:0/2420160543 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400ccc0 con 0x7f7d0810a180 2024-04-26T21:58:48.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.084+0000 7f7cfeffd640 1 -- 172.21.15.55:0/2420160543 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d040054d0 con 0x7f7d0810a180 2024-04-26T21:58:48.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.084+0000 7f7cfeffd640 1 -- 172.21.15.55:0/2420160543 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04010430 con 0x7f7d0810a180 2024-04-26T21:58:48.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.086+0000 7f7d0fe47640 1 -- 172.21.15.55:0/2420160543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0819ea10 con 0x7f7d0810a180 2024-04-26T21:58:48.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.086+0000 7f7d0fe47640 1 -- 172.21.15.55:0/2420160543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0819ef10 con 0x7f7d0810a180 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.094+0000 7f7d127fd740 1 Processor -- start 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.095+0000 7f7cfeffd640 1 -- 172.21.15.55:0/2420160543 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d040046c0 con 0x7f7d0810a180 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.095+0000 7f7d127fd740 1 -- start start 2024-04-26T21:58:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.095+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2bf4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.096+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf50b0 0x558ab2bf5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.096+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bfc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.096+0000 7f7d127fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558ab2b58d80 con 0x558ab2bf3ee0 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.096+0000 7f7d127fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ab2b128b0 con 0x558ab2bf5a70 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.096+0000 7f7d127fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ab2b54d40 con 0x558ab2bf50b0 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.097+0000 7f7cfeffd640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cf40a3e40 0x7f7cf40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.098+0000 7f7d0dbe3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf50b0 0x558ab2bf5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.098+0000 7f7cfeffd640 1 -- 172.21.15.55:0/2420160543 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7d040d0a70 con 0x7f7d0810a180 2024-04-26T21:58:48.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.099+0000 7f7d0d3e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bfc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.099+0000 7f7d0d3e2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bfc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39802/0 (socket says 172.21.15.55:39802) 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.099+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/2176165911 learned_addr learned my addr 172.21.15.55:0/2176165911 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:48.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.101+0000 7f7d0e3e4640 1 --2- 172.21.15.55:0/2176165911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2bf4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.104+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/2176165911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf50b0 msgr2=0x558ab2bf5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.104+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/2176165911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf50b0 0x558ab2bf5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.104+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/2176165911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 msgr2=0x558ab2bf4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.104+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/2176165911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2bf4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.104+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/2176165911 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab2ae3a40 con 0x558ab2bf5a70 2024-04-26T21:58:48.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.107+0000 7f7d0e3e4640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cf40a3e40 0x7f7cf40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.109+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/2176165911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bfc300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d040a5290 tx=0x7f7d0409d310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b1a51ad4d6e6d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:48.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.109+0000 7f7d0e3e4640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cf40a3e40 0x7f7cf40a6300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009830 tx=0x7f7cf802a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:48.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.115+0000 7f7cf3fff640 1 -- 172.21.15.55:0/2176165911 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040a57c0 con 0x558ab2bf5a70 2024-04-26T21:58:48.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.116+0000 7f7cf3fff640 1 -- 172.21.15.55:0/2176165911 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d0409ae50 con 0x558ab2bf5a70 2024-04-26T21:58:48.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.116+0000 7f7cf3fff640 1 -- 172.21.15.55:0/2176165911 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040988a0 con 0x558ab2bf5a70 2024-04-26T21:58:48.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.118+0000 7f7d127fd740 1 -- 172.21.15.55:0/2176165911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 msgr2=0x558ab2bfc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.118+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2176165911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bfc300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d040a5290 tx=0x7f7d0409d310 comp rx=0 tx=0).stop 2024-04-26T21:58:48.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.120+0000 7f7d127fd740 1 -- 172.21.15.55:0/2176165911 shutdown_connections 2024-04-26T21:58:48.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.120+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2176165911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bfc300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.120+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2176165911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf50b0 0x558ab2bf5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.120+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2176165911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2bf4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.120+0000 7f7d127fd740 1 -- 172.21.15.55:0/2176165911 >> 172.21.15.55:0/2176165911 conn(0x558ab2bef3d0 msgr2=0x558ab2bef7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:48.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.121+0000 7f7d127fd740 1 -- 172.21.15.55:0/2176165911 shutdown_connections 2024-04-26T21:58:48.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.122+0000 7f7d127fd740 1 -- 172.21.15.55:0/2176165911 wait complete. 2024-04-26T21:58:48.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.122+0000 7f7d127fd740 1 -- start start 2024-04-26T21:58:48.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.122+0000 7f7d127fd740 1 Processor -- start 2024-04-26T21:58:48.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2c0f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 0x558ab2c0f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf5a70 0x558ab2c16900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d127fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558ab2c0d9d0 con 0x558ab2bf3ee0 2024-04-26T21:58:48.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d127fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ab2adfd10 con 0x558ab2bf50b0 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d127fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558ab2c0db50 con 0x558ab2bf5a70 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d127fd740 10 client.0 _ll_register_callbacks cb 0x558ab2b0c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d0dbe3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2c0f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d0dbe3640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2c0f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34850/0 (socket says 172.21.15.55:34850) 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.123+0000 7f7d0dbe3640 1 -- 172.21.15.55:0/1770892022 learned_addr learned my addr 172.21.15.55:0/1770892022 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 0x558ab2c0f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7d0dbe3640 1 -- 172.21.15.55:0/1770892022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf5a70 msgr2=0x558ab2c16900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7d0dbe3640 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf5a70 0x558ab2c16900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7d0dbe3640 1 -- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 msgr2=0x558ab2c0f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7d0dbe3640 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 0x558ab2c0f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7d0dbe3640 1 -- 172.21.15.55:0/1770892022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab29347d0 con 0x558ab2bf3ee0 2024-04-26T21:58:48.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 0x558ab2c0f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:48.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7d0dbe3640 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2c0f380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7d00010850 tx=0x7f7d00010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e91b6eea6cc5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:48.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.124+0000 7f7cf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:48.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.125+0000 7f7cf17fa640 1 -- 172.21.15.55:0/1770892022 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00017070 con 0x558ab2bf3ee0 2024-04-26T21:58:48.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.125+0000 7f7cf17fa640 1 -- 172.21.15.55:0/1770892022 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7d00004d20 con 0x558ab2bf3ee0 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.125+0000 7f7d127fd740 1 -- 172.21.15.55:0/1770892022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0409ae50 con 0x558ab2bf3ee0 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.125+0000 7f7cf17fa640 1 -- 172.21.15.55:0/1770892022 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7d0000ea60 con 0x558ab2bf3ee0 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.125+0000 7f7d127fd740 10 client.16311 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7d127fd740 1 -- 172.21.15.55:0/1770892022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558ab286a3f0 con 0x558ab2bf3ee0 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7cf17fa640 1 -- 172.21.15.55:0/1770892022 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d0409ae50 con 0x558ab2bf3ee0 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7cf17fa640 1 -- 172.21.15.55:0/1770892022 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7d00025420 con 0x558ab2bf3ee0 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7d127fd740 10 client.16311 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7d127fd740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7d127fd740 20 client.16311 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7d127fd740 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558ab2c1ae50 0x558ab2c1d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7d127fd740 4 client.16311 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:48.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.126+0000 7f7d127fd740 1 -- 172.21.15.55:0/1770892022 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ab2b66990 con 0x558ab2c1ae50 2024-04-26T21:58:48.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.129+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558ab2c1ae50 0x558ab2c1d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.135+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558ab2c1ae50 0x558ab2c1d2e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9608c9e9e14abc server_cookie=bc18751fe3040961 in_seq=0 out_seq=0 2024-04-26T21:58:48.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.136+0000 7f7cf17fa640 10 client.16311 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:48.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.142+0000 7f7cf17fa640 1 -- 172.21.15.55:0/1770892022 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ab2b66990 con 0x558ab2c1ae50 2024-04-26T21:58:48.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.142+0000 7f7cf17fa640 10 client.16311 handle_command_reply: tid=0 2024-04-26T21:58:48.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.142+0000 7f7d127fd740 1 client.16311 shutdown 2024-04-26T21:58:48.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 -- 172.21.15.55:0/1770892022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 msgr2=0x558ab2c0f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2c0f380 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7d00010850 tx=0x7f7d00010d10 comp rx=0 tx=0).stop 2024-04-26T21:58:48.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 -- 172.21.15.55:0/1770892022 shutdown_connections 2024-04-26T21:58:48.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf5a70 0x558ab2c16900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 0x558ab2c0f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558ab2c1ae50 0x558ab2c1d2e0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 --2- 172.21.15.55:0/1770892022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf3ee0 0x558ab2c0f380 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 -- 172.21.15.55:0/1770892022 >> 172.21.15.55:0/1770892022 conn(0x558ab2bef3d0 msgr2=0x558ab2bee550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7cf17fa640 0 client.16311 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 -- 172.21.15.55:0/1770892022 shutdown_connections 2024-04-26T21:58:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.143+0000 7f7d127fd740 1 -- 172.21.15.55:0/1770892022 wait complete. 2024-04-26T21:58:48.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.144+0000 7f7d127fd740 20 client.16311 trim_cache size 0 max 16384 2024-04-26T21:58:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d127fd740 1 Processor -- start 2024-04-26T21:58:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d127fd740 1 -- start start 2024-04-26T21:58:48.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf3ee0 0x558ab2bedcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf50b0 0x558ab2bee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bee740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d127fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7d081a1db0 con 0x558ab2bf50b0 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d127fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0819f680 con 0x558ab2bf5a70 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d127fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08072e60 con 0x558ab2bf3ee0 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.180+0000 7f7d0d3e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf50b0 0x558ab2bee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.181+0000 7f7d0d3e2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf50b0 0x558ab2bee200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34862/0 (socket says 172.21.15.55:34862) 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.181+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/595817968 learned_addr learned my addr 172.21.15.55:0/595817968 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.181+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/595817968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf3ee0 msgr2=0x558ab2bedcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.181+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/595817968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf3ee0 0x558ab2bedcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.181+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/595817968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 msgr2=0x558ab2bee740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.181+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/595817968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bee740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.181+0000 7f7d0d3e2640 1 -- 172.21.15.55:0/595817968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab282a0f0 con 0x558ab2bf50b0 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.181+0000 7f7d0d3e2640 1 --2- 172.21.15.55:0/595817968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf50b0 0x558ab2bee200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400c6a0 tx=0x7f7d040a1080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f939e6aaabca0036 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:48.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.182+0000 7f7cf37fe640 1 -- 172.21.15.55:0/595817968 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0409ae50 con 0x558ab2bf50b0 2024-04-26T21:58:48.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.182+0000 7f7cf37fe640 1 -- 172.21.15.55:0/595817968 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d040bd0c0 con 0x558ab2bf50b0 2024-04-26T21:58:48.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.182+0000 7f7cf37fe640 1 -- 172.21.15.55:0/595817968 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04098b00 con 0x558ab2bf50b0 2024-04-26T21:58:48.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 -- 172.21.15.55:0/595817968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf50b0 msgr2=0x558ab2bee200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 --2- 172.21.15.55:0/595817968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf50b0 0x558ab2bee200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400c6a0 tx=0x7f7d040a1080 comp rx=0 tx=0).stop 2024-04-26T21:58:48.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 -- 172.21.15.55:0/595817968 shutdown_connections 2024-04-26T21:58:48.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 --2- 172.21.15.55:0/595817968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf5a70 0x558ab2bee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 --2- 172.21.15.55:0/595817968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf50b0 0x558ab2bee200 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 --2- 172.21.15.55:0/595817968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf3ee0 0x558ab2bedcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 -- 172.21.15.55:0/595817968 >> 172.21.15.55:0/595817968 conn(0x558ab2bef3d0 msgr2=0x558ab2bea480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:48.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 -- 172.21.15.55:0/595817968 shutdown_connections 2024-04-26T21:58:48.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.183+0000 7f7d127fd740 1 -- 172.21.15.55:0/595817968 wait complete. 2024-04-26T21:58:48.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.184+0000 7f7d127fd740 1 -- start start 2024-04-26T21:58:48.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.185+0000 7f7d127fd740 1 Processor -- start 2024-04-26T21:58:48.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf3ee0 0x558ab2c11110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 0x558ab2c11650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d127fd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf5a70 0x558ab2c14600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d127fd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080734d0 con 0x558ab2bf5a70 2024-04-26T21:58:48.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d127fd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08074940 con 0x558ab2bf50b0 2024-04-26T21:58:48.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d127fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08073ba0 con 0x558ab2bf3ee0 2024-04-26T21:58:48.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d0e3e4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf5a70 0x558ab2c14600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d0e3e4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf5a70 0x558ab2c14600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34868/0 (socket says 172.21.15.55:34868) 2024-04-26T21:58:48.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.186+0000 7f7d0e3e4640 1 -- 172.21.15.55:0/2462300606 learned_addr learned my addr 172.21.15.55:0/2462300606 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:48.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.187+0000 7f7d127fd740 10 client.0 _ll_register_callbacks cb 0x7f7d000099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:48.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.187+0000 7f7d0e3e4640 1 -- 172.21.15.55:0/2462300606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf3ee0 msgr2=0x558ab2c11110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:48.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.187+0000 7f7d0e3e4640 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf3ee0 0x558ab2c11110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.187+0000 7f7d0e3e4640 1 -- 172.21.15.55:0/2462300606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 msgr2=0x558ab2c11650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:48.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.187+0000 7f7d0e3e4640 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 0x558ab2c11650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.187+0000 7f7d0e3e4640 1 -- 172.21.15.55:0/2462300606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab2b194b0 con 0x558ab2bf5a70 2024-04-26T21:58:48.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.188+0000 7f7d0e3e4640 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf5a70 0x558ab2c14600 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8035040 tx=0x7f7cf8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06caaa77abf7b04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:48.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.188+0000 7f7cf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:48.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.188+0000 7f7cf17fa640 1 -- 172.21.15.55:0/2462300606 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf804d060 con 0x558ab2bf5a70 2024-04-26T21:58:48.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.188+0000 7f7cf17fa640 1 -- 172.21.15.55:0/2462300606 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7cf8005460 con 0x558ab2bf5a70 2024-04-26T21:58:48.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.188+0000 7f7d127fd740 1 -- 172.21.15.55:0/2462300606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d040bd0c0 con 0x558ab2bf5a70 2024-04-26T21:58:48.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.189+0000 7f7cf17fa640 1 -- 172.21.15.55:0/2462300606 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7cf8004490 con 0x558ab2bf5a70 2024-04-26T21:58:48.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.189+0000 7f7d127fd740 10 client.16317 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:48.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.189+0000 7f7d127fd740 1 -- 172.21.15.55:0/2462300606 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558ab2c15db0 con 0x558ab2bf5a70 2024-04-26T21:58:48.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.191+0000 7f7cf17fa640 1 -- 172.21.15.55:0/2462300606 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d040bd0c0 con 0x558ab2bf5a70 2024-04-26T21:58:48.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.192+0000 7f7cf17fa640 1 -- 172.21.15.55:0/2462300606 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7cf8003830 con 0x558ab2bf5a70 2024-04-26T21:58:48.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.192+0000 7f7d127fd740 10 client.16317 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:48.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.192+0000 7f7d127fd740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:48.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.192+0000 7f7d127fd740 20 client.16317 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:48.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.192+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558ab2c81830 0x558ab2c83c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.192+0000 7f7d127fd740 4 client.16317 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:58:48.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.192+0000 7f7d127fd740 1 -- 172.21.15.55:0/2462300606 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ab2b60800 con 0x558ab2c81830 2024-04-26T21:58:48.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.195+0000 7f7d0dbe3640 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558ab2c81830 0x558ab2c83c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.200+0000 7f7d0dbe3640 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558ab2c81830 0x558ab2c83c50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b411ca8b29749a server_cookie=aa4bbc51fb452193 in_seq=0 out_seq=0 2024-04-26T21:58:48.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.200+0000 7f7cf17fa640 10 client.16317 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:48.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.205+0000 7f7cf17fa640 1 -- 172.21.15.55:0/2462300606 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558ab2b60800 con 0x558ab2c81830 2024-04-26T21:58:48.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.205+0000 7f7cf17fa640 10 client.16317 handle_command_reply: tid=0 2024-04-26T21:58:48.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.205+0000 7f7d127fd740 1 client.16317 shutdown 2024-04-26T21:58:48.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.205+0000 7f7d127fd740 1 -- 172.21.15.55:0/2462300606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf5a70 msgr2=0x558ab2c14600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.205+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf5a70 0x558ab2c14600 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8035040 tx=0x7f7cf8005f90 comp rx=0 tx=0).stop 2024-04-26T21:58:48.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 1 -- 172.21.15.55:0/2462300606 shutdown_connections 2024-04-26T21:58:48.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558ab2c81830 0x558ab2c83c50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558ab2bf5a70 0x558ab2c14600 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ab2bf50b0 0x558ab2c11650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7cf17fa640 0 client.16317 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 1 --2- 172.21.15.55:0/2462300606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ab2bf3ee0 0x558ab2c11110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 1 -- 172.21.15.55:0/2462300606 >> 172.21.15.55:0/2462300606 conn(0x558ab2bef3d0 msgr2=0x558ab2bf23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 1 -- 172.21.15.55:0/2462300606 shutdown_connections 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 1 -- 172.21.15.55:0/2462300606 wait complete. 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.206+0000 7f7d127fd740 20 client.16317 trim_cache size 0 max 16384 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:58:48.209 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:58:48.210 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 -- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cf40a3e40 msgr2=0x7f7cf40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cf40a3e40 0x7f7cf40a6300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009830 tx=0x7f7cf802a380 comp rx=0 tx=0).stop 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 -- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810a180 msgr2=0x7f7d08080610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810a180 0x7f7d08080610 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9c0 tx=0x7f7d0400ee80 comp rx=0 tx=0).stop 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 -- 172.21.15.55:0/2420160543 shutdown_connections 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cf40a3e40 0x7f7cf40a6300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d0810ab40 0x7f7d0819e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7d0810a180 0x7f7d08080610 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 --2- 172.21.15.55:0/2420160543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d08108f90 0x7f7d080800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.208+0000 7f7d0fe47640 1 -- 172.21.15.55:0/2420160543 >> 172.21.15.55:0/2420160543 conn(0x7f7d08077f30 msgr2=0x7f7d0810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.209+0000 7f7d0fe47640 1 -- 172.21.15.55:0/2420160543 shutdown_connections 2024-04-26T21:58:48.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.209+0000 7f7d0fe47640 1 -- 172.21.15.55:0/2420160543 wait complete. 2024-04-26T21:58:48.227 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:58:48.227 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:58:48.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 -- 172.21.15.55:0/3560843056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c074760 msgr2=0x7fb00c10b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 --2- 172.21.15.55:0/3560843056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c074760 0x7fb00c10b940 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb000009900 tx=0x7fb00002a410 comp rx=0 tx=0).stop 2024-04-26T21:58:48.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 -- 172.21.15.55:0/3560843056 shutdown_connections 2024-04-26T21:58:48.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 --2- 172.21.15.55:0/3560843056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb00c10ea10 0x7fb00c119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 --2- 172.21.15.55:0/3560843056 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c10bf10 0x7fb00c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 --2- 172.21.15.55:0/3560843056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c074760 0x7fb00c10b940 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 -- 172.21.15.55:0/3560843056 >> 172.21.15.55:0/3560843056 conn(0x7fb00c06f690 msgr2=0x7fb00c071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:48.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 -- 172.21.15.55:0/3560843056 shutdown_connections 2024-04-26T21:58:48.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.621+0000 7fb011262640 1 -- 172.21.15.55:0/3560843056 wait complete. 2024-04-26T21:58:48.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.622+0000 7fb011262640 1 Processor -- start 2024-04-26T21:58:48.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.622+0000 7fb011262640 1 -- start start 2024-04-26T21:58:48.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.622+0000 7fb011262640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb00c074760 0x7fb00c1a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.623+0000 7fb011262640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c10bf10 0x7fb00c1a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.623+0000 7fb011262640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c10ea10 0x7fb00c1aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.623+0000 7fb011262640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c11d180 con 0x7fb00c10bf10 2024-04-26T21:58:48.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.623+0000 7fb011262640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c11d000 con 0x7fb00c10ea10 2024-04-26T21:58:48.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.623+0000 7fb011262640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c11d300 con 0x7fb00c074760 2024-04-26T21:58:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.623+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c10bf10 0x7fb00c1a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.623+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c10bf10 0x7fb00c1a7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34884/0 (socket says 172.21.15.55:34884) 2024-04-26T21:58:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.623+0000 7fb00b7fe640 1 -- 172.21.15.55:0/651048764 learned_addr learned my addr 172.21.15.55:0/651048764 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.624+0000 7fb00b7fe640 1 -- 172.21.15.55:0/651048764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb00c074760 msgr2=0x7fb00c1a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.624+0000 7fb00b7fe640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb00c074760 0x7fb00c1a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.624+0000 7fb00b7fe640 1 -- 172.21.15.55:0/651048764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c10ea10 msgr2=0x7fb00c1aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:48.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.624+0000 7fb010a61640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c10ea10 0x7fb00c1aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.624+0000 7fb00b7fe640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c10ea10 0x7fb00c1aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.624+0000 7fb00b7fe640 1 -- 172.21.15.55:0/651048764 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c1af3c0 con 0x7fb00c10bf10 2024-04-26T21:58:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.624+0000 7fb010a61640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c10ea10 0x7fb00c1aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:48.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.625+0000 7fb00b7fe640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c10bf10 0x7fb00c1a7d30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb00400ba50 tx=0x7fb00400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:48.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.626+0000 7fb0097fa640 1 -- 172.21.15.55:0/651048764 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004019070 con 0x7fb00c10bf10 2024-04-26T21:58:48.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.626+0000 7fb0097fa640 1 -- 172.21.15.55:0/651048764 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00400cc10 con 0x7fb00c10bf10 2024-04-26T21:58:48.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.626+0000 7fb0097fa640 1 -- 172.21.15.55:0/651048764 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040075d0 con 0x7fb00c10bf10 2024-04-26T21:58:48.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.626+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1af6b0 con 0x7fb00c10bf10 2024-04-26T21:58:48.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.626+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1afbb0 con 0x7fb00c10bf10 2024-04-26T21:58:48.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.628+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c072190 con 0x7fb00c10bf10 2024-04-26T21:58:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.634+0000 7fb0097fa640 1 -- 172.21.15.55:0/651048764 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb004014440 con 0x7fb00c10bf10 2024-04-26T21:58:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.634+0000 7fb0097fa640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7faff40a3d70 0x7faff40a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.635+0000 7fb0097fa640 1 -- 172.21.15.55:0/651048764 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb0040d0350 con 0x7fb00c10bf10 2024-04-26T21:58:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.636+0000 7fb00bfff640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7faff40a3d70 0x7faff40a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:48.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.638+0000 7fb00bfff640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7faff40a3d70 0x7faff40a6230 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb0000096c0 tx=0x7fb000000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:48.641+0000 7fb0097fa640 1 -- 172.21.15.55:0/651048764 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb00409cc40 con 0x7fb00c10bf10 2024-04-26T21:58:49.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.356+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb00c02dd80 con 0x7fb00c10bf10 2024-04-26T21:58:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.359+0000 7fb0097fa640 1 -- 172.21.15.55:0/651048764 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fb00409c540 con 0x7fb00c10bf10 2024-04-26T21:58:49.362 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:58:49.362 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:58:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T21:58:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.373+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7faff40a3d70 msgr2=0x7faff40a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.373+0000 7fb011262640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7faff40a3d70 0x7faff40a6230 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb0000096c0 tx=0x7fb000000bc0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.373+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c10bf10 msgr2=0x7fb00c1a7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.373+0000 7fb011262640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c10bf10 0x7fb00c1a7d30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb00400ba50 tx=0x7fb00400bf10 comp rx=0 tx=0).stop 2024-04-26T21:58:49.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.375+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 shutdown_connections 2024-04-26T21:58:49.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.376+0000 7fb011262640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7faff40a3d70 0x7faff40a6230 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.376+0000 7fb011262640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c10ea10 0x7fb00c1aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.376+0000 7fb011262640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00c10bf10 0x7fb00c1a7d30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.376+0000 7fb011262640 1 --2- 172.21.15.55:0/651048764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb00c074760 0x7fb00c1a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.376+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 >> 172.21.15.55:0/651048764 conn(0x7fb00c06f690 msgr2=0x7fb00c10b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.376+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 shutdown_connections 2024-04-26T21:58:49.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.376+0000 7fb011262640 1 -- 172.21.15.55:0/651048764 wait complete. 2024-04-26T21:58:49.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:49 smithi055 ceph-mon[29221]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 908 MiB/s wr, 1.80k op/s 2024-04-26T21:58:49.441 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:58:49.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:49 smithi064 ceph-mon[32765]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 908 MiB/s wr, 1.80k op/s 2024-04-26T21:58:49.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:49 smithi165 ceph-mon[33296]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 908 MiB/s wr, 1.80k op/s 2024-04-26T21:58:49.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.798+0000 7f7cce15d640 1 -- 172.21.15.55:0/2593650970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cc8074750 msgr2=0x7f7cc810ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.798+0000 7f7cce15d640 1 --2- 172.21.15.55:0/2593650970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cc8074750 0x7f7cc810ca50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009900 tx=0x7f7cb802a440 comp rx=0 tx=0).stop 2024-04-26T21:58:49.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.801+0000 7f7cce15d640 1 -- 172.21.15.55:0/2593650970 shutdown_connections 2024-04-26T21:58:49.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.801+0000 7f7cce15d640 1 --2- 172.21.15.55:0/2593650970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc810fb00 0x7f7cc8111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.801+0000 7f7cce15d640 1 --2- 172.21.15.55:0/2593650970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cc810d020 0x7f7cc810f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.801+0000 7f7cce15d640 1 --2- 172.21.15.55:0/2593650970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cc8074750 0x7f7cc810ca50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.801+0000 7f7cce15d640 1 -- 172.21.15.55:0/2593650970 >> 172.21.15.55:0/2593650970 conn(0x7f7cc806f780 msgr2=0x7f7cc8071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.805+0000 7f7cce15d640 1 -- 172.21.15.55:0/2593650970 shutdown_connections 2024-04-26T21:58:49.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.810+0000 7f7cce15d640 1 -- 172.21.15.55:0/2593650970 wait complete. 2024-04-26T21:58:49.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.811+0000 7f7cce15d640 1 Processor -- start 2024-04-26T21:58:49.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.813+0000 7f7cce15d640 1 -- start start 2024-04-26T21:58:49.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.813+0000 7f7cce15d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cc8074750 0x7f7cc810a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.813+0000 7f7cce15d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cc810d020 0x7f7cc81a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.813+0000 7f7cce15d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc810fb00 0x7f7cc81a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.813+0000 7f7cce15d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc8115a10 con 0x7f7cc810d020 2024-04-26T21:58:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.813+0000 7f7cce15d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc8115890 con 0x7f7cc810fb00 2024-04-26T21:58:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.813+0000 7f7cce15d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc8115b90 con 0x7f7cc8074750 2024-04-26T21:58:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.815+0000 7f7ccd95c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc810fb00 0x7f7cc81a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.815+0000 7f7ccd95c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc810fb00 0x7f7cc81a41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39874/0 (socket says 172.21.15.55:39874) 2024-04-26T21:58:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.815+0000 7f7ccd95c640 1 -- 172.21.15.55:0/1200600564 learned_addr learned my addr 172.21.15.55:0/1200600564 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:49.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.815+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cc8074750 0x7f7cc810a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.816+0000 7f7ccd95c640 1 -- 172.21.15.55:0/1200600564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cc8074750 msgr2=0x7f7cc810a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.816+0000 7f7ccd95c640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cc8074750 0x7f7cc810a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.816+0000 7f7ccd95c640 1 -- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cc810d020 msgr2=0x7f7cc81a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.816+0000 7f7ccd95c640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cc810d020 0x7f7cc81a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.816+0000 7f7ccd95c640 1 -- 172.21.15.55:0/1200600564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc81a4aa0 con 0x7f7cc810fb00 2024-04-26T21:58:49.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.816+0000 7f7ccd95c640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc810fb00 0x7f7cc81a41e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c970 tx=0x7f7cc400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:49.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.817+0000 7f7cbe7fc640 1 -- 172.21.15.55:0/1200600564 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4007bb0 con 0x7f7cc810fb00 2024-04-26T21:58:49.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.818+0000 7f7cce15d640 1 -- 172.21.15.55:0/1200600564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc81a4d90 con 0x7f7cc810fb00 2024-04-26T21:58:49.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.818+0000 7f7cce15d640 1 -- 172.21.15.55:0/1200600564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc81a52e0 con 0x7f7cc810fb00 2024-04-26T21:58:49.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.818+0000 7f7cbe7fc640 1 -- 172.21.15.55:0/1200600564 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc4007d50 con 0x7f7cc810fb00 2024-04-26T21:58:49.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.818+0000 7f7cbe7fc640 1 -- 172.21.15.55:0/1200600564 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4018730 con 0x7f7cc810fb00 2024-04-26T21:58:49.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.833+0000 7f7cbe7fc640 1 -- 172.21.15.55:0/1200600564 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7cc40213f0 con 0x7f7cc810fb00 2024-04-26T21:58:49.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.835+0000 7f7cbe7fc640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cb40a3e40 0x7f7cb40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.835+0000 7f7cbe7fc640 1 -- 172.21.15.55:0/1200600564 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7cc40d1550 con 0x7f7cc810fb00 2024-04-26T21:58:49.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.839+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cb40a3e40 0x7f7cb40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.841+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cb40a3e40 0x7f7cb40a6300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8000c00 tx=0x7f7cb8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.844+0000 7f7cd21b9740 1 Processor -- start 2024-04-26T21:58:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.844+0000 7f7cd21b9740 1 -- start start 2024-04-26T21:58:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.844+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eabae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.845+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 0x55e42eabc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.845+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eac2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.845+0000 7f7cd21b9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e42ea1ed80 con 0x55e42eabc5c0 2024-04-26T21:58:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.845+0000 7f7cd21b9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e42e9d8a20 con 0x55e42eabaa30 2024-04-26T21:58:49.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.845+0000 7f7cd21b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e42ea1ad40 con 0x55e42eabbc00 2024-04-26T21:58:49.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.846+0000 7f7ccd15b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eabae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.846+0000 7f7ccd15b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eabae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39876/0 (socket says 172.21.15.55:39876) 2024-04-26T21:58:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.846+0000 7f7ccd15b640 1 -- 172.21.15.55:0/1737292008 learned_addr learned my addr 172.21.15.55:0/1737292008 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:49.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.847+0000 7f7ccd95c640 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eac2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.847+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 0x55e42eabc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.848+0000 7f7ccd15b640 1 -- 172.21.15.55:0/1737292008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 msgr2=0x55e42eabc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:49.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.848+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 0x55e42eabc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.848+0000 7f7ccd15b640 1 -- 172.21.15.55:0/1737292008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 msgr2=0x55e42eac2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:49.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.848+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eac2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.848+0000 7f7ccd15b640 1 -- 172.21.15.55:0/1737292008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42e9e3c30 con 0x55e42eabaa30 2024-04-26T21:58:49.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.851+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eabae50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009900 tx=0x7f7cb803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b57d3a2b8b0a85a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:49.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.853+0000 7f7cb37fe640 1 -- 172.21.15.55:0/1737292008 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8031440 con 0x55e42eabaa30 2024-04-26T21:58:49.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.853+0000 7f7cb37fe640 1 -- 172.21.15.55:0/1737292008 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb8031a90 con 0x55e42eabaa30 2024-04-26T21:58:49.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.854+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1737292008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 msgr2=0x55e42eabae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.854+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eabae50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8009900 tx=0x7f7cb803f000 comp rx=0 tx=0).stop 2024-04-26T21:58:49.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.861+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1737292008 shutdown_connections 2024-04-26T21:58:49.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.861+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eac2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.861+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 0x55e42eabc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.861+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1737292008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eabae50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.861+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1737292008 >> 172.21.15.55:0/1737292008 conn(0x55e42e9fb910 msgr2=0x55e42eab6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.865+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1737292008 shutdown_connections 2024-04-26T21:58:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.865+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1737292008 wait complete. 2024-04-26T21:58:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.866+0000 7f7cd21b9740 1 -- start start 2024-04-26T21:58:49.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.866+0000 7f7cd21b9740 1 Processor -- start 2024-04-26T21:58:49.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.867+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eac4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.867+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 0x55e42eac51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.867+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eacc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.867+0000 7f7cd21b9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e42eac3610 con 0x55e42eabc5c0 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.867+0000 7f7cd21b9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e42e9a5dc0 con 0x55e42eabaa30 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.867+0000 7f7cd21b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e42eac3790 con 0x55e42eabbc00 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.867+0000 7f7cd21b9740 10 client.0 _ll_register_callbacks cb 0x55e42e9c1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.868+0000 7f7ccd95c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eacc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.868+0000 7f7ccd95c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eacc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34918/0 (socket says 172.21.15.55:34918) 2024-04-26T21:58:49.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.868+0000 7f7ccd95c640 1 -- 172.21.15.55:0/1923378752 learned_addr learned my addr 172.21.15.55:0/1923378752 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:49.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.869+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eac4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.869+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 0x55e42eac51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.870+0000 7f7ccd95c640 1 -- 172.21.15.55:0/1923378752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 msgr2=0x55e42eac51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:49.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.870+0000 7f7ccd95c640 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 0x55e42eac51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.870+0000 7f7ccd95c640 1 -- 172.21.15.55:0/1923378752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 msgr2=0x55e42eac4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:49.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.870+0000 7f7ccd95c640 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eac4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.871+0000 7f7ccd95c640 1 -- 172.21.15.55:0/1923378752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42e7303f0 con 0x55e42eabc5c0 2024-04-26T21:58:49.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.881+0000 7f7ccd95c640 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eacc230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7cc409a000 tx=0x7f7cc409dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffb9ca5dfe592a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:49.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.882+0000 7f7cb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:49.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.882+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/1923378752 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40bb060 con 0x55e42eabc5c0 2024-04-26T21:58:49.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.883+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1923378752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb8031a90 con 0x55e42eabc5c0 2024-04-26T21:58:49.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.884+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/1923378752 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7cc40a6010 con 0x55e42eabc5c0 2024-04-26T21:58:49.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.884+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/1923378752 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7cc40b4070 con 0x55e42eabc5c0 2024-04-26T21:58:49.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.884+0000 7f7cd21b9740 10 client.16329 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:49.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.884+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1923378752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e42e7fa7d0 con 0x55e42eabc5c0 2024-04-26T21:58:49.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.886+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/1923378752 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb8031a90 con 0x55e42eabc5c0 2024-04-26T21:58:49.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.888+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/1923378752 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7cc40bc360 con 0x55e42eabc5c0 2024-04-26T21:58:49.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.888+0000 7f7cd21b9740 10 client.16329 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:49.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.888+0000 7f7cd21b9740 10 client.16329 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:58:49.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.888+0000 7f7cd21b9740 20 client.16329 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:49.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.888+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55e42ead1120 0x55e42ead35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.888+0000 7f7cd21b9740 4 client.16329 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:49.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.888+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1923378752 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e42ea2c990 con 0x55e42ead1120 2024-04-26T21:58:49.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.890+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55e42ead1120 0x55e42ead35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.895+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55e42ead1120 0x55e42ead35a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=532a6877bdb8f698 server_cookie=1be0a76e849e4fb0 in_seq=0 out_seq=0 2024-04-26T21:58:49.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.895+0000 7f7cb0ff9640 10 client.16329 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:58:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.898+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/1923378752 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e42ea2c990 con 0x55e42ead1120 2024-04-26T21:58:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cb0ff9640 10 client.16329 handle_command_reply: tid=0 2024-04-26T21:58:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 client.16329 shutdown 2024-04-26T21:58:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1923378752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 msgr2=0x55e42eacc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eacc230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7cc409a000 tx=0x7f7cc409dd30 comp rx=0 tx=0).stop 2024-04-26T21:58:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1923378752 shutdown_connections 2024-04-26T21:58:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55e42ead1120 0x55e42ead35a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabc5c0 0x55e42eacc230 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabbc00 0x55e42eac51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/1923378752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eac4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1923378752 >> 172.21.15.55:0/1923378752 conn(0x55e42e9fb910 msgr2=0x55e42eab53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.899+0000 7f7cb0ff9640 0 client.16329 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:58:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.900+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1923378752 shutdown_connections 2024-04-26T21:58:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.900+0000 7f7cd21b9740 1 -- 172.21.15.55:0/1923378752 wait complete. 2024-04-26T21:58:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.900+0000 7f7cd21b9740 20 client.16329 trim_cache size 0 max 16384 2024-04-26T21:58:49.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.937+0000 7f7cd21b9740 1 Processor -- start 2024-04-26T21:58:49.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.937+0000 7f7cd21b9740 1 -- start start 2024-04-26T21:58:49.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.938+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eab6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.938+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabbc00 0x55e42eacc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.938+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 0x55e42eab9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.939+0000 7f7ccd15b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eab6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.939+0000 7f7ccc95a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabbc00 0x55e42eacc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.939+0000 7f7ccc95a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabbc00 0x55e42eacc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34932/0 (socket says 172.21.15.55:34932) 2024-04-26T21:58:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.939+0000 7f7ccd95c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 0x55e42eab9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.939+0000 7f7ccc95a640 1 -- 172.21.15.55:0/3662418504 learned_addr learned my addr 172.21.15.55:0/3662418504 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.940+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3662418504 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac010d00 con 0x55e42eabbc00 2024-04-26T21:58:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.940+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3662418504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac00e5d0 con 0x55e42eabaa30 2024-04-26T21:58:49.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.940+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3662418504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac00c4d0 con 0x55e42eabc5c0 2024-04-26T21:58:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.941+0000 7f7ccc95a640 1 -- 172.21.15.55:0/3662418504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 msgr2=0x55e42eab9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.941+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/3662418504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 0x55e42eab9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.941+0000 7f7ccc95a640 1 -- 172.21.15.55:0/3662418504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 msgr2=0x55e42eab6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.941+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/3662418504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eab6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.941+0000 7f7ccc95a640 1 -- 172.21.15.55:0/3662418504 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e42eab6fe0 con 0x55e42eabbc00 2024-04-26T21:58:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.941+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/3662418504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabbc00 0x55e42eacc1f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0007cf0 tx=0x7f7cc000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600054d56b832f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.942+0000 7f7cb2ffd640 1 -- 172.21.15.55:0/3662418504 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0004d20 con 0x55e42eabbc00 2024-04-26T21:58:49.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.942+0000 7f7cb2ffd640 1 -- 172.21.15.55:0/3662418504 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc00054d0 con 0x55e42eabbc00 2024-04-26T21:58:49.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.942+0000 7f7cb2ffd640 1 -- 172.21.15.55:0/3662418504 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0011880 con 0x55e42eabbc00 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.943+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3662418504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabbc00 msgr2=0x55e42eacc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.943+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3662418504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabbc00 0x55e42eacc1f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0007cf0 tx=0x7f7cc000f730 comp rx=0 tx=0).stop 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.944+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3662418504 shutdown_connections 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.944+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3662418504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 0x55e42eab9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.944+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3662418504 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabbc00 0x55e42eacc1f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.944+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3662418504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabaa30 0x55e42eab6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.944+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3662418504 >> 172.21.15.55:0/3662418504 conn(0x55e42e9fb910 msgr2=0x55e42eab5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.944+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3662418504 shutdown_connections 2024-04-26T21:58:49.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.944+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3662418504 wait complete. 2024-04-26T21:58:49.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.945+0000 7f7cd21b9740 1 -- start start 2024-04-26T21:58:49.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.946+0000 7f7cd21b9740 1 Processor -- start 2024-04-26T21:58:49.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabaa30 0x55e42eac9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabbc00 0x55e42eac97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7cd21b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 0x55e42eb356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7cd21b9740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac00ca90 con 0x55e42eabaa30 2024-04-26T21:58:49.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7ccd15b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabaa30 0x55e42eac9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7ccd15b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabaa30 0x55e42eac9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34944/0 (socket says 172.21.15.55:34944) 2024-04-26T21:58:49.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7ccd15b640 1 -- 172.21.15.55:0/3041509112 learned_addr learned my addr 172.21.15.55:0/3041509112 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:49.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac00df00 con 0x55e42eabbc00 2024-04-26T21:58:49.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac00d160 con 0x55e42eabc5c0 2024-04-26T21:58:49.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.947+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabbc00 0x55e42eac97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.948+0000 7f7cd21b9740 10 client.0 _ll_register_callbacks cb 0x7f7cc4007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.948+0000 7f7ccd15b640 1 -- 172.21.15.55:0/3041509112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 msgr2=0x55e42eb356d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.948+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 0x55e42eb356d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.948+0000 7f7ccd15b640 1 -- 172.21.15.55:0/3041509112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabbc00 msgr2=0x55e42eac97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.948+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabbc00 0x55e42eac97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.948+0000 7f7ccd15b640 1 -- 172.21.15.55:0/3041509112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42eb35cc0 con 0x55e42eabaa30 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.948+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabbc00 0x55e42eac97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.949+0000 7f7ccd15b640 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabaa30 0x55e42eac9280 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8004650 tx=0x7f7cb8005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32b3e486b61df6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.949+0000 7f7cb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.949+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/3041509112 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb805f470 con 0x55e42eabaa30 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.949+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/3041509112 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7cb805f610 con 0x55e42eabaa30 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.950+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc00054d0 con 0x55e42eabaa30 2024-04-26T21:58:49.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.950+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/3041509112 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7cb805f810 con 0x55e42eabaa30 2024-04-26T21:58:49.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.950+0000 7f7cd21b9740 10 client.16335 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:49.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.950+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e42eb36f90 con 0x55e42eabaa30 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.951+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/3041509112 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc00054d0 con 0x55e42eabaa30 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.951+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/3041509112 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7cb803d420 con 0x55e42eabaa30 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.951+0000 7f7cd21b9740 10 client.16335 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.951+0000 7f7cd21b9740 10 client.16335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.951+0000 7f7cd21b9740 20 client.16335 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.951+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55e42eb375c0 0x55e42eb399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.951+0000 7f7cd21b9740 4 client.16335 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.951+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e42ea26800 con 0x55e42eb375c0 2024-04-26T21:58:49.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.952+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55e42eb375c0 0x55e42eb399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:49.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.954+0000 7f7ccc95a640 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55e42eb375c0 0x55e42eb399b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97df36659f5e112a server_cookie=7755ff7b8099ceda in_seq=0 out_seq=0 2024-04-26T21:58:49.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.954+0000 7f7cb0ff9640 10 client.16335 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:58:49.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.954+0000 7f7cb0ff9640 1 -- 172.21.15.55:0/3041509112 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e42ea26800 con 0x55e42eb375c0 2024-04-26T21:58:49.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.954+0000 7f7cb0ff9640 10 client.16335 handle_command_reply: tid=0 2024-04-26T21:58:49.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.954+0000 7f7cd21b9740 1 client.16335 shutdown 2024-04-26T21:58:49.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabaa30 msgr2=0x55e42eac9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabaa30 0x55e42eac9280 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8004650 tx=0x7f7cb8005370 comp rx=0 tx=0).stop 2024-04-26T21:58:49.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 shutdown_connections 2024-04-26T21:58:49.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55e42eb375c0 0x55e42eb399b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e42eabc5c0 0x55e42eb356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e42eabbc00 0x55e42eac97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 --2- 172.21.15.55:0/3041509112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e42eabaa30 0x55e42eac9280 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cb0ff9640 0 client.16335 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:58:49.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 >> 172.21.15.55:0/3041509112 conn(0x55e42e9fb910 msgr2=0x55e42eab9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 shutdown_connections 2024-04-26T21:58:49.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 1 -- 172.21.15.55:0/3041509112 wait complete. 2024-04-26T21:58:49.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.955+0000 7f7cd21b9740 20 client.16335 trim_cache size 0 max 16384 2024-04-26T21:58:49.957 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 -- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cb40a3e40 msgr2=0x7f7cb40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cb40a3e40 0x7f7cb40a6300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8000c00 tx=0x7f7cb8005fd0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 -- 172.21.15.55:0/1200600564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc810fb00 msgr2=0x7f7cc81a41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc810fb00 0x7f7cc81a41e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c970 tx=0x7f7cc400ce30 comp rx=0 tx=0).stop 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 -- 172.21.15.55:0/1200600564 shutdown_connections 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7cb40a3e40 0x7f7cb40a6300 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7cc810fb00 0x7f7cc81a41e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cc810d020 0x7f7cc81a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 --2- 172.21.15.55:0/1200600564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7cc8074750 0x7f7cc810a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:49.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.956+0000 7f7cce15d640 1 -- 172.21.15.55:0/1200600564 >> 172.21.15.55:0/1200600564 conn(0x7f7cc806f780 msgr2=0x7f7cc810b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:49.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.957+0000 7f7cce15d640 1 -- 172.21.15.55:0/1200600564 shutdown_connections 2024-04-26T21:58:49.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:49.957+0000 7f7cce15d640 1 -- 172.21.15.55:0/1200600564 wait complete. 2024-04-26T21:58:49.992 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:58:50.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.372+0000 7f017bcd1640 1 -- 172.21.15.55:0/3515734518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0174101b40 msgr2=0x7f0174101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.372+0000 7f017bcd1640 1 --2- 172.21.15.55:0/3515734518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0174101b40 0x7f0174101fa0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0170009900 tx=0x7f017002a410 comp rx=0 tx=0).stop 2024-04-26T21:58:50.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.373+0000 7f017bcd1640 1 -- 172.21.15.55:0/3515734518 shutdown_connections 2024-04-26T21:58:50.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.373+0000 7f017bcd1640 1 --2- 172.21.15.55:0/3515734518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01741024e0 0x7f0174102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.373+0000 7f017bcd1640 1 --2- 172.21.15.55:0/3515734518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0174101b40 0x7f0174101fa0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.373+0000 7f017bcd1640 1 --2- 172.21.15.55:0/3515734518 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01741011e0 0x7f0174101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.373+0000 7f017bcd1640 1 -- 172.21.15.55:0/3515734518 >> 172.21.15.55:0/3515734518 conn(0x7f01740fcde0 msgr2=0x7f01740ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.374+0000 7f017bcd1640 1 -- 172.21.15.55:0/3515734518 shutdown_connections 2024-04-26T21:58:50.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.374+0000 7f017bcd1640 1 -- 172.21.15.55:0/3515734518 wait complete. 2024-04-26T21:58:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.374+0000 7f017bcd1640 1 Processor -- start 2024-04-26T21:58:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f017bcd1640 1 -- start start 2024-04-26T21:58:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f017bcd1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01741011e0 0x7f017419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f017bcd1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0174101b40 0x7f017419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f017bcd1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01741024e0 0x7f01741a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f017bcd1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0174110750 con 0x7f01741011e0 2024-04-26T21:58:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f017bcd1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f01741105d0 con 0x7f01741024e0 2024-04-26T21:58:50.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f017bcd1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01741108d0 con 0x7f0174101b40 2024-04-26T21:58:50.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f0179a6d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01741011e0 0x7f017419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f0179a6d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01741011e0 0x7f017419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34962/0 (socket says 172.21.15.55:34962) 2024-04-26T21:58:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.375+0000 7f0179a6d640 1 -- 172.21.15.55:0/2439336761 learned_addr learned my addr 172.21.15.55:0/2439336761 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.376+0000 7f0179a6d640 1 -- 172.21.15.55:0/2439336761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0174101b40 msgr2=0x7f017419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.376+0000 7f0179a6d640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0174101b40 0x7f017419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.376+0000 7f0179a6d640 1 -- 172.21.15.55:0/2439336761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01741024e0 msgr2=0x7f01741a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.376+0000 7f0179a6d640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01741024e0 0x7f01741a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.376+0000 7f0179a6d640 1 -- 172.21.15.55:0/2439336761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01741a2a80 con 0x7f01741011e0 2024-04-26T21:58:50.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.376+0000 7f0179a6d640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01741011e0 0x7f017419ae60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f016400d950 tx=0x7f016400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.377+0000 7f016affd640 1 -- 172.21.15.55:0/2439336761 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164017070 con 0x7f01741011e0 2024-04-26T21:58:50.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.377+0000 7f017bcd1640 1 -- 172.21.15.55:0/2439336761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01741a2d70 con 0x7f01741011e0 2024-04-26T21:58:50.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.377+0000 7f017bcd1640 1 -- 172.21.15.55:0/2439336761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741a3300 con 0x7f01741011e0 2024-04-26T21:58:50.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.377+0000 7f016affd640 1 -- 172.21.15.55:0/2439336761 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0164009c30 con 0x7f01741011e0 2024-04-26T21:58:50.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.377+0000 7f016affd640 1 -- 172.21.15.55:0/2439336761 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016400f620 con 0x7f01741011e0 2024-04-26T21:58:50.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.381+0000 7f017e9ef740 1 Processor -- start 2024-04-26T21:58:50.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.381+0000 7f017e9ef740 1 -- start start 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.382+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f2ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.382+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 0x564cc7f2fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.382+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f36c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.382+0000 7f017e9ef740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564cc7e92d80 con 0x564cc7f2e820 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.382+0000 7f017e9ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564cc7e4ca20 con 0x564cc7f303b0 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.382+0000 7f017e9ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564cc7e8ed40 con 0x564cc7f2f9f0 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.383+0000 7f017926c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f36c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.383+0000 7f017926c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f36c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39942/0 (socket says 172.21.15.55:39942) 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.383+0000 7f017926c640 1 -- 172.21.15.55:0/871090664 learned_addr learned my addr 172.21.15.55:0/871090664 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.383+0000 7f016affd640 1 -- 172.21.15.55:0/2439336761 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0164010460 con 0x7f01741011e0 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.383+0000 7f017a26e640 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f2ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.383+0000 7f0179a6d640 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 0x564cc7f2fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.384+0000 7f017a26e640 1 -- 172.21.15.55:0/871090664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 msgr2=0x564cc7f2fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.384+0000 7f017a26e640 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 0x564cc7f2fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.384+0000 7f017a26e640 1 -- 172.21.15.55:0/871090664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 msgr2=0x564cc7f36c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.384+0000 7f017a26e640 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f36c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.384+0000 7f017a26e640 1 -- 172.21.15.55:0/871090664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cc7e57c30 con 0x564cc7f2e820 2024-04-26T21:58:50.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.384+0000 7f017926c640 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f36c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:50.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.384+0000 7f017a26e640 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f2ec40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f016c009ce0 tx=0x7f016c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926a32442a6db562 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.384+0000 7f016affd640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01600a3d40 0x7f01600a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.385+0000 7f0179a6d640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01600a3d40 0x7f01600a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.385+0000 7f016affd640 1 -- 172.21.15.55:0/2439336761 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f01640d0200 con 0x7f01741011e0 2024-04-26T21:58:50.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.385+0000 7f015ffff640 1 -- 172.21.15.55:0/871090664 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c018070 con 0x564cc7f2e820 2024-04-26T21:58:50.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.385+0000 7f015ffff640 1 -- 172.21.15.55:0/871090664 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016c0119d0 con 0x564cc7f2e820 2024-04-26T21:58:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.385+0000 7f015ffff640 1 -- 172.21.15.55:0/871090664 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c013950 con 0x564cc7f2e820 2024-04-26T21:58:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 -- 172.21.15.55:0/871090664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 msgr2=0x564cc7f2ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f2ec40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f016c009ce0 tx=0x7f016c00b7e0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f0179a6d640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01600a3d40 0x7f01600a6200 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0164098740 tx=0x7f016406ee20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 -- 172.21.15.55:0/871090664 shutdown_connections 2024-04-26T21:58:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f36c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 0x564cc7f2fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 --2- 172.21.15.55:0/871090664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f2ec40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 -- 172.21.15.55:0/871090664 >> 172.21.15.55:0/871090664 conn(0x564cc7e6f910 msgr2=0x564cc7f2a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 -- 172.21.15.55:0/871090664 shutdown_connections 2024-04-26T21:58:50.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.387+0000 7f017e9ef740 1 -- 172.21.15.55:0/871090664 wait complete. 2024-04-26T21:58:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.388+0000 7f017e9ef740 1 -- start start 2024-04-26T21:58:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.388+0000 7f017e9ef740 1 Processor -- start 2024-04-26T21:58:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.389+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 0x564cc7f41340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.389+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 0x564cc7f41880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.389+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.389+0000 7f017e9ef740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564cc7f373c0 con 0x564cc7f303b0 2024-04-26T21:58:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.389+0000 7f017e9ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564cc7e19dc0 con 0x564cc7f2f9f0 2024-04-26T21:58:50.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.389+0000 7f017e9ef740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564cc7f37540 con 0x564cc7f2e820 2024-04-26T21:58:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.389+0000 7f017e9ef740 10 client.0 _ll_register_callbacks cb 0x564cc7e35600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.391+0000 7f017926c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 0x564cc7f41880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.391+0000 7f017926c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 0x564cc7f41880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39946/0 (socket says 172.21.15.55:39946) 2024-04-26T21:58:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.391+0000 7f017926c640 1 -- 172.21.15.55:0/546171507 learned_addr learned my addr 172.21.15.55:0/546171507 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.391+0000 7f017a26e640 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 0x564cc7f41340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.391+0000 7f017a26e640 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.394+0000 7f017a26e640 1 -- 172.21.15.55:0/546171507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 msgr2=0x564cc7f41340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.394+0000 7f017a26e640 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 0x564cc7f41340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.394+0000 7f017a26e640 1 -- 172.21.15.55:0/546171507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 msgr2=0x564cc7f41880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.394+0000 7f017a26e640 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 0x564cc7f41880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.394+0000 7f017a26e640 1 -- 172.21.15.55:0/546171507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cc7ba43f0 con 0x564cc7f303b0 2024-04-26T21:58:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.395+0000 7f017a26e640 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f488c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f016c009ce0 tx=0x7f016c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37d3da0d8bfa6c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.395+0000 7f015d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:50.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.396+0000 7f015d7fa640 1 -- 172.21.15.55:0/546171507 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c026060 con 0x564cc7f303b0 2024-04-26T21:58:50.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.396+0000 7f015d7fa640 1 -- 172.21.15.55:0/546171507 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f016c013900 con 0x564cc7f303b0 2024-04-26T21:58:50.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.397+0000 7f017e9ef740 1 -- 172.21.15.55:0/546171507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f016c0119d0 con 0x564cc7f303b0 2024-04-26T21:58:50.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.398+0000 7f015d7fa640 1 -- 172.21.15.55:0/546171507 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f016c018040 con 0x564cc7f303b0 2024-04-26T21:58:50.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.398+0000 7f017e9ef740 10 client.16347 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:50.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.398+0000 7f017e9ef740 1 -- 172.21.15.55:0/546171507 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564cc7c6e7d0 con 0x564cc7f303b0 2024-04-26T21:58:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.400+0000 7f015d7fa640 1 -- 172.21.15.55:0/546171507 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016c0119d0 con 0x564cc7f303b0 2024-04-26T21:58:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.400+0000 7f015d7fa640 1 -- 172.21.15.55:0/546171507 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f016c02aa70 con 0x564cc7f303b0 2024-04-26T21:58:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.401+0000 7f017e9ef740 10 client.16347 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.401+0000 7f017e9ef740 10 client.16347 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:58:50.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.401+0000 7f017e9ef740 20 client.16347 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:50.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.401+0000 7f017e9ef740 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564cc7f5df50 0x564cc7f60340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.401+0000 7f017e9ef740 4 client.16347 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:50.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.401+0000 7f017e9ef740 1 -- 172.21.15.55:0/546171507 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cc7ea0990 con 0x564cc7f5df50 2024-04-26T21:58:50.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.402+0000 7f017926c640 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564cc7f5df50 0x564cc7f60340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.404+0000 7f017926c640 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564cc7f5df50 0x564cc7f60340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc49b1ca234ad25d server_cookie=daafdcf2b4e9a80d in_seq=0 out_seq=0 2024-04-26T21:58:50.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.404+0000 7f015d7fa640 10 client.16347 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:58:50.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.406+0000 7f015d7fa640 1 -- 172.21.15.55:0/546171507 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cc7ea0990 con 0x564cc7f5df50 2024-04-26T21:58:50.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.406+0000 7f015d7fa640 10 client.16347 handle_command_reply: tid=0 2024-04-26T21:58:50.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.406+0000 7f017e9ef740 1 client.16347 shutdown 2024-04-26T21:58:50.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.406+0000 7f017e9ef740 1 -- 172.21.15.55:0/546171507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 msgr2=0x564cc7f488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.406+0000 7f017e9ef740 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f488c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f016c009ce0 tx=0x7f016c000f40 comp rx=0 tx=0).stop 2024-04-26T21:58:50.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 1 -- 172.21.15.55:0/546171507 shutdown_connections 2024-04-26T21:58:50.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564cc7f5df50 0x564cc7f60340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f488c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 0x564cc7f41880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 1 --2- 172.21.15.55:0/546171507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 0x564cc7f41340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 1 -- 172.21.15.55:0/546171507 >> 172.21.15.55:0/546171507 conn(0x564cc7e6f910 msgr2=0x564cc7f29280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f015d7fa640 0 client.16347 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:58:50.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 1 -- 172.21.15.55:0/546171507 shutdown_connections 2024-04-26T21:58:50.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 1 -- 172.21.15.55:0/546171507 wait complete. 2024-04-26T21:58:50.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.407+0000 7f017e9ef740 20 client.16347 trim_cache size 0 max 16384 2024-04-26T21:58:50.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:50 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/651048764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:50.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.438+0000 7f017e9ef740 1 Processor -- start 2024-04-26T21:58:50.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.438+0000 7f017e9ef740 1 -- start start 2024-04-26T21:58:50.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 0x564cc7f2cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 0x564cc7f487e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f2d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017e9ef740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0158010f80 con 0x564cc7f303b0 2024-04-26T21:58:50.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017a26e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f2d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017a26e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f2d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34986/0 (socket says 172.21.15.55:34986) 2024-04-26T21:58:50.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017a26e640 1 -- 172.21.15.55:0/4170428997 learned_addr learned my addr 172.21.15.55:0/4170428997 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017e9ef740 1 -- 172.21.15.55:0/4170428997 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f015800e850 con 0x564cc7f2f9f0 2024-04-26T21:58:50.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.439+0000 7f017e9ef740 1 -- 172.21.15.55:0/4170428997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f015800c6a0 con 0x564cc7f2e820 2024-04-26T21:58:50.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.440+0000 7f017a26e640 1 -- 172.21.15.55:0/4170428997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 msgr2=0x564cc7f2cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.440+0000 7f017a26e640 1 --2- 172.21.15.55:0/4170428997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 0x564cc7f2cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.440+0000 7f017a26e640 1 -- 172.21.15.55:0/4170428997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 msgr2=0x564cc7f487e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:50.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.440+0000 7f017a26e640 1 --2- 172.21.15.55:0/4170428997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 0x564cc7f487e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.440+0000 7f017a26e640 1 -- 172.21.15.55:0/4170428997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cc7f424c0 con 0x564cc7f303b0 2024-04-26T21:58:50.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.440+0000 7f017a26e640 1 --2- 172.21.15.55:0/4170428997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f2d320 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f016c009f40 tx=0x7f016c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fd644a3193e2da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.441+0000 7f015f7fe640 1 -- 172.21.15.55:0/4170428997 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c026040 con 0x564cc7f303b0 2024-04-26T21:58:50.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.441+0000 7f015f7fe640 1 -- 172.21.15.55:0/4170428997 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016c0118c0 con 0x564cc7f303b0 2024-04-26T21:58:50.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.441+0000 7f015f7fe640 1 -- 172.21.15.55:0/4170428997 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c030080 con 0x564cc7f303b0 2024-04-26T21:58:50.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.442+0000 7f017e9ef740 1 -- 172.21.15.55:0/4170428997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 msgr2=0x564cc7f2d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.442+0000 7f017e9ef740 1 --2- 172.21.15.55:0/4170428997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f2d320 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f016c009f40 tx=0x7f016c004ad0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.443+0000 7f017e9ef740 1 -- 172.21.15.55:0/4170428997 shutdown_connections 2024-04-26T21:58:50.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.443+0000 7f017e9ef740 1 --2- 172.21.15.55:0/4170428997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f303b0 0x564cc7f2d320 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.443+0000 7f017e9ef740 1 --2- 172.21.15.55:0/4170428997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f2f9f0 0x564cc7f487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.443+0000 7f017e9ef740 1 --2- 172.21.15.55:0/4170428997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2e820 0x564cc7f2cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.443+0000 7f017e9ef740 1 -- 172.21.15.55:0/4170428997 >> 172.21.15.55:0/4170428997 conn(0x564cc7e6f910 msgr2=0x564cc7f332b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.444+0000 7f017e9ef740 1 -- 172.21.15.55:0/4170428997 shutdown_connections 2024-04-26T21:58:50.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.446+0000 7f017e9ef740 1 -- 172.21.15.55:0/4170428997 wait complete. 2024-04-26T21:58:50.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.446+0000 7f017e9ef740 1 -- start start 2024-04-26T21:58:50.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.447+0000 7f017e9ef740 1 Processor -- start 2024-04-26T21:58:50.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.448+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.449+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 0x564cc7f45f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.449+0000 7f017e9ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f46f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.449+0000 7f017a26e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.449+0000 7f017a26e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34990/0 (socket says 172.21.15.55:34990) 2024-04-26T21:58:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.449+0000 7f017a26e640 1 -- 172.21.15.55:0/1509419644 learned_addr learned my addr 172.21.15.55:0/1509419644 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.449+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f015800cd10 con 0x564cc7f2e820 2024-04-26T21:58:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.449+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f015800e180 con 0x564cc7f303b0 2024-04-26T21:58:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.450+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f015800d3e0 con 0x564cc7f2f9f0 2024-04-26T21:58:50.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.450+0000 7f017e9ef740 10 client.0 _ll_register_callbacks cb 0x7f016c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:50.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.450+0000 7f017926c640 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 0x564cc7f45f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.452+0000 7f017a26e640 1 -- 172.21.15.55:0/1509419644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 msgr2=0x564cc7f45f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.452+0000 7f017a26e640 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 0x564cc7f45f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.452+0000 7f017a26e640 1 -- 172.21.15.55:0/1509419644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 msgr2=0x564cc7f46f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.452+0000 7f017a26e640 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f46f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.452+0000 7f017a26e640 1 -- 172.21.15.55:0/1509419644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cc7f47530 con 0x564cc7f2e820 2024-04-26T21:58:50.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.454+0000 7f017a26e640 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f459f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f016c009760 tx=0x7f016c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1159192a9ba2593b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.454+0000 7f015d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:50.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.455+0000 7f015d7fa640 1 -- 172.21.15.55:0/1509419644 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c026070 con 0x564cc7f2e820 2024-04-26T21:58:50.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.455+0000 7f015d7fa640 1 -- 172.21.15.55:0/1509419644 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f016c003c10 con 0x564cc7f2e820 2024-04-26T21:58:50.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.455+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f016c0118c0 con 0x564cc7f2e820 2024-04-26T21:58:50.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.456+0000 7f015d7fa640 1 -- 172.21.15.55:0/1509419644 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f016c0098d0 con 0x564cc7f2e820 2024-04-26T21:58:50.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.456+0000 7f017e9ef740 10 client.16353 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:50.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.456+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564cc7f48810 con 0x564cc7f2e820 2024-04-26T21:58:50.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.457+0000 7f015d7fa640 1 -- 172.21.15.55:0/1509419644 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016c003e10 con 0x564cc7f2e820 2024-04-26T21:58:50.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.457+0000 7f015d7fa640 1 -- 172.21.15.55:0/1509419644 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f016c02a5b0 con 0x564cc7f2e820 2024-04-26T21:58:50.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.459+0000 7f017e9ef740 10 client.16353 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:50.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.460+0000 7f017e9ef740 10 client.16353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:58:50.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.460+0000 7f017e9ef740 20 client.16353 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:50.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.460+0000 7f017e9ef740 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564cc7fc4730 0x564cc7fc6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.460+0000 7f017e9ef740 4 client.16353 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:50.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.460+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f016c0118c0 con 0x564cc7fc4730 2024-04-26T21:58:50.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.465+0000 7f017926c640 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564cc7fc4730 0x564cc7fc6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.484+0000 7f017926c640 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564cc7fc4730 0x564cc7fc6b50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76d16b01824c750e server_cookie=220ea2af557b39d1 in_seq=0 out_seq=0 2024-04-26T21:58:50.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.484+0000 7f015d7fa640 10 client.16353 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:58:50.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.486+0000 7f015d7fa640 1 -- 172.21.15.55:0/1509419644 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f016c0118c0 con 0x564cc7fc4730 2024-04-26T21:58:50.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.486+0000 7f015d7fa640 10 client.16353 handle_command_reply: tid=0 2024-04-26T21:58:50.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.487+0000 7f017e9ef740 1 client.16353 shutdown 2024-04-26T21:58:50.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.501+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 msgr2=0x564cc7f459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.502+0000 7f017e9ef740 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f459f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f016c009760 tx=0x7f016c011b00 comp rx=0 tx=0).stop 2024-04-26T21:58:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.506+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 shutdown_connections 2024-04-26T21:58:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.506+0000 7f017e9ef740 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564cc7fc4730 0x564cc7fc6b50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.506+0000 7f017e9ef740 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cc7f303b0 0x564cc7f46f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.506+0000 7f017e9ef740 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564cc7f2f9f0 0x564cc7f45f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.506+0000 7f017e9ef740 1 --2- 172.21.15.55:0/1509419644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564cc7f2e820 0x564cc7f459f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.506+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 >> 172.21.15.55:0/1509419644 conn(0x564cc7e6f910 msgr2=0x564cc7f289f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:50 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/651048764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.509+0000 7f015d7fa640 0 client.16353 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:58:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.509+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 shutdown_connections 2024-04-26T21:58:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.509+0000 7f017e9ef740 1 -- 172.21.15.55:0/1509419644 wait complete. 2024-04-26T21:58:50.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.521+0000 7f017e9ef740 20 client.16353 trim_cache size 0 max 16384 2024-04-26T21:58:50.524 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:58:50.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.534+0000 7f017bcd1640 1 -- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01600a3d40 msgr2=0x7f01600a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.534+0000 7f017bcd1640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01600a3d40 0x7f01600a6200 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0164098740 tx=0x7f016406ee20 comp rx=0 tx=0).stop 2024-04-26T21:58:50.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.534+0000 7f017bcd1640 1 -- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01741011e0 msgr2=0x7f017419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.534+0000 7f017bcd1640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01741011e0 0x7f017419ae60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f016400d950 tx=0x7f016400de10 comp rx=0 tx=0).stop 2024-04-26T21:58:50.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.534+0000 7f017bcd1640 1 -- 172.21.15.55:0/2439336761 shutdown_connections 2024-04-26T21:58:50.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.535+0000 7f017bcd1640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01600a3d40 0x7f01600a6200 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0164098740 tx=0x7f016406ee20 comp rx=0 tx=0).stop 2024-04-26T21:58:50.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.535+0000 7f017bcd1640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01741024e0 0x7f01741a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.535+0000 7f017bcd1640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0174101b40 0x7f017419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.535+0000 7f017bcd1640 1 --2- 172.21.15.55:0/2439336761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01741011e0 0x7f017419ae60 secure :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f016400d950 tx=0x7f016400de10 comp rx=0 tx=0).stop 2024-04-26T21:58:50.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.536+0000 7f017bcd1640 1 -- 172.21.15.55:0/2439336761 >> 172.21.15.55:0/2439336761 conn(0x7f01740fcde0 msgr2=0x7f01740fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.543+0000 7f017bcd1640 1 -- 172.21.15.55:0/2439336761 shutdown_connections 2024-04-26T21:58:50.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.547+0000 7f017bcd1640 1 -- 172.21.15.55:0/2439336761 wait complete. 2024-04-26T21:58:50.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:50 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/651048764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:58:50.611 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:58:50.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.889+0000 7f4daf789640 1 -- 172.21.15.55:0/3187582713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8101260 msgr2=0x7f4da8101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.889+0000 7f4daf789640 1 --2- 172.21.15.55:0/3187582713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8101260 0x7f4da8101680 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009900 tx=0x7f4da402a410 comp rx=0 tx=0).stop 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.889+0000 7f4daf789640 1 -- 172.21.15.55:0/3187582713 shutdown_connections 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.889+0000 7f4daf789640 1 --2- 172.21.15.55:0/3187582713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4da8102560 0x7f4da81029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.889+0000 7f4daf789640 1 --2- 172.21.15.55:0/3187582713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8101bc0 0x7f4da8102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.889+0000 7f4daf789640 1 --2- 172.21.15.55:0/3187582713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8101260 0x7f4da8101680 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.889+0000 7f4daf789640 1 -- 172.21.15.55:0/3187582713 >> 172.21.15.55:0/3187582713 conn(0x7f4da80fce20 msgr2=0x7f4da80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.890+0000 7f4daf789640 1 -- 172.21.15.55:0/3187582713 shutdown_connections 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.890+0000 7f4daf789640 1 -- 172.21.15.55:0/3187582713 wait complete. 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.890+0000 7f4daf789640 1 Processor -- start 2024-04-26T21:58:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4daf789640 1 -- start start 2024-04-26T21:58:50.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4daf789640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8101260 0x7f4da819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4daf789640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4da8101bc0 0x7f4da819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4daf789640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8102560 0x7f4da81a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4daf789640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81107f0 con 0x7f4da8102560 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4daf789640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8110670 con 0x7f4da8101260 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4daf789640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8110970 con 0x7f4da8101bc0 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4dadd26640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8102560 0x7f4da81a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4dadd26640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8102560 0x7f4da81a24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53668/0 (socket says 172.21.15.55:53668) 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.891+0000 7f4dadd26640 1 -- 172.21.15.55:0/3415456513 learned_addr learned my addr 172.21.15.55:0/3415456513 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.892+0000 7f4dadd26640 1 -- 172.21.15.55:0/3415456513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4da8101bc0 msgr2=0x7f4da819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.892+0000 7f4dadd26640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4da8101bc0 0x7f4da819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.892+0000 7f4dadd26640 1 -- 172.21.15.55:0/3415456513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8101260 msgr2=0x7f4da819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.892+0000 7f4dadd26640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8101260 0x7f4da819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.892+0000 7f4dadd26640 1 -- 172.21.15.55:0/3415456513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da81a2b40 con 0x7f4da8102560 2024-04-26T21:58:50.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.892+0000 7f4dadd26640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8102560 0x7f4da81a24d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4da000b970 tx=0x7f4da000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.893+0000 7f4d9e7fc640 1 -- 172.21.15.55:0/3415456513 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0019070 con 0x7f4da8102560 2024-04-26T21:58:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.893+0000 7f4d9e7fc640 1 -- 172.21.15.55:0/3415456513 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da0014440 con 0x7f4da8102560 2024-04-26T21:58:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.893+0000 7f4daf789640 1 -- 172.21.15.55:0/3415456513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da81a2e30 con 0x7f4da8102560 2024-04-26T21:58:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.893+0000 7f4d9e7fc640 1 -- 172.21.15.55:0/3415456513 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da00075d0 con 0x7f4da8102560 2024-04-26T21:58:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.893+0000 7f4daf789640 1 -- 172.21.15.55:0/3415456513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da81a3420 con 0x7f4da8102560 2024-04-26T21:58:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.897+0000 7f4d9e7fc640 1 -- 172.21.15.55:0/3415456513 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4da001c070 con 0x7f4da8102560 2024-04-26T21:58:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.899+0000 7f4d9e7fc640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4d940a3d40 0x7f4d940a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.899+0000 7f4d9e7fc640 1 -- 172.21.15.55:0/3415456513 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4da00cfa70 con 0x7f4da8102560 2024-04-26T21:58:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4db257f740 1 Processor -- start 2024-04-26T21:58:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4dad525640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4d940a3d40 0x7f4d940a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4db257f740 1 -- start start 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4db257f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a8030 0x5652785a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4db257f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9200 0x5652785a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4db257f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a9bc0 0x5652785b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4db257f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x565278503d80 con 0x5652785a8030 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4db257f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5652784bda20 con 0x5652785a9bc0 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4db257f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5652784ffd40 con 0x5652785a9200 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4dacd24640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9200 0x5652785a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4dacd24640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9200 0x5652785a9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:46122/0 (socket says 172.21.15.55:46122) 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.900+0000 7f4dacd24640 1 -- 172.21.15.55:0/1233721083 learned_addr learned my addr 172.21.15.55:0/1233721083 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.901+0000 7f4dacd24640 1 -- 172.21.15.55:0/1233721083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a9bc0 msgr2=0x5652785b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.901+0000 7f4dacd24640 1 --2- 172.21.15.55:0/1233721083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a9bc0 0x5652785b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.901+0000 7f4dacd24640 1 -- 172.21.15.55:0/1233721083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a8030 msgr2=0x5652785a8450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:58:50.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.901+0000 7f4dacd24640 1 --2- 172.21.15.55:0/1233721083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a8030 0x5652785a8450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.901+0000 7f4dacd24640 1 -- 172.21.15.55:0/1233721083 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652784c8c30 con 0x5652785a9200 2024-04-26T21:58:50.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.902+0000 7f4dad525640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4d940a3d40 0x7f4d940a6200 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4da4031500 tx=0x7f4da4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.902+0000 7f4dacd24640 1 --2- 172.21.15.55:0/1233721083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9200 0x5652785a9680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4d980097b0 tx=0x7f4d98009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb638642188cd2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.902+0000 7f4d937fe640 1 -- 172.21.15.55:0/1233721083 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98015020 con 0x5652785a9200 2024-04-26T21:58:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.903+0000 7f4d937fe640 1 -- 172.21.15.55:0/1233721083 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d98011420 con 0x5652785a9200 2024-04-26T21:58:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.903+0000 7f4d937fe640 1 -- 172.21.15.55:0/1233721083 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d980105e0 con 0x5652785a9200 2024-04-26T21:58:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 -- 172.21.15.55:0/1233721083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9200 msgr2=0x5652785a9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 --2- 172.21.15.55:0/1233721083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9200 0x5652785a9680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4d980097b0 tx=0x7f4d98009c70 comp rx=0 tx=0).stop 2024-04-26T21:58:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 -- 172.21.15.55:0/1233721083 shutdown_connections 2024-04-26T21:58:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 --2- 172.21.15.55:0/1233721083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a9bc0 0x5652785b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 --2- 172.21.15.55:0/1233721083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9200 0x5652785a9680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 --2- 172.21.15.55:0/1233721083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a8030 0x5652785a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 -- 172.21.15.55:0/1233721083 >> 172.21.15.55:0/1233721083 conn(0x5652784e0910 msgr2=0x5652785a3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 -- 172.21.15.55:0/1233721083 shutdown_connections 2024-04-26T21:58:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.905+0000 7f4db257f740 1 -- 172.21.15.55:0/1233721083 wait complete. 2024-04-26T21:58:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.906+0000 7f4db257f740 1 -- start start 2024-04-26T21:58:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.906+0000 7f4db257f740 1 Processor -- start 2024-04-26T21:58:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.906+0000 7f4db257f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a8030 0x5652785b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.906+0000 7f4db257f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a9200 0x5652785b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.906+0000 7f4db257f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9bc0 0x5652785b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.906+0000 7f4db257f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5652785b0c70 con 0x5652785a9bc0 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.907+0000 7f4db257f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56527848adc0 con 0x5652785a9200 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.907+0000 7f4db257f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5652785b0df0 con 0x5652785a8030 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.907+0000 7f4db257f740 10 client.0 _ll_register_callbacks cb 0x5652784a6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.907+0000 7f4dadd26640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9bc0 0x5652785b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.907+0000 7f4dadd26640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9bc0 0x5652785b98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53680/0 (socket says 172.21.15.55:53680) 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.907+0000 7f4dadd26640 1 -- 172.21.15.55:0/2378041337 learned_addr learned my addr 172.21.15.55:0/2378041337 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.908+0000 7f4dadd26640 1 -- 172.21.15.55:0/2378041337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a8030 msgr2=0x5652785b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.908+0000 7f4dadd26640 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a8030 0x5652785b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.908+0000 7f4dadd26640 1 -- 172.21.15.55:0/2378041337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a9200 msgr2=0x5652785b28b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.908+0000 7f4dadd26640 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a9200 0x5652785b28b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.908+0000 7f4dadd26640 1 -- 172.21.15.55:0/2378041337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652782153f0 con 0x5652785a9bc0 2024-04-26T21:58:50.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.908+0000 7f4dadd26640 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9bc0 0x5652785b98f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4da0098630 tx=0x7f4da009bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ba4ae11e2c1315 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.908+0000 7f4d90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:50.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.909+0000 7f4d90ff9640 1 -- 172.21.15.55:0/2378041337 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da00cd5d0 con 0x5652785a9bc0 2024-04-26T21:58:50.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.909+0000 7f4d90ff9640 1 -- 172.21.15.55:0/2378041337 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4da00bc070 con 0x5652785a9bc0 2024-04-26T21:58:50.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.909+0000 7f4db257f740 1 -- 172.21.15.55:0/2378041337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d98011420 con 0x5652785a9bc0 2024-04-26T21:58:50.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.910+0000 7f4d90ff9640 1 -- 172.21.15.55:0/2378041337 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4da00b51d0 con 0x5652785a9bc0 2024-04-26T21:58:50.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.910+0000 7f4db257f740 10 client.16362 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:50.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.910+0000 7f4db257f740 1 -- 172.21.15.55:0/2378041337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5652782df7d0 con 0x5652785a9bc0 2024-04-26T21:58:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.910+0000 7f4d90ff9640 1 -- 172.21.15.55:0/2378041337 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d98011420 con 0x5652785a9bc0 2024-04-26T21:58:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.911+0000 7f4d90ff9640 1 -- 172.21.15.55:0/2378041337 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4da009b0b0 con 0x5652785a9bc0 2024-04-26T21:58:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.911+0000 7f4db257f740 10 client.16362 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.911+0000 7f4db257f740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.911+0000 7f4db257f740 20 client.16362 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.911+0000 7f4db257f740 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5652785be2b0 0x5652785c0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.911+0000 7f4db257f740 4 client.16362 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.911+0000 7f4db257f740 1 -- 172.21.15.55:0/2378041337 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565278511990 con 0x5652785be2b0 2024-04-26T21:58:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.912+0000 7f4dad525640 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5652785be2b0 0x5652785c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.916+0000 7f4d90ff9640 10 client.16362 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:50.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.916+0000 7f4dad525640 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5652785be2b0 0x5652785c0730 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3782b234ecf95e8f server_cookie=9ee3986064c5dbeb in_seq=0 out_seq=0 2024-04-26T21:58:50.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.920+0000 7f4d90ff9640 1 -- 172.21.15.55:0/2378041337 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565278511990 con 0x5652785be2b0 2024-04-26T21:58:50.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.920+0000 7f4d90ff9640 10 client.16362 handle_command_reply: tid=0 2024-04-26T21:58:50.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.920+0000 7f4db257f740 1 client.16362 shutdown 2024-04-26T21:58:50.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 -- 172.21.15.55:0/2378041337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9bc0 msgr2=0x5652785b98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9bc0 0x5652785b98f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4da0098630 tx=0x7f4da009bd20 comp rx=0 tx=0).stop 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 -- 172.21.15.55:0/2378041337 shutdown_connections 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5652785be2b0 0x5652785c0730 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9bc0 0x5652785b98f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a9200 0x5652785b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 --2- 172.21.15.55:0/2378041337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a8030 0x5652785b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 -- 172.21.15.55:0/2378041337 >> 172.21.15.55:0/2378041337 conn(0x5652784e0910 msgr2=0x56527859a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4d90ff9640 0 client.16362 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 -- 172.21.15.55:0/2378041337 shutdown_connections 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 1 -- 172.21.15.55:0/2378041337 wait complete. 2024-04-26T21:58:50.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.921+0000 7f4db257f740 20 client.16362 trim_cache size 0 max 16384 2024-04-26T21:58:50.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.949+0000 7f4db257f740 1 Processor -- start 2024-04-26T21:58:50.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.950+0000 7f4db257f740 1 -- start start 2024-04-26T21:58:50.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.950+0000 7f4db257f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a8030 0x5652785c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.950+0000 7f4db257f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 0x5652785b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.951+0000 7f4db257f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.951+0000 7f4db257f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81aa840 con 0x5652785a9200 2024-04-26T21:58:50.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.951+0000 7f4db257f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81a8110 con 0x5652785a8030 2024-04-26T21:58:50.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.951+0000 7f4db257f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81a5f60 con 0x5652785a9bc0 2024-04-26T21:58:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.956+0000 7f4dadd26640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.956+0000 7f4dacd24640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 0x5652785b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.956+0000 7f4dacd24640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 0x5652785b98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53694/0 (socket says 172.21.15.55:53694) 2024-04-26T21:58:50.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.956+0000 7f4dacd24640 1 -- 172.21.15.55:0/2321299620 learned_addr learned my addr 172.21.15.55:0/2321299620 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:50.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.959+0000 7f4dad525640 1 --2- 172.21.15.55:0/2321299620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a8030 0x5652785c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.967+0000 7f4dad525640 1 -- 172.21.15.55:0/2321299620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 msgr2=0x5652785a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.967+0000 7f4dad525640 1 --2- 172.21.15.55:0/2321299620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785a6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.967+0000 7f4dad525640 1 -- 172.21.15.55:0/2321299620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 msgr2=0x5652785b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:50.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.967+0000 7f4dad525640 1 --2- 172.21.15.55:0/2321299620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 0x5652785b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.967+0000 7f4dad525640 1 -- 172.21.15.55:0/2321299620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652785968b0 con 0x5652785a8030 2024-04-26T21:58:50.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.976+0000 7f4dad525640 1 --2- 172.21.15.55:0/2321299620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a8030 0x5652785c02c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009230 tx=0x7f4da4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5feddb75f2103b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:50.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.984+0000 7f4d92ffd640 1 -- 172.21.15.55:0/2321299620 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4035040 con 0x5652785a8030 2024-04-26T21:58:50.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.984+0000 7f4d92ffd640 1 -- 172.21.15.55:0/2321299620 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da4053070 con 0x5652785a8030 2024-04-26T21:58:50.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.986+0000 7f4db257f740 1 -- 172.21.15.55:0/2321299620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a8030 msgr2=0x5652785c02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:50.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.986+0000 7f4db257f740 1 --2- 172.21.15.55:0/2321299620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a8030 0x5652785c02c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009230 tx=0x7f4da4005fd0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.992+0000 7f4db257f740 1 -- 172.21.15.55:0/2321299620 shutdown_connections 2024-04-26T21:58:50.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.992+0000 7f4db257f740 1 --2- 172.21.15.55:0/2321299620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.992+0000 7f4db257f740 1 --2- 172.21.15.55:0/2321299620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 0x5652785b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.992+0000 7f4db257f740 1 --2- 172.21.15.55:0/2321299620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785a8030 0x5652785c02c0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009230 tx=0x7f4da4005fd0 comp rx=0 tx=0).stop 2024-04-26T21:58:50.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.992+0000 7f4db257f740 1 -- 172.21.15.55:0/2321299620 >> 172.21.15.55:0/2321299620 conn(0x5652784e0910 msgr2=0x5652785acb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:50.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.992+0000 7f4db257f740 1 -- 172.21.15.55:0/2321299620 shutdown_connections 2024-04-26T21:58:50.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.992+0000 7f4db257f740 1 -- 172.21.15.55:0/2321299620 wait complete. 2024-04-26T21:58:50.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.993+0000 7f4db257f740 1 -- start start 2024-04-26T21:58:50.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.993+0000 7f4db257f740 1 Processor -- start 2024-04-26T21:58:50.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.994+0000 7f4db257f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 0x5652785b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.994+0000 7f4db257f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785b7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.994+0000 7f4db257f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785b8550 0x5652785b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:50.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.994+0000 7f4db257f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81a65d0 con 0x5652785a9200 2024-04-26T21:58:50.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.994+0000 7f4db257f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81a7a40 con 0x5652785b8550 2024-04-26T21:58:50.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.994+0000 7f4db257f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81a6ca0 con 0x5652785a9bc0 2024-04-26T21:58:50.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.994+0000 7f4db257f740 10 client.0 _ll_register_callbacks cb 0x7f4da0097f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:50.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.995+0000 7f4dacd24640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:50.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.995+0000 7f4dacd24640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:46154/0 (socket says 172.21.15.55:46154) 2024-04-26T21:58:50.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.995+0000 7f4dacd24640 1 -- 172.21.15.55:0/3437259395 learned_addr learned my addr 172.21.15.55:0/3437259395 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:51.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.995+0000 7f4dadd26640 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785b8550 0x5652785b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.998+0000 7f4dadd26640 1 -- 172.21.15.55:0/3437259395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 msgr2=0x5652785b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.998+0000 7f4dadd26640 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.998+0000 7f4dadd26640 1 -- 172.21.15.55:0/3437259395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 msgr2=0x5652785b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:51.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.998+0000 7f4dadd26640 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 0x5652785b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:50.998+0000 7f4dadd26640 1 -- 172.21.15.55:0/3437259395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652785b8f10 con 0x5652785b8550 2024-04-26T21:58:51.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.007+0000 7f4dadd26640 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785b8550 0x5652785b89d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4da00a01d0 tx=0x7f4da009b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4e370032c4ad9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.009+0000 7f4d90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:58:51.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.016+0000 7f4d90ff9640 1 -- 172.21.15.55:0/3437259395 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da009b5c0 con 0x5652785b8550 2024-04-26T21:58:51.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.017+0000 7f4d90ff9640 1 -- 172.21.15.55:0/3437259395 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4da0009040 con 0x5652785b8550 2024-04-26T21:58:51.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.018+0000 7f4db257f740 1 -- 172.21.15.55:0/3437259395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56527850b800 con 0x5652785b8550 2024-04-26T21:58:51.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.019+0000 7f4d90ff9640 1 -- 172.21.15.55:0/3437259395 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4da00b51d0 con 0x5652785b8550 2024-04-26T21:58:51.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.020+0000 7f4db257f740 10 client.25679 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:51.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.020+0000 7f4db257f740 1 -- 172.21.15.55:0/3437259395 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56527859af20 con 0x5652785b8550 2024-04-26T21:58:51.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.021+0000 7f4d90ff9640 1 -- 172.21.15.55:0/3437259395 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56527850b800 con 0x5652785b8550 2024-04-26T21:58:51.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.021+0000 7f4d90ff9640 1 -- 172.21.15.55:0/3437259395 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4da007aad0 con 0x5652785b8550 2024-04-26T21:58:51.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.021+0000 7f4db257f740 10 client.25679 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:51.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.021+0000 7f4db257f740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:51.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.021+0000 7f4db257f740 20 client.25679 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:51.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.021+0000 7f4db257f740 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56527859b870 0x56527859dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:51.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.021+0000 7f4db257f740 4 client.25679 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:58:51.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.021+0000 7f4db257f740 1 -- 172.21.15.55:0/3437259395 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565278486610 con 0x56527859b870 2024-04-26T21:58:51.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.026+0000 7f4dad525640 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56527859b870 0x56527859dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.048+0000 7f4d90ff9640 10 client.25679 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.048+0000 7f4dad525640 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56527859b870 0x56527859dc60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3bfda16e2123957 server_cookie=80da0efed4b6e0c7 in_seq=0 out_seq=0 2024-04-26T21:58:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.054+0000 7f4d90ff9640 1 -- 172.21.15.55:0/3437259395 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565278486610 con 0x56527859b870 2024-04-26T21:58:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.054+0000 7f4d90ff9640 10 client.25679 handle_command_reply: tid=0 2024-04-26T21:58:51.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.057+0000 7f4db257f740 1 client.25679 shutdown 2024-04-26T21:58:51.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.061+0000 7f4db257f740 1 -- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785b8550 msgr2=0x5652785b89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.061+0000 7f4db257f740 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785b8550 0x5652785b89d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4da00a01d0 tx=0x7f4da009b8d0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.061+0000 7f4db257f740 1 -- 172.21.15.55:0/3437259395 shutdown_connections 2024-04-26T21:58:51.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.061+0000 7f4db257f740 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5652785b8550 0x5652785b89d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.061+0000 7f4db257f740 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5652785a9bc0 0x5652785b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.061+0000 7f4db257f740 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56527859b870 0x56527859dc60 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.061+0000 7f4db257f740 1 --2- 172.21.15.55:0/3437259395 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5652785a9200 0x5652785b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.061+0000 7f4db257f740 1 -- 172.21.15.55:0/3437259395 >> 172.21.15.55:0/3437259395 conn(0x5652784e0910 msgr2=0x5652785999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.064+0000 7f4d90ff9640 0 client.25679 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:51.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.070+0000 7f4db257f740 1 -- 172.21.15.55:0/3437259395 shutdown_connections 2024-04-26T21:58:51.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.070+0000 7f4db257f740 1 -- 172.21.15.55:0/3437259395 wait complete. 2024-04-26T21:58:51.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.075+0000 7f4db257f740 20 client.25679 trim_cache size 0 max 16384 2024-04-26T21:58:51.087 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:58:51.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.086+0000 7f4daf789640 1 -- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4d940a3d40 msgr2=0x7f4d940a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.086+0000 7f4daf789640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4d940a3d40 0x7f4d940a6200 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4da4031500 tx=0x7f4da4000bc0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.087+0000 7f4daf789640 1 -- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8102560 msgr2=0x7f4da81a24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:51.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.087+0000 7f4daf789640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8102560 0x7f4da81a24d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4da000b970 tx=0x7f4da000be30 comp rx=0 tx=0).stop 2024-04-26T21:58:51.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.088+0000 7f4daf789640 1 -- 172.21.15.55:0/3415456513 shutdown_connections 2024-04-26T21:58:51.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.088+0000 7f4daf789640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4d940a3d40 0x7f4d940a6200 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.088+0000 7f4daf789640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da8102560 0x7f4da81a24d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.088+0000 7f4daf789640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4da8101bc0 0x7f4da819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.088+0000 7f4daf789640 1 --2- 172.21.15.55:0/3415456513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8101260 0x7f4da819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:51.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.088+0000 7f4daf789640 1 -- 172.21.15.55:0/3415456513 >> 172.21.15.55:0/3415456513 conn(0x7f4da80fce20 msgr2=0x7f4da80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:51.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.093+0000 7f4daf789640 1 -- 172.21.15.55:0/3415456513 shutdown_connections 2024-04-26T21:58:51.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:51.093+0000 7f4daf789640 1 -- 172.21.15.55:0/3415456513 wait complete. 2024-04-26T21:58:51.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:51 smithi055 ceph-mon[29221]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 866 MiB/s wr, 1.45k op/s 2024-04-26T21:58:51.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:51 smithi064 ceph-mon[32765]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 866 MiB/s wr, 1.45k op/s 2024-04-26T21:58:51.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:51 smithi165 ceph-mon[33296]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 866 MiB/s wr, 1.45k op/s 2024-04-26T21:58:52.153 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:58:52.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:58:52.154 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:58:52.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 -- 172.21.15.55:0/198717142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 msgr2=0x7fbc20105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 --2- 172.21.15.55:0/198717142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 0x7fbc20105880 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009900 tx=0x7fbc1402a410 comp rx=0 tx=0).stop 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 -- 172.21.15.55:0/198717142 shutdown_connections 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 --2- 172.21.15.55:0/198717142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc201086f0 0x7fbc2010aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 --2- 172.21.15.55:0/198717142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20105dc0 0x7fbc201081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 --2- 172.21.15.55:0/198717142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 0x7fbc20105880 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 -- 172.21.15.55:0/198717142 >> 172.21.15.55:0/198717142 conn(0x7fbc200fcde0 msgr2=0x7fbc200ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 -- 172.21.15.55:0/198717142 shutdown_connections 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.517+0000 7fbc26b24640 1 -- 172.21.15.55:0/198717142 wait complete. 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.518+0000 7fbc26b24640 1 Processor -- start 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.518+0000 7fbc26b24640 1 -- start start 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc26b24640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 0x7fbc20198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc26b24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20105dc0 0x7fbc201992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc26b24640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc201086f0 0x7fbc201a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc26b24640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010e5e0 con 0x7fbc20103490 2024-04-26T21:58:52.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc26b24640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010e460 con 0x7fbc20105dc0 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc26b24640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010e760 con 0x7fbc201086f0 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc248c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 0x7fbc20198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc248c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 0x7fbc20198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53724/0 (socket says 172.21.15.55:53724) 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc248c0640 1 -- 172.21.15.55:0/609277592 learned_addr learned my addr 172.21.15.55:0/609277592 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc248c0640 1 -- 172.21.15.55:0/609277592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc201086f0 msgr2=0x7fbc201a0320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc248c0640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc201086f0 0x7fbc201a0320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc248c0640 1 -- 172.21.15.55:0/609277592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20105dc0 msgr2=0x7fbc201992b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc248c0640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20105dc0 0x7fbc201992b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.519+0000 7fbc248c0640 1 -- 172.21.15.55:0/609277592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc201a0990 con 0x7fbc20103490 2024-04-26T21:58:52.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.520+0000 7fbc248c0640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 0x7fbc20198d70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbc14031500 tx=0x7fbc14003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.520+0000 7fbc1dffb640 1 -- 172.21.15.55:0/609277592 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1403f070 con 0x7fbc20103490 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.520+0000 7fbc26b24640 1 -- 172.21.15.55:0/609277592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc201a0c20 con 0x7fbc20103490 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.520+0000 7fbc1dffb640 1 -- 172.21.15.55:0/609277592 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc1403b460 con 0x7fbc20103490 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.520+0000 7fbc1dffb640 1 -- 172.21.15.55:0/609277592 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14035030 con 0x7fbc20103490 2024-04-26T21:58:52.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.520+0000 7fbc26b24640 1 -- 172.21.15.55:0/609277592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc201a1160 con 0x7fbc20103490 2024-04-26T21:58:52.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.524+0000 7fbc29841740 1 Processor -- start 2024-04-26T21:58:52.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.524+0000 7fbc29841740 1 -- start start 2024-04-26T21:58:52.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.524+0000 7fbc29841740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.524+0000 7fbc29841740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.524+0000 7fbc29841740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7cf410 0x55d1cc7d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.524+0000 7fbc29841740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc72eb00 con 0x55d1cc7cf410 2024-04-26T21:58:52.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.524+0000 7fbc29841740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc6ca920 con 0x55d1cc7c98b0 2024-04-26T21:58:52.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.524+0000 7fbc29841740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc72aac0 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.525+0000 7fbc1ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7cf410 0x55d1cc7d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.525+0000 7fbc1ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7cf410 0x55d1cc7d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53730/0 (socket says 172.21.15.55:53730) 2024-04-26T21:58:52.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.525+0000 7fbc1ffff640 1 -- 172.21.15.55:0/2408354334 learned_addr learned my addr 172.21.15.55:0/2408354334 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:52.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.525+0000 7fbc1dffb640 1 -- 172.21.15.55:0/609277592 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbc14042070 con 0x7fbc20103490 2024-04-26T21:58:52.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.526+0000 7fbc1ffff640 1 -- 172.21.15.55:0/2408354334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7ca2a0 msgr2=0x55d1cc7ceda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:52.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.526+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/2408354334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7ceda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.526+0000 7fbc1ffff640 1 -- 172.21.15.55:0/2408354334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7c98b0 msgr2=0x55d1cc7c9cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:52.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.526+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/2408354334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.526+0000 7fbc1ffff640 1 -- 172.21.15.55:0/2408354334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cc5087e0 con 0x55d1cc7cf410 2024-04-26T21:58:52.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.527+0000 7fbc1dffb640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbc100a3c40 0x7fbc100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.527+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/2408354334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7cf410 0x55d1cc7d18a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0096f0 tx=0x7fbc0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a842974ab6bec35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.527+0000 7fbc1dffb640 1 -- 172.21.15.55:0/609277592 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbc140f1040 con 0x7fbc20103490 2024-04-26T21:58:52.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.527+0000 7fbc250c1640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbc100a3c40 0x7fbc100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.527+0000 7fbc0affd640 1 -- 172.21.15.55:0/2408354334 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c015020 con 0x55d1cc7cf410 2024-04-26T21:58:52.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.528+0000 7fbc0affd640 1 -- 172.21.15.55:0/2408354334 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc0c00bd80 con 0x55d1cc7cf410 2024-04-26T21:58:52.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.528+0000 7fbc0affd640 1 -- 172.21.15.55:0/2408354334 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c010380 con 0x55d1cc7cf410 2024-04-26T21:58:52.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc250c1640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbc100a3c40 0x7fbc100a6100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55d1cc7cab00 tx=0x7fbc18009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc29841740 1 -- 172.21.15.55:0/2408354334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7cf410 msgr2=0x55d1cc7d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc29841740 1 --2- 172.21.15.55:0/2408354334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7cf410 0x55d1cc7d18a0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0096f0 tx=0x7fbc0c009bb0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc29841740 1 -- 172.21.15.55:0/2408354334 shutdown_connections 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc29841740 1 --2- 172.21.15.55:0/2408354334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7cf410 0x55d1cc7d18a0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc29841740 1 --2- 172.21.15.55:0/2408354334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc29841740 1 --2- 172.21.15.55:0/2408354334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc29841740 1 -- 172.21.15.55:0/2408354334 >> 172.21.15.55:0/2408354334 conn(0x55d1cc7c4f20 msgr2=0x55d1cc7c5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.529+0000 7fbc29841740 1 -- 172.21.15.55:0/2408354334 shutdown_connections 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.530+0000 7fbc29841740 1 -- 172.21.15.55:0/2408354334 wait complete. 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.530+0000 7fbc29841740 1 -- start start 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.531+0000 7fbc29841740 1 Processor -- start 2024-04-26T21:58:52.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.531+0000 7fbc29841740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7d7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.531+0000 7fbc29841740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.531+0000 7fbc29841740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7cf410 0x55d1cc7df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.531+0000 7fbc29841740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc7d2460 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.532+0000 7fbc29841740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc6c85d0 con 0x55d1cc7cf410 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.532+0000 7fbc29841740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc7d25e0 con 0x55d1cc7c98b0 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.532+0000 7fbc29841740 10 client.0 _ll_register_callbacks cb 0x55d1cc6b6080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.532+0000 7fbc1ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.532+0000 7fbc1ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7d84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53732/0 (socket says 172.21.15.55:53732) 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.532+0000 7fbc1ffff640 1 -- 172.21.15.55:0/2555969664 learned_addr learned my addr 172.21.15.55:0/2555969664 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbc1ffff640 1 -- 172.21.15.55:0/2555969664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7c98b0 msgr2=0x55d1cc7d7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbc248c0640 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7d7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7d7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbc1ffff640 1 -- 172.21.15.55:0/2555969664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7cf410 msgr2=0x55d1cc7df4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7cf410 0x55d1cc7df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbc1ffff640 1 -- 172.21.15.55:0/2555969664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cc568f70 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbc248c0640 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7d7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:52.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7d84b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c010 tx=0x7fbc0c012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a734310a0c79b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbbfffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:52.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.533+0000 7fbbfffff640 1 -- 172.21.15.55:0/2555969664 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c020070 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbbfffff640 1 -- 172.21.15.55:0/2555969664 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbc0c00bad0 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbc29841740 1 -- 172.21.15.55:0/2555969664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc0c00bd80 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbbfffff640 1 -- 172.21.15.55:0/2555969664 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbc0c004040 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbc29841740 10 client.16377 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:52.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbc29841740 1 -- 172.21.15.55:0/2555969664 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d1cc400170 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbbfffff640 1 -- 172.21.15.55:0/2555969664 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc0c00bd80 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbbfffff640 1 -- 172.21.15.55:0/2555969664 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fbc0c01a420 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbc29841740 10 client.16377 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbc29841740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbc29841740 20 client.16377 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.534+0000 7fbc29841740 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d1cc7e3710 0x55d1cc7e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.535+0000 7fbc29841740 4 client.16377 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.535+0000 7fbc29841740 1 -- 172.21.15.55:0/2555969664 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1cc73c710 con 0x55d1cc7e3710 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.535+0000 7fbc248c0640 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d1cc7e3710 0x55d1cc7e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.537+0000 7fbbfffff640 10 client.16377 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:52.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.537+0000 7fbc248c0640 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d1cc7e3710 0x55d1cc7e5b40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa9105b4a404e53 server_cookie=d6b681ae752134ac in_seq=0 out_seq=0 2024-04-26T21:58:52.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbbfffff640 1 -- 172.21.15.55:0/2555969664 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1cc73c710 con 0x55d1cc7e3710 2024-04-26T21:58:52.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbbfffff640 10 client.16377 handle_command_reply: tid=0 2024-04-26T21:58:52.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 client.16377 shutdown 2024-04-26T21:58:52.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 -- 172.21.15.55:0/2555969664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7ca2a0 msgr2=0x55d1cc7d84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7d84b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c010 tx=0x7fbc0c012f00 comp rx=0 tx=0).stop 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 -- 172.21.15.55:0/2555969664 shutdown_connections 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7cf410 0x55d1cc7df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d1cc7e3710 0x55d1cc7e5b40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7d84b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 --2- 172.21.15.55:0/2555969664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7d7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbbfffff640 0 client.16377 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 -- 172.21.15.55:0/2555969664 >> 172.21.15.55:0/2555969664 conn(0x55d1cc7c4f20 msgr2=0x55d1cc7c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 -- 172.21.15.55:0/2555969664 shutdown_connections 2024-04-26T21:58:52.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.541+0000 7fbc29841740 1 -- 172.21.15.55:0/2555969664 wait complete. 2024-04-26T21:58:52.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.542+0000 7fbc29841740 20 client.16377 trim_cache size 0 max 16384 2024-04-26T21:58:52.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.563+0000 7fbc29841740 1 Processor -- start 2024-04-26T21:58:52.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.563+0000 7fbc29841740 1 -- start start 2024-04-26T21:58:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc29841740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc29841740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7df4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc248c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc248c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53746/0 (socket says 172.21.15.55:53746) 2024-04-26T21:58:52.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc248c0640 1 -- 172.21.15.55:0/2048310465 learned_addr learned my addr 172.21.15.55:0/2048310465 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:52.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc29841740 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 0x55d1cc7cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc29841740 1 -- 172.21.15.55:0/2048310465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbc201aa700 con 0x55d1cc7c98b0 2024-04-26T21:58:52.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc29841740 1 -- 172.21.15.55:0/2048310465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc201a7fd0 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc29841740 1 -- 172.21.15.55:0/2048310465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc201a5e20 con 0x55d1cc7cf410 2024-04-26T21:58:52.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7df4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc248c0640 1 -- 172.21.15.55:0/2048310465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 msgr2=0x55d1cc7cb480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.564+0000 7fbc248c0640 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 0x55d1cc7cb480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.565+0000 7fbc248c0640 1 -- 172.21.15.55:0/2048310465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 msgr2=0x55d1cc7df4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.565+0000 7fbc248c0640 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7df4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.565+0000 7fbc248c0640 1 -- 172.21.15.55:0/2048310465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cc7c4d50 con 0x55d1cc7c98b0 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.565+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7df4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.565+0000 7fbc248c0640 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9c90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbc14005c40 tx=0x7fbc140b9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb98154eed1a894 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.565+0000 7fbc0a7fc640 1 -- 172.21.15.55:0/2048310465 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc140d6070 con 0x55d1cc7c98b0 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.565+0000 7fbc0a7fc640 1 -- 172.21.15.55:0/2048310465 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc140c1530 con 0x55d1cc7c98b0 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.565+0000 7fbc0a7fc640 1 -- 172.21.15.55:0/2048310465 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14049040 con 0x55d1cc7c98b0 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.566+0000 7fbc29841740 1 -- 172.21.15.55:0/2048310465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 msgr2=0x55d1cc7c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.566+0000 7fbc29841740 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9c90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbc14005c40 tx=0x7fbc140b9540 comp rx=0 tx=0).stop 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.566+0000 7fbc29841740 1 -- 172.21.15.55:0/2048310465 shutdown_connections 2024-04-26T21:58:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.566+0000 7fbc29841740 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 0x55d1cc7cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.566+0000 7fbc29841740 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7df4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.566+0000 7fbc29841740 1 --2- 172.21.15.55:0/2048310465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7c9c90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.566+0000 7fbc29841740 1 -- 172.21.15.55:0/2048310465 >> 172.21.15.55:0/2048310465 conn(0x55d1cc7c4f20 msgr2=0x55d1cc7ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.566+0000 7fbc29841740 1 -- 172.21.15.55:0/2048310465 shutdown_connections 2024-04-26T21:58:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.567+0000 7fbc29841740 1 -- 172.21.15.55:0/2048310465 wait complete. 2024-04-26T21:58:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.567+0000 7fbc29841740 1 -- start start 2024-04-26T21:58:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.567+0000 7fbc29841740 1 Processor -- start 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc29841740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7dcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc29841740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc29841740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 0x55d1cc7de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc29841740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbc201a6490 con 0x55d1cc7c98b0 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc29841740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc201a7900 con 0x55d1cc7ca2a0 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc29841740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbc201a6b60 con 0x55d1cc7cf410 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc29841740 10 client.0 _ll_register_callbacks cb 0x7fbc0c009f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc248c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7dcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc248c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7dcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53762/0 (socket says 172.21.15.55:53762) 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc248c0640 1 -- 172.21.15.55:0/3425166334 learned_addr learned my addr 172.21.15.55:0/3425166334 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.568+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.569+0000 7fbc250c1640 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 0x55d1cc7de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.569+0000 7fbc248c0640 1 -- 172.21.15.55:0/3425166334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 msgr2=0x55d1cc7de040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.569+0000 7fbc248c0640 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 0x55d1cc7de040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.569+0000 7fbc248c0640 1 -- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 msgr2=0x55d1cc7dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.569+0000 7fbc248c0640 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7dd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.569+0000 7fbc248c0640 1 -- 172.21.15.55:0/3425166334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cc7de630 con 0x55d1cc7c98b0 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.569+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.570+0000 7fbc248c0640 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7dcb40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbc140c58c0 tx=0x7fbc14004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b787cb618a7db46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.570+0000 7fbbfffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.570+0000 7fbbfffff640 1 -- 172.21.15.55:0/3425166334 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14049040 con 0x55d1cc7c98b0 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.570+0000 7fbbfffff640 1 -- 172.21.15.55:0/3425166334 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbc1403b8d0 con 0x55d1cc7c98b0 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.570+0000 7fbc29841740 1 -- 172.21.15.55:0/3425166334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc140c1530 con 0x55d1cc7c98b0 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.570+0000 7fbbfffff640 1 -- 172.21.15.55:0/3425166334 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbc140d6040 con 0x55d1cc7c98b0 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.570+0000 7fbc29841740 10 client.16383 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.570+0000 7fbc29841740 1 -- 172.21.15.55:0/3425166334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d1cc849cc0 con 0x55d1cc7c98b0 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbbfffff640 1 -- 172.21.15.55:0/3425166334 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc140c5df0 con 0x55d1cc7c98b0 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbbfffff640 1 -- 172.21.15.55:0/3425166334 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fbc140c5960 con 0x55d1cc7c98b0 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbc29841740 10 client.16383 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbc29841740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbc29841740 20 client.16383 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbc29841740 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d1cc84a4d0 0x55d1cc84c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbc29841740 4 client.16383 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbc29841740 1 -- 172.21.15.55:0/3425166334 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fbc140c1530 con 0x55d1cc84a4d0 2024-04-26T21:58:52.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.571+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d1cc84a4d0 0x55d1cc84c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.572+0000 7fbc1ffff640 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d1cc84a4d0 0x55d1cc84c8f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b402df51823a6d server_cookie=3325083e3bed1554 in_seq=0 out_seq=0 2024-04-26T21:58:52.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:52.572+0000 7fbbfffff640 10 client.16383 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:53.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:53 smithi055 ceph-mon[29221]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.20k op/s 2024-04-26T21:58:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:53 smithi064 ceph-mon[32765]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.20k op/s 2024-04-26T21:58:53.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbbfffff640 1 -- 172.21.15.55:0/3425166334 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fbc140c1530 con 0x55d1cc84a4d0 2024-04-26T21:58:53.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbbfffff640 10 client.16383 handle_command_reply: tid=0 2024-04-26T21:58:53.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 client.16383 shutdown 2024-04-26T21:58:53.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 -- 172.21.15.55:0/3425166334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 msgr2=0x55d1cc7dcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7dcb40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbc140c58c0 tx=0x7fbc14004020 comp rx=0 tx=0).stop 2024-04-26T21:58:53.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 -- 172.21.15.55:0/3425166334 shutdown_connections 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1cc7cf410 0x55d1cc7de040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d1cc7ca2a0 0x55d1cc7dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d1cc84a4d0 0x55d1cc84c8f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 --2- 172.21.15.55:0/3425166334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d1cc7c98b0 0x55d1cc7dcb40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbc29841740 1 -- 172.21.15.55:0/3425166334 >> 172.21.15.55:0/3425166334 conn(0x55d1cc7c4f20 msgr2=0x55d1cc7d7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.516+0000 7fbbfffff640 0 client.16383 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.517+0000 7fbc29841740 1 -- 172.21.15.55:0/3425166334 shutdown_connections 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.517+0000 7fbc29841740 1 -- 172.21.15.55:0/3425166334 wait complete. 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.517+0000 7fbc29841740 20 client.16383 trim_cache size 0 max 16384 2024-04-26T21:58:53.518 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:58:53.519 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:58:53.519 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "e1db6ae2-ed41-43f3-b3d9-70108d943182", 2024-04-26T21:58:53.519 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:58:53.519 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.517+0000 7fbc26b24640 1 -- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbc100a3c40 msgr2=0x7fbc100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.517+0000 7fbc26b24640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbc100a3c40 0x7fbc100a6100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55d1cc7cab00 tx=0x7fbc18009250 comp rx=0 tx=0).stop 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.517+0000 7fbc26b24640 1 -- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 msgr2=0x7fbc20198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.517+0000 7fbc26b24640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 0x7fbc20198d70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbc14031500 tx=0x7fbc14003d50 comp rx=0 tx=0).stop 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.518+0000 7fbc26b24640 1 -- 172.21.15.55:0/609277592 shutdown_connections 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.518+0000 7fbc26b24640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbc100a3c40 0x7fbc100a6100 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.518+0000 7fbc26b24640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc201086f0 0x7fbc201a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.518+0000 7fbc26b24640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc20105dc0 0x7fbc201992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.518+0000 7fbc26b24640 1 --2- 172.21.15.55:0/609277592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20103490 0x7fbc20198d70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.518+0000 7fbc26b24640 1 -- 172.21.15.55:0/609277592 >> 172.21.15.55:0/609277592 conn(0x7fbc200fcde0 msgr2=0x7fbc20105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.518+0000 7fbc26b24640 1 -- 172.21.15.55:0/609277592 shutdown_connections 2024-04-26T21:58:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.518+0000 7fbc26b24640 1 -- 172.21.15.55:0/609277592 wait complete. 2024-04-26T21:58:53.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:58:53.541 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:58:53.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:53 smithi165 ceph-mon[33296]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.20k op/s 2024-04-26T21:58:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.869+0000 7fca85c12640 1 -- 172.21.15.55:0/3738180650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8010a960 msgr2=0x7fca801111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.869+0000 7fca85c12640 1 --2- 172.21.15.55:0/3738180650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8010a960 0x7fca801111f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fca78009ca0 tx=0x7fca7802a8f0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.873+0000 7fca85c12640 1 -- 172.21.15.55:0/3738180650 shutdown_connections 2024-04-26T21:58:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.873+0000 7fca85c12640 1 --2- 172.21.15.55:0/3738180650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8010a960 0x7fca801111f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.873+0000 7fca85c12640 1 --2- 172.21.15.55:0/3738180650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80109fa0 0x7fca8010a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.873+0000 7fca85c12640 1 --2- 172.21.15.55:0/3738180650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca80108db0 0x7fca801091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.873+0000 7fca85c12640 1 -- 172.21.15.55:0/3738180650 >> 172.21.15.55:0/3738180650 conn(0x7fca80077d40 msgr2=0x7fca8007a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.875+0000 7fca85c12640 1 -- 172.21.15.55:0/3738180650 shutdown_connections 2024-04-26T21:58:53.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.876+0000 7fca85c12640 1 -- 172.21.15.55:0/3738180650 wait complete. 2024-04-26T21:58:53.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.876+0000 7fca85c12640 1 Processor -- start 2024-04-26T21:58:53.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.877+0000 7fca85c12640 1 -- start start 2024-04-26T21:58:53.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.877+0000 7fca85c12640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca80108db0 0x7fca80080000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.877+0000 7fca85c12640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca80109fa0 0x7fca80080540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.877+0000 7fca85c12640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8010a960 0x7fca8019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.877+0000 7fca85c12640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fca80114a40 con 0x7fca80109fa0 2024-04-26T21:58:53.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.877+0000 7fca85c12640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca801148c0 con 0x7fca8010a960 2024-04-26T21:58:53.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.877+0000 7fca85c12640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca80114bc0 con 0x7fca80108db0 2024-04-26T21:58:53.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.878+0000 7fca84c10640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca80108db0 0x7fca80080000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.878+0000 7fca84c10640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca80108db0 0x7fca80080000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:46224/0 (socket says 172.21.15.55:46224) 2024-04-26T21:58:53.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.878+0000 7fca84c10640 1 -- 172.21.15.55:0/2821791464 learned_addr learned my addr 172.21.15.55:0/2821791464 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.879+0000 7fca84c10640 1 -- 172.21.15.55:0/2821791464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8010a960 msgr2=0x7fca8019e140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:58:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.879+0000 7fca84c10640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8010a960 0x7fca8019e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.879+0000 7fca84c10640 1 -- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca80109fa0 msgr2=0x7fca80080540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.879+0000 7fca84c10640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca80109fa0 0x7fca80080540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.879+0000 7fca84c10640 1 -- 172.21.15.55:0/2821791464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca80083730 con 0x7fca80108db0 2024-04-26T21:58:53.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.881+0000 7fca84c10640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca80108db0 0x7fca80080000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fca7000b470 tx=0x7fca7000b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:53.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.890+0000 7fca7dffb640 1 -- 172.21.15.55:0/2821791464 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70015020 con 0x7fca80108db0 2024-04-26T21:58:53.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.891+0000 7fca7dffb640 1 -- 172.21.15.55:0/2821791464 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca70011d40 con 0x7fca80108db0 2024-04-26T21:58:53.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.891+0000 7fca7dffb640 1 -- 172.21.15.55:0/2821791464 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7000f340 con 0x7fca80108db0 2024-04-26T21:58:53.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.892+0000 7fca85c12640 1 -- 172.21.15.55:0/2821791464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8019e740 con 0x7fca80108db0 2024-04-26T21:58:53.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.893+0000 7fca85c12640 1 -- 172.21.15.55:0/2821791464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8019ec90 con 0x7fca80108db0 2024-04-26T21:58:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.905+0000 7fca89b8b740 1 Processor -- start 2024-04-26T21:58:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.906+0000 7fca7dffb640 1 -- 172.21.15.55:0/2821791464 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fca70018020 con 0x7fca80108db0 2024-04-26T21:58:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.907+0000 7fca89b8b740 1 -- start start 2024-04-26T21:58:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.907+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.907+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.907+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.907+0000 7fca89b8b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56333850fd80 con 0x5633385acc40 2024-04-26T21:58:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.907+0000 7fca89b8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5633384c98b0 con 0x5633385ab0b0 2024-04-26T21:58:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.907+0000 7fca89b8b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56333850bd40 con 0x5633385ac280 2024-04-26T21:58:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.908+0000 7fca7dffb640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca680a3dc0 0x7fca680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.909+0000 7fca7dffb640 1 -- 172.21.15.55:0/2821791464 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fca700cfec0 con 0x7fca80108db0 2024-04-26T21:58:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.910+0000 7fca84c10640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.910+0000 7fca84c10640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:46232/0 (socket says 172.21.15.55:46232) 2024-04-26T21:58:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.910+0000 7fca84c10640 1 -- 172.21.15.55:0/3352876637 learned_addr learned my addr 172.21.15.55:0/3352876637 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.912+0000 7fca85411640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca680a3dc0 0x7fca680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.912+0000 7fca85411640 1 --2- 172.21.15.55:0/3352876637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.916+0000 7fca84c10640 1 -- 172.21.15.55:0/3352876637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 msgr2=0x5633385ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.916+0000 7fca84c10640 1 --2- 172.21.15.55:0/3352876637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.916+0000 7fca84c10640 1 -- 172.21.15.55:0/3352876637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 msgr2=0x5633385b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.916+0000 7fca84c10640 1 --2- 172.21.15.55:0/3352876637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.916+0000 7fca85411640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca680a3dc0 0x7fca680a6280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca78004af0 tx=0x7fca78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.916+0000 7fca84c10640 1 -- 172.21.15.55:0/3352876637 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333849aa40 con 0x5633385ac280 2024-04-26T21:58:53.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.926+0000 7fca84c10640 1 --2- 172.21.15.55:0/3352876637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ac700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fca70004790 tx=0x7fca7009c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8aade92514c1876 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.927+0000 7fca66ffd640 1 -- 172.21.15.55:0/3352876637 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca700a4d20 con 0x5633385ac280 2024-04-26T21:58:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.927+0000 7fca66ffd640 1 -- 172.21.15.55:0/3352876637 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca7009a3b0 con 0x5633385ac280 2024-04-26T21:58:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.929+0000 7fca89b8b740 1 -- 172.21.15.55:0/3352876637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 msgr2=0x5633385ac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.929+0000 7fca89b8b740 1 --2- 172.21.15.55:0/3352876637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ac700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fca70004790 tx=0x7fca7009c870 comp rx=0 tx=0).stop 2024-04-26T21:58:53.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.930+0000 7fca89b8b740 1 -- 172.21.15.55:0/3352876637 shutdown_connections 2024-04-26T21:58:53.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.930+0000 7fca89b8b740 1 --2- 172.21.15.55:0/3352876637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.930+0000 7fca89b8b740 1 --2- 172.21.15.55:0/3352876637 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ac700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.931+0000 7fca89b8b740 1 --2- 172.21.15.55:0/3352876637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.931+0000 7fca89b8b740 1 -- 172.21.15.55:0/3352876637 >> 172.21.15.55:0/3352876637 conn(0x5633385a65a0 msgr2=0x5633385a69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.933+0000 7fca89b8b740 1 -- 172.21.15.55:0/3352876637 shutdown_connections 2024-04-26T21:58:53.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.934+0000 7fca89b8b740 1 -- 172.21.15.55:0/3352876637 wait complete. 2024-04-26T21:58:53.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.935+0000 7fca89b8b740 1 -- start start 2024-04-26T21:58:53.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.935+0000 7fca89b8b740 1 Processor -- start 2024-04-26T21:58:53.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.936+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.936+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.936+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385bca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.936+0000 7fca89b8b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5633385b3c40 con 0x5633385acc40 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.936+0000 7fca89b8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563338496d10 con 0x5633385ab0b0 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.936+0000 7fca89b8b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5633385b3dc0 con 0x5633385ac280 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.936+0000 7fca89b8b740 10 client.0 _ll_register_callbacks cb 0x5633384c3210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.938+0000 7fca85411640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385bca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.938+0000 7fca85411640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385bca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53788/0 (socket says 172.21.15.55:53788) 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.938+0000 7fca85411640 1 -- 172.21.15.55:0/2200164105 learned_addr learned my addr 172.21.15.55:0/2200164105 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.938+0000 7fca85411640 1 -- 172.21.15.55:0/2200164105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 msgr2=0x5633385b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.938+0000 7fca85411640 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.938+0000 7fca85411640 1 -- 172.21.15.55:0/2200164105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 msgr2=0x5633385b54f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:58:53.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.938+0000 7fca85411640 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385b54f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.938+0000 7fca85411640 1 -- 172.21.15.55:0/2200164105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633382eb7d0 con 0x5633385acc40 2024-04-26T21:58:53.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.939+0000 7fca85411640 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385bca70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fca78009ca0 tx=0x7fca780315c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1761874e2a6ea1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.939+0000 7fca5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.939+0000 7fca5ffff640 1 -- 172.21.15.55:0/2200164105 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78003e00 con 0x5633385acc40 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.940+0000 7fca89b8b740 1 -- 172.21.15.55:0/2200164105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7009a3b0 con 0x5633385acc40 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.940+0000 7fca5ffff640 1 -- 172.21.15.55:0/2200164105 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fca7802fb10 con 0x5633385acc40 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.940+0000 7fca5ffff640 1 -- 172.21.15.55:0/2200164105 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fca7802fd10 con 0x5633385acc40 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.940+0000 7fca89b8b740 10 client.16389 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.940+0000 7fca89b8b740 1 -- 172.21.15.55:0/2200164105 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5633382213f0 con 0x5633385acc40 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.941+0000 7fca5ffff640 1 -- 172.21.15.55:0/2200164105 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca7009a3b0 con 0x5633385acc40 2024-04-26T21:58:53.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.941+0000 7fca5ffff640 1 -- 172.21.15.55:0/2200164105 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fca78052470 con 0x5633385acc40 2024-04-26T21:58:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.941+0000 7fca89b8b740 10 client.16389 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.941+0000 7fca89b8b740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.941+0000 7fca89b8b740 20 client.16389 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.941+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5633385d2480 0x5633385d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.941+0000 7fca89b8b740 4 client.16389 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:58:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.941+0000 7fca89b8b740 1 -- 172.21.15.55:0/2200164105 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56333851d990 con 0x5633385d2480 2024-04-26T21:58:53.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.943+0000 7fca84c10640 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5633385d2480 0x5633385d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:53.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.954+0000 7fca84c10640 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5633385d2480 0x5633385d4940 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af827027fb1ba4c8 server_cookie=d7f851feffc09ebb in_seq=0 out_seq=0 2024-04-26T21:58:53.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.954+0000 7fca5ffff640 10 client.16389 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:53.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.956+0000 7fca5ffff640 1 -- 172.21.15.55:0/2200164105 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56333851d990 con 0x5633385d2480 2024-04-26T21:58:53.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.956+0000 7fca5ffff640 10 client.16389 handle_command_reply: tid=0 2024-04-26T21:58:53.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.956+0000 7fca89b8b740 1 client.16389 shutdown 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.969+0000 7fca89b8b740 1 -- 172.21.15.55:0/2200164105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 msgr2=0x5633385bca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.969+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385bca70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fca78009ca0 tx=0x7fca780315c0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 1 -- 172.21.15.55:0/2200164105 shutdown_connections 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5633385d2480 0x5633385d4940 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385acc40 0x5633385bca70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2200164105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 1 -- 172.21.15.55:0/2200164105 >> 172.21.15.55:0/2200164105 conn(0x5633385a65a0 msgr2=0x5633385a56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca5ffff640 0 client.16389 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:53.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 1 -- 172.21.15.55:0/2200164105 shutdown_connections 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 1 -- 172.21.15.55:0/2200164105 wait complete. 2024-04-26T21:58:53.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:53.970+0000 7fca89b8b740 20 client.16389 trim_cache size 0 max 16384 2024-04-26T21:58:54.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.005+0000 7fca89b8b740 1 Processor -- start 2024-04-26T21:58:54.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.005+0000 7fca89b8b740 1 -- start start 2024-04-26T21:58:54.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.005+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.005+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ac280 0x5633385bca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.005+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385acc40 0x5633385a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.005+0000 7fca89b8b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c010fd0 con 0x5633385ac280 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.005+0000 7fca89b8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c00e8a0 con 0x5633385ab0b0 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.005+0000 7fca89b8b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c00c6f0 con 0x5633385acc40 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca7ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ac280 0x5633385bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca7ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ac280 0x5633385bca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53792/0 (socket says 172.21.15.55:53792) 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca7ffff640 1 -- 172.21.15.55:0/2375958911 learned_addr learned my addr 172.21.15.55:0/2375958911 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca84c10640 1 --2- 172.21.15.55:0/2375958911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca7ffff640 1 -- 172.21.15.55:0/2375958911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385acc40 msgr2=0x5633385a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca7ffff640 1 --2- 172.21.15.55:0/2375958911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385acc40 0x5633385a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca7ffff640 1 -- 172.21.15.55:0/2375958911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 msgr2=0x5633385b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:54.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca7ffff640 1 --2- 172.21.15.55:0/2375958911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.006+0000 7fca7ffff640 1 -- 172.21.15.55:0/2375958911 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633381e10f0 con 0x5633385ac280 2024-04-26T21:58:54.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.007+0000 7fca7ffff640 1 --2- 172.21.15.55:0/2375958911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ac280 0x5633385bca30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fca740095a0 tx=0x7fca74009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0df09b9427cee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:54.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.007+0000 7fca667fc640 1 -- 172.21.15.55:0/2375958911 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74011830 con 0x5633385ac280 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.007+0000 7fca667fc640 1 -- 172.21.15.55:0/2375958911 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca74016070 con 0x5633385ac280 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.007+0000 7fca667fc640 1 -- 172.21.15.55:0/2375958911 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74010da0 con 0x5633385ac280 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.008+0000 7fca89b8b740 1 -- 172.21.15.55:0/2375958911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ac280 msgr2=0x5633385bca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.008+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2375958911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ac280 0x5633385bca30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fca740095a0 tx=0x7fca74009a60 comp rx=0 tx=0).stop 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.008+0000 7fca89b8b740 1 -- 172.21.15.55:0/2375958911 shutdown_connections 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.008+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2375958911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385acc40 0x5633385a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.008+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2375958911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ac280 0x5633385bca30 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.008+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2375958911 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385ab0b0 0x5633385b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.008+0000 7fca89b8b740 1 -- 172.21.15.55:0/2375958911 >> 172.21.15.55:0/2375958911 conn(0x5633385a65a0 msgr2=0x5633385b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:54.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.009+0000 7fca89b8b740 1 -- 172.21.15.55:0/2375958911 shutdown_connections 2024-04-26T21:58:54.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.009+0000 7fca89b8b740 1 -- 172.21.15.55:0/2375958911 wait complete. 2024-04-26T21:58:54.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.009+0000 7fca89b8b740 1 -- start start 2024-04-26T21:58:54.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.010+0000 7fca89b8b740 1 Processor -- start 2024-04-26T21:58:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.010+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ab0b0 0x5633385ba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.010+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.010+0000 7fca89b8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385acc40 0x5633385bb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.010+0000 7fca89b8b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c00cd60 con 0x5633385ab0b0 2024-04-26T21:58:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca89b8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c00e1d0 con 0x5633385acc40 2024-04-26T21:58:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca89b8b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c00d430 con 0x5633385ac280 2024-04-26T21:58:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca89b8b740 10 client.0 _ll_register_callbacks cb 0x7fca78004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:58:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca84c10640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ab0b0 0x5633385ba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca84c10640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ab0b0 0x5633385ba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53796/0 (socket says 172.21.15.55:53796) 2024-04-26T21:58:54.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca84c10640 1 -- 172.21.15.55:0/2007217209 learned_addr learned my addr 172.21.15.55:0/2007217209 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:58:54.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca85411640 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385acc40 0x5633385bb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca84c10640 1 -- 172.21.15.55:0/2007217209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 msgr2=0x5633385ba780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:54.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca84c10640 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca84c10640 1 -- 172.21.15.55:0/2007217209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385acc40 msgr2=0x5633385bb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:58:54.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.011+0000 7fca84c10640 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385acc40 0x5633385bb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.012+0000 7fca84c10640 1 -- 172.21.15.55:0/2007217209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633384d04b0 con 0x5633385ab0b0 2024-04-26T21:58:54.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.012+0000 7fca84c10640 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ab0b0 0x5633385ba240 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fca70011040 tx=0x7fca7009c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e308ff3f3d6eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:58:54.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.012+0000 7fca5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:58:54.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.013+0000 7fca5ffff640 1 -- 172.21.15.55:0/2007217209 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7009a3b0 con 0x5633385ab0b0 2024-04-26T21:58:54.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.013+0000 7fca5ffff640 1 -- 172.21.15.55:0/2007217209 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fca700a00b0 con 0x5633385ab0b0 2024-04-26T21:58:54.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.013+0000 7fca89b8b740 1 -- 172.21.15.55:0/2007217209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca74016070 con 0x5633385ab0b0 2024-04-26T21:58:54.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.014+0000 7fca5ffff640 1 -- 172.21.15.55:0/2007217209 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fca700cdbc0 con 0x5633385ab0b0 2024-04-26T21:58:54.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.016+0000 7fca89b8b740 10 client.16395 fetch_fsmap learned FSMap version 23 2024-04-26T21:58:54.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.016+0000 7fca89b8b740 1 -- 172.21.15.55:0/2007217209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563338638a80 con 0x5633385ab0b0 2024-04-26T21:58:54.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.016+0000 7fca5ffff640 1 -- 172.21.15.55:0/2007217209 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca74016070 con 0x5633385ab0b0 2024-04-26T21:58:54.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.017+0000 7fca5ffff640 1 -- 172.21.15.55:0/2007217209 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fca7009b500 con 0x5633385ab0b0 2024-04-26T21:58:54.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.017+0000 7fca89b8b740 10 client.16395 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:58:54.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.017+0000 7fca89b8b740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:58:54.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.017+0000 7fca89b8b740 20 client.16395 populate_metadata read hostname 'smithi055' 2024-04-26T21:58:54.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.017+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563338638f90 0x56333863b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:58:54.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.017+0000 7fca89b8b740 4 client.16395 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:58:54.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.017+0000 7fca89b8b740 1 -- 172.21.15.55:0/2007217209 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563338517800 con 0x563338638f90 2024-04-26T21:58:54.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.019+0000 7fca7ffff640 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563338638f90 0x56333863b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:58:54.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.024+0000 7fca5ffff640 10 client.16395 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:54.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.024+0000 7fca7ffff640 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563338638f90 0x56333863b380 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ace1d72a8a65b04f server_cookie=a48e5045e4397a92 in_seq=0 out_seq=0 2024-04-26T21:58:54.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.026+0000 7fca5ffff640 1 -- 172.21.15.55:0/2007217209 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x563338517800 con 0x563338638f90 2024-04-26T21:58:54.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.026+0000 7fca5ffff640 10 client.16395 handle_command_reply: tid=0 2024-04-26T21:58:54.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.026+0000 7fca89b8b740 1 client.16395 shutdown 2024-04-26T21:58:54.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca89b8b740 1 -- 172.21.15.55:0/2007217209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ab0b0 msgr2=0x5633385ba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ab0b0 0x5633385ba240 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fca70011040 tx=0x7fca7009c870 comp rx=0 tx=0).stop 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca89b8b740 1 -- 172.21.15.55:0/2007217209 shutdown_connections 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633385acc40 0x5633385bb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633385ac280 0x5633385ba780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563338638f90 0x56333863b380 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca89b8b740 1 --2- 172.21.15.55:0/2007217209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5633385ab0b0 0x5633385ba240 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca5ffff640 0 client.16395 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.027+0000 7fca89b8b740 1 -- 172.21.15.55:0/2007217209 >> 172.21.15.55:0/2007217209 conn(0x5633385a65a0 msgr2=0x5633385a4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.028+0000 7fca89b8b740 1 -- 172.21.15.55:0/2007217209 shutdown_connections 2024-04-26T21:58:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.028+0000 7fca89b8b740 1 -- 172.21.15.55:0/2007217209 wait complete. 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.028+0000 7fca89b8b740 20 client.16395 trim_cache size 0 max 16384 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stdout: "e1db6ae2-ed41-43f3-b3d9-70108d943182": { 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "e1db6ae2-ed41-43f3-b3d9-70108d943182", 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:58:54.030 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:58:54.031 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:58:54.031 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:58:54.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 -- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca680a3dc0 msgr2=0x7fca680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca680a3dc0 0x7fca680a6280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca78004af0 tx=0x7fca78005f90 comp rx=0 tx=0).stop 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 -- 172.21.15.55:0/2821791464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca80108db0 msgr2=0x7fca80080000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca80108db0 0x7fca80080000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fca7000b470 tx=0x7fca7000b930 comp rx=0 tx=0).stop 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 -- 172.21.15.55:0/2821791464 shutdown_connections 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fca680a3dc0 0x7fca680a6280 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca8010a960 0x7fca8019e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca80109fa0 0x7fca80080540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 --2- 172.21.15.55:0/2821791464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca80108db0 0x7fca80080000 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 -- 172.21.15.55:0/2821791464 >> 172.21.15.55:0/2821791464 conn(0x7fca80077d40 msgr2=0x7fca8010d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 -- 172.21.15.55:0/2821791464 shutdown_connections 2024-04-26T21:58:54.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:58:54.029+0000 7fca85c12640 1 -- 172.21.15.55:0/2821791464 wait complete. 2024-04-26T21:58:54.059 INFO:tasks.cephfs.filesystem:scrub status for tag:e1db6ae2-ed41-43f3-b3d9-70108d943182 - {'path': '/', 'tag': 'e1db6ae2-ed41-43f3-b3d9-70108d943182', 'options': 'recursive,force'} 2024-04-26T21:58:54.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:54 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:58:54.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:54 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:54.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:54 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:58:54.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:54 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:54.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:54 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:58:54.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:54 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:55 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:58:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:55 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:55 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:58:55.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:55 smithi064 ceph-mon[32765]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 886 MiB/s wr, 1.19k op/s 2024-04-26T21:58:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:55 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:58:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:55 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:55 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:58:55.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:55 smithi165 ceph-mon[33296]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 886 MiB/s wr, 1.19k op/s 2024-04-26T21:58:55.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:55 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:58:55.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:55 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:55.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:55 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:58:55.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:55 smithi055 ceph-mon[29221]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 886 MiB/s wr, 1.19k op/s 2024-04-26T21:58:57.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:57 smithi064 ceph-mon[32765]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 1.23k op/s 2024-04-26T21:58:57.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:57 smithi165 ceph-mon[33296]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 1.23k op/s 2024-04-26T21:58:57.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:57 smithi055 ceph-mon[29221]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 1.23k op/s 2024-04-26T21:58:58.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:58 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:58.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:58 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:58.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:58 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:58:59.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:58:59 smithi064 ceph-mon[32765]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 911 MiB/s wr, 1.39k op/s 2024-04-26T21:58:59.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:58:59 smithi165 ceph-mon[33296]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 911 MiB/s wr, 1.39k op/s 2024-04-26T21:58:59.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:58:59 smithi055 ceph-mon[29221]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 911 MiB/s wr, 1.39k op/s 2024-04-26T21:59:01.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:01 smithi064 ceph-mon[32765]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 861 MiB/s wr, 1.27k op/s 2024-04-26T21:59:01.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:01 smithi165 ceph-mon[33296]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 861 MiB/s wr, 1.27k op/s 2024-04-26T21:59:01.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:01 smithi055 ceph-mon[29221]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 861 MiB/s wr, 1.27k op/s 2024-04-26T21:59:02.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:02 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:59:02.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:02 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:59:02.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:02 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:59:02.690 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:02 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:59:02.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:02 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:59:02.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:02 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:59:03.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:03 smithi165 ceph-mon[33296]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 842 MiB/s wr, 1.21k op/s 2024-04-26T21:59:03.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:03 smithi055 ceph-mon[29221]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 842 MiB/s wr, 1.21k op/s 2024-04-26T21:59:03.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:03 smithi064 ceph-mon[32765]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 842 MiB/s wr, 1.21k op/s 2024-04-26T21:59:05.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:05 smithi165 ceph-mon[33296]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 843 MiB/s wr, 1.21k op/s 2024-04-26T21:59:05.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:05 smithi055 ceph-mon[29221]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 843 MiB/s wr, 1.21k op/s 2024-04-26T21:59:05.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:05 smithi064 ceph-mon[32765]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 843 MiB/s wr, 1.21k op/s 2024-04-26T21:59:07.558 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:07 smithi064 ceph-mon[32765]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 832 MiB/s wr, 1.31k op/s 2024-04-26T21:59:07.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:07 smithi165 ceph-mon[33296]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 832 MiB/s wr, 1.31k op/s 2024-04-26T21:59:07.693 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:07 smithi055 ceph-mon[29221]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 832 MiB/s wr, 1.31k op/s 2024-04-26T21:59:09.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:09 smithi165 ceph-mon[33296]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 909 MiB/s wr, 1.58k op/s 2024-04-26T21:59:09.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:09 smithi055 ceph-mon[29221]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 909 MiB/s wr, 1.58k op/s 2024-04-26T21:59:09.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:09 smithi064 ceph-mon[32765]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 909 MiB/s wr, 1.58k op/s 2024-04-26T21:59:11.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:11 smithi165 ceph-mon[33296]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 849 MiB/s wr, 1.51k op/s 2024-04-26T21:59:11.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:11 smithi055 ceph-mon[29221]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 849 MiB/s wr, 1.51k op/s 2024-04-26T21:59:11.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:11 smithi064 ceph-mon[32765]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 849 MiB/s wr, 1.51k op/s 2024-04-26T21:59:13.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:13 smithi165 ceph-mon[33296]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 841 MiB/s wr, 1.46k op/s 2024-04-26T21:59:13.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:13 smithi055 ceph-mon[29221]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 841 MiB/s wr, 1.46k op/s 2024-04-26T21:59:13.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:13 smithi064 ceph-mon[32765]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 841 MiB/s wr, 1.46k op/s 2024-04-26T21:59:15.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:15 smithi055 ceph-mon[29221]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.47k op/s 2024-04-26T21:59:15.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:15 smithi064 ceph-mon[32765]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.47k op/s 2024-04-26T21:59:15.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:15 smithi165 ceph-mon[33296]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.47k op/s 2024-04-26T21:59:17.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:17 smithi055 ceph-mon[29221]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 879 MiB/s wr, 1.58k op/s 2024-04-26T21:59:17.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:17 smithi064 ceph-mon[32765]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 879 MiB/s wr, 1.58k op/s 2024-04-26T21:59:17.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:17 smithi165 ceph-mon[33296]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 879 MiB/s wr, 1.58k op/s 2024-04-26T21:59:19.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:19 smithi055 ceph-mon[29221]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 899 MiB/s wr, 1.47k op/s 2024-04-26T21:59:19.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:19 smithi064 ceph-mon[32765]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 899 MiB/s wr, 1.47k op/s 2024-04-26T21:59:19.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:19 smithi165 ceph-mon[33296]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 899 MiB/s wr, 1.47k op/s 2024-04-26T21:59:21.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:21 smithi055 ceph-mon[29221]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 837 MiB/s wr, 1.16k op/s 2024-04-26T21:59:21.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:21 smithi064 ceph-mon[32765]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 837 MiB/s wr, 1.16k op/s 2024-04-26T21:59:21.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:21 smithi165 ceph-mon[33296]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 837 MiB/s wr, 1.16k op/s 2024-04-26T21:59:23.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:23 smithi055 ceph-mon[29221]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.09k op/s 2024-04-26T21:59:23.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:23 smithi064 ceph-mon[32765]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.09k op/s 2024-04-26T21:59:23.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:23 smithi165 ceph-mon[33296]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.09k op/s 2024-04-26T21:59:24.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:59:24.061 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:59:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.504+0000 7f8c48903640 1 -- 172.21.15.55:0/1798586498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c4410ec70 msgr2=0x7f8c4410f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.504+0000 7f8c48903640 1 --2- 172.21.15.55:0/1798586498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c4410ec70 0x7f8c4410f120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009be0 tx=0x7f8c3c02a6f0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.506+0000 7f8c48903640 1 -- 172.21.15.55:0/1798586498 shutdown_connections 2024-04-26T21:59:24.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.506+0000 7f8c48903640 1 --2- 172.21.15.55:0/1798586498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c4410ec70 0x7f8c4410f120 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.506+0000 7f8c48903640 1 --2- 172.21.15.55:0/1798586498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c4410c2a0 0x7f8c4410e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.506+0000 7f8c48903640 1 --2- 172.21.15.55:0/1798586498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c44109930 0x7f8c4410bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.506+0000 7f8c48903640 1 -- 172.21.15.55:0/1798586498 >> 172.21.15.55:0/1798586498 conn(0x7f8c4406f7c0 msgr2=0x7f8c44071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.512+0000 7f8c48903640 1 -- 172.21.15.55:0/1798586498 shutdown_connections 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.515+0000 7f8c48903640 1 -- 172.21.15.55:0/1798586498 wait complete. 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.516+0000 7f8c48903640 1 Processor -- start 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.516+0000 7f8c48903640 1 -- start start 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.516+0000 7f8c48903640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c44109930 0x7f8c441192d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.516+0000 7f8c48903640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c4410c2a0 0x7f8c44119810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.516+0000 7f8c48903640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c4410ec70 0x7f8c4411a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.516+0000 7f8c48903640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44111900 con 0x7f8c44109930 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.516+0000 7f8c48903640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44111780 con 0x7f8c4410c2a0 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.516+0000 7f8c48903640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44111a80 con 0x7f8c4410ec70 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.517+0000 7f8c4259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c44109930 0x7f8c441192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.517+0000 7f8c4259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c44109930 0x7f8c441192d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49464/0 (socket says 172.21.15.55:49464) 2024-04-26T21:59:24.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.517+0000 7f8c4259c640 1 -- 172.21.15.55:0/1109129322 learned_addr learned my addr 172.21.15.55:0/1109129322 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:24.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.517+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c4410ec70 0x7f8c4411a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.518+0000 7f8c4259c640 1 -- 172.21.15.55:0/1109129322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c4410ec70 msgr2=0x7f8c4411a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.518+0000 7f8c4259c640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c4410ec70 0x7f8c4411a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.518+0000 7f8c4259c640 1 -- 172.21.15.55:0/1109129322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c4410c2a0 msgr2=0x7f8c44119810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:59:24.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.518+0000 7f8c4259c640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c4410c2a0 0x7f8c44119810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.518+0000 7f8c4259c640 1 -- 172.21.15.55:0/1109129322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4411af30 con 0x7f8c44109930 2024-04-26T21:59:24.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.519+0000 7f8c4259c640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c44109930 0x7f8c441192d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400e940 tx=0x7f8c3400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:24.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.520+0000 7f8c337fe640 1 -- 172.21.15.55:0/1109129322 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400ccc0 con 0x7f8c44109930 2024-04-26T21:59:24.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.521+0000 7f8c48903640 1 -- 172.21.15.55:0/1109129322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4411b220 con 0x7f8c44109930 2024-04-26T21:59:24.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.521+0000 7f8c48903640 1 -- 172.21.15.55:0/1109129322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4407ab90 con 0x7f8c44109930 2024-04-26T21:59:24.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.521+0000 7f8c337fe640 1 -- 172.21.15.55:0/1109129322 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c340054d0 con 0x7f8c44109930 2024-04-26T21:59:24.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.522+0000 7f8c337fe640 1 -- 172.21.15.55:0/1109129322 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34010430 con 0x7f8c44109930 2024-04-26T21:59:24.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.526+0000 7f8c4b618740 1 Processor -- start 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.527+0000 7f8c4b618740 1 -- start start 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.527+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 0x555f10c1e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.527+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c1f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.527+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1fa70 0x555f10c26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.527+0000 7f8c4b618740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555f10b82d80 con 0x555f10c1fa70 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.527+0000 7f8c4b618740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555f10b3c8b0 con 0x555f10c1f0b0 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.527+0000 7f8c4b618740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555f10b7ed40 con 0x555f10c1dee0 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.529+0000 7f8c42d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 0x555f10c1e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.530+0000 7f8c337fe640 1 -- 172.21.15.55:0/1109129322 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8c340046c0 con 0x7f8c44109930 2024-04-26T21:59:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.531+0000 7f8c337fe640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8c2c0a3dc0 0x7f8c2c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.532+0000 7f8c41d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1fa70 0x555f10c26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.532+0000 7f8c41d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1fa70 0x555f10c26300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49478/0 (socket says 172.21.15.55:49478) 2024-04-26T21:59:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.532+0000 7f8c41d9b640 1 -- 172.21.15.55:0/4013428356 learned_addr learned my addr 172.21.15.55:0/4013428356 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.533+0000 7f8c41d9b640 1 -- 172.21.15.55:0/4013428356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 msgr2=0x555f10c1e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.533+0000 7f8c41d9b640 1 --2- 172.21.15.55:0/4013428356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 0x555f10c1e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.533+0000 7f8c41d9b640 1 -- 172.21.15.55:0/4013428356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 msgr2=0x555f10c1f530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.533+0000 7f8c41d9b640 1 --2- 172.21.15.55:0/4013428356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c1f530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.533+0000 7f8c41d9b640 1 -- 172.21.15.55:0/4013428356 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f10b0da40 con 0x555f10c1fa70 2024-04-26T21:59:24.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.534+0000 7f8c41d9b640 1 --2- 172.21.15.55:0/4013428356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1fa70 0x555f10c26300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800ba50 tx=0x7f8c3800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2849f7ecff409e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:24.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.535+0000 7f8c30ff9640 1 -- 172.21.15.55:0/4013428356 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38016070 con 0x555f10c1fa70 2024-04-26T21:59:24.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.535+0000 7f8c30ff9640 1 -- 172.21.15.55:0/4013428356 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3800dc20 con 0x555f10c1fa70 2024-04-26T21:59:24.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.537+0000 7f8c4b618740 1 -- 172.21.15.55:0/4013428356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1fa70 msgr2=0x555f10c26300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.537+0000 7f8c4b618740 1 --2- 172.21.15.55:0/4013428356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1fa70 0x555f10c26300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800ba50 tx=0x7f8c3800bf10 comp rx=0 tx=0).stop 2024-04-26T21:59:24.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.537+0000 7f8c337fe640 1 -- 172.21.15.55:0/1109129322 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8c3409a940 con 0x7f8c44109930 2024-04-26T21:59:24.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.537+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8c2c0a3dc0 0x7f8c2c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.538+0000 7f8c4b618740 1 -- 172.21.15.55:0/4013428356 shutdown_connections 2024-04-26T21:59:24.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.538+0000 7f8c4b618740 1 --2- 172.21.15.55:0/4013428356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1fa70 0x555f10c26300 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.538+0000 7f8c4b618740 1 --2- 172.21.15.55:0/4013428356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c1f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.538+0000 7f8c4b618740 1 --2- 172.21.15.55:0/4013428356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 0x555f10c1e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.538+0000 7f8c4b618740 1 -- 172.21.15.55:0/4013428356 >> 172.21.15.55:0/4013428356 conn(0x555f10c193d0 msgr2=0x555f10c197d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:24.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.539+0000 7f8c4b618740 1 -- 172.21.15.55:0/4013428356 shutdown_connections 2024-04-26T21:59:24.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.539+0000 7f8c4b618740 1 -- 172.21.15.55:0/4013428356 wait complete. 2024-04-26T21:59:24.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.540+0000 7f8c4b618740 1 -- start start 2024-04-26T21:59:24.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.540+0000 7f8c4b618740 1 Processor -- start 2024-04-26T21:59:24.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.542+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 0x555f10c283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.542+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.542+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.542+0000 7f8c4b618740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555f10c26ad0 con 0x555f10c2d910 2024-04-26T21:59:24.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.542+0000 7f8c4b618740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555f10b09d10 con 0x555f10c1f0b0 2024-04-26T21:59:24.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.542+0000 7f8c4b618740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555f10c26c50 con 0x555f10c1dee0 2024-04-26T21:59:24.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.542+0000 7f8c4b618740 10 client.0 _ll_register_callbacks cb 0x555f10b36210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:24.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.543+0000 7f8c42d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.543+0000 7f8c42d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c2fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49494/0 (socket says 172.21.15.55:49494) 2024-04-26T21:59:24.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.543+0000 7f8c42d9d640 1 -- 172.21.15.55:0/2093364170 learned_addr learned my addr 172.21.15.55:0/2093364170 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:24.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.545+0000 7f8c41d9b640 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.546+0000 7f8c4259c640 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 0x555f10c283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.546+0000 7f8c42d9d640 1 -- 172.21.15.55:0/2093364170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 msgr2=0x555f10c283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:24.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.546+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 0x555f10c283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.546+0000 7f8c42d9d640 1 -- 172.21.15.55:0/2093364170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 msgr2=0x555f10c288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:24.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.546+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c288e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.546+0000 7f8c42d9d640 1 -- 172.21.15.55:0/2093364170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f1095e7d0 con 0x555f10c2d910 2024-04-26T21:59:24.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.547+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8c2c0a3dc0 0x7f8c2c0a6280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8c4406c650 tx=0x7f8c3c030700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:24.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.547+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c2fd00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0099a0 tx=0x7f8c3c045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3479cf0d83e535 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:24.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.558+0000 7f8c29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:24.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.561+0000 7f8c29ffb640 1 -- 172.21.15.55:0/2093364170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c036040 con 0x555f10c2d910 2024-04-26T21:59:24.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.561+0000 7f8c4b618740 1 -- 172.21.15.55:0/2093364170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f10b90990 con 0x555f10c2d910 2024-04-26T21:59:24.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.561+0000 7f8c29ffb640 1 -- 172.21.15.55:0/2093364170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8c3c0040c0 con 0x555f10c2d910 2024-04-26T21:59:24.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.564+0000 7f8c29ffb640 1 -- 172.21.15.55:0/2093364170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8c3c0042c0 con 0x555f10c2d910 2024-04-26T21:59:24.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.565+0000 7f8c4b618740 10 client.16407 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:24.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.565+0000 7f8c4b618740 1 -- 172.21.15.55:0/2093364170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555f108943f0 con 0x555f10c2d910 2024-04-26T21:59:24.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.572+0000 7f8c29ffb640 1 -- 172.21.15.55:0/2093364170 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555f10b90990 con 0x555f10c2d910 2024-04-26T21:59:24.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.572+0000 7f8c29ffb640 1 -- 172.21.15.55:0/2093364170 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8c3c03fc60 con 0x555f10c2d910 2024-04-26T21:59:24.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.572+0000 7f8c4b618740 10 client.16407 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:24.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.572+0000 7f8c4b618740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:59:24.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.572+0000 7f8c4b618740 20 client.16407 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:24.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.577+0000 7f8c4b618740 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x555f10c34420 0x555f10c36850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.577+0000 7f8c4b618740 4 client.16407 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:59:24.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.577+0000 7f8c4b618740 1 -- 172.21.15.55:0/2093364170 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f10b8a800 con 0x555f10c34420 2024-04-26T21:59:24.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.579+0000 7f8c4259c640 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x555f10c34420 0x555f10c36850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.588+0000 7f8c4259c640 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x555f10c34420 0x555f10c36850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67dd401a3dc3884c server_cookie=876654db81033061 in_seq=0 out_seq=0 2024-04-26T21:59:24.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.589+0000 7f8c29ffb640 10 client.16407 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:24.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.592+0000 7f8c29ffb640 1 -- 172.21.15.55:0/2093364170 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f10b8a800 con 0x555f10c34420 2024-04-26T21:59:24.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.592+0000 7f8c29ffb640 10 client.16407 handle_command_reply: tid=0 2024-04-26T21:59:24.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.592+0000 7f8c4b618740 1 client.16407 shutdown 2024-04-26T21:59:24.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.594+0000 7f8c4b618740 1 -- 172.21.15.55:0/2093364170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 msgr2=0x555f10c2fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.594+0000 7f8c4b618740 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c2fd00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0099a0 tx=0x7f8c3c045000 comp rx=0 tx=0).stop 2024-04-26T21:59:24.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.600+0000 7f8c4b618740 1 -- 172.21.15.55:0/2093364170 shutdown_connections 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.600+0000 7f8c4b618740 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x555f10c34420 0x555f10c36850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.600+0000 7f8c4b618740 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c2fd00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.600+0000 7f8c4b618740 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.600+0000 7f8c4b618740 1 --2- 172.21.15.55:0/2093364170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1dee0 0x555f10c283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.600+0000 7f8c4b618740 1 -- 172.21.15.55:0/2093364170 >> 172.21.15.55:0/2093364170 conn(0x555f10c193d0 msgr2=0x555f10c1c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.600+0000 7f8c29ffb640 0 client.16407 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.600+0000 7f8c4b618740 1 -- 172.21.15.55:0/2093364170 shutdown_connections 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.601+0000 7f8c4b618740 1 -- 172.21.15.55:0/2093364170 wait complete. 2024-04-26T21:59:24.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.602+0000 7f8c4b618740 20 client.16407 trim_cache size 0 max 16384 2024-04-26T21:59:24.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.638+0000 7f8c4b618740 1 Processor -- start 2024-04-26T21:59:24.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.638+0000 7f8c4b618740 1 -- start start 2024-04-26T21:59:24.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1dee0 0x555f10c27670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c27bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c2d910 0x555f10c1c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4b618740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c24010fd0 con 0x555f10c1dee0 2024-04-26T21:59:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4b618740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2400e8a0 con 0x555f10c1f0b0 2024-04-26T21:59:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4b618740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2400c6f0 con 0x555f10c2d910 2024-04-26T21:59:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1dee0 0x555f10c27670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1dee0 0x555f10c27670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49510/0 (socket says 172.21.15.55:49510) 2024-04-26T21:59:24.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4259c640 1 -- 172.21.15.55:0/3410923630 learned_addr learned my addr 172.21.15.55:0/3410923630 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:24.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4259c640 1 -- 172.21.15.55:0/3410923630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c2d910 msgr2=0x555f10c1c2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:24.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4259c640 1 --2- 172.21.15.55:0/3410923630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c2d910 0x555f10c1c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4259c640 1 -- 172.21.15.55:0/3410923630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 msgr2=0x555f10c27bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:24.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4259c640 1 --2- 172.21.15.55:0/3410923630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c27bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.639+0000 7f8c4259c640 1 -- 172.21.15.55:0/3410923630 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f108540f0 con 0x555f10c1dee0 2024-04-26T21:59:24.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.640+0000 7f8c4259c640 1 --2- 172.21.15.55:0/3410923630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1dee0 0x555f10c27670 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8c34099270 tx=0x7f8c3400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c79dca657285ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:24.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.640+0000 7f8c2bfff640 1 -- 172.21.15.55:0/3410923630 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c340a0c50 con 0x555f10c1dee0 2024-04-26T21:59:24.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.640+0000 7f8c2bfff640 1 -- 172.21.15.55:0/3410923630 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c340a5510 con 0x555f10c1dee0 2024-04-26T21:59:24.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.640+0000 7f8c2bfff640 1 -- 172.21.15.55:0/3410923630 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c340a5080 con 0x555f10c1dee0 2024-04-26T21:59:24.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.641+0000 7f8c4b618740 1 -- 172.21.15.55:0/3410923630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1dee0 msgr2=0x555f10c27670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.641+0000 7f8c4b618740 1 --2- 172.21.15.55:0/3410923630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1dee0 0x555f10c27670 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8c34099270 tx=0x7f8c3400ef30 comp rx=0 tx=0).stop 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.641+0000 7f8c4b618740 1 -- 172.21.15.55:0/3410923630 shutdown_connections 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.641+0000 7f8c4b618740 1 --2- 172.21.15.55:0/3410923630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c2d910 0x555f10c1c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.641+0000 7f8c4b618740 1 --2- 172.21.15.55:0/3410923630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1f0b0 0x555f10c27bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.641+0000 7f8c4b618740 1 --2- 172.21.15.55:0/3410923630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c1dee0 0x555f10c27670 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.641+0000 7f8c4b618740 1 -- 172.21.15.55:0/3410923630 >> 172.21.15.55:0/3410923630 conn(0x555f10c193d0 msgr2=0x555f10c22af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.641+0000 7f8c4b618740 1 -- 172.21.15.55:0/3410923630 shutdown_connections 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.642+0000 7f8c4b618740 1 -- 172.21.15.55:0/3410923630 wait complete. 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.642+0000 7f8c4b618740 1 -- start start 2024-04-26T21:59:24.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.643+0000 7f8c4b618740 1 Processor -- start 2024-04-26T21:59:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.643+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1dee0 0x555f10c45300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.643+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1f0b0 0x555f10c45840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.643+0000 7f8c4b618740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c46850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.644+0000 7f8c4b618740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2400cd60 con 0x555f10c2d910 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.644+0000 7f8c4b618740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2400e1d0 con 0x555f10c1dee0 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.644+0000 7f8c4b618740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2400d430 con 0x555f10c1f0b0 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.644+0000 7f8c4b618740 10 client.0 _ll_register_callbacks cb 0x7f8c34005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.644+0000 7f8c42d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c46850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.644+0000 7f8c42d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c46850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49514/0 (socket says 172.21.15.55:49514) 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.644+0000 7f8c42d9d640 1 -- 172.21.15.55:0/347863011 learned_addr learned my addr 172.21.15.55:0/347863011 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.645+0000 7f8c42d9d640 1 -- 172.21.15.55:0/347863011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1f0b0 msgr2=0x555f10c45840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.645+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1f0b0 0x555f10c45840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.645+0000 7f8c42d9d640 1 -- 172.21.15.55:0/347863011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1dee0 msgr2=0x555f10c45300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.645+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1dee0 0x555f10c45300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.645+0000 7f8c42d9d640 1 -- 172.21.15.55:0/347863011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f10b434b0 con 0x555f10c2d910 2024-04-26T21:59:24.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.646+0000 7f8c42d9d640 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c46850 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c04b510 tx=0x7f8c3c004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50128686a06f528 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:24.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.646+0000 7f8c29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:24.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.647+0000 7f8c29ffb640 1 -- 172.21.15.55:0/347863011 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c004460 con 0x555f10c2d910 2024-04-26T21:59:24.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.647+0000 7f8c4b618740 1 -- 172.21.15.55:0/347863011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c340a5510 con 0x555f10c2d910 2024-04-26T21:59:24.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.647+0000 7f8c29ffb640 1 -- 172.21.15.55:0/347863011 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8c3c004600 con 0x555f10c2d910 2024-04-26T21:59:24.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.648+0000 7f8c29ffb640 1 -- 172.21.15.55:0/347863011 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8c3c004820 con 0x555f10c2d910 2024-04-26T21:59:24.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.648+0000 7f8c4b618740 10 client.16413 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:24.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.648+0000 7f8c4b618740 1 -- 172.21.15.55:0/347863011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555f10c47f60 con 0x555f10c2d910 2024-04-26T21:59:24.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.649+0000 7f8c29ffb640 1 -- 172.21.15.55:0/347863011 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c03f900 con 0x555f10c2d910 2024-04-26T21:59:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.649+0000 7f8c29ffb640 1 -- 172.21.15.55:0/347863011 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8c3c031440 con 0x555f10c2d910 2024-04-26T21:59:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.649+0000 7f8c4b618740 10 client.16413 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.649+0000 7f8c4b618740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:59:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.649+0000 7f8c4b618740 20 client.16413 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.649+0000 7f8c4b618740 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x555f10c9ab90 0x555f10c9cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.649+0000 7f8c4b618740 4 client.16413 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:59:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.649+0000 7f8c4b618740 1 -- 172.21.15.55:0/347863011 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8c340a5510 con 0x555f10c9ab90 2024-04-26T21:59:24.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.650+0000 7f8c4259c640 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x555f10c9ab90 0x555f10c9cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:24.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.657+0000 7f8c29ffb640 10 client.16413 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:24.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.657+0000 7f8c4259c640 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x555f10c9ab90 0x555f10c9cfb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a837054b479fa26a server_cookie=b87bcfcdaf2345f9 in_seq=0 out_seq=0 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c29ffb640 1 -- 172.21.15.55:0/347863011 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f8c340a5510 con 0x555f10c9ab90 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c29ffb640 10 client.16413 handle_command_reply: tid=0 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 client.16413 shutdown 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 -- 172.21.15.55:0/347863011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 msgr2=0x555f10c46850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c46850 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c04b510 tx=0x7f8c3c004170 comp rx=0 tx=0).stop 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 -- 172.21.15.55:0/347863011 shutdown_connections 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x555f10c9ab90 0x555f10c9cfb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c29ffb640 0 client.16413 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555f10c2d910 0x555f10c46850 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f10c1f0b0 0x555f10c45840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 --2- 172.21.15.55:0/347863011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555f10c1dee0 0x555f10c45300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 -- 172.21.15.55:0/347863011 >> 172.21.15.55:0/347863011 conn(0x555f10c193d0 msgr2=0x555f10c27670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 -- 172.21.15.55:0/347863011 shutdown_connections 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.661+0000 7f8c4b618740 1 -- 172.21.15.55:0/347863011 wait complete. 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.662+0000 7f8c4b618740 20 client.16413 trim_cache size 0 max 16384 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T21:59:24.663 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.662+0000 7f8c48903640 1 -- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8c2c0a3dc0 msgr2=0x7f8c2c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.662+0000 7f8c48903640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8c2c0a3dc0 0x7f8c2c0a6280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8c4406c650 tx=0x7f8c3c030700 comp rx=0 tx=0).stop 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.662+0000 7f8c48903640 1 -- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c44109930 msgr2=0x7f8c441192d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.662+0000 7f8c48903640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c44109930 0x7f8c441192d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400e940 tx=0x7f8c3400ee00 comp rx=0 tx=0).stop 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.663+0000 7f8c48903640 1 -- 172.21.15.55:0/1109129322 shutdown_connections 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.663+0000 7f8c48903640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8c2c0a3dc0 0x7f8c2c0a6280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.663+0000 7f8c48903640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c4410ec70 0x7f8c4411a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.663+0000 7f8c48903640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8c4410c2a0 0x7f8c44119810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.663+0000 7f8c48903640 1 --2- 172.21.15.55:0/1109129322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c44109930 0x7f8c441192d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.663+0000 7f8c48903640 1 -- 172.21.15.55:0/1109129322 >> 172.21.15.55:0/1109129322 conn(0x7f8c4406f7c0 msgr2=0x7f8c4410cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.663+0000 7f8c48903640 1 -- 172.21.15.55:0/1109129322 shutdown_connections 2024-04-26T21:59:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:24.663+0000 7f8c48903640 1 -- 172.21.15.55:0/1109129322 wait complete. 2024-04-26T21:59:24.687 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T21:59:24.687 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:59:25.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.051+0000 7fd6e16c4640 1 -- 172.21.15.55:0/2279782023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10fcf0 msgr2=0x7fd6dc1121b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:25.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.051+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/2279782023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10fcf0 0x7fd6dc1121b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009ca0 tx=0x7fd6d402a8d0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.052+0000 7fd6e16c4640 1 -- 172.21.15.55:0/2279782023 shutdown_connections 2024-04-26T21:59:25.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.052+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/2279782023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10fcf0 0x7fd6dc1121b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.052+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/2279782023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc10d370 0x7fd6dc10f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.052+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/2279782023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc10aa40 0x7fd6dc10ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.052+0000 7fd6e16c4640 1 -- 172.21.15.55:0/2279782023 >> 172.21.15.55:0/2279782023 conn(0x7fd6dc06f7d0 msgr2=0x7fd6dc071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:25.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.055+0000 7fd6e16c4640 1 -- 172.21.15.55:0/2279782023 shutdown_connections 2024-04-26T21:59:25.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.058+0000 7fd6e16c4640 1 -- 172.21.15.55:0/2279782023 wait complete. 2024-04-26T21:59:25.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.059+0000 7fd6e16c4640 1 Processor -- start 2024-04-26T21:59:25.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.060+0000 7fd6e16c4640 1 -- start start 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.060+0000 7fd6e16c4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10aa40 0x7fd6dc1a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.060+0000 7fd6e16c4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc10d370 0x7fd6dc1a8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.061+0000 7fd6daffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10aa40 0x7fd6dc1a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.061+0000 7fd6daffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10aa40 0x7fd6dc1a7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49534/0 (socket says 172.21.15.55:49534) 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.061+0000 7fd6e16c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc10fcf0 0x7fd6dc1aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.061+0000 7fd6e16c4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc114b10 con 0x7fd6dc10aa40 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.061+0000 7fd6e16c4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc114990 con 0x7fd6dc10fcf0 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.061+0000 7fd6e16c4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6dc114c90 con 0x7fd6dc10d370 2024-04-26T21:59:25.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.061+0000 7fd6daffd640 1 -- 172.21.15.55:0/3151016570 learned_addr learned my addr 172.21.15.55:0/3151016570 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:25.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.062+0000 7fd6daffd640 1 -- 172.21.15.55:0/3151016570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc10d370 msgr2=0x7fd6dc1a8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:25.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.062+0000 7fd6daffd640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc10d370 0x7fd6dc1a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.062+0000 7fd6daffd640 1 -- 172.21.15.55:0/3151016570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc10fcf0 msgr2=0x7fd6dc1aefe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:59:25.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.062+0000 7fd6daffd640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc10fcf0 0x7fd6dc1aefe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.062+0000 7fd6daffd640 1 -- 172.21.15.55:0/3151016570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6dc1af680 con 0x7fd6dc10aa40 2024-04-26T21:59:25.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.063+0000 7fd6daffd640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10aa40 0x7fd6dc1a7ad0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b470 tx=0x7fd6cc00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:25.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.064+0000 7fd6c3fff640 1 -- 172.21.15.55:0/3151016570 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc015020 con 0x7fd6dc10aa40 2024-04-26T21:59:25.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.064+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6dc1af970 con 0x7fd6dc10aa40 2024-04-26T21:59:25.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.064+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6dc1aff90 con 0x7fd6dc10aa40 2024-04-26T21:59:25.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.064+0000 7fd6c3fff640 1 -- 172.21.15.55:0/3151016570 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6cc011d40 con 0x7fd6dc10aa40 2024-04-26T21:59:25.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.064+0000 7fd6c3fff640 1 -- 172.21.15.55:0/3151016570 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc01ad70 con 0x7fd6dc10aa40 2024-04-26T21:59:25.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.066+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c8005290 con 0x7fd6dc10aa40 2024-04-26T21:59:25.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.068+0000 7fd6c3fff640 1 -- 172.21.15.55:0/3151016570 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd6cc018020 con 0x7fd6dc10aa40 2024-04-26T21:59:25.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.069+0000 7fd6c3fff640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d00 0x7fd6c40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:25.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.069+0000 7fd6da7fc640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d00 0x7fd6c40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:25.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.070+0000 7fd6c3fff640 1 -- 172.21.15.55:0/3151016570 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd6cc0d04d0 con 0x7fd6dc10aa40 2024-04-26T21:59:25.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.071+0000 7fd6da7fc640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d00 0x7fd6c40a61c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000a8f0 tx=0x7fd6d0005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:25.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.078+0000 7fd6c3fff640 1 -- 172.21.15.55:0/3151016570 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd6cc01f020 con 0x7fd6dc10aa40 2024-04-26T21:59:25.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:25 smithi055 ceph-mon[29221]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 830 MiB/s wr, 1.11k op/s 2024-04-26T21:59:25.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:25 smithi064 ceph-mon[32765]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 830 MiB/s wr, 1.11k op/s 2024-04-26T21:59:25.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.805+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6c8005580 con 0x7fd6dc10aa40 2024-04-26T21:59:25.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.815+0000 7fd6c3fff640 1 -- 172.21.15.55:0/3151016570 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fd6cc00d040 con 0x7fd6dc10aa40 2024-04-26T21:59:25.821 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T21:59:25.821 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T21:59:25.821 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T21:59:25.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:25 smithi165 ceph-mon[33296]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 830 MiB/s wr, 1.11k op/s 2024-04-26T21:59:25.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.829+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d00 msgr2=0x7fd6c40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:25.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.829+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d00 0x7fd6c40a61c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000a8f0 tx=0x7fd6d0005bf0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.830+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10aa40 msgr2=0x7fd6dc1a7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:25.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.830+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10aa40 0x7fd6dc1a7ad0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b470 tx=0x7fd6cc00b930 comp rx=0 tx=0).stop 2024-04-26T21:59:25.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.836+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 shutdown_connections 2024-04-26T21:59:25.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.836+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd6c40a3d00 0x7fd6c40a61c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.836+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6dc10fcf0 0x7fd6dc1aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.836+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6dc10d370 0x7fd6dc1a8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.836+0000 7fd6e16c4640 1 --2- 172.21.15.55:0/3151016570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd6dc10aa40 0x7fd6dc1a7ad0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:25.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.836+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 >> 172.21.15.55:0/3151016570 conn(0x7fd6dc06f7d0 msgr2=0x7fd6dc10ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:25.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.839+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 shutdown_connections 2024-04-26T21:59:25.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:25.840+0000 7fd6e16c4640 1 -- 172.21.15.55:0/3151016570 wait complete. 2024-04-26T21:59:25.897 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T21:59:26.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.277+0000 7f7da547b640 1 -- 172.21.15.55:0/4208599890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 msgr2=0x7f7da010f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.277+0000 7f7da547b640 1 --2- 172.21.15.55:0/4208599890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 0x7f7da010f120 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009ca0 tx=0x7f7d9802a8e0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.278+0000 7f7da547b640 1 -- 172.21.15.55:0/4208599890 shutdown_connections 2024-04-26T21:59:26.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.278+0000 7f7da547b640 1 --2- 172.21.15.55:0/4208599890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 0x7f7da010f120 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.278+0000 7f7da547b640 1 --2- 172.21.15.55:0/4208599890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da010c2a0 0x7f7da010e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.278+0000 7f7da547b640 1 --2- 172.21.15.55:0/4208599890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7da0109930 0x7f7da010bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.278+0000 7f7da547b640 1 -- 172.21.15.55:0/4208599890 >> 172.21.15.55:0/4208599890 conn(0x7f7da006f7c0 msgr2=0x7f7da0071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.279+0000 7f7da547b640 1 -- 172.21.15.55:0/4208599890 shutdown_connections 2024-04-26T21:59:26.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.280+0000 7f7da547b640 1 -- 172.21.15.55:0/4208599890 wait complete. 2024-04-26T21:59:26.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.281+0000 7f7da547b640 1 Processor -- start 2024-04-26T21:59:26.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.281+0000 7f7da547b640 1 -- start start 2024-04-26T21:59:26.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.281+0000 7f7da547b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da0109930 0x7f7da0116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.282+0000 7f7da547b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7da010c2a0 0x7f7da01165c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.282+0000 7f7da547b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 0x7f7da01b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.282+0000 7f7da547b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7da011d370 con 0x7f7da010ec70 2024-04-26T21:59:26.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.282+0000 7f7da547b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7da011d1f0 con 0x7f7da0109930 2024-04-26T21:59:26.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.282+0000 7f7da547b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7da011d4f0 con 0x7f7da010c2a0 2024-04-26T21:59:26.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.282+0000 7f7d9f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 0x7f7da01b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.282+0000 7f7d9f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 0x7f7da01b7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49542/0 (socket says 172.21.15.55:49542) 2024-04-26T21:59:26.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.282+0000 7f7d9f7fe640 1 -- 172.21.15.55:0/1158248670 learned_addr learned my addr 172.21.15.55:0/1158248670 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.283+0000 7f7d9e7fc640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7da010c2a0 0x7f7da01165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.284+0000 7f7d9f7fe640 1 -- 172.21.15.55:0/1158248670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7da010c2a0 msgr2=0x7f7da01165c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.284+0000 7f7d9f7fe640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7da010c2a0 0x7f7da01165c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.284+0000 7f7d9f7fe640 1 -- 172.21.15.55:0/1158248670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da0109930 msgr2=0x7f7da0116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.284+0000 7f7d9f7fe640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da0109930 0x7f7da0116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.284+0000 7f7d9f7fe640 1 -- 172.21.15.55:0/1158248670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da01b7fc0 con 0x7f7da010ec70 2024-04-26T21:59:26.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.285+0000 7f7d9f7fe640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 0x7f7da01b7a80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7d980098d0 tx=0x7f7d98003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.285+0000 7f7d87fff640 1 -- 172.21.15.55:0/1158248670 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98004040 con 0x7f7da010ec70 2024-04-26T21:59:26.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.285+0000 7f7da547b640 1 -- 172.21.15.55:0/1158248670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da01b8190 con 0x7f7da010ec70 2024-04-26T21:59:26.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.285+0000 7f7d87fff640 1 -- 172.21.15.55:0/1158248670 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d98004690 con 0x7f7da010ec70 2024-04-26T21:59:26.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.285+0000 7f7da547b640 1 -- 172.21.15.55:0/1158248670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da01b8680 con 0x7f7da010ec70 2024-04-26T21:59:26.287 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.286+0000 7f7d87fff640 1 -- 172.21.15.55:0/1158248670 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9803a400 con 0x7f7da010ec70 2024-04-26T21:59:26.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.289+0000 7f7da8192740 1 Processor -- start 2024-04-26T21:59:26.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.291+0000 7f7d87fff640 1 -- 172.21.15.55:0/1158248670 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7d980305e0 con 0x7f7da010ec70 2024-04-26T21:59:26.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.291+0000 7f7da8192740 1 -- start start 2024-04-26T21:59:26.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7da8192740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7da8192740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7da8192740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7da8192740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55aae2030d80 con 0x55aae20cc820 2024-04-26T21:59:26.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7da8192740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aae1feaa20 con 0x55aae20ce3b0 2024-04-26T21:59:26.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7da8192740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55aae202cd40 con 0x55aae20cd9f0 2024-04-26T21:59:26.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7d9e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7d9e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49550/0 (socket says 172.21.15.55:49550) 2024-04-26T21:59:26.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7d9e7fc640 1 -- 172.21.15.55:0/2079731983 learned_addr learned my addr 172.21.15.55:0/2079731983 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7d9e7fc640 1 -- 172.21.15.55:0/2079731983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 msgr2=0x55aae20cde70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:26.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7d9e7fc640 1 --2- 172.21.15.55:0/2079731983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20cde70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.292+0000 7f7d9e7fc640 1 -- 172.21.15.55:0/2079731983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 msgr2=0x55aae20d4c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:26.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.293+0000 7f7d9e7fc640 1 --2- 172.21.15.55:0/2079731983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20d4c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.293+0000 7f7d9e7fc640 1 -- 172.21.15.55:0/2079731983 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae1ff5c30 con 0x55aae20cc820 2024-04-26T21:59:26.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.293+0000 7f7d9e7fc640 1 --2- 172.21.15.55:0/2079731983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20ccc40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400bca0 tx=0x7f7d9400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935e2e244ad866c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.294+0000 7f7d857fa640 1 -- 172.21.15.55:0/2079731983 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94015070 con 0x55aae20cc820 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.294+0000 7f7d857fa640 1 -- 172.21.15.55:0/2079731983 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9401a910 con 0x55aae20cc820 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.294+0000 7f7d857fa640 1 -- 172.21.15.55:0/2079731983 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940118d0 con 0x55aae20cc820 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.295+0000 7f7d87fff640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7d880a3d40 0x7f7d880a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.295+0000 7f7da8192740 1 -- 172.21.15.55:0/2079731983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 msgr2=0x55aae20ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.295+0000 7f7da8192740 1 --2- 172.21.15.55:0/2079731983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20ccc40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400bca0 tx=0x7f7d9400f730 comp rx=0 tx=0).stop 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.295+0000 7f7da8192740 1 -- 172.21.15.55:0/2079731983 shutdown_connections 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.295+0000 7f7da8192740 1 --2- 172.21.15.55:0/2079731983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20d4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.295+0000 7f7da8192740 1 --2- 172.21.15.55:0/2079731983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.295+0000 7f7da8192740 1 --2- 172.21.15.55:0/2079731983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20ccc40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.295+0000 7f7da8192740 1 -- 172.21.15.55:0/2079731983 >> 172.21.15.55:0/2079731983 conn(0x55aae200d910 msgr2=0x55aae20c8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.296+0000 7f7da8192740 1 -- 172.21.15.55:0/2079731983 shutdown_connections 2024-04-26T21:59:26.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.296+0000 7f7d9effd640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7d880a3d40 0x7f7d880a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.296+0000 7f7da8192740 1 -- 172.21.15.55:0/2079731983 wait complete. 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.296+0000 7f7da8192740 1 -- start start 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.296+0000 7f7da8192740 1 Processor -- start 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7da8192740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20cc820 0x55aae20d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7da8192740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7da8192740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20ce3b0 0x55aae20de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7da8192740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55aae20d53c0 con 0x55aae20ce3b0 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7da8192740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aae1fb7dc0 con 0x55aae20cc820 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7da8192740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55aae20d5540 con 0x55aae20cd9f0 2024-04-26T21:59:26.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7da8192740 10 client.0 _ll_register_callbacks cb 0x55aae1fd3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7d9f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20ce3b0 0x55aae20de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7d9f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20ce3b0 0x55aae20de0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49556/0 (socket says 172.21.15.55:49556) 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.297+0000 7f7d9effd640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7d880a3d40 0x7f7d880a6200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55aae20c8fd0 tx=0x7f7d90006be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.298+0000 7f7d9f7fe640 1 -- 172.21.15.55:0/3875763703 learned_addr learned my addr 172.21.15.55:0/3875763703 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.298+0000 7f7d87fff640 1 -- 172.21.15.55:0/1158248670 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7d980f1ae0 con 0x7f7da010ec70 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.298+0000 7f7d9f7fe640 1 -- 172.21.15.55:0/3875763703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 msgr2=0x55aae20d7070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.298+0000 7f7d9f7fe640 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20d7070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.298+0000 7f7d9f7fe640 1 -- 172.21.15.55:0/3875763703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20cc820 msgr2=0x55aae20d6b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.298+0000 7f7d9f7fe640 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20cc820 0x55aae20d6b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.305 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.298+0000 7f7d9f7fe640 1 -- 172.21.15.55:0/3875763703 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae1d423f0 con 0x55aae20ce3b0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.300+0000 7f7d82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.300+0000 7f7d9f7fe640 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20ce3b0 0x55aae20de0b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d980c6410 tx=0x7f7d980099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa293a2768b0d8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.300+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3875763703 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d980bd630 con 0x55aae20ce3b0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.300+0000 7f7da8192740 1 -- 172.21.15.55:0/3875763703 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d9401a910 con 0x55aae20ce3b0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.300+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3875763703 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7d980bbfd0 con 0x55aae20ce3b0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.301+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3875763703 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7d98040280 con 0x55aae20ce3b0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.301+0000 7f7da8192740 10 client.16431 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.301+0000 7f7da8192740 1 -- 172.21.15.55:0/3875763703 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55aae1e0c7d0 con 0x55aae20ce3b0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.301+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3875763703 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d98036030 con 0x55aae20ce3b0 2024-04-26T21:59:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.301+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3875763703 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7d98041d50 con 0x55aae20ce3b0 2024-04-26T21:59:26.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.301+0000 7f7da8192740 10 client.16431 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:26.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.302+0000 7f7da8192740 10 client.16431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:59:26.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.302+0000 7f7da8192740 20 client.16431 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:26.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.302+0000 7f7da8192740 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55aae20e2990 0x55aae20e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.302+0000 7f7da8192740 4 client.16431 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:59:26.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.302+0000 7f7da8192740 1 -- 172.21.15.55:0/3875763703 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7d9401a910 con 0x55aae20e2990 2024-04-26T21:59:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.303+0000 7f7d9effd640 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55aae20e2990 0x55aae20e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.308+0000 7f7d82ff5640 10 client.16431 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:59:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.308+0000 7f7d9effd640 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55aae20e2990 0x55aae20e4e10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbc049d5cb60f9b3 server_cookie=6ccbf782d29f4cb6 in_seq=0 out_seq=0 2024-04-26T21:59:26.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.310+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3875763703 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f7d9401a910 con 0x55aae20e2990 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.310+0000 7f7d82ff5640 10 client.16431 handle_command_reply: tid=0 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.310+0000 7f7da8192740 1 client.16431 shutdown 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.310+0000 7f7da8192740 1 -- 172.21.15.55:0/3875763703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20ce3b0 msgr2=0x55aae20de0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.310+0000 7f7da8192740 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20ce3b0 0x55aae20de0b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d980c6410 tx=0x7f7d980099c0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 1 -- 172.21.15.55:0/3875763703 shutdown_connections 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55aae20e2990 0x55aae20e4e10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7d82ff5640 0 client.16431 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20ce3b0 0x55aae20de0b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20d7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 1 --2- 172.21.15.55:0/3875763703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20cc820 0x55aae20d6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 1 -- 172.21.15.55:0/3875763703 >> 172.21.15.55:0/3875763703 conn(0x55aae200d910 msgr2=0x55aae20c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 1 -- 172.21.15.55:0/3875763703 shutdown_connections 2024-04-26T21:59:26.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 1 -- 172.21.15.55:0/3875763703 wait complete. 2024-04-26T21:59:26.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.311+0000 7f7da8192740 20 client.16431 trim_cache size 0 max 16384 2024-04-26T21:59:26.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.341+0000 7f7da8192740 1 Processor -- start 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.341+0000 7f7da8192740 1 -- start start 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7da8192740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20cb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7da8192740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7da8192740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7da8192740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01c3cd0 con 0x55aae20cc820 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7da8192740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01c15a0 con 0x55aae20ce3b0 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7d9effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7da8192740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01bf3f0 con 0x55aae20cd9f0 2024-04-26T21:59:26.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7d9effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49562/0 (socket says 172.21.15.55:49562) 2024-04-26T21:59:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7d9effd640 1 -- 172.21.15.55:0/2674629735 learned_addr learned my addr 172.21.15.55:0/2674629735 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7d9effd640 1 -- 172.21.15.55:0/2674629735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 msgr2=0x55aae20cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7d9effd640 1 --2- 172.21.15.55:0/2674629735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7d9effd640 1 -- 172.21.15.55:0/2674629735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 msgr2=0x55aae20de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7d9effd640 1 --2- 172.21.15.55:0/2674629735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.342+0000 7f7d9effd640 1 -- 172.21.15.55:0/2674629735 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae20c7f30 con 0x55aae20cc820 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.343+0000 7f7d9effd640 1 --2- 172.21.15.55:0/2674629735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20cb720 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7d90013960 tx=0x7f7d90012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad9bd1feeb8caea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.343+0000 7f7d84ff9640 1 -- 172.21.15.55:0/2674629735 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90025030 con 0x55aae20cc820 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.343+0000 7f7d84ff9640 1 -- 172.21.15.55:0/2674629735 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d90012d40 con 0x55aae20cc820 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.343+0000 7f7d84ff9640 1 -- 172.21.15.55:0/2674629735 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90020400 con 0x55aae20cc820 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.344+0000 7f7da8192740 1 -- 172.21.15.55:0/2674629735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 msgr2=0x55aae20cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.344+0000 7f7da8192740 1 --2- 172.21.15.55:0/2674629735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20cb720 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7d90013960 tx=0x7f7d90012000 comp rx=0 tx=0).stop 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.344+0000 7f7da8192740 1 -- 172.21.15.55:0/2674629735 shutdown_connections 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.344+0000 7f7da8192740 1 --2- 172.21.15.55:0/2674629735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.344+0000 7f7da8192740 1 --2- 172.21.15.55:0/2674629735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cd9f0 0x55aae20cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.344+0000 7f7da8192740 1 --2- 172.21.15.55:0/2674629735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cc820 0x55aae20cb720 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.344+0000 7f7da8192740 1 -- 172.21.15.55:0/2674629735 >> 172.21.15.55:0/2674629735 conn(0x55aae200d910 msgr2=0x55aae20cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.344+0000 7f7da8192740 1 -- 172.21.15.55:0/2674629735 shutdown_connections 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 -- 172.21.15.55:0/2674629735 wait complete. 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 -- start start 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 Processor -- start 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cc820 0x55aae20dbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cd9f0 0x55aae20dc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01bfa60 con 0x55aae20cd9f0 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01c0ed0 con 0x55aae20ce3b0 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.345+0000 7f7da8192740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7da01c0130 con 0x55aae20cc820 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7da8192740 10 client.0 _ll_register_callbacks cb 0x7f7d980308d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cd9f0 0x55aae20dc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cd9f0 0x55aae20dc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49570/0 (socket says 172.21.15.55:49570) 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 -- 172.21.15.55:0/3007068470 learned_addr learned my addr 172.21.15.55:0/3007068470 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9effd640 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cc820 0x55aae20dbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 -- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cc820 msgr2=0x55aae20dbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cc820 0x55aae20dbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 -- 172.21.15.55:0/3007068470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 msgr2=0x55aae20dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 -- 172.21.15.55:0/3007068470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae20dd900 con 0x55aae20cd9f0 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9effd640 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cc820 0x55aae20dbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d9e7fc640 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cd9f0 0x55aae20dc370 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7d94005d00 tx=0x7f7d9400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e42cabecd4aaa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.346+0000 7f7d82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.347+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3007068470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94015070 con 0x55aae20cd9f0 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.347+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3007068470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7d94005550 con 0x55aae20cd9f0 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.347+0000 7f7da8192740 1 -- 172.21.15.55:0/3007068470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d90012d40 con 0x55aae20cd9f0 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.347+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3007068470 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7d94004640 con 0x55aae20cd9f0 2024-04-26T21:59:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.347+0000 7f7da8192740 10 client.16437 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:26.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.347+0000 7f7da8192740 1 -- 172.21.15.55:0/3007068470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55aae2149560 con 0x55aae20cd9f0 2024-04-26T21:59:26.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.348+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3007068470 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d90012d40 con 0x55aae20cd9f0 2024-04-26T21:59:26.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.348+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3007068470 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7d9401a840 con 0x55aae20cd9f0 2024-04-26T21:59:26.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.348+0000 7f7da8192740 10 client.16437 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.348+0000 7f7da8192740 10 client.16437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T21:59:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.348+0000 7f7da8192740 20 client.16437 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.348+0000 7f7da8192740 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55aae20e2990 0x55aae214bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.348+0000 7f7da8192740 4 client.16437 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:59:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.348+0000 7f7da8192740 1 -- 172.21.15.55:0/3007068470 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aae203e990 con 0x55aae20e2990 2024-04-26T21:59:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.349+0000 7f7d9effd640 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55aae20e2990 0x55aae214bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.352+0000 7f7d82ff5640 10 client.16437 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T21:59:26.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.352+0000 7f7d9effd640 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55aae20e2990 0x55aae214bb60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecb23d8fa109fee8 server_cookie=d5d61a4ddf5cf061 in_seq=0 out_seq=0 2024-04-26T21:59:26.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.353+0000 7f7d82ff5640 1 -- 172.21.15.55:0/3007068470 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aae203e990 con 0x55aae20e2990 2024-04-26T21:59:26.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.353+0000 7f7d82ff5640 10 client.16437 handle_command_reply: tid=0 2024-04-26T21:59:26.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.353+0000 7f7da8192740 1 client.16437 shutdown 2024-04-26T21:59:26.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7da8192740 1 -- 172.21.15.55:0/3007068470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cd9f0 msgr2=0x55aae20dc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7da8192740 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cd9f0 0x55aae20dc370 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7d94005d00 tx=0x7f7d9400bfa0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7da8192740 1 -- 172.21.15.55:0/3007068470 shutdown_connections 2024-04-26T21:59:26.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7da8192740 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55aae20e2990 0x55aae214bb60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7d82ff5640 0 client.16437 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T21:59:26.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7da8192740 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aae20ce3b0 0x55aae20dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7da8192740 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55aae20cd9f0 0x55aae20dc370 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7da8192740 1 --2- 172.21.15.55:0/3007068470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aae20cc820 0x55aae20dbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.354+0000 7f7da8192740 1 -- 172.21.15.55:0/3007068470 >> 172.21.15.55:0/3007068470 conn(0x55aae200d910 msgr2=0x55aae20cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.355+0000 7f7da8192740 1 -- 172.21.15.55:0/3007068470 shutdown_connections 2024-04-26T21:59:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.355+0000 7f7da8192740 1 -- 172.21.15.55:0/3007068470 wait complete. 2024-04-26T21:59:26.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.355+0000 7f7da8192740 20 client.16437 trim_cache size 0 max 16384 2024-04-26T21:59:26.356 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:59:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 -- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7d880a3d40 msgr2=0x7f7d880a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7d880a3d40 0x7f7d880a6200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55aae20c8fd0 tx=0x7f7d90006be0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 -- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 msgr2=0x7f7da01b7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 0x7f7da01b7a80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7d980098d0 tx=0x7f7d98003ba0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 -- 172.21.15.55:0/1158248670 shutdown_connections 2024-04-26T21:59:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7d880a3d40 0x7f7d880a6200 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7da010ec70 0x7f7da01b7a80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7da010c2a0 0x7f7da01165c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 --2- 172.21.15.55:0/1158248670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7da0109930 0x7f7da0116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 -- 172.21.15.55:0/1158248670 >> 172.21.15.55:0/1158248670 conn(0x7f7da006f7c0 msgr2=0x7f7da010af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 -- 172.21.15.55:0/1158248670 shutdown_connections 2024-04-26T21:59:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.357+0000 7f7da547b640 1 -- 172.21.15.55:0/1158248670 wait complete. 2024-04-26T21:59:26.398 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T21:59:26.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:26 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3151016570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:26.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:26 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3151016570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:26.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 -- 172.21.15.55:0/2043262581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 msgr2=0x7fbbf0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 --2- 172.21.15.55:0/2043262581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 0x7fbbf0102ee0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc009900 tx=0x7fbbdc02a410 comp rx=0 tx=0).stop 2024-04-26T21:59:26.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 -- 172.21.15.55:0/2043262581 shutdown_connections 2024-04-26T21:59:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 --2- 172.21.15.55:0/2043262581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf0103420 0x7fbbf0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 --2- 172.21.15.55:0/2043262581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 0x7fbbf0102ee0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 --2- 172.21.15.55:0/2043262581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf0101870 0x7fbbf0101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 -- 172.21.15.55:0/2043262581 >> 172.21.15.55:0/2043262581 conn(0x7fbbf00fcde0 msgr2=0x7fbbf00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 -- 172.21.15.55:0/2043262581 shutdown_connections 2024-04-26T21:59:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.823+0000 7fbbf716a640 1 -- 172.21.15.55:0/2043262581 wait complete. 2024-04-26T21:59:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.824+0000 7fbbf716a640 1 Processor -- start 2024-04-26T21:59:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.824+0000 7fbbf716a640 1 -- start start 2024-04-26T21:59:26.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.825+0000 7fbbf716a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf0101870 0x7fbbf0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.825+0000 7fbbf716a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 0x7fbbf0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.825+0000 7fbbf716a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf0103420 0x7fbbf01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.825+0000 7fbbf716a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf010d460 con 0x7fbbf0102a60 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.825+0000 7fbbf716a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf010d2e0 con 0x7fbbf0101870 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.826+0000 7fbbf716a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf010d5e0 con 0x7fbbf0103420 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.826+0000 7fbbeffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 0x7fbbf0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.826+0000 7fbbeffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 0x7fbbf0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49580/0 (socket says 172.21.15.55:49580) 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.826+0000 7fbbeffff640 1 -- 172.21.15.55:0/1146545040 learned_addr learned my addr 172.21.15.55:0/1146545040 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.826+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf0101870 0x7fbbf0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:26 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3151016570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.827+0000 7fbbeffff640 1 -- 172.21.15.55:0/1146545040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf0103420 msgr2=0x7fbbf01a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.827+0000 7fbbeffff640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf0103420 0x7fbbf01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.827+0000 7fbbeffff640 1 -- 172.21.15.55:0/1146545040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf0101870 msgr2=0x7fbbf0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.827+0000 7fbbeffff640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf0101870 0x7fbbf0198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.827+0000 7fbbeffff640 1 -- 172.21.15.55:0/1146545040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf01a0830 con 0x7fbbf0102a60 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.827+0000 7fbbeffff640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 0x7fbbf0199160 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc031500 tx=0x7fbbdc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.827+0000 7fbbedffb640 1 -- 172.21.15.55:0/1146545040 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc03f070 con 0x7fbbf0102a60 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.827+0000 7fbbf716a640 1 -- 172.21.15.55:0/1146545040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf01a0ac0 con 0x7fbbf0102a60 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.828+0000 7fbbedffb640 1 -- 172.21.15.55:0/1146545040 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc03b460 con 0x7fbbf0102a60 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.828+0000 7fbbedffb640 1 -- 172.21.15.55:0/1146545040 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc035030 con 0x7fbbf0102a60 2024-04-26T21:59:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.828+0000 7fbbf716a640 1 -- 172.21.15.55:0/1146545040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf01a1080 con 0x7fbbf0102a60 2024-04-26T21:59:26.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.831+0000 7fbbedffb640 1 -- 172.21.15.55:0/1146545040 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbbdc042070 con 0x7fbbf0102a60 2024-04-26T21:59:26.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbedffb640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbbe00a3c40 0x7fbbe00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf9e82740 1 Processor -- start 2024-04-26T21:59:26.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf9e82740 1 -- start start 2024-04-26T21:59:26.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbedffb640 1 -- 172.21.15.55:0/1146545040 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbbdc0f0ff0 con 0x7fbbf0102a60 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff6fc30 0x55fccff70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff70e00 0x55fccff71280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff717c0 0x55fccff78050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fccfecfd80 con 0x55fccff717c0 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fccfe89a20 con 0x55fccff6fc30 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbbe00a3c40 0x7fbbe00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.833+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fccfecbd40 con 0x55fccff70e00 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.834+0000 7fbbf5707640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff717c0 0x55fccff78050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.834+0000 7fbbf5707640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff717c0 0x55fccff78050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49584/0 (socket says 172.21.15.55:49584) 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.834+0000 7fbbf5707640 1 -- 172.21.15.55:0/2478622123 learned_addr learned my addr 172.21.15.55:0/2478622123 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.834+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbbe00a3c40 0x7fbbe00a6100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbbf019a0f0 tx=0x7fbbe4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.834+0000 7fbbf5707640 1 -- 172.21.15.55:0/2478622123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff70e00 msgr2=0x55fccff71280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.834+0000 7fbbf5707640 1 --2- 172.21.15.55:0/2478622123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff70e00 0x55fccff71280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.834+0000 7fbbf5707640 1 -- 172.21.15.55:0/2478622123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff6fc30 msgr2=0x55fccff70050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:26.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.834+0000 7fbbf5707640 1 --2- 172.21.15.55:0/2478622123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff6fc30 0x55fccff70050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.835+0000 7fbbf5707640 1 -- 172.21.15.55:0/2478622123 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fccfe94c30 con 0x55fccff717c0 2024-04-26T21:59:26.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.835+0000 7fbbf5707640 1 --2- 172.21.15.55:0/2478622123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff717c0 0x55fccff78050 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8007cf0 tx=0x7fbbe800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f372a21208161cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.835+0000 7fbbdaffd640 1 -- 172.21.15.55:0/2478622123 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8014070 con 0x55fccff717c0 2024-04-26T21:59:26.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.835+0000 7fbbdaffd640 1 -- 172.21.15.55:0/2478622123 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe800fb00 con 0x55fccff717c0 2024-04-26T21:59:26.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.835+0000 7fbbdaffd640 1 -- 172.21.15.55:0/2478622123 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800ead0 con 0x55fccff717c0 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.836+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2478622123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff717c0 msgr2=0x55fccff78050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.837+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2478622123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff717c0 0x55fccff78050 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8007cf0 tx=0x7fbbe800c6a0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.837+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2478622123 shutdown_connections 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.837+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2478622123 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff717c0 0x55fccff78050 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.837+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2478622123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff70e00 0x55fccff71280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.837+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2478622123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff6fc30 0x55fccff70050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.837+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2478622123 >> 172.21.15.55:0/2478622123 conn(0x55fccfeac910 msgr2=0x55fccff67120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.837+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2478622123 shutdown_connections 2024-04-26T21:59:26.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.838+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2478622123 wait complete. 2024-04-26T21:59:26.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.838+0000 7fbbf9e82740 1 -- start start 2024-04-26T21:59:26.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.838+0000 7fbbf9e82740 1 Processor -- start 2024-04-26T21:59:26.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff79ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff70e00 0x55fccff7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff81460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fccff787d0 con 0x55fccff6fc30 2024-04-26T21:59:26.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fccfe56dc0 con 0x55fccff70e00 2024-04-26T21:59:26.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fccff78950 con 0x55fccff717c0 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf9e82740 10 client.0 _ll_register_callbacks cb 0x55fccfe72600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf4f06640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff79ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf4f06640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff79ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49586/0 (socket says 172.21.15.55:49586) 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf4f06640 1 -- 172.21.15.55:0/2574739033 learned_addr learned my addr 172.21.15.55:0/2574739033 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf4f06640 1 -- 172.21.15.55:0/2574739033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 msgr2=0x55fccff81460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf5707640 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff81460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff81460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf4f06640 1 -- 172.21.15.55:0/2574739033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff70e00 msgr2=0x55fccff7a420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff70e00 0x55fccff7a420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.839+0000 7fbbf4f06640 1 -- 172.21.15.55:0/2574739033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fccfbe13f0 con 0x55fccff6fc30 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.840+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff79ee0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400ce20 tx=0x7fbbe40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd1950d71dc6749 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.840+0000 7fbbcffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:26.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.840+0000 7fbbf5707640 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff81460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.840+0000 7fbbcffff640 1 -- 172.21.15.55:0/2574739033 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4023060 con 0x55fccff6fc30 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.840+0000 7fbbcffff640 1 -- 172.21.15.55:0/2574739033 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbbe401e600 con 0x55fccff6fc30 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2574739033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe800fb00 con 0x55fccff6fc30 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbcffff640 1 -- 172.21.15.55:0/2574739033 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbbe401e800 con 0x55fccff6fc30 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbf9e82740 10 client.16449 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2574739033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fccfcab7d0 con 0x55fccff6fc30 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbcffff640 1 -- 172.21.15.55:0/2574739033 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe800fb00 con 0x55fccff6fc30 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbcffff640 1 -- 172.21.15.55:0/2574739033 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fbbe40114d0 con 0x55fccff6fc30 2024-04-26T21:59:26.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbf9e82740 10 client.16449 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:26.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbf9e82740 10 client.16449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:59:26.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbf9e82740 20 client.16449 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:26.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.841+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fccff85e20 0x55fccff882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.842+0000 7fbbf9e82740 4 client.16449 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:59:26.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.842+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2574739033 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fccfedd990 con 0x55fccff85e20 2024-04-26T21:59:26.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.842+0000 7fbbeffff640 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fccff85e20 0x55fccff882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.845+0000 7fbbeffff640 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fccff85e20 0x55fccff882e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d56d42b41abacf71 server_cookie=9942001993237c9e in_seq=0 out_seq=0 2024-04-26T21:59:26.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.845+0000 7fbbcffff640 10 client.16449 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:59:26.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.848+0000 7fbbcffff640 1 -- 172.21.15.55:0/2574739033 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fccfedd990 con 0x55fccff85e20 2024-04-26T21:59:26.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.848+0000 7fbbcffff640 10 client.16449 handle_command_reply: tid=0 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.848+0000 7fbbf9e82740 1 client.16449 shutdown 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2574739033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 msgr2=0x55fccff79ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff79ee0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400ce20 tx=0x7fbbe40047d0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2574739033 shutdown_connections 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fccff85e20 0x55fccff882e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff81460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff70e00 0x55fccff7a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/2574739033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff79ee0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2574739033 >> 172.21.15.55:0/2574739033 conn(0x55fccfeac910 msgr2=0x55fccff66280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbcffff640 0 client.16449 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2574739033 shutdown_connections 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 1 -- 172.21.15.55:0/2574739033 wait complete. 2024-04-26T21:59:26.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.849+0000 7fbbf9e82740 20 client.16449 trim_cache size 0 max 16384 2024-04-26T21:59:26.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.885+0000 7fbbf9e82740 1 Processor -- start 2024-04-26T21:59:26.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.895+0000 7fbbf9e82740 1 -- start start 2024-04-26T21:59:26.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.896+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff6fc30 0x55fccff66280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.896+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff70e00 0x55fccff813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.896+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff717c0 0x55fccff8cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.896+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01aa5d0 con 0x55fccff70e00 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.896+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01a7ea0 con 0x55fccff717c0 2024-04-26T21:59:26.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.896+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01a5cf0 con 0x55fccff6fc30 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.900+0000 7fbbf4f06640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff6fc30 0x55fccff66280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.901+0000 7fbbeffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff70e00 0x55fccff813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.901+0000 7fbbf4f06640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff6fc30 0x55fccff66280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:35868/0 (socket says 172.21.15.55:35868) 2024-04-26T21:59:26.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.901+0000 7fbbf4f06640 1 -- 172.21.15.55:0/1735704497 learned_addr learned my addr 172.21.15.55:0/1735704497 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.904+0000 7fbbeffff640 1 -- 172.21.15.55:0/1735704497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff6fc30 msgr2=0x55fccff66280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.904+0000 7fbbeffff640 1 --2- 172.21.15.55:0/1735704497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff6fc30 0x55fccff66280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.904+0000 7fbbeffff640 1 -- 172.21.15.55:0/1735704497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff717c0 msgr2=0x55fccff8cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.904+0000 7fbbeffff640 1 --2- 172.21.15.55:0/1735704497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff717c0 0x55fccff8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.904+0000 7fbbeffff640 1 -- 172.21.15.55:0/1735704497 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fccff70880 con 0x55fccff70e00 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.904+0000 7fbbeffff640 1 --2- 172.21.15.55:0/1735704497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff70e00 0x55fccff813f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc0eea00 tx=0x7fbbdc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433eff092cb89f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.905+0000 7fbbda7fc640 1 -- 172.21.15.55:0/1735704497 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0d5070 con 0x55fccff70e00 2024-04-26T21:59:26.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.905+0000 7fbbda7fc640 1 -- 172.21.15.55:0/1735704497 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc0c1580 con 0x55fccff70e00 2024-04-26T21:59:26.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.906+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1735704497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff70e00 msgr2=0x55fccff813f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.906+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1735704497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff70e00 0x55fccff813f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc0eea00 tx=0x7fbbdc02a410 comp rx=0 tx=0).stop 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.911+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1735704497 shutdown_connections 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.911+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1735704497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff717c0 0x55fccff8cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.911+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1735704497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff70e00 0x55fccff813f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.911+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1735704497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff6fc30 0x55fccff66280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.911+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1735704497 >> 172.21.15.55:0/1735704497 conn(0x55fccfeac910 msgr2=0x55fccff7a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.916+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1735704497 shutdown_connections 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.923+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1735704497 wait complete. 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.923+0000 7fbbf9e82740 1 -- start start 2024-04-26T21:59:26.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.923+0000 7fbbf9e82740 1 Processor -- start 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.924+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff7b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.924+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff70e00 0x55fccff7bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.924+0000 7fbbf9e82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff80590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.924+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01a6360 con 0x55fccff6fc30 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.924+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01a77d0 con 0x55fccff70e00 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.924+0000 7fbbf9e82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01a6a30 con 0x55fccff717c0 2024-04-26T21:59:26.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.924+0000 7fbbf9e82740 10 client.0 _ll_register_callbacks cb 0x7fbbe400a4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:26.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.926+0000 7fbbf4f06640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff7b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.926+0000 7fbbf4f06640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff7b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49600/0 (socket says 172.21.15.55:49600) 2024-04-26T21:59:26.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.926+0000 7fbbf4f06640 1 -- 172.21.15.55:0/1642898620 learned_addr learned my addr 172.21.15.55:0/1642898620 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:26.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.928+0000 7fbbf5707640 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff80590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.929+0000 7fbbf4f06640 1 -- 172.21.15.55:0/1642898620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 msgr2=0x55fccff80590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:26.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.929+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff80590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.929+0000 7fbbf4f06640 1 -- 172.21.15.55:0/1642898620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff70e00 msgr2=0x55fccff7bb50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:26.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.929+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff70e00 0x55fccff7bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.929+0000 7fbbf4f06640 1 -- 172.21.15.55:0/1642898620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fccff80bd0 con 0x55fccff6fc30 2024-04-26T21:59:26.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.932+0000 7fbbf4f06640 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff7b610 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400a140 tx=0x7fbbe401e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a03a14e60992f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:26.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.938+0000 7fbbcffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:26.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.938+0000 7fbbcffff640 1 -- 172.21.15.55:0/1642898620 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe40117a0 con 0x55fccff6fc30 2024-04-26T21:59:26.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.938+0000 7fbbcffff640 1 -- 172.21.15.55:0/1642898620 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbbe40059c0 con 0x55fccff6fc30 2024-04-26T21:59:26.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.939+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1642898620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc0c1580 con 0x55fccff6fc30 2024-04-26T21:59:26.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.951+0000 7fbbcffff640 1 -- 172.21.15.55:0/1642898620 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbbe4005bc0 con 0x55fccff6fc30 2024-04-26T21:59:26.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.951+0000 7fbbf9e82740 10 client.16455 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:26.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.951+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1642898620 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fcd0002200 con 0x55fccff6fc30 2024-04-26T21:59:26.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.958+0000 7fbbcffff640 1 -- 172.21.15.55:0/1642898620 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbdc0c1580 con 0x55fccff6fc30 2024-04-26T21:59:26.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.958+0000 7fbbcffff640 1 -- 172.21.15.55:0/1642898620 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fbbe4006530 con 0x55fccff6fc30 2024-04-26T21:59:26.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.958+0000 7fbbf9e82740 10 client.16455 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.958+0000 7fbbf9e82740 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T21:59:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.958+0000 7fbbf9e82740 20 client.16455 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.958+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fcd0002b80 0x55fcd0004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.958+0000 7fbbf9e82740 4 client.16455 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:59:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.958+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1642898620 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fccfed7800 con 0x55fcd0002b80 2024-04-26T21:59:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.960+0000 7fbbeffff640 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fcd0002b80 0x55fcd0004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:26.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.975+0000 7fbbeffff640 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fcd0002b80 0x55fcd0004fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=376e52797a18bfc7 server_cookie=cccc60bfc899586a in_seq=0 out_seq=0 2024-04-26T21:59:26.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.976+0000 7fbbcffff640 10 client.16455 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T21:59:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.977+0000 7fbbcffff640 1 -- 172.21.15.55:0/1642898620 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fccfed7800 con 0x55fcd0002b80 2024-04-26T21:59:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.977+0000 7fbbcffff640 10 client.16455 handle_command_reply: tid=0 2024-04-26T21:59:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.981+0000 7fbbf9e82740 1 client.16455 shutdown 2024-04-26T21:59:26.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.982+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1642898620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 msgr2=0x55fccff7b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:26.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.982+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff7b610 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400a140 tx=0x7fbbe401e6b0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.983+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1642898620 shutdown_connections 2024-04-26T21:59:26.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.983+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fcd0002b80 0x55fcd0004fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.983+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fccff717c0 0x55fccff80590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.983+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fccff70e00 0x55fccff7bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.983+0000 7fbbf9e82740 1 --2- 172.21.15.55:0/1642898620 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fccff6fc30 0x55fccff7b610 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:26.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.983+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1642898620 >> 172.21.15.55:0/1642898620 conn(0x55fccfeac910 msgr2=0x55fccff66280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:26.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.984+0000 7fbbcffff640 0 client.16455 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T21:59:26.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.991+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1642898620 shutdown_connections 2024-04-26T21:59:26.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.994+0000 7fbbf9e82740 1 -- 172.21.15.55:0/1642898620 wait complete. 2024-04-26T21:59:26.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:26.997+0000 7fbbf9e82740 20 client.16455 trim_cache size 0 max 16384 2024-04-26T21:59:27.007 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:59:27.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.026+0000 7fbbf716a640 1 -- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbbe00a3c40 msgr2=0x7fbbe00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.026+0000 7fbbf716a640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbbe00a3c40 0x7fbbe00a6100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbbf019a0f0 tx=0x7fbbe4008040 comp rx=0 tx=0).stop 2024-04-26T21:59:27.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.026+0000 7fbbf716a640 1 -- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 msgr2=0x7fbbf0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.026+0000 7fbbf716a640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 0x7fbbf0199160 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc031500 tx=0x7fbbdc003d50 comp rx=0 tx=0).stop 2024-04-26T21:59:27.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.029+0000 7fbbf716a640 1 -- 172.21.15.55:0/1146545040 shutdown_connections 2024-04-26T21:59:27.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.029+0000 7fbbf716a640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbbe00a3c40 0x7fbbe00a6100 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.029+0000 7fbbf716a640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbbf0103420 0x7fbbf01a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.029+0000 7fbbf716a640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbbf0102a60 0x7fbbf0199160 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.029+0000 7fbbf716a640 1 --2- 172.21.15.55:0/1146545040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbf0101870 0x7fbbf0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.029+0000 7fbbf716a640 1 -- 172.21.15.55:0/1146545040 >> 172.21.15.55:0/1146545040 conn(0x7fbbf00fcde0 msgr2=0x7fbbf00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:27.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.030+0000 7fbbf716a640 1 -- 172.21.15.55:0/1146545040 shutdown_connections 2024-04-26T21:59:27.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.031+0000 7fbbf716a640 1 -- 172.21.15.55:0/1146545040 wait complete. 2024-04-26T21:59:27.092 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T21:59:27.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.359+0000 7ff5a1ffb640 1 -- 172.21.15.55:0/3044394488 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594031cd0 con 0x7ff5a41086d0 2024-04-26T21:59:27.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.360+0000 7ff5a92b1640 1 -- 172.21.15.55:0/3044394488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 msgr2=0x7ff5a410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.360+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/3044394488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 0x7ff5a410aac0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff594009ca0 tx=0x7ff59402a8e0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.361+0000 7ff5a92b1640 1 -- 172.21.15.55:0/3044394488 shutdown_connections 2024-04-26T21:59:27.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.361+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/3044394488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 0x7ff5a410aac0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.361+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/3044394488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a4105da0 0x7ff5a4108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.361+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/3044394488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a4103470 0x7ff5a4105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.361+0000 7ff5a92b1640 1 -- 172.21.15.55:0/3044394488 >> 172.21.15.55:0/3044394488 conn(0x7ff5a40fce20 msgr2=0x7ff5a40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.361+0000 7ff5a92b1640 1 -- 172.21.15.55:0/3044394488 shutdown_connections 2024-04-26T21:59:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.361+0000 7ff5a92b1640 1 -- 172.21.15.55:0/3044394488 wait complete. 2024-04-26T21:59:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.362+0000 7ff5a92b1640 1 Processor -- start 2024-04-26T21:59:27.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.362+0000 7ff5a92b1640 1 -- start start 2024-04-26T21:59:27.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a92b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a4103470 0x7ff5a419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a92b1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a4105da0 0x7ff5a419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a92b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 0x7ff5a41a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a92b1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41107b0 con 0x7ff5a41086d0 2024-04-26T21:59:27.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a92b1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4110630 con 0x7ff5a4103470 2024-04-26T21:59:27.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a92b1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4110930 con 0x7ff5a4105da0 2024-04-26T21:59:27.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 0x7ff5a41a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a27fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a4105da0 0x7ff5a419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 0x7ff5a41a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49622/0 (socket says 172.21.15.55:49622) 2024-04-26T21:59:27.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.363+0000 7ff5a37fe640 1 -- 172.21.15.55:0/1465366989 learned_addr learned my addr 172.21.15.55:0/1465366989 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:27.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.364+0000 7ff5a37fe640 1 -- 172.21.15.55:0/1465366989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a4105da0 msgr2=0x7ff5a419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.364+0000 7ff5a37fe640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a4105da0 0x7ff5a419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.364+0000 7ff5a37fe640 1 -- 172.21.15.55:0/1465366989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a4103470 msgr2=0x7ff5a419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.364+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a4103470 0x7ff5a419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.364+0000 7ff5a37fe640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a4103470 0x7ff5a419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.364+0000 7ff5a37fe640 1 -- 172.21.15.55:0/1465366989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a41a2ba0 con 0x7ff5a41086d0 2024-04-26T21:59:27.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.364+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a4105da0 0x7ff5a419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:27.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.364+0000 7ff5a37fe640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 0x7ff5a41a2530 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff594009dc0 tx=0x7ff594003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:27.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.365+0000 7ff59bfff640 1 -- 172.21.15.55:0/1465366989 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940042f0 con 0x7ff5a41086d0 2024-04-26T21:59:27.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.365+0000 7ff59bfff640 1 -- 172.21.15.55:0/1465366989 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5940305e0 con 0x7ff5a41086d0 2024-04-26T21:59:27.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.365+0000 7ff59bfff640 1 -- 172.21.15.55:0/1465366989 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59403b640 con 0x7ff5a41086d0 2024-04-26T21:59:27.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.365+0000 7ff5a92b1640 1 -- 172.21.15.55:0/1465366989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a41a2e30 con 0x7ff5a41086d0 2024-04-26T21:59:27.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.366+0000 7ff5a92b1640 1 -- 172.21.15.55:0/1465366989 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a41a31e0 con 0x7ff5a41086d0 2024-04-26T21:59:27.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.371+0000 7ff59bfff640 1 -- 172.21.15.55:0/1465366989 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff594004490 con 0x7ff5a41086d0 2024-04-26T21:59:27.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.373+0000 7ff5abfc5740 1 Processor -- start 2024-04-26T21:59:27.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.373+0000 7ff59bfff640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5900a3d90 0x7ff5900a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.373+0000 7ff5abfc5740 1 -- start start 2024-04-26T21:59:27.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.373+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76c030 0x55f74c76c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 0x55f74c76d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c774450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f74c6c7d80 con 0x55f74c76c030 2024-04-26T21:59:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f74c681a20 con 0x55f74c76d200 2024-04-26T21:59:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f74c6c3d40 con 0x55f74c76dbc0 2024-04-26T21:59:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff59bfff640 1 -- 172.21.15.55:0/1465366989 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff5940f1d80 con 0x7ff5a41086d0 2024-04-26T21:59:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5a37fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c774450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76c030 0x55f74c76c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5a37fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c774450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:35910/0 (socket says 172.21.15.55:35910) 2024-04-26T21:59:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5a37fe640 1 -- 172.21.15.55:0/1073158547 learned_addr learned my addr 172.21.15.55:0/1073158547 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.374+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5900a3d90 0x7ff5900a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.375+0000 7ff5a2ffd640 1 -- 172.21.15.55:0/1073158547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 msgr2=0x55f74c774450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.375+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/1073158547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c774450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.375+0000 7ff5a2ffd640 1 -- 172.21.15.55:0/1073158547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 msgr2=0x55f74c76d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.375+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/1073158547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 0x55f74c76d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.375+0000 7ff5a2ffd640 1 -- 172.21.15.55:0/1073158547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74c68cc30 con 0x55f74c76c030 2024-04-26T21:59:27.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.376+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/1073158547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76c030 0x55f74c76c450 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00aa90 tx=0x7ff59c00af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8915fd12e0de38a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:27.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.389+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5900a3d90 0x7ff5900a6250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff58c006fd0 tx=0x7ff58c00a130 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:27.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.395+0000 7ff5997fa640 1 -- 172.21.15.55:0/1073158547 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c015070 con 0x55f74c76c030 2024-04-26T21:59:27.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.395+0000 7ff5997fa640 1 -- 172.21.15.55:0/1073158547 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c0113f0 con 0x55f74c76c030 2024-04-26T21:59:27.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.395+0000 7ff5997fa640 1 -- 172.21.15.55:0/1073158547 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0185c0 con 0x55f74c76c030 2024-04-26T21:59:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.423+0000 7ff5abfc5740 1 -- 172.21.15.55:0/1073158547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76c030 msgr2=0x55f74c76c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.423+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/1073158547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76c030 0x55f74c76c450 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00aa90 tx=0x7ff59c00af50 comp rx=0 tx=0).stop 2024-04-26T21:59:27.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.424+0000 7ff5abfc5740 1 -- 172.21.15.55:0/1073158547 shutdown_connections 2024-04-26T21:59:27.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.424+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/1073158547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c774450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.424+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/1073158547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 0x55f74c76d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.424+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/1073158547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76c030 0x55f74c76c450 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.424+0000 7ff5abfc5740 1 -- 172.21.15.55:0/1073158547 >> 172.21.15.55:0/1073158547 conn(0x55f74c6a4910 msgr2=0x55f74c75f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:27.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.428+0000 7ff5abfc5740 1 -- 172.21.15.55:0/1073158547 shutdown_connections 2024-04-26T21:59:27.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.431+0000 7ff5abfc5740 1 -- 172.21.15.55:0/1073158547 wait complete. 2024-04-26T21:59:27.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.431+0000 7ff5abfc5740 1 -- start start 2024-04-26T21:59:27.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.432+0000 7ff5abfc5740 1 Processor -- start 2024-04-26T21:59:27.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.432+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76c030 0x55f74c764dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.432+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 0x55f74c765300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.432+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76dbc0 0x55f74c76a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.432+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f74c763600 con 0x55f74c76dbc0 2024-04-26T21:59:27.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.432+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f74c64edc0 con 0x55f74c76d200 2024-04-26T21:59:27.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.432+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f74c763780 con 0x55f74c76c030 2024-04-26T21:59:27.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.432+0000 7ff5abfc5740 10 client.0 _ll_register_callbacks cb 0x55f74c66a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:27.433 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:27 smithi055 ceph-mon[29221]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 859 MiB/s wr, 1.12k op/s 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.433+0000 7ff5a37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76dbc0 0x55f74c76a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.433+0000 7ff5a37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76dbc0 0x55f74c76a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49640/0 (socket says 172.21.15.55:49640) 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.433+0000 7ff5a37fe640 1 -- 172.21.15.55:0/2970080393 learned_addr learned my addr 172.21.15.55:0/2970080393 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:27.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.435+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 0x55f74c765300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.435+0000 7ff5a37fe640 1 -- 172.21.15.55:0/2970080393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76c030 msgr2=0x55f74c764dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:59:27.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.435+0000 7ff5a37fe640 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76c030 0x55f74c764dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.435+0000 7ff5a37fe640 1 -- 172.21.15.55:0/2970080393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 msgr2=0x55f74c765300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:27.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.435+0000 7ff5a37fe640 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 0x55f74c765300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.435+0000 7ff5a37fe640 1 -- 172.21.15.55:0/2970080393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74c3d93f0 con 0x55f74c76dbc0 2024-04-26T21:59:27.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.440+0000 7ff5a37fe640 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76dbc0 0x55f74c76a330 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff594005fa0 tx=0x7ff5940be820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e4f001bb012e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:27.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.449+0000 7ff58a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:27.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.450+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2970080393 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940d7060 con 0x55f74c76dbc0 2024-04-26T21:59:27.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.450+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2970080393 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff594009800 con 0x55f74c76dbc0 2024-04-26T21:59:27.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.450+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2970080393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59c0113f0 con 0x55f74c76dbc0 2024-04-26T21:59:27.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.454+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2970080393 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff5940c6c40 con 0x55f74c76dbc0 2024-04-26T21:59:27.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.456+0000 7ff5abfc5740 10 client.16467 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:27.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.456+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2970080393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f74c4a37d0 con 0x55f74c76dbc0 2024-04-26T21:59:27.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.461+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2970080393 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c0113f0 con 0x55f74c76dbc0 2024-04-26T21:59:27.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.461+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2970080393 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff5940ba6d0 con 0x55f74c76dbc0 2024-04-26T21:59:27.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.461+0000 7ff5abfc5740 10 client.16467 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:27.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.461+0000 7ff5abfc5740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:59:27.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.461+0000 7ff5abfc5740 20 client.16467 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:27.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.461+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f74c79bb80 0x55f74c79e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.461+0000 7ff5abfc5740 4 client.16467 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:59:27.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.461+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2970080393 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f74c6d5990 con 0x55f74c79bb80 2024-04-26T21:59:27.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.473+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f74c79bb80 0x55f74c79e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.487+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f74c79bb80 0x55f74c79e040 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c595a6e2b1c773d server_cookie=eb17d6fba7ea9a7b in_seq=0 out_seq=0 2024-04-26T21:59:27.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.487+0000 7ff58a7fc640 10 client.16467 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:27.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.492+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2970080393 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f74c6d5990 con 0x55f74c79bb80 2024-04-26T21:59:27.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.492+0000 7ff58a7fc640 10 client.16467 handle_command_reply: tid=0 2024-04-26T21:59:27.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.492+0000 7ff5abfc5740 1 client.16467 shutdown 2024-04-26T21:59:27.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2970080393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76dbc0 msgr2=0x55f74c76a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76dbc0 0x55f74c76a330 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff594005fa0 tx=0x7ff5940be820 comp rx=0 tx=0).stop 2024-04-26T21:59:27.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2970080393 shutdown_connections 2024-04-26T21:59:27.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f74c79bb80 0x55f74c79e040 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76dbc0 0x55f74c76a330 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76d200 0x55f74c765300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2970080393 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76c030 0x55f74c764dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2970080393 >> 172.21.15.55:0/2970080393 conn(0x55f74c6a4910 msgr2=0x55f74c75e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:27.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.500+0000 7ff58a7fc640 0 client.16467 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:27.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.501+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2970080393 shutdown_connections 2024-04-26T21:59:27.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.502+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2970080393 wait complete. 2024-04-26T21:59:27.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.503+0000 7ff5abfc5740 20 client.16467 trim_cache size 0 max 16384 2024-04-26T21:59:27.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.553+0000 7ff5abfc5740 1 Processor -- start 2024-04-26T21:59:27.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.553+0000 7ff5abfc5740 1 -- start start 2024-04-26T21:59:27.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.554+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 0x55f74c770ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.554+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.554+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c762010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.555+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40736b0 con 0x55f74c76d200 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.555+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4070f80 con 0x55f74c76c030 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.555+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a406edf0 con 0x55f74c76dbc0 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.555+0000 7ff5a37fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c762010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.555+0000 7ff5a37fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c762010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:35916/0 (socket says 172.21.15.55:35916) 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.555+0000 7ff5a37fe640 1 -- 172.21.15.55:0/3692472298 learned_addr learned my addr 172.21.15.55:0/3692472298 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.556+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/3692472298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.557+0000 7ff5a27fc640 1 -- 172.21.15.55:0/3692472298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 msgr2=0x55f74c762010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.557+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/3692472298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c762010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.557+0000 7ff5a27fc640 1 -- 172.21.15.55:0/3692472298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 msgr2=0x55f74c770ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:27.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.557+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/3692472298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 0x55f74c770ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.557+0000 7ff5a27fc640 1 -- 172.21.15.55:0/3692472298 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f74c76bc30 con 0x55f74c76d200 2024-04-26T21:59:27.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.560+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/3692472298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7710e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x55f74c7662a0 tx=0x7ff58c009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac35d06f513a0c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:27.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.562+0000 7ff598ff9640 1 -- 172.21.15.55:0/3692472298 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58c024070 con 0x55f74c76d200 2024-04-26T21:59:27.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.562+0000 7ff598ff9640 1 -- 172.21.15.55:0/3692472298 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff58c01b040 con 0x55f74c76d200 2024-04-26T21:59:27.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.564+0000 7ff5abfc5740 1 -- 172.21.15.55:0/3692472298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 msgr2=0x55f74c7710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.565+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/3692472298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7710e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x55f74c7662a0 tx=0x7ff58c009000 comp rx=0 tx=0).stop 2024-04-26T21:59:27.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.567+0000 7ff5abfc5740 1 -- 172.21.15.55:0/3692472298 shutdown_connections 2024-04-26T21:59:27.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.567+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/3692472298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c762010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.567+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/3692472298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7710e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.567+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/3692472298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 0x55f74c770ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.567+0000 7ff5abfc5740 1 -- 172.21.15.55:0/3692472298 >> 172.21.15.55:0/3692472298 conn(0x55f74c6a4910 msgr2=0x55f74c764dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:27.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.570+0000 7ff5abfc5740 1 -- 172.21.15.55:0/3692472298 shutdown_connections 2024-04-26T21:59:27.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.570+0000 7ff5abfc5740 1 -- 172.21.15.55:0/3692472298 wait complete. 2024-04-26T21:59:27.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.571+0000 7ff5abfc5740 1 -- start start 2024-04-26T21:59:27.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.572+0000 7ff5abfc5740 1 Processor -- start 2024-04-26T21:59:27.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.573+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 0x55f74c7ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.574+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.574+0000 7ff5abfc5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c7ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.574+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a406f440 con 0x55f74c76d200 2024-04-26T21:59:27.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.574+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a40708b0 con 0x55f74c76c030 2024-04-26T21:59:27.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.574+0000 7ff5abfc5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a406fb10 con 0x55f74c76dbc0 2024-04-26T21:59:27.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.575+0000 7ff5abfc5740 10 client.0 _ll_register_callbacks cb 0x7ff5940b9ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.576+0000 7ff5a27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.576+0000 7ff5a27fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49656/0 (socket says 172.21.15.55:49656) 2024-04-26T21:59:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.576+0000 7ff5a27fc640 1 -- 172.21.15.55:0/2394156945 learned_addr learned my addr 172.21.15.55:0/2394156945 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.577+0000 7ff5a37fe640 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c7ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.577+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 0x55f74c7ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.578+0000 7ff5a2ffd640 1 -- 172.21.15.55:0/2394156945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 msgr2=0x55f74c7ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.578+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c7ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.578+0000 7ff5a2ffd640 1 -- 172.21.15.55:0/2394156945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 msgr2=0x55f74c7ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:27.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.578+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.579+0000 7ff5a2ffd640 1 -- 172.21.15.55:0/2394156945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f74c7aec40 con 0x55f74c76c030 2024-04-26T21:59:27.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.580+0000 7ff5a2ffd640 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 0x55f74c7ad1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff59c004700 tx=0x7ff59c00a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b316c70c6ecc8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:27.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.580+0000 7ff58a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:59:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.581+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2394156945 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c015040 con 0x55f74c76c030 2024-04-26T21:59:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.581+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2394156945 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff59c023bc0 con 0x55f74c76c030 2024-04-26T21:59:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.582+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2394156945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58c01b040 con 0x55f74c76c030 2024-04-26T21:59:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.583+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2394156945 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff59c023de0 con 0x55f74c76c030 2024-04-26T21:59:27.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.587+0000 7ff5abfc5740 10 client.25703 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:27.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.588+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2394156945 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f74c8021d0 con 0x55f74c76c030 2024-04-26T21:59:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.589+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2394156945 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c00c910 con 0x55f74c76c030 2024-04-26T21:59:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.589+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2394156945 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff59c023470 con 0x55f74c76c030 2024-04-26T21:59:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.589+0000 7ff5abfc5740 10 client.25703 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.589+0000 7ff5abfc5740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:59:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.590+0000 7ff5abfc5740 20 client.25703 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.590+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f74c802740 0x55f74c804b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.590+0000 7ff5abfc5740 4 client.25703 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T21:59:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.590+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2394156945 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff58c01b040 con 0x55f74c802740 2024-04-26T21:59:27.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.596+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f74c802740 0x55f74c804b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:27.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.599+0000 7ff5a27fc640 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f74c802740 0x55f74c804b60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f733afccea163d server_cookie=8f0426a276fab0b7 in_seq=0 out_seq=0 2024-04-26T21:59:27.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.599+0000 7ff58a7fc640 10 client.25703 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:27.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.603+0000 7ff58a7fc640 1 -- 172.21.15.55:0/2394156945 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff58c01b040 con 0x55f74c802740 2024-04-26T21:59:27.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.603+0000 7ff58a7fc640 10 client.25703 handle_command_reply: tid=0 2024-04-26T21:59:27.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.603+0000 7ff5abfc5740 1 client.25703 shutdown 2024-04-26T21:59:27.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 msgr2=0x55f74c7ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 0x55f74c7ad1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff59c004700 tx=0x7ff59c00a180 comp rx=0 tx=0).stop 2024-04-26T21:59:27.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2394156945 shutdown_connections 2024-04-26T21:59:27.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f74c76dbc0 0x55f74c7ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f74c802740 0x55f74c804b60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f74c76d200 0x55f74c7ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff5abfc5740 1 --2- 172.21.15.55:0/2394156945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f74c76c030 0x55f74c7ad1a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2394156945 >> 172.21.15.55:0/2394156945 conn(0x55f74c6a4910 msgr2=0x55f74c770ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.605+0000 7ff58a7fc640 0 client.25703 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:27.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.606+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2394156945 shutdown_connections 2024-04-26T21:59:27.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.606+0000 7ff5abfc5740 1 -- 172.21.15.55:0/2394156945 wait complete. 2024-04-26T21:59:27.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.606+0000 7ff5abfc5740 20 client.25703 trim_cache size 0 max 16384 2024-04-26T21:59:27.608 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T21:59:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.613+0000 7ff5a92b1640 1 -- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5900a3d90 msgr2=0x7ff5900a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.613+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5900a3d90 0x7ff5900a6250 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff58c006fd0 tx=0x7ff58c00a130 comp rx=0 tx=0).stop 2024-04-26T21:59:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.613+0000 7ff5a92b1640 1 -- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 msgr2=0x7ff5a41a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:27.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.613+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 0x7ff5a41a2530 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff594009dc0 tx=0x7ff594003f90 comp rx=0 tx=0).stop 2024-04-26T21:59:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.614+0000 7ff5a92b1640 1 -- 172.21.15.55:0/1465366989 shutdown_connections 2024-04-26T21:59:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.614+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff5900a3d90 0x7ff5900a6250 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.614+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5a41086d0 0x7ff5a41a2530 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.614+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5a4105da0 0x7ff5a419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.614+0000 7ff5a92b1640 1 --2- 172.21.15.55:0/1465366989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5a4103470 0x7ff5a419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.614+0000 7ff5a92b1640 1 -- 172.21.15.55:0/1465366989 >> 172.21.15.55:0/1465366989 conn(0x7ff5a40fce20 msgr2=0x7ff5a4103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:27.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.615+0000 7ff5a92b1640 1 -- 172.21.15.55:0/1465366989 shutdown_connections 2024-04-26T21:59:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:27.618+0000 7ff5a92b1640 1 -- 172.21.15.55:0/1465366989 wait complete. 2024-04-26T21:59:27.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:27 smithi064 ceph-mon[32765]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 859 MiB/s wr, 1.12k op/s 2024-04-26T21:59:27.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:27 smithi165 ceph-mon[33296]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 859 MiB/s wr, 1.12k op/s 2024-04-26T21:59:28.681 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T21:59:28.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:59:28.681 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T21:59:29.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/3895051158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 msgr2=0x7fa8a410bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/3895051158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 0x7fa8a410bd60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa894009900 tx=0x7fa89402a440 comp rx=0 tx=0).stop 2024-04-26T21:59:29.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/3895051158 shutdown_connections 2024-04-26T21:59:29.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/3895051158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a410ec70 0x7fa8a410f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/3895051158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8a410c2a0 0x7fa8a410e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/3895051158 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 0x7fa8a410bd60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/3895051158 >> 172.21.15.55:0/3895051158 conn(0x7fa8a406f7c0 msgr2=0x7fa8a4071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:29.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/3895051158 shutdown_connections 2024-04-26T21:59:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.029+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/3895051158 wait complete. 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.030+0000 7fa8a8eb8640 1 Processor -- start 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a8eb8640 1 -- start start 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a8eb8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 0x7fa8a41a7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a8eb8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a410c2a0 0x7fa8a41a8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a8eb8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8a410ec70 0x7fa8a41af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a8eb8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a411d370 con 0x7fa8a410ec70 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a8eb8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a411d1f0 con 0x7fa8a410c2a0 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a8eb8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a411d4f0 con 0x7fa8a4109930 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 0x7fa8a41a7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 0x7fa8a41a7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:35960/0 (socket says 172.21.15.55:35960) 2024-04-26T21:59:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a259c640 1 -- 172.21.15.55:0/4117778472 learned_addr learned my addr 172.21.15.55:0/4117778472 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a410c2a0 0x7fa8a41a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.031+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8a410ec70 0x7fa8a41af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8a259c640 1 -- 172.21.15.55:0/4117778472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a410c2a0 msgr2=0x7fa8a41a8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8a259c640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a410c2a0 0x7fa8a41a8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8a259c640 1 -- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8a410ec70 msgr2=0x7fa8a41af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8a259c640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8a410ec70 0x7fa8a41af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8a259c640 1 -- 172.21.15.55:0/4117778472 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a41af6c0 con 0x7fa8a4109930 2024-04-26T21:59:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8a259c640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 0x7fa8a41a7af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa894031470 tx=0x7fa894003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8937fe640 1 -- 172.21.15.55:0/4117778472 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89403f070 con 0x7fa8a4109930 2024-04-26T21:59:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8937fe640 1 -- 172.21.15.55:0/4117778472 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa894031bd0 con 0x7fa8a4109930 2024-04-26T21:59:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.032+0000 7fa8937fe640 1 -- 172.21.15.55:0/4117778472 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894035030 con 0x7fa8a4109930 2024-04-26T21:59:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.033+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/4117778472 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a41af950 con 0x7fa8a4109930 2024-04-26T21:59:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.033+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/4117778472 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a41afdf0 con 0x7fa8a4109930 2024-04-26T21:59:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8abbcc740 1 Processor -- start 2024-04-26T21:59:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8abbcc740 1 -- start start 2024-04-26T21:59:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8937fe640 1 -- 172.21.15.55:0/4117778472 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa894042070 con 0x7fa8a4109930 2024-04-26T21:59:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad63acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad63fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad6428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564bad59fb00 con 0x564bad63b2a0 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564bad53b920 con 0x564bad63a8b0 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.037+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564bad59bac0 con 0x564bad640410 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.038+0000 7fa8a1d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad6428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.038+0000 7fa8a1d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad6428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:35974/0 (socket says 172.21.15.55:35974) 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.038+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/2019645169 learned_addr learned my addr 172.21.15.55:0/2019645169 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.038+0000 7fa8937fe640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa88c0a3d40 0x7fa88c0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.038+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/2019645169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad63acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.039+0000 7fa8937fe640 1 -- 172.21.15.55:0/4117778472 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa8940f0e20 con 0x7fa8a4109930 2024-04-26T21:59:29.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.039+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/2019645169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 msgr2=0x564bad63acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:29.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.039+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/2019645169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad63acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.039+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/2019645169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 msgr2=0x564bad63fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:29.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.039+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa88c0a3d40 0x7fa88c0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.039+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/2019645169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad63fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.039+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/2019645169 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bad3797e0 con 0x564bad640410 2024-04-26T21:59:29.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.041+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/2019645169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad6428a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0103a0 tx=0x7fa89c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=629594ad8a537005 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:29.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.041+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa88c0a3d40 0x7fa88c0a6200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x564bad63bb00 tx=0x7fa8980046e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:29.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.053+0000 7fa890ff9640 1 -- 172.21.15.55:0/2019645169 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c016070 con 0x564bad640410 2024-04-26T21:59:29.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.053+0000 7fa890ff9640 1 -- 172.21.15.55:0/2019645169 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa89c012bc0 con 0x564bad640410 2024-04-26T21:59:29.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.053+0000 7fa890ff9640 1 -- 172.21.15.55:0/2019645169 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c01bd10 con 0x564bad640410 2024-04-26T21:59:29.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.066+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2019645169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 msgr2=0x564bad6428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.066+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2019645169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad6428a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0103a0 tx=0x7fa89c010860 comp rx=0 tx=0).stop 2024-04-26T21:59:29.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.079+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2019645169 shutdown_connections 2024-04-26T21:59:29.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.079+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2019645169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad6428a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.079+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2019645169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad63fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.079+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2019645169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad63acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.079+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2019645169 >> 172.21.15.55:0/2019645169 conn(0x564bad635f20 msgr2=0x564bad636320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:29.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.084+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2019645169 shutdown_connections 2024-04-26T21:59:29.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.086+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2019645169 wait complete. 2024-04-26T21:59:29.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.086+0000 7fa8abbcc740 1 -- start start 2024-04-26T21:59:29.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.087+0000 7fa8abbcc740 1 Processor -- start 2024-04-26T21:59:29.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.087+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad644bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.087+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad6450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.087+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad64c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.087+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564bad6433c0 con 0x564bad63b2a0 2024-04-26T21:59:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.087+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564bad5395d0 con 0x564bad63a8b0 2024-04-26T21:59:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.087+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564bad643540 con 0x564bad640410 2024-04-26T21:59:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.087+0000 7fa8abbcc740 10 client.0 _ll_register_callbacks cb 0x564bad527080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:29.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.093+0000 7fa8a1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad6450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.093+0000 7fa8a1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad6450f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49702/0 (socket says 172.21.15.55:49702) 2024-04-26T21:59:29.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.093+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/661011582 learned_addr learned my addr 172.21.15.55:0/661011582 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:29.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.093+0000 7fa8a259c640 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad644bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.093+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad64c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.095+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/661011582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 msgr2=0x564bad64c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:29.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.095+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad64c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.095+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/661011582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 msgr2=0x564bad644bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:29.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.095+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad644bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.095+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/661011582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bad3d9f70 con 0x564bad63b2a0 2024-04-26T21:59:29.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.102+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad6450f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa89c01b790 tx=0x7fa89c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f730098308c4988e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:29.105 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.103+0000 7fa889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:29.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.107+0000 7fa889ffb640 1 -- 172.21.15.55:0/661011582 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c016040 con 0x564bad63b2a0 2024-04-26T21:59:29.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.107+0000 7fa889ffb640 1 -- 172.21.15.55:0/661011582 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa89c01ba50 con 0x564bad63b2a0 2024-04-26T21:59:29.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.107+0000 7fa8abbcc740 1 -- 172.21.15.55:0/661011582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa89c012bc0 con 0x564bad63b2a0 2024-04-26T21:59:29.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.117+0000 7fa889ffb640 1 -- 172.21.15.55:0/661011582 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa89c01bc90 con 0x564bad63b2a0 2024-04-26T21:59:29.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.117+0000 7fa8abbcc740 10 client.16482 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:29.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.117+0000 7fa8abbcc740 1 -- 172.21.15.55:0/661011582 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564bad271170 con 0x564bad63b2a0 2024-04-26T21:59:29.125 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.123+0000 7fa889ffb640 1 -- 172.21.15.55:0/661011582 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa89c012bc0 con 0x564bad63b2a0 2024-04-26T21:59:29.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.124+0000 7fa889ffb640 1 -- 172.21.15.55:0/661011582 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fa89c02c030 con 0x564bad63b2a0 2024-04-26T21:59:29.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.124+0000 7fa8abbcc740 10 client.16482 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:29.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.124+0000 7fa8abbcc740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:59:29.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.124+0000 7fa8abbcc740 20 client.16482 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:29.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.124+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564bad6502e0 0x564bad652720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.124+0000 7fa8abbcc740 4 client.16482 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:59:29.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.124+0000 7fa8abbcc740 1 -- 172.21.15.55:0/661011582 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bad5ad710 con 0x564bad6502e0 2024-04-26T21:59:29.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.129+0000 7fa8a259c640 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564bad6502e0 0x564bad652720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.149+0000 7fa889ffb640 10 client.16482 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:29.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.149+0000 7fa8a259c640 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564bad6502e0 0x564bad652720 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989ebb2cd7ac8a1a server_cookie=5136a0bfee0e1167 in_seq=0 out_seq=0 2024-04-26T21:59:29.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.164+0000 7fa889ffb640 1 -- 172.21.15.55:0/661011582 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bad5ad710 con 0x564bad6502e0 2024-04-26T21:59:29.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.164+0000 7fa889ffb640 10 client.16482 handle_command_reply: tid=0 2024-04-26T21:59:29.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.164+0000 7fa8abbcc740 1 client.16482 shutdown 2024-04-26T21:59:29.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.166+0000 7fa8abbcc740 1 -- 172.21.15.55:0/661011582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 msgr2=0x564bad6450f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.166+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad6450f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa89c01b790 tx=0x7fa89c000f40 comp rx=0 tx=0).stop 2024-04-26T21:59:29.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa8abbcc740 1 -- 172.21.15.55:0/661011582 shutdown_connections 2024-04-26T21:59:29.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad640410 0x564bad64c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564bad6502e0 0x564bad652720 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad6450f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/661011582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad644bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa8abbcc740 1 -- 172.21.15.55:0/661011582 >> 172.21.15.55:0/661011582 conn(0x564bad635f20 msgr2=0x564bad6350e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:29.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa889ffb640 0 client.16482 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:29.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa8abbcc740 1 -- 172.21.15.55:0/661011582 shutdown_connections 2024-04-26T21:59:29.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.168+0000 7fa8abbcc740 1 -- 172.21.15.55:0/661011582 wait complete. 2024-04-26T21:59:29.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.169+0000 7fa8abbcc740 20 client.16482 trim_cache size 0 max 16384 2024-04-26T21:59:29.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.247+0000 7fa8abbcc740 1 Processor -- start 2024-04-26T21:59:29.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.248+0000 7fa8abbcc740 1 -- start start 2024-04-26T21:59:29.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.248+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad638c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.252+0000 7fa8a259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad638c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.252 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.252+0000 7fa8a259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad638c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:36696/0 (socket says 172.21.15.55:36696) 2024-04-26T21:59:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.254+0000 7fa8a259c640 1 -- 172.21.15.55:0/3790563722 learned_addr learned my addr 172.21.15.55:0/3790563722 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:29.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.254+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63b2a0 0x564bad64c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.254+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad640410 0x564bad639140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.254+0000 7fa8abbcc740 1 -- 172.21.15.55:0/3790563722 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41bb630 con 0x564bad640410 2024-04-26T21:59:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.254+0000 7fa8abbcc740 1 -- 172.21.15.55:0/3790563722 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41b8f00 con 0x564bad63a8b0 2024-04-26T21:59:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.254+0000 7fa8abbcc740 1 -- 172.21.15.55:0/3790563722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41b6d50 con 0x564bad63b2a0 2024-04-26T21:59:29.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.254+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad640410 0x564bad639140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.257+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63b2a0 0x564bad64c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.260+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/3790563722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 msgr2=0x564bad638c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.260+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad638c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.260+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/3790563722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad640410 msgr2=0x564bad639140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.260+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad640410 0x564bad639140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.260+0000 7fa8a1d9b640 1 -- 172.21.15.55:0/3790563722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bad635d50 con 0x564bad63b2a0 2024-04-26T21:59:29.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.261+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad640410 0x564bad639140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:59:29.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.263+0000 7fa8a1d9b640 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63b2a0 0x564bad64c0f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa89c01b7e0 tx=0x7fa89c0049e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe8578f35fa8beb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:29.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.265+0000 7fa88bfff640 1 -- 172.21.15.55:0/3790563722 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c016040 con 0x564bad63b2a0 2024-04-26T21:59:29.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.265+0000 7fa88bfff640 1 -- 172.21.15.55:0/3790563722 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa89c0057f0 con 0x564bad63b2a0 2024-04-26T21:59:29.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.265+0000 7fa88bfff640 1 -- 172.21.15.55:0/3790563722 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c02d510 con 0x564bad63b2a0 2024-04-26T21:59:29.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.267+0000 7fa8abbcc740 1 -- 172.21.15.55:0/3790563722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63b2a0 msgr2=0x564bad64c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.267+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63b2a0 0x564bad64c0f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa89c01b7e0 tx=0x7fa89c0049e0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.267+0000 7fa8abbcc740 1 -- 172.21.15.55:0/3790563722 shutdown_connections 2024-04-26T21:59:29.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.267+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad640410 0x564bad639140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.267+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63b2a0 0x564bad64c0f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.267+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/3790563722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad63a8b0 0x564bad638c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.267+0000 7fa8abbcc740 1 -- 172.21.15.55:0/3790563722 >> 172.21.15.55:0/3790563722 conn(0x564bad635f20 msgr2=0x564bad63c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:29.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.269+0000 7fa8abbcc740 1 -- 172.21.15.55:0/3790563722 shutdown_connections 2024-04-26T21:59:29.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.270+0000 7fa8abbcc740 1 -- 172.21.15.55:0/3790563722 wait complete. 2024-04-26T21:59:29.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.271+0000 7fa8abbcc740 1 -- start start 2024-04-26T21:59:29.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.271+0000 7fa8abbcc740 1 Processor -- start 2024-04-26T21:59:29.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.272+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63a8b0 0x564bad649ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.272+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad649ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.272+0000 7fa8abbcc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad640410 0x564bad64af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.273+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41b73c0 con 0x564bad63b2a0 2024-04-26T21:59:29.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.273+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41b8830 con 0x564bad640410 2024-04-26T21:59:29.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.273+0000 7fa8abbcc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41b7a90 con 0x564bad63a8b0 2024-04-26T21:59:29.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.273+0000 7fa8abbcc740 10 client.0 _ll_register_callbacks cb 0x7fa89c01b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:29.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.273+0000 7fa8a259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63a8b0 0x564bad649ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.273+0000 7fa8a259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63a8b0 0x564bad649ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36006/0 (socket says 172.21.15.55:36006) 2024-04-26T21:59:29.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.273+0000 7fa8a259c640 1 -- 172.21.15.55:0/2453697451 learned_addr learned my addr 172.21.15.55:0/2453697451 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:29.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.274+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad640410 0x564bad64af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.275+0000 7fa8a2d9d640 1 -- 172.21.15.55:0/2453697451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63a8b0 msgr2=0x564bad649ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:29.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.275+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63a8b0 0x564bad649ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.275+0000 7fa8a2d9d640 1 -- 172.21.15.55:0/2453697451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 msgr2=0x564bad649ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:29.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.275+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad649ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.275+0000 7fa8a2d9d640 1 -- 172.21.15.55:0/2453697451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bad64b530 con 0x564bad640410 2024-04-26T21:59:29.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.277+0000 7fa8a2d9d640 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad640410 0x564bad64af40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa898009700 tx=0x7fa898004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f421d27de084747 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:29.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.279+0000 7fa889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T21:59:29.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.281+0000 7fa889ffb640 1 -- 172.21.15.55:0/2453697451 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898026070 con 0x564bad640410 2024-04-26T21:59:29.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.281+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2453697451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa89c0057f0 con 0x564bad640410 2024-04-26T21:59:29.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.281+0000 7fa889ffb640 1 -- 172.21.15.55:0/2453697451 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa8980159a0 con 0x564bad640410 2024-04-26T21:59:29.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.286+0000 7fa889ffb640 1 -- 172.21.15.55:0/2453697451 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa89800f040 con 0x564bad640410 2024-04-26T21:59:29.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.287+0000 7fa8abbcc740 10 client.25718 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:29.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.287+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2453697451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564bad6b6b40 con 0x564bad640410 2024-04-26T21:59:29.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.289+0000 7fa889ffb640 1 -- 172.21.15.55:0/2453697451 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa89c0057f0 con 0x564bad640410 2024-04-26T21:59:29.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.289+0000 7fa889ffb640 1 -- 172.21.15.55:0/2453697451 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fa898015070 con 0x564bad640410 2024-04-26T21:59:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.292+0000 7fa8abbcc740 10 client.25718 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.293+0000 7fa8abbcc740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:59:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.293+0000 7fa8abbcc740 20 client.25718 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.293+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564bad6b74a0 0x564bad6b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.293+0000 7fa8abbcc740 4 client.25718 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T21:59:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.293+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2453697451 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564bad5a7580 con 0x564bad6b74a0 2024-04-26T21:59:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.295+0000 7fa8a259c640 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564bad6b74a0 0x564bad6b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.297+0000 7fa8a259c640 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564bad6b74a0 0x564bad6b98e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a710ac4452fb5036 server_cookie=b35c1ec37c61b42e in_seq=0 out_seq=0 2024-04-26T21:59:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.297+0000 7fa889ffb640 10 client.25718 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:29.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:29 smithi064 ceph-mon[32765]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 853 MiB/s wr, 1.04k op/s 2024-04-26T21:59:29.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:29 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:29.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:29 smithi165 ceph-mon[33296]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 853 MiB/s wr, 1.04k op/s 2024-04-26T21:59:29.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:29 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:29.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.886+0000 7fa889ffb640 1 -- 172.21.15.55:0/2453697451 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564bad5a7580 con 0x564bad6b74a0 2024-04-26T21:59:29.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.886+0000 7fa889ffb640 10 client.25718 handle_command_reply: tid=0 2024-04-26T21:59:29.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.886+0000 7fa8abbcc740 1 client.25718 shutdown 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.887+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad640410 msgr2=0x564bad64af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.887+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad640410 0x564bad64af40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa898009700 tx=0x7fa898004750 comp rx=0 tx=0).stop 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.887+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2453697451 shutdown_connections 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.887+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564bad640410 0x564bad64af40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.887+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564bad6b74a0 0x564bad6b98e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.887+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564bad63b2a0 0x564bad649ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.887+0000 7fa8abbcc740 1 --2- 172.21.15.55:0/2453697451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bad63a8b0 0x564bad649ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.887+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2453697451 >> 172.21.15.55:0/2453697451 conn(0x564bad635f20 msgr2=0x564bad638c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:29.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.888+0000 7fa889ffb640 0 client.25718 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:29.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.896+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2453697451 shutdown_connections 2024-04-26T21:59:29.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.896+0000 7fa8abbcc740 1 -- 172.21.15.55:0/2453697451 wait complete. 2024-04-26T21:59:29.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.897+0000 7fa8abbcc740 20 client.25718 trim_cache size 0 max 16384 2024-04-26T21:59:29.903 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:59:29.903 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T21:59:29.903 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "578e999a-2a27-43fa-bc58-53cb1bd1a0f4", 2024-04-26T21:59:29.903 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T21:59:29.903 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.902+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa88c0a3d40 msgr2=0x7fa88c0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.902+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa88c0a3d40 0x7fa88c0a6200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x564bad63bb00 tx=0x7fa8980046e0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.902+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/4117778472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 msgr2=0x7fa8a41a7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.902+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 0x7fa8a41a7af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa894031470 tx=0x7fa894003c10 comp rx=0 tx=0).stop 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.903+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/4117778472 shutdown_connections 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.903+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa88c0a3d40 0x7fa88c0a6200 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.903+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8a410ec70 0x7fa8a41af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.903+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa8a410c2a0 0x7fa8a41a8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.903+0000 7fa8a8eb8640 1 --2- 172.21.15.55:0/4117778472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4109930 0x7fa8a41a7af0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:29.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.903+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/4117778472 >> 172.21.15.55:0/4117778472 conn(0x7fa8a406f7c0 msgr2=0x7fa8a410c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:29.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.904+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/4117778472 shutdown_connections 2024-04-26T21:59:29.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:29.904+0000 7fa8a8eb8640 1 -- 172.21.15.55:0/4117778472 wait complete. 2024-04-26T21:59:29.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:29 smithi055 ceph-mon[29221]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 853 MiB/s wr, 1.04k op/s 2024-04-26T21:59:29.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:29 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:29.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T21:59:29.953 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T21:59:30.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.376+0000 7f01fcba6640 1 -- 172.21.15.55:0/4120549077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f8109f10 msgr2=0x7f01f810a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:30.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.376+0000 7f01fcba6640 1 --2- 172.21.15.55:0/4120549077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f8109f10 0x7f01f810a390 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009900 tx=0x7f01ec02a440 comp rx=0 tx=0).stop 2024-04-26T21:59:30.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.376+0000 7f01fcba6640 1 -- 172.21.15.55:0/4120549077 shutdown_connections 2024-04-26T21:59:30.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.376+0000 7f01fcba6640 1 --2- 172.21.15.55:0/4120549077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01f810a8d0 0x7f01f810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.376+0000 7f01fcba6640 1 --2- 172.21.15.55:0/4120549077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f8109f10 0x7f01f810a390 secure :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009900 tx=0x7f01ec02a440 comp rx=0 tx=0).stop 2024-04-26T21:59:30.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.376+0000 7f01fcba6640 1 --2- 172.21.15.55:0/4120549077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01f8073c10 0x7f01f8074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.376+0000 7f01fcba6640 1 -- 172.21.15.55:0/4120549077 >> 172.21.15.55:0/4120549077 conn(0x7f01f806f780 msgr2=0x7f01f8071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:30.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.380+0000 7f01fcba6640 1 -- 172.21.15.55:0/4120549077 shutdown_connections 2024-04-26T21:59:30.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.381+0000 7f01fcba6640 1 -- 172.21.15.55:0/4120549077 wait complete. 2024-04-26T21:59:30.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.383+0000 7f01fcba6640 1 Processor -- start 2024-04-26T21:59:30.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.384+0000 7f01fcba6640 1 -- start start 2024-04-26T21:59:30.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.385+0000 7f01fcba6640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f8073c10 0x7f01f8116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.385+0000 7f01fcba6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01f810a8d0 0x7f01f81167b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.385+0000 7f01fcba6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01f81b57f0 0x7f01f81b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.385+0000 7f01fcba6640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f01f811d110 con 0x7f01f8073c10 2024-04-26T21:59:30.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.385+0000 7f01fcba6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f01f811cf90 con 0x7f01f81b57f0 2024-04-26T21:59:30.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.385+0000 7f01fcba6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01f811d290 con 0x7f01f810a8d0 2024-04-26T21:59:30.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.387+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01f81b57f0 0x7f01f81b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.387+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01f81b57f0 0x7f01f81b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:36730/0 (socket says 172.21.15.55:36730) 2024-04-26T21:59:30.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.387+0000 7f01f7fff640 1 -- 172.21.15.55:0/1364453632 learned_addr learned my addr 172.21.15.55:0/1364453632 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:30.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.388+0000 7f01f7fff640 1 -- 172.21.15.55:0/1364453632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01f810a8d0 msgr2=0x7f01f81167b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T21:59:30.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.388+0000 7f01f7fff640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01f810a8d0 0x7f01f81167b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.388+0000 7f01f7fff640 1 -- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f8073c10 msgr2=0x7f01f8116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:30.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.388+0000 7f01f7fff640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f8073c10 0x7f01f8116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.388+0000 7f01f7fff640 1 -- 172.21.15.55:0/1364453632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f81b8250 con 0x7f01f81b57f0 2024-04-26T21:59:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.392+0000 7f01f7fff640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01f81b57f0 0x7f01f81b7be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01e800e970 tx=0x7f01e800ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.393+0000 7f01f4ff9640 1 -- 172.21.15.55:0/1364453632 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e800ccc0 con 0x7f01f81b57f0 2024-04-26T21:59:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.394+0000 7f01f4ff9640 1 -- 172.21.15.55:0/1364453632 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01e800ce60 con 0x7f01f81b57f0 2024-04-26T21:59:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.394+0000 7f01f4ff9640 1 -- 172.21.15.55:0/1364453632 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8010430 con 0x7f01f81b57f0 2024-04-26T21:59:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.394+0000 7f01fcba6640 1 -- 172.21.15.55:0/1364453632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f81b84e0 con 0x7f01f81b57f0 2024-04-26T21:59:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.394+0000 7f01fcba6640 1 -- 172.21.15.55:0/1364453632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f81b8a00 con 0x7f01f81b57f0 2024-04-26T21:59:30.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.401+0000 7f01f4ff9640 1 -- 172.21.15.55:0/1364453632 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f01e80046c0 con 0x7f01f81b57f0 2024-04-26T21:59:30.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.402+0000 7f01f4ff9640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01e00a3d80 0x7f01e00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.403+0000 7f01f4ff9640 1 -- 172.21.15.55:0/1364453632 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f01e80d09d0 con 0x7f01f81b57f0 2024-04-26T21:59:30.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.409+0000 7f01f77fe640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01e00a3d80 0x7f01e00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.468+0000 7f0200bf4740 1 Processor -- start 2024-04-26T21:59:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f0200bf4740 1 -- start start 2024-04-26T21:59:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfaf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0280 0x5602ddfb0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f0200bf4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5602ddf13d80 con 0x5602ddfb0280 2024-04-26T21:59:30.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f0200bf4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5602ddecd8b0 con 0x5602ddfb0c40 2024-04-26T21:59:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f0200bf4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5602ddf0fd40 con 0x5602ddfaf0b0 2024-04-26T21:59:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f01f6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0280 0x5602ddfb0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.471+0000 7f01f6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0280 0x5602ddfb0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49764/0 (socket says 172.21.15.55:49764) 2024-04-26T21:59:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.472+0000 7f01f6ffd640 1 -- 172.21.15.55:0/3066184228 learned_addr learned my addr 172.21.15.55:0/3066184228 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.474+0000 7f01f7fff640 1 --2- 172.21.15.55:0/3066184228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.475+0000 7f01f6ffd640 1 -- 172.21.15.55:0/3066184228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 msgr2=0x5602ddfaf4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T21:59:30.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.475+0000 7f01f6ffd640 1 --2- 172.21.15.55:0/3066184228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfaf4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.475+0000 7f01f6ffd640 1 -- 172.21.15.55:0/3066184228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 msgr2=0x5602ddfb74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:30.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.475+0000 7f01f6ffd640 1 --2- 172.21.15.55:0/3066184228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.475+0000 7f01f6ffd640 1 -- 172.21.15.55:0/3066184228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602dde9ea40 con 0x5602ddfb0280 2024-04-26T21:59:30.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.476+0000 7f01f77fe640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01e00a3d80 0x7f01e00a6240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01f0007c30 tx=0x7f01f0005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:30.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.477+0000 7f01f6ffd640 1 --2- 172.21.15.55:0/3066184228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0280 0x5602ddfb0700 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009850 tx=0x7f01ec009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07d80de1d3c9294 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:30.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.479+0000 7f01ddffb640 1 -- 172.21.15.55:0/3066184228 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec040070 con 0x5602ddfb0280 2024-04-26T21:59:30.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.479+0000 7f01ddffb640 1 -- 172.21.15.55:0/3066184228 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ec004cc0 con 0x5602ddfb0280 2024-04-26T21:59:30.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 -- 172.21.15.55:0/3066184228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0280 msgr2=0x5602ddfb0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:30.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 --2- 172.21.15.55:0/3066184228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0280 0x5602ddfb0700 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009850 tx=0x7f01ec009640 comp rx=0 tx=0).stop 2024-04-26T21:59:30.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 -- 172.21.15.55:0/3066184228 shutdown_connections 2024-04-26T21:59:30.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 --2- 172.21.15.55:0/3066184228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 --2- 172.21.15.55:0/3066184228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0280 0x5602ddfb0700 secure :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009850 tx=0x7f01ec009640 comp rx=0 tx=0).stop 2024-04-26T21:59:30.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 --2- 172.21.15.55:0/3066184228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfaf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 -- 172.21.15.55:0/3066184228 >> 172.21.15.55:0/3066184228 conn(0x5602ddfaa5a0 msgr2=0x5602ddfaa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:30.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 -- 172.21.15.55:0/3066184228 shutdown_connections 2024-04-26T21:59:30.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.481+0000 7f0200bf4740 1 -- 172.21.15.55:0/3066184228 wait complete. 2024-04-26T21:59:30.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.482+0000 7f0200bf4740 1 -- start start 2024-04-26T21:59:30.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.483+0000 7f0200bf4740 1 Processor -- start 2024-04-26T21:59:30.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.483+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfb94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.483+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.483+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602ddfc0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.483+0000 7f0200bf4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5602ddfb7c40 con 0x5602ddfbea30 2024-04-26T21:59:30.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.483+0000 7f0200bf4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5602dde9ad10 con 0x5602ddfb0c40 2024-04-26T21:59:30.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.483+0000 7f0200bf4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5602ddfb7dc0 con 0x5602ddfaf0b0 2024-04-26T21:59:30.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.483+0000 7f0200bf4740 10 client.0 _ll_register_callbacks cb 0x5602ddec7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:30.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.484+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602ddfc0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.484+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602ddfc0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49778/0 (socket says 172.21.15.55:49778) 2024-04-26T21:59:30.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.484+0000 7f01f7fff640 1 -- 172.21.15.55:0/2902799550 learned_addr learned my addr 172.21.15.55:0/2902799550 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:30.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.484+0000 7f01f77fe640 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfb94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.484+0000 7f01f6ffd640 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.485+0000 7f01f7fff640 1 -- 172.21.15.55:0/2902799550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 msgr2=0x5602ddfb94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:30.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.485+0000 7f01f7fff640 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfb94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.485+0000 7f01f7fff640 1 -- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 msgr2=0x5602ddfb9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:30.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.485+0000 7f01f7fff640 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.485+0000 7f01f7fff640 1 -- 172.21.15.55:0/2902799550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ddcef7d0 con 0x5602ddfbea30 2024-04-26T21:59:30.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.485+0000 7f01f6ffd640 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:59:30.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.489+0000 7f01f7fff640 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602ddfc0e20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f01e800d010 tx=0x7f01e809cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12953cbdbd7667a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:30.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.489+0000 7f01d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:30.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.491+0000 7f01d6ffd640 1 -- 172.21.15.55:0/2902799550 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80a5490 con 0x5602ddfbea30 2024-04-26T21:59:30.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.491+0000 7f0200bf4740 1 -- 172.21.15.55:0/2902799550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602ddf21990 con 0x5602ddfbea30 2024-04-26T21:59:30.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.492+0000 7f01d6ffd640 1 -- 172.21.15.55:0/2902799550 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f01e809bf00 con 0x5602ddfbea30 2024-04-26T21:59:30.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.492+0000 7f01d6ffd640 1 -- 172.21.15.55:0/2902799550 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f01e809c100 con 0x5602ddfbea30 2024-04-26T21:59:30.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.493+0000 7f0200bf4740 10 client.16494 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:30.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.493+0000 7f0200bf4740 1 -- 172.21.15.55:0/2902799550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5602ddc253f0 con 0x5602ddfbea30 2024-04-26T21:59:30.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.496+0000 7f01d6ffd640 1 -- 172.21.15.55:0/2902799550 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5602ddf21990 con 0x5602ddfbea30 2024-04-26T21:59:30.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.496+0000 7f01d6ffd640 1 -- 172.21.15.55:0/2902799550 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f01e80a1ff0 con 0x5602ddfbea30 2024-04-26T21:59:30.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.496+0000 7f0200bf4740 10 client.16494 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:30.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.496+0000 7f0200bf4740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:59:30.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.496+0000 7f0200bf4740 20 client.16494 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:30.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.496+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5602ddfc5450 0x5602ddfc78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.496+0000 7f0200bf4740 4 client.16494 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:59:30.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.496+0000 7f0200bf4740 1 -- 172.21.15.55:0/2902799550 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602ddf1b800 con 0x5602ddfc5450 2024-04-26T21:59:30.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.497+0000 7f01f77fe640 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5602ddfc5450 0x5602ddfc78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.498+0000 7f01f77fe640 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5602ddfc5450 0x5602ddfc78d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5544e146e8b1d2 server_cookie=5e7181976224b94 in_seq=0 out_seq=0 2024-04-26T21:59:30.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.498+0000 7f01d6ffd640 10 client.16494 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:30.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.501+0000 7f01d6ffd640 1 -- 172.21.15.55:0/2902799550 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602ddf1b800 con 0x5602ddfc5450 2024-04-26T21:59:30.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.502+0000 7f01d6ffd640 10 client.16494 handle_command_reply: tid=0 2024-04-26T21:59:30.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.502+0000 7f0200bf4740 1 client.16494 shutdown 2024-04-26T21:59:30.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 -- 172.21.15.55:0/2902799550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 msgr2=0x5602ddfc0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:30.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602ddfc0e20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f01e800d010 tx=0x7f01e809cbf0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 -- 172.21.15.55:0/2902799550 shutdown_connections 2024-04-26T21:59:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5602ddfc5450 0x5602ddfc78d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602ddfc0e20 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfb0c40 0x5602ddfb9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2902799550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfb94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 -- 172.21.15.55:0/2902799550 >> 172.21.15.55:0/2902799550 conn(0x5602ddfaa5a0 msgr2=0x5602ddfb0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f01d6ffd640 0 client.16494 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 -- 172.21.15.55:0/2902799550 shutdown_connections 2024-04-26T21:59:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.503+0000 7f0200bf4740 1 -- 172.21.15.55:0/2902799550 wait complete. 2024-04-26T21:59:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.504+0000 7f0200bf4740 20 client.16494 trim_cache size 0 max 16384 2024-04-26T21:59:30.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.559+0000 7f0200bf4740 1 Processor -- start 2024-04-26T21:59:30.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.559+0000 7f0200bf4740 1 -- start start 2024-04-26T21:59:30.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.559+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfb3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.559+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0c40 0x5602ddfb4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.559+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfbea30 0x5602ddfa8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.559+0000 7f0200bf4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4010fd0 con 0x5602ddfb0c40 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.559+0000 7f0200bf4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f01e400e8a0 con 0x5602ddfbea30 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.559+0000 7f0200bf4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01e400c6f0 con 0x5602ddfaf0b0 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.560+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfbea30 0x5602ddfa8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.560+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfbea30 0x5602ddfa8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:36750/0 (socket says 172.21.15.55:36750) 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.560+0000 7f01f7fff640 1 -- 172.21.15.55:0/2038506378 learned_addr learned my addr 172.21.15.55:0/2038506378 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.561+0000 7f01f77fe640 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfb3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.561+0000 7f01f7fff640 1 -- 172.21.15.55:0/2038506378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 msgr2=0x5602ddfb3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.561+0000 7f01f7fff640 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfb3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.561+0000 7f01f7fff640 1 -- 172.21.15.55:0/2038506378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0c40 msgr2=0x5602ddfb4170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.561+0000 7f01f6ffd640 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0c40 0x5602ddfb4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.561+0000 7f01f7fff640 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0c40 0x5602ddfb4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.561+0000 7f01f7fff640 1 -- 172.21.15.55:0/2038506378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ddbe50f0 con 0x5602ddfbea30 2024-04-26T21:59:30.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.562+0000 7f01f7fff640 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfbea30 0x5602ddfa8e20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01e80a1020 tx=0x7f01e8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24dcecfeb523662d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:30.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.563+0000 7f01dd7fa640 1 -- 172.21.15.55:0/2038506378 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e809d1b0 con 0x5602ddfbea30 2024-04-26T21:59:30.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.563+0000 7f01dd7fa640 1 -- 172.21.15.55:0/2038506378 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01e80b6020 con 0x5602ddfbea30 2024-04-26T21:59:30.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.564+0000 7f0200bf4740 1 -- 172.21.15.55:0/2038506378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfbea30 msgr2=0x5602ddfa8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:30.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.565+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfbea30 0x5602ddfa8e20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f01e80a1020 tx=0x7f01e8009310 comp rx=0 tx=0).stop 2024-04-26T21:59:30.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.565+0000 7f0200bf4740 1 -- 172.21.15.55:0/2038506378 shutdown_connections 2024-04-26T21:59:30.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.565+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfbea30 0x5602ddfa8e20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.565+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfb0c40 0x5602ddfb4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.565+0000 7f0200bf4740 1 --2- 172.21.15.55:0/2038506378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfaf0b0 0x5602ddfb3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.565+0000 7f0200bf4740 1 -- 172.21.15.55:0/2038506378 >> 172.21.15.55:0/2038506378 conn(0x5602ddfaa5a0 msgr2=0x5602ddfb94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:30.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.566+0000 7f0200bf4740 1 -- 172.21.15.55:0/2038506378 shutdown_connections 2024-04-26T21:59:30.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.566+0000 7f0200bf4740 1 -- 172.21.15.55:0/2038506378 wait complete. 2024-04-26T21:59:30.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.567+0000 7f0200bf4740 1 -- start start 2024-04-26T21:59:30.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.567+0000 7f0200bf4740 1 Processor -- start 2024-04-26T21:59:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.568+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfaf0b0 0x5602ddfd3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.568+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfb0c40 0x5602ddfd4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.568+0000 7f0200bf4740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602de029520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.568+0000 7f0200bf4740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f01e400cd60 con 0x5602ddfbea30 2024-04-26T21:59:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.568+0000 7f0200bf4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f01e400e1d0 con 0x5602ddfaf0b0 2024-04-26T21:59:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.568+0000 7f0200bf4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01e400d430 con 0x5602ddfb0c40 2024-04-26T21:59:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.568+0000 7f0200bf4740 10 client.0 _ll_register_callbacks cb 0x7f01e809c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.569+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602de029520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.569+0000 7f01f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602de029520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49796/0 (socket says 172.21.15.55:49796) 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.569+0000 7f01f7fff640 1 -- 172.21.15.55:0/1837914129 learned_addr learned my addr 172.21.15.55:0/1837914129 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.570+0000 7f01f7fff640 1 -- 172.21.15.55:0/1837914129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfb0c40 msgr2=0x5602ddfd4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.570+0000 7f01f6ffd640 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfb0c40 0x5602ddfd4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.570+0000 7f01f7fff640 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfb0c40 0x5602ddfd4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.570+0000 7f01f7fff640 1 -- 172.21.15.55:0/1837914129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfaf0b0 msgr2=0x5602ddfd3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.570+0000 7f01f7fff640 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfaf0b0 0x5602ddfd3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.570+0000 7f01f7fff640 1 -- 172.21.15.55:0/1837914129 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602dded44b0 con 0x5602ddfbea30 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.571+0000 7f01f7fff640 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602de029520 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f01e809c7b0 tx=0x7f01e809c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c065d1b5bb5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:59:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.571+0000 7f01d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T21:59:30.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.572+0000 7f01d6ffd640 1 -- 172.21.15.55:0/1837914129 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80a9020 con 0x5602ddfbea30 2024-04-26T21:59:30.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.572+0000 7f0200bf4740 1 -- 172.21.15.55:0/1837914129 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01e80b6020 con 0x5602ddfbea30 2024-04-26T21:59:30.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.572+0000 7f01d6ffd640 1 -- 172.21.15.55:0/1837914129 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f01e80870b0 con 0x5602ddfbea30 2024-04-26T21:59:30.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.573+0000 7f01d6ffd640 1 -- 172.21.15.55:0/1837914129 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f01e8098b70 con 0x5602ddfbea30 2024-04-26T21:59:30.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.573+0000 7f0200bf4740 10 client.16497 fetch_fsmap learned FSMap version 23 2024-04-26T21:59:30.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.573+0000 7f0200bf4740 1 -- 172.21.15.55:0/1837914129 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5602dded44b0 con 0x5602ddfbea30 2024-04-26T21:59:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.574+0000 7f01d6ffd640 1 -- 172.21.15.55:0/1837914129 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01e80b6020 con 0x5602ddfbea30 2024-04-26T21:59:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.575+0000 7f01d6ffd640 1 -- 172.21.15.55:0/1837914129 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f01e80b1080 con 0x5602ddfbea30 2024-04-26T21:59:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.576+0000 7f0200bf4740 10 client.16497 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T21:59:30.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.576+0000 7f0200bf4740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T21:59:30.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.576+0000 7f0200bf4740 20 client.16497 populate_metadata read hostname 'smithi055' 2024-04-26T21:59:30.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.576+0000 7f0200bf4740 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5602de02b060 0x5602de02d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:59:30.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.576+0000 7f0200bf4740 4 client.16497 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T21:59:30.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.576+0000 7f0200bf4740 1 -- 172.21.15.55:0/1837914129 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5602dde964a0 con 0x5602de02b060 2024-04-26T21:59:30.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.577+0000 7f01f77fe640 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5602de02b060 0x5602de02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:59:30.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.579+0000 7f01f77fe640 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5602de02b060 0x5602de02d450 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fded0cf1b43a15c server_cookie=c420b0746bc1b732 in_seq=0 out_seq=0 2024-04-26T21:59:30.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.579+0000 7f01d6ffd640 10 client.16497 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:30.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.584+0000 7f01d6ffd640 1 -- 172.21.15.55:0/1837914129 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x5602dde964a0 con 0x5602de02b060 2024-04-26T21:59:30.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.585+0000 7f01d6ffd640 10 client.16497 handle_command_reply: tid=0 2024-04-26T21:59:30.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.585+0000 7f0200bf4740 1 client.16497 shutdown 2024-04-26T21:59:30.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.586+0000 7f0200bf4740 1 -- 172.21.15.55:0/1837914129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 msgr2=0x5602de029520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:30.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.586+0000 7f0200bf4740 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602de029520 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f01e809c7b0 tx=0x7f01e809c6b0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.586+0000 7f0200bf4740 1 -- 172.21.15.55:0/1837914129 shutdown_connections 2024-04-26T21:59:30.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.586+0000 7f0200bf4740 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5602de02b060 0x5602de02d450 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.586+0000 7f0200bf4740 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5602ddfbea30 0x5602de029520 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.586+0000 7f0200bf4740 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5602ddfb0c40 0x5602ddfd4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.586+0000 7f0200bf4740 1 --2- 172.21.15.55:0/1837914129 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5602ddfaf0b0 0x5602ddfd3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.586+0000 7f0200bf4740 1 -- 172.21.15.55:0/1837914129 >> 172.21.15.55:0/1837914129 conn(0x5602ddfaa5a0 msgr2=0x5602ddfa8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:30.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.587+0000 7f01d6ffd640 0 client.16497 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T21:59:30.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.590+0000 7f0200bf4740 1 -- 172.21.15.55:0/1837914129 shutdown_connections 2024-04-26T21:59:30.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.590+0000 7f0200bf4740 1 -- 172.21.15.55:0/1837914129 wait complete. 2024-04-26T21:59:30.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.590+0000 7f0200bf4740 20 client.16497 trim_cache size 0 max 16384 2024-04-26T21:59:30.593 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T21:59:30.593 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-26T21:59:30.594 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T21:59:30.594 INFO:teuthology.orchestra.run.smithi055.stdout: "578e999a-2a27-43fa-bc58-53cb1bd1a0f4": { 2024-04-26T21:59:30.594 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T21:59:30.594 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "578e999a-2a27-43fa-bc58-53cb1bd1a0f4", 2024-04-26T21:59:30.594 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T21:59:30.594 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:59:30.594 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T21:59:30.594 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T21:59:30.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.610+0000 7f01fcba6640 1 -- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01e00a3d80 msgr2=0x7f01e00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:30.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.610+0000 7f01fcba6640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01e00a3d80 0x7f01e00a6240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01f0007c30 tx=0x7f01f0005f70 comp rx=0 tx=0).stop 2024-04-26T21:59:30.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.610+0000 7f01fcba6640 1 -- 172.21.15.55:0/1364453632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01f81b57f0 msgr2=0x7f01f81b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:59:30.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.610+0000 7f01fcba6640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01f81b57f0 0x7f01f81b7be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01e800e970 tx=0x7f01e800ed70 comp rx=0 tx=0).stop 2024-04-26T21:59:30.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.613+0000 7f01fcba6640 1 -- 172.21.15.55:0/1364453632 shutdown_connections 2024-04-26T21:59:30.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.613+0000 7f01fcba6640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01e00a3d80 0x7f01e00a6240 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.613+0000 7f01fcba6640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f01f81b57f0 0x7f01f81b7be0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.613+0000 7f01fcba6640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01f810a8d0 0x7f01f81167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.613+0000 7f01fcba6640 1 --2- 172.21.15.55:0/1364453632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f8073c10 0x7f01f8116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:59:30.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.613+0000 7f01fcba6640 1 -- 172.21.15.55:0/1364453632 >> 172.21.15.55:0/1364453632 conn(0x7f01f806f780 msgr2=0x7f01f8071500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:59:30.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.617+0000 7f01fcba6640 1 -- 172.21.15.55:0/1364453632 shutdown_connections 2024-04-26T21:59:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T21:59:30.617+0000 7f01fcba6640 1 -- 172.21.15.55:0/1364453632 wait complete. 2024-04-26T21:59:30.676 INFO:tasks.cephfs.filesystem:scrub status for tag:578e999a-2a27-43fa-bc58-53cb1bd1a0f4 - {'path': '/', 'tag': '578e999a-2a27-43fa-bc58-53cb1bd1a0f4', 'options': 'recursive,force'} 2024-04-26T21:59:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:30 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T21:59:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:30 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:30.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:30 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:59:30.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:30 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T21:59:30.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:30 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:30.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:30 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:59:30.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:30 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T21:59:30.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:30 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:30.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:30 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:59:31.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:31 smithi064 ceph-mon[32765]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 1.04k op/s 2024-04-26T21:59:31.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:31 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T21:59:31.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:31 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:31.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:31 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:31 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:31 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:31 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:31 smithi165 ceph-mon[33296]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 1.04k op/s 2024-04-26T21:59:31.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:31 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T21:59:31.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:31 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:31.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:31 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:31 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:31 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.828 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:31 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:31 smithi055 ceph-mon[29221]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 1.04k op/s 2024-04-26T21:59:31.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:31 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T21:59:31.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:31 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:31.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:31 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:31 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:31 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:31.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:31 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:59:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:32 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-04-26T21:59:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:32 smithi064 ceph-mon[32765]: Unable to set osd_memory_target on smithi055 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-26T21:59:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:32.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:32 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:59:32.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:32 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-04-26T21:59:32.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:32 smithi165 ceph-mon[33296]: Unable to set osd_memory_target on smithi055 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-26T21:59:32.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:32.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:32.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:32 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:59:32.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:32 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi055 to 177.2M 2024-04-26T21:59:32.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:32 smithi055 ceph-mon[29221]: Unable to set osd_memory_target on smithi055 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-04-26T21:59:32.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:32.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:32.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:32 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T21:59:33.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:33 smithi064 ceph-mon[32765]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 848 MiB/s wr, 1.11k op/s 2024-04-26T21:59:33.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:33 smithi165 ceph-mon[33296]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 848 MiB/s wr, 1.11k op/s 2024-04-26T21:59:33.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:33 smithi055 ceph-mon[29221]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 848 MiB/s wr, 1.11k op/s 2024-04-26T21:59:34.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:34 smithi165 ceph-mon[33296]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.17k op/s 2024-04-26T21:59:34.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:34 smithi055 ceph-mon[29221]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.17k op/s 2024-04-26T21:59:35.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:34 smithi064 ceph-mon[32765]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 840 MiB/s wr, 1.17k op/s 2024-04-26T21:59:35.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:35 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:35.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:35 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:36.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:35 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T21:59:36.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:36 smithi165 ceph-mon[33296]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 876 MiB/s wr, 1.28k op/s 2024-04-26T21:59:36.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:36 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:59:36.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:36 smithi055 ceph-mon[29221]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 876 MiB/s wr, 1.28k op/s 2024-04-26T21:59:36.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:36 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:59:37.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:36 smithi064 ceph-mon[32765]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 876 MiB/s wr, 1.28k op/s 2024-04-26T21:59:37.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:36 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:59:37.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:37 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T21:59:37.907 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:37 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T21:59:37.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:37 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T21:59:38.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:38 smithi055 ceph-mon[29221]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 883 MiB/s wr, 1.34k op/s 2024-04-26T21:59:39.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:38 smithi064 ceph-mon[32765]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 883 MiB/s wr, 1.34k op/s 2024-04-26T21:59:39.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:38 smithi165 ceph-mon[33296]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 883 MiB/s wr, 1.34k op/s 2024-04-26T21:59:41.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:40 smithi064 ceph-mon[32765]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 901 MiB/s wr, 1.58k op/s 2024-04-26T21:59:41.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:40 smithi165 ceph-mon[33296]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 901 MiB/s wr, 1.58k op/s 2024-04-26T21:59:41.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:40 smithi055 ceph-mon[29221]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 901 MiB/s wr, 1.58k op/s 2024-04-26T21:59:42.996 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:42 smithi055 ceph-mon[29221]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 853 MiB/s wr, 1.64k op/s 2024-04-26T21:59:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:42 smithi064 ceph-mon[32765]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 853 MiB/s wr, 1.64k op/s 2024-04-26T21:59:43.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:42 smithi165 ceph-mon[33296]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 853 MiB/s wr, 1.64k op/s 2024-04-26T21:59:45.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:44 smithi064 ceph-mon[32765]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.68k op/s 2024-04-26T21:59:45.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:44 smithi165 ceph-mon[33296]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.68k op/s 2024-04-26T21:59:45.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:44 smithi055 ceph-mon[29221]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.68k op/s 2024-04-26T21:59:47.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:46 smithi165 ceph-mon[33296]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 877 MiB/s wr, 1.64k op/s 2024-04-26T21:59:47.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:46 smithi055 ceph-mon[29221]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 877 MiB/s wr, 1.64k op/s 2024-04-26T21:59:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:46 smithi064 ceph-mon[32765]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 877 MiB/s wr, 1.64k op/s 2024-04-26T21:59:49.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:48 smithi165 ceph-mon[33296]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.64k op/s 2024-04-26T21:59:49.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:48 smithi055 ceph-mon[29221]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.64k op/s 2024-04-26T21:59:49.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:48 smithi064 ceph-mon[32765]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.64k op/s 2024-04-26T21:59:51.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:50 smithi165 ceph-mon[33296]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.72k op/s 2024-04-26T21:59:51.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:50 smithi055 ceph-mon[29221]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.72k op/s 2024-04-26T21:59:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:50 smithi064 ceph-mon[32765]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.72k op/s 2024-04-26T21:59:53.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:52 smithi165 ceph-mon[33296]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 841 MiB/s wr, 1.46k op/s 2024-04-26T21:59:53.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:52 smithi055 ceph-mon[29221]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 841 MiB/s wr, 1.46k op/s 2024-04-26T21:59:53.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:52 smithi064 ceph-mon[32765]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 841 MiB/s wr, 1.46k op/s 2024-04-26T21:59:55.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:54 smithi165 ceph-mon[33296]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 835 MiB/s wr, 1.30k op/s 2024-04-26T21:59:55.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:54 smithi055 ceph-mon[29221]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 835 MiB/s wr, 1.30k op/s 2024-04-26T21:59:55.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:54 smithi064 ceph-mon[32765]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 835 MiB/s wr, 1.30k op/s 2024-04-26T21:59:57.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:56 smithi165 ceph-mon[33296]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.25k op/s 2024-04-26T21:59:57.195 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:56 smithi055 ceph-mon[29221]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.25k op/s 2024-04-26T21:59:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:56 smithi064 ceph-mon[32765]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.25k op/s 2024-04-26T21:59:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 21:59:58 smithi165 ceph-mon[33296]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 856 MiB/s wr, 1.26k op/s 2024-04-26T21:59:59.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 21:59:58 smithi055 ceph-mon[29221]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 856 MiB/s wr, 1.26k op/s 2024-04-26T21:59:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 21:59:58 smithi064 ceph-mon[32765]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 856 MiB/s wr, 1.26k op/s 2024-04-26T22:00:00.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:00:00.677 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:00:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:00 smithi055 ceph-mon[29221]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 877 MiB/s wr, 1.33k op/s 2024-04-26T22:00:01.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:00 smithi055 ceph-mon[29221]: overall HEALTH_OK 2024-04-26T22:00:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:00 smithi064 ceph-mon[32765]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 877 MiB/s wr, 1.33k op/s 2024-04-26T22:00:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:00 smithi064 ceph-mon[32765]: overall HEALTH_OK 2024-04-26T22:00:01.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.264+0000 7fd733fff640 1 -- 172.21.15.55:0/1485829033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a9b0 msgr2=0x7fd73410f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.264+0000 7fd733fff640 1 --2- 172.21.15.55:0/1485829033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a9b0 0x7fd73410f270 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009ca0 tx=0x7fd72c02a8d0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.268+0000 7fd733fff640 1 -- 172.21.15.55:0/1485829033 shutdown_connections 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.268+0000 7fd733fff640 1 --2- 172.21.15.55:0/1485829033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a9b0 0x7fd73410f270 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.268+0000 7fd733fff640 1 --2- 172.21.15.55:0/1485829033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd73410a010 0x7fd73410a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.268+0000 7fd733fff640 1 --2- 172.21.15.55:0/1485829033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734073d70 0x7fd734074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.268+0000 7fd733fff640 1 -- 172.21.15.55:0/1485829033 >> 172.21.15.55:0/1485829033 conn(0x7fd73406f840 msgr2=0x7fd734071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.272+0000 7fd733fff640 1 -- 172.21.15.55:0/1485829033 shutdown_connections 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.273+0000 7fd733fff640 1 -- 172.21.15.55:0/1485829033 wait complete. 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.274+0000 7fd733fff640 1 Processor -- start 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.274+0000 7fd733fff640 1 -- start start 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.275+0000 7fd733fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734073d70 0x7fd734118fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.275+0000 7fd733fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a010 0x7fd734119510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.275+0000 7fd733fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd73410a9b0 0x7fd73411a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.275+0000 7fd733fff640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd734111510 con 0x7fd73410a9b0 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.275+0000 7fd733fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd734111390 con 0x7fd734073d70 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.275+0000 7fd733fff640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd734111690 con 0x7fd73410a010 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.276+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd73410a9b0 0x7fd73411a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.276+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd73410a9b0 0x7fd73411a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46052/0 (socket says 172.21.15.55:46052) 2024-04-26T22:00:01.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.276+0000 7fd7337fe640 1 -- 172.21.15.55:0/1397252582 learned_addr learned my addr 172.21.15.55:0/1397252582 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:01.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.281+0000 7fd732ffd640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734073d70 0x7fd734118fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.281+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a010 0x7fd734119510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.285+0000 7fd7327fc640 1 -- 172.21.15.55:0/1397252582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734073d70 msgr2=0x7fd734118fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.285+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734073d70 0x7fd734118fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.285+0000 7fd7327fc640 1 -- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd73410a9b0 msgr2=0x7fd73411a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.285+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd73410a9b0 0x7fd73411a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.285+0000 7fd7327fc640 1 -- 172.21.15.55:0/1397252582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73411abc0 con 0x7fd73410a010 2024-04-26T22:00:01.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.289+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a010 0x7fd734119510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd72400d6d0 tx=0x7fd72400dad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.291+0000 7fd71bfff640 1 -- 172.21.15.55:0/1397252582 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724017070 con 0x7fd73410a010 2024-04-26T22:00:01.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.291+0000 7fd71bfff640 1 -- 172.21.15.55:0/1397252582 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7240047a0 con 0x7fd73410a010 2024-04-26T22:00:01.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.291+0000 7fd71bfff640 1 -- 172.21.15.55:0/1397252582 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724009a40 con 0x7fd73410a010 2024-04-26T22:00:01.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.291+0000 7fd733fff640 1 -- 172.21.15.55:0/1397252582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73411aeb0 con 0x7fd73410a010 2024-04-26T22:00:01.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.291+0000 7fd733fff640 1 -- 172.21.15.55:0/1397252582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73411b3b0 con 0x7fd73410a010 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.296+0000 7fd73c590740 1 Processor -- start 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.300+0000 7fd73c590740 1 -- start start 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.300+0000 7fd73c590740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a620b0 0x55cbc1a624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.300+0000 7fd73c590740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 0x55cbc1a63700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.300+0000 7fd73c590740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63c40 0x55cbc1a6a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.301+0000 7fd73c590740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cbc19c6d80 con 0x55cbc1a63c40 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.301+0000 7fd73c590740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cbc19808b0 con 0x55cbc1a63280 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.301+0000 7fd73c590740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cbc19c2d40 con 0x55cbc1a620b0 2024-04-26T22:00:01.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.301+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a620b0 0x55cbc1a624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.301+0000 7fd7337fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a620b0 0x55cbc1a624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48740/0 (socket says 172.21.15.55:48740) 2024-04-26T22:00:01.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.301+0000 7fd7337fe640 1 -- 172.21.15.55:0/2813711510 learned_addr learned my addr 172.21.15.55:0/2813711510 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:01.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.304+0000 7fd7337fe640 1 -- 172.21.15.55:0/2813711510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 msgr2=0x55cbc1a63700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:01.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.304+0000 7fd7337fe640 1 --2- 172.21.15.55:0/2813711510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 0x55cbc1a63700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.304+0000 7fd7337fe640 1 -- 172.21.15.55:0/2813711510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63c40 msgr2=0x55cbc1a6a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.304+0000 7fd7337fe640 1 --2- 172.21.15.55:0/2813711510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63c40 0x55cbc1a6a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.304+0000 7fd7337fe640 1 -- 172.21.15.55:0/2813711510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc1951a40 con 0x55cbc1a620b0 2024-04-26T22:00:01.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.305+0000 7fd71bfff640 1 -- 172.21.15.55:0/1397252582 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd724009be0 con 0x7fd73410a010 2024-04-26T22:00:01.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.306+0000 7fd71bfff640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd71c0a3e40 0x7fd71c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.307+0000 7fd71bfff640 1 -- 172.21.15.55:0/1397252582 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd7240cfe00 con 0x7fd73410a010 2024-04-26T22:00:01.309 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.307+0000 7fd732ffd640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd71c0a3e40 0x7fd71c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.309+0000 7fd732ffd640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd71c0a3e40 0x7fd71c0a6300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd72800abc0 tx=0x7fd72800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.309+0000 7fd7337fe640 1 --2- 172.21.15.55:0/2813711510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a620b0 0x55cbc1a624d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009700 tx=0x7fd72c009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa703498cf5d5240 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.312 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.310+0000 7fd7197fa640 1 -- 172.21.15.55:0/2813711510 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c049070 con 0x55cbc1a620b0 2024-04-26T22:00:01.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.313+0000 7fd7197fa640 1 -- 172.21.15.55:0/2813711510 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd72c036040 con 0x55cbc1a620b0 2024-04-26T22:00:01.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.315+0000 7fd73c590740 1 -- 172.21.15.55:0/2813711510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a620b0 msgr2=0x55cbc1a624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.315+0000 7fd73c590740 1 --2- 172.21.15.55:0/2813711510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a620b0 0x55cbc1a624d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009700 tx=0x7fd72c009730 comp rx=0 tx=0).stop 2024-04-26T22:00:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.316+0000 7fd73c590740 1 -- 172.21.15.55:0/2813711510 shutdown_connections 2024-04-26T22:00:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.316+0000 7fd73c590740 1 --2- 172.21.15.55:0/2813711510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63c40 0x55cbc1a6a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.316+0000 7fd73c590740 1 --2- 172.21.15.55:0/2813711510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 0x55cbc1a63700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.316+0000 7fd73c590740 1 --2- 172.21.15.55:0/2813711510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a620b0 0x55cbc1a624d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.316+0000 7fd73c590740 1 -- 172.21.15.55:0/2813711510 >> 172.21.15.55:0/2813711510 conn(0x55cbc1a5d5a0 msgr2=0x55cbc1a5d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:00 smithi165 ceph-mon[33296]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 877 MiB/s wr, 1.33k op/s 2024-04-26T22:00:01.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:00 smithi165 ceph-mon[33296]: overall HEALTH_OK 2024-04-26T22:00:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.320+0000 7fd73c590740 1 -- 172.21.15.55:0/2813711510 shutdown_connections 2024-04-26T22:00:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.320+0000 7fd73c590740 1 -- 172.21.15.55:0/2813711510 wait complete. 2024-04-26T22:00:01.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.321+0000 7fd73c590740 1 -- start start 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.321+0000 7fd73c590740 1 Processor -- start 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.321+0000 7fd73c590740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 0x55cbc1a6c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.321+0000 7fd73c590740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 0x55cbc1a6caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.321+0000 7fd73c590740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a71ad0 0x55cbc1a73ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.321+0000 7fd73c590740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cbc1a6ac90 con 0x55cbc1a71ad0 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.321+0000 7fd73c590740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cbc194dd10 con 0x55cbc1a63280 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.321+0000 7fd73c590740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cbc1a6ae10 con 0x55cbc1a63c40 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.322+0000 7fd73c590740 10 client.0 _ll_register_callbacks cb 0x55cbc197a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.326+0000 7fd7327fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 0x55cbc1a6caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.326+0000 7fd7327fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 0x55cbc1a6caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48756/0 (socket says 172.21.15.55:48756) 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.326+0000 7fd7327fc640 1 -- 172.21.15.55:0/1285874191 learned_addr learned my addr 172.21.15.55:0/1285874191 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:01.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.326+0000 7fd732ffd640 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 0x55cbc1a6c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.328+0000 7fd7337fe640 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a71ad0 0x55cbc1a73ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.330+0000 7fd732ffd640 1 -- 172.21.15.55:0/1285874191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 msgr2=0x55cbc1a6caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.330+0000 7fd732ffd640 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 0x55cbc1a6caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.330+0000 7fd732ffd640 1 -- 172.21.15.55:0/1285874191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a71ad0 msgr2=0x55cbc1a73ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.330+0000 7fd732ffd640 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a71ad0 0x55cbc1a73ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.330+0000 7fd732ffd640 1 -- 172.21.15.55:0/1285874191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc17a27d0 con 0x55cbc1a63280 2024-04-26T22:00:01.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.335+0000 7fd732ffd640 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 0x55cbc1a6c560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55cbc1a5e470 tx=0x7fd728004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79088ac05b7a618a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.337+0000 7fd716ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:00:01.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.337+0000 7fd716ff5640 1 -- 172.21.15.55:0/1285874191 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728016070 con 0x55cbc1a63280 2024-04-26T22:00:01.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.338+0000 7fd73c590740 1 -- 172.21.15.55:0/1285874191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbc19d4990 con 0x55cbc1a63280 2024-04-26T22:00:01.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.338+0000 7fd716ff5640 1 -- 172.21.15.55:0/1285874191 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd72801bc90 con 0x55cbc1a63280 2024-04-26T22:00:01.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.341+0000 7fd716ff5640 1 -- 172.21.15.55:0/1285874191 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd728010e40 con 0x55cbc1a63280 2024-04-26T22:00:01.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.341+0000 7fd73c590740 10 client.25739 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:01.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.341+0000 7fd73c590740 1 -- 172.21.15.55:0/1285874191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cbc16d83f0 con 0x55cbc1a63280 2024-04-26T22:00:01.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.344+0000 7fd716ff5640 1 -- 172.21.15.55:0/1285874191 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cbc19d4990 con 0x55cbc1a63280 2024-04-26T22:00:01.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.344+0000 7fd716ff5640 1 -- 172.21.15.55:0/1285874191 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd728031450 con 0x55cbc1a63280 2024-04-26T22:00:01.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.344+0000 7fd73c590740 10 client.25739 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.344+0000 7fd73c590740 10 client.25739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.344+0000 7fd73c590740 20 client.25739 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.344+0000 7fd73c590740 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cbc1a788b0 0x55cbc1a7acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.344+0000 7fd73c590740 4 client.25739 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:01.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.344+0000 7fd73c590740 1 -- 172.21.15.55:0/1285874191 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbc19ce800 con 0x55cbc1a788b0 2024-04-26T22:00:01.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.348+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cbc1a788b0 0x55cbc1a7acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.359+0000 7fd716ff5640 10 client.25739 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:01.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.359+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cbc1a788b0 0x55cbc1a7acf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912552660df663e9 server_cookie=6f0f10804fce40cf in_seq=0 out_seq=0 2024-04-26T22:00:01.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.363+0000 7fd716ff5640 1 -- 172.21.15.55:0/1285874191 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbc19ce800 con 0x55cbc1a788b0 2024-04-26T22:00:01.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.363+0000 7fd716ff5640 10 client.25739 handle_command_reply: tid=0 2024-04-26T22:00:01.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.363+0000 7fd73c590740 1 client.25739 shutdown 2024-04-26T22:00:01.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.365+0000 7fd73c590740 1 -- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 msgr2=0x55cbc1a6c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.365+0000 7fd73c590740 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 0x55cbc1a6c560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55cbc1a5e470 tx=0x7fd728004730 comp rx=0 tx=0).stop 2024-04-26T22:00:01.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.367+0000 7fd73c590740 1 -- 172.21.15.55:0/1285874191 shutdown_connections 2024-04-26T22:00:01.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.367+0000 7fd73c590740 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cbc1a788b0 0x55cbc1a7acf0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.367+0000 7fd73c590740 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a71ad0 0x55cbc1a73ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.367+0000 7fd73c590740 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 0x55cbc1a6caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.367+0000 7fd73c590740 1 --2- 172.21.15.55:0/1285874191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63280 0x55cbc1a6c560 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.367+0000 7fd73c590740 1 -- 172.21.15.55:0/1285874191 >> 172.21.15.55:0/1285874191 conn(0x55cbc1a5d5a0 msgr2=0x55cbc1a5d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.368+0000 7fd716ff5640 0 client.25739 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:01.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.373+0000 7fd73c590740 1 -- 172.21.15.55:0/1285874191 shutdown_connections 2024-04-26T22:00:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.374+0000 7fd73c590740 1 -- 172.21.15.55:0/1285874191 wait complete. 2024-04-26T22:00:01.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.379+0000 7fd73c590740 20 client.25739 trim_cache size 0 max 16384 2024-04-26T22:00:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.423+0000 7fd73c590740 1 Processor -- start 2024-04-26T22:00:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.423+0000 7fd73c590740 1 -- start start 2024-04-26T22:00:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.423+0000 7fd73c590740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a5d5a0 0x55cbc1a6c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.423+0000 7fd73c590740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a5be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.423+0000 7fd73c590740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63c40 0x55cbc1a5c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.423+0000 7fd7327fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a5be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.424+0000 7fd7327fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a5be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46078/0 (socket says 172.21.15.55:46078) 2024-04-26T22:00:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.424+0000 7fd7327fc640 1 -- 172.21.15.55:0/1155858207 learned_addr learned my addr 172.21.15.55:0/1155858207 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.424+0000 7fd73c590740 1 -- 172.21.15.55:0/1155858207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341c3d20 con 0x55cbc1a63280 2024-04-26T22:00:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.424+0000 7fd73c590740 1 -- 172.21.15.55:0/1155858207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341c15f0 con 0x55cbc1a63c40 2024-04-26T22:00:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.424+0000 7fd73c590740 1 -- 172.21.15.55:0/1155858207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341bf440 con 0x55cbc1a5d5a0 2024-04-26T22:00:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.425+0000 7fd7327fc640 1 -- 172.21.15.55:0/1155858207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a5d5a0 msgr2=0x55cbc1a6c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.425+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1155858207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a5d5a0 0x55cbc1a6c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.425+0000 7fd7327fc640 1 -- 172.21.15.55:0/1155858207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63c40 msgr2=0x55cbc1a5c360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.425+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1155858207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63c40 0x55cbc1a5c360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.425+0000 7fd7327fc640 1 -- 172.21.15.55:0/1155858207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc16980f0 con 0x55cbc1a63280 2024-04-26T22:00:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.425+0000 7fd7327fc640 1 --2- 172.21.15.55:0/1155858207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a5be20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd7240cf010 tx=0x7fd724004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed5c5a5cdef7544 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.426+0000 7fd718ff9640 1 -- 172.21.15.55:0/1155858207 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724020040 con 0x55cbc1a63280 2024-04-26T22:00:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.426+0000 7fd718ff9640 1 -- 172.21.15.55:0/1155858207 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7240ba070 con 0x55cbc1a63280 2024-04-26T22:00:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.426+0000 7fd718ff9640 1 -- 172.21.15.55:0/1155858207 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724098550 con 0x55cbc1a63280 2024-04-26T22:00:01.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 -- 172.21.15.55:0/1155858207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 msgr2=0x55cbc1a5be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 --2- 172.21.15.55:0/1155858207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a5be20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd7240cf010 tx=0x7fd724004f20 comp rx=0 tx=0).stop 2024-04-26T22:00:01.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 -- 172.21.15.55:0/1155858207 shutdown_connections 2024-04-26T22:00:01.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 --2- 172.21.15.55:0/1155858207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a63c40 0x55cbc1a5c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 --2- 172.21.15.55:0/1155858207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a5be20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 --2- 172.21.15.55:0/1155858207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a5d5a0 0x55cbc1a6c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 -- 172.21.15.55:0/1155858207 >> 172.21.15.55:0/1155858207 conn(0x55cbc1a62630 msgr2=0x55cbc1a6c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 -- 172.21.15.55:0/1155858207 shutdown_connections 2024-04-26T22:00:01.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.427+0000 7fd73c590740 1 -- 172.21.15.55:0/1155858207 wait complete. 2024-04-26T22:00:01.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.428+0000 7fd73c590740 1 -- start start 2024-04-26T22:00:01.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.428+0000 7fd73c590740 1 Processor -- start 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd73c590740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a5d5a0 0x55cbc1a87b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd73c590740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a88050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd73c590740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 0x55cbc1a8b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd73c590740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341bfab0 con 0x55cbc1a63280 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd73c590740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341c0f20 con 0x55cbc1a5d5a0 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd73c590740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd7341c0180 con 0x55cbc1a63c40 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd73c590740 10 client.0 _ll_register_callbacks cb 0x7fd72800a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a88050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a88050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46082/0 (socket says 172.21.15.55:46082) 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 -- 172.21.15.55:0/2435207380 learned_addr learned my addr 172.21.15.55:0/2435207380 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:01.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 -- 172.21.15.55:0/2435207380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 msgr2=0x55cbc1a8b070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:01.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 0x55cbc1a8b070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 -- 172.21.15.55:0/2435207380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a5d5a0 msgr2=0x55cbc1a87b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:01.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a5d5a0 0x55cbc1a87b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 -- 172.21.15.55:0/2435207380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc19874b0 con 0x55cbc1a63280 2024-04-26T22:00:01.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd7327fc640 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a88050 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd7240988b0 tx=0x7fd724000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff99d06398382e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.429+0000 7fd716ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:01.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.430+0000 7fd716ff5640 1 -- 172.21.15.55:0/2435207380 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724020040 con 0x55cbc1a63280 2024-04-26T22:00:01.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.430+0000 7fd73c590740 1 -- 172.21.15.55:0/2435207380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7240ba070 con 0x55cbc1a63280 2024-04-26T22:00:01.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.430+0000 7fd716ff5640 1 -- 172.21.15.55:0/2435207380 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd724098550 con 0x55cbc1a63280 2024-04-26T22:00:01.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.430+0000 7fd716ff5640 1 -- 172.21.15.55:0/2435207380 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd7240a4b50 con 0x55cbc1a63280 2024-04-26T22:00:01.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.430+0000 7fd73c590740 10 client.16512 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:01.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.430+0000 7fd73c590740 1 -- 172.21.15.55:0/2435207380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cbc1adea70 con 0x55cbc1a63280 2024-04-26T22:00:01.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.431+0000 7fd716ff5640 1 -- 172.21.15.55:0/2435207380 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7240ba070 con 0x55cbc1a63280 2024-04-26T22:00:01.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.431+0000 7fd716ff5640 1 -- 172.21.15.55:0/2435207380 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd724097c30 con 0x55cbc1a63280 2024-04-26T22:00:01.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.431+0000 7fd73c590740 10 client.16512 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:01.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.431+0000 7fd73c590740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:01.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.431+0000 7fd73c590740 20 client.16512 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:01.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.432+0000 7fd73c590740 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cbc1a71ad0 0x55cbc1ae13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.432+0000 7fd73c590740 4 client.16512 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:00:01.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.432+0000 7fd73c590740 1 -- 172.21.15.55:0/2435207380 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbc19494a0 con 0x55cbc1a71ad0 2024-04-26T22:00:01.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.435+0000 7fd732ffd640 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cbc1a71ad0 0x55cbc1ae13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.439+0000 7fd732ffd640 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cbc1a71ad0 0x55cbc1ae13f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8251b9007d4d0cdd server_cookie=46cacefe3e8d1eb in_seq=0 out_seq=0 2024-04-26T22:00:01.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.439+0000 7fd716ff5640 10 client.16512 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:01.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.441+0000 7fd716ff5640 1 -- 172.21.15.55:0/2435207380 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cbc19494a0 con 0x55cbc1a71ad0 2024-04-26T22:00:01.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.441+0000 7fd716ff5640 10 client.16512 handle_command_reply: tid=0 2024-04-26T22:00:01.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 client.16512 shutdown 2024-04-26T22:00:01.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 -- 172.21.15.55:0/2435207380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 msgr2=0x55cbc1a88050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a88050 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd7240988b0 tx=0x7fd724000bc0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 -- 172.21.15.55:0/2435207380 shutdown_connections 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cbc1a63c40 0x55cbc1a8b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55cbc1a71ad0 0x55cbc1ae13f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cbc1a63280 0x55cbc1a88050 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 --2- 172.21.15.55:0/2435207380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cbc1a5d5a0 0x55cbc1a87b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd716ff5640 0 client.16512 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 -- 172.21.15.55:0/2435207380 >> 172.21.15.55:0/2435207380 conn(0x55cbc1a62630 msgr2=0x55cbc1a604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 -- 172.21.15.55:0/2435207380 shutdown_connections 2024-04-26T22:00:01.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.442+0000 7fd73c590740 1 -- 172.21.15.55:0/2435207380 wait complete. 2024-04-26T22:00:01.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.443+0000 7fd73c590740 20 client.16512 trim_cache size 0 max 16384 2024-04-26T22:00:01.445 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:00:01.445 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:00:01.445 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:00:01.445 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.443+0000 7fd733fff640 1 -- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd71c0a3e40 msgr2=0x7fd71c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.443+0000 7fd733fff640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd71c0a3e40 0x7fd71c0a6300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd72800abc0 tx=0x7fd72800a2d0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.443+0000 7fd733fff640 1 -- 172.21.15.55:0/1397252582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a010 msgr2=0x7fd734119510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.443+0000 7fd733fff640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a010 0x7fd734119510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd72400d6d0 tx=0x7fd72400dad0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.444+0000 7fd733fff640 1 -- 172.21.15.55:0/1397252582 shutdown_connections 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.444+0000 7fd733fff640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd71c0a3e40 0x7fd71c0a6300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.444+0000 7fd733fff640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd73410a9b0 0x7fd73411a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.444+0000 7fd733fff640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd73410a010 0x7fd734119510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.444+0000 7fd733fff640 1 --2- 172.21.15.55:0/1397252582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd734073d70 0x7fd734118fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.444+0000 7fd733fff640 1 -- 172.21.15.55:0/1397252582 >> 172.21.15.55:0/1397252582 conn(0x7fd73406f840 msgr2=0x7fd73410d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.444+0000 7fd733fff640 1 -- 172.21.15.55:0/1397252582 shutdown_connections 2024-04-26T22:00:01.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.444+0000 7fd733fff640 1 -- 172.21.15.55:0/1397252582 wait complete. 2024-04-26T22:00:01.468 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:00:01.469 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:01.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.799+0000 7f513fe72640 1 -- 172.21.15.55:0/2255443580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513810c2a0 msgr2=0x7f513810e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.799+0000 7f513fe72640 1 --2- 172.21.15.55:0/2255443580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513810c2a0 0x7f513810e730 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5134009900 tx=0x7f513402a410 comp rx=0 tx=0).stop 2024-04-26T22:00:01.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.800+0000 7f513fe72640 1 -- 172.21.15.55:0/2255443580 shutdown_connections 2024-04-26T22:00:01.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.800+0000 7f513fe72640 1 --2- 172.21.15.55:0/2255443580 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513810ec70 0x7f513810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.800+0000 7f513fe72640 1 --2- 172.21.15.55:0/2255443580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f513810c2a0 0x7f513810e730 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.800+0000 7f513fe72640 1 --2- 172.21.15.55:0/2255443580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5138109930 0x7f513810bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.800+0000 7f513fe72640 1 -- 172.21.15.55:0/2255443580 >> 172.21.15.55:0/2255443580 conn(0x7f513806f7d0 msgr2=0x7f5138071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:01.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.803+0000 7f513fe72640 1 -- 172.21.15.55:0/2255443580 shutdown_connections 2024-04-26T22:00:01.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.803+0000 7f513fe72640 1 -- 172.21.15.55:0/2255443580 wait complete. 2024-04-26T22:00:01.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.804+0000 7f513fe72640 1 Processor -- start 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.804+0000 7f513fe72640 1 -- start start 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.804+0000 7f513fe72640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5138109930 0x7f51381a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.804+0000 7f513fe72640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513810c2a0 0x7f51381a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.804+0000 7f513fe72640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513810ec70 0x7f51381aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.804+0000 7f513fe72640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f513811d2e0 con 0x7f513810c2a0 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.805+0000 7f513fe72640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f513811d160 con 0x7f5138109930 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.805+0000 7f513fe72640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f513811d460 con 0x7f513810ec70 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.805+0000 7f513e40f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513810ec70 0x7f51381aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.805+0000 7f513e40f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513810ec70 0x7f51381aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48800/0 (socket says 172.21.15.55:48800) 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.805+0000 7f513e40f640 1 -- 172.21.15.55:0/1914171005 learned_addr learned my addr 172.21.15.55:0/1914171005 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.806+0000 7f513e40f640 1 -- 172.21.15.55:0/1914171005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5138109930 msgr2=0x7f51381a7ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.806+0000 7f513e40f640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5138109930 0x7f51381a7ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.806+0000 7f513e40f640 1 -- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513810c2a0 msgr2=0x7f51381a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:01.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.806+0000 7f513e40f640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513810c2a0 0x7f51381a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:01.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.806+0000 7f513e40f640 1 -- 172.21.15.55:0/1914171005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51381af640 con 0x7f513810ec70 2024-04-26T22:00:01.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.807+0000 7f513e40f640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513810ec70 0x7f51381aefd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f513000cab0 tx=0x7f513000cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:01.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.816+0000 7f512effd640 1 -- 172.21.15.55:0/1914171005 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513000ae20 con 0x7f513810ec70 2024-04-26T22:00:01.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.816+0000 7f512effd640 1 -- 172.21.15.55:0/1914171005 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513000f900 con 0x7f513810ec70 2024-04-26T22:00:01.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.818+0000 7f512effd640 1 -- 172.21.15.55:0/1914171005 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130018a50 con 0x7f513810ec70 2024-04-26T22:00:01.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.818+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51381af930 con 0x7f513810ec70 2024-04-26T22:00:01.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.818+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51381afe00 con 0x7f513810ec70 2024-04-26T22:00:01.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.824+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51380722d0 con 0x7f513810ec70 2024-04-26T22:00:01.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.824+0000 7f512effd640 1 -- 172.21.15.55:0/1914171005 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f513000f450 con 0x7f513810ec70 2024-04-26T22:00:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.825+0000 7f512effd640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51240a3c20 0x7f51240a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.826+0000 7f512effd640 1 -- 172.21.15.55:0/1914171005 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f51300d0d30 con 0x7f513810ec70 2024-04-26T22:00:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.829+0000 7f513dc0e640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51240a3c20 0x7f51240a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:01.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.835+0000 7f512effd640 1 -- 172.21.15.55:0/1914171005 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f51300d5070 con 0x7f513810ec70 2024-04-26T22:00:01.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:01.835+0000 7f513dc0e640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51240a3c20 0x7f51240a60e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5128009a60 tx=0x7f5128005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.454+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f513802dad0 con 0x7f513810ec70 2024-04-26T22:00:02.463 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:00:02.464 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:00:02.465 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:00:02.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.462+0000 7f512effd640 1 -- 172.21.15.55:0/1914171005 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f513001d060 con 0x7f513810ec70 2024-04-26T22:00:02.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.467+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51240a3c20 msgr2=0x7f51240a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.467+0000 7f513fe72640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51240a3c20 0x7f51240a60e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5128009a60 tx=0x7f5128005f90 comp rx=0 tx=0).stop 2024-04-26T22:00:02.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513810ec70 msgr2=0x7f51381aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513810ec70 0x7f51381aefd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f513000cab0 tx=0x7f513000cf70 comp rx=0 tx=0).stop 2024-04-26T22:00:02.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 shutdown_connections 2024-04-26T22:00:02.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51240a3c20 0x7f51240a60e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513810ec70 0x7f51381aefd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f513810c2a0 0x7f51381a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 --2- 172.21.15.55:0/1914171005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5138109930 0x7f51381a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 >> 172.21.15.55:0/1914171005 conn(0x7f513806f7d0 msgr2=0x7f5138109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.468+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 shutdown_connections 2024-04-26T22:00:02.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.469+0000 7f513fe72640 1 -- 172.21.15.55:0/1914171005 wait complete. 2024-04-26T22:00:02.498 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:00:02.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.796+0000 7f7f27fff640 1 -- 172.21.15.55:0/1125461458 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20004030 con 0x7f7f3010c2a0 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.797+0000 7f7f3776a640 1 -- 172.21.15.55:0/1125461458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010c2a0 msgr2=0x7f7f3010e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.797+0000 7f7f3776a640 1 --2- 172.21.15.55:0/1125461458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010c2a0 0x7f7f3010e730 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7f2001a910 tx=0x7f7f2003b470 comp rx=0 tx=0).stop 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.797+0000 7f7f3776a640 1 -- 172.21.15.55:0/1125461458 shutdown_connections 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.797+0000 7f7f3776a640 1 --2- 172.21.15.55:0/1125461458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f3010ec70 0x7f7f3010f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.797+0000 7f7f3776a640 1 --2- 172.21.15.55:0/1125461458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010c2a0 0x7f7f3010e730 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.797+0000 7f7f3776a640 1 --2- 172.21.15.55:0/1125461458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f30109930 0x7f7f3010bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.797+0000 7f7f3776a640 1 -- 172.21.15.55:0/1125461458 >> 172.21.15.55:0/1125461458 conn(0x7f7f3006f7c0 msgr2=0x7f7f30071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.798+0000 7f7f3776a640 1 -- 172.21.15.55:0/1125461458 shutdown_connections 2024-04-26T22:00:02.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.798+0000 7f7f3776a640 1 -- 172.21.15.55:0/1125461458 wait complete. 2024-04-26T22:00:02.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.799+0000 7f7f3776a640 1 Processor -- start 2024-04-26T22:00:02.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.799+0000 7f7f3776a640 1 -- start start 2024-04-26T22:00:02.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.800+0000 7f7f3776a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f30109930 0x7f7f3019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.800+0000 7f7f3776a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3010c2a0 0x7f7f3019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.800+0000 7f7f3776a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010ec70 0x7f7f3007ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.800+0000 7f7f3776a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30112b50 con 0x7f7f3010ec70 2024-04-26T22:00:02.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.800+0000 7f7f3776a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f301129d0 con 0x7f7f30109930 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.800+0000 7f7f3776a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30112cd0 con 0x7f7f3010c2a0 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.801+0000 7f7f35d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010ec70 0x7f7f3007ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.801+0000 7f7f35d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010ec70 0x7f7f3007ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46104/0 (socket says 172.21.15.55:46104) 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.801+0000 7f7f35d07640 1 -- 172.21.15.55:0/3427230777 learned_addr learned my addr 172.21.15.55:0/3427230777 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.802+0000 7f7f35d07640 1 -- 172.21.15.55:0/3427230777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3010c2a0 msgr2=0x7f7f3019f770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.802+0000 7f7f35d07640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3010c2a0 0x7f7f3019f770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.802+0000 7f7f35d07640 1 -- 172.21.15.55:0/3427230777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f30109930 msgr2=0x7f7f3019f230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.802+0000 7f7f35d07640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f30109930 0x7f7f3019f230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.802+0000 7f7f35d07640 1 -- 172.21.15.55:0/3427230777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3007b230 con 0x7f7f3010ec70 2024-04-26T22:00:02.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.804+0000 7f7f35d07640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010ec70 0x7f7f3007ab90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9d0 tx=0x7f7f2800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.805+0000 7f7f267fc640 1 -- 172.21.15.55:0/3427230777 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800cd80 con 0x7f7f3010ec70 2024-04-26T22:00:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.806+0000 7f7f267fc640 1 -- 172.21.15.55:0/3427230777 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f28005020 con 0x7f7f3010ec70 2024-04-26T22:00:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.806+0000 7f7f267fc640 1 -- 172.21.15.55:0/3427230777 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280116d0 con 0x7f7f3010ec70 2024-04-26T22:00:02.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.809+0000 7f7f3776a640 1 -- 172.21.15.55:0/3427230777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3007b520 con 0x7f7f3010ec70 2024-04-26T22:00:02.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.809+0000 7f7f3776a640 1 -- 172.21.15.55:0/3427230777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3007b9f0 con 0x7f7f3010ec70 2024-04-26T22:00:02.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.825+0000 7f7f3a122740 1 Processor -- start 2024-04-26T22:00:02.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.826+0000 7f7f267fc640 1 -- 172.21.15.55:0/3427230777 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7f280051d0 con 0x7f7f3010ec70 2024-04-26T22:00:02.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.828+0000 7f7f267fc640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7f1c0a3e10 0x7f7f1c0a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.830+0000 7f7f267fc640 1 -- 172.21.15.55:0/3427230777 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7f280d0ab0 con 0x7f7f3010ec70 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.832+0000 7f7f3a122740 1 -- start start 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.832+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 0x556c33901c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.832+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339029f0 0x556c33902e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.832+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c33909c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.832+0000 7f7f3a122740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556c33865d80 con 0x556c339033b0 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.832+0000 7f7f3a122740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556c3381fa20 con 0x556c339029f0 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.832+0000 7f7f3a122740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556c33861d40 con 0x556c33901820 2024-04-26T22:00:02.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.835+0000 7f7f34d05640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7f1c0a3e10 0x7f7f1c0a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.836+0000 7f7f35d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c33909c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.837+0000 7f7f35d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c33909c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46106/0 (socket says 172.21.15.55:46106) 2024-04-26T22:00:02.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.837+0000 7f7f35d07640 1 -- 172.21.15.55:0/2913984058 learned_addr learned my addr 172.21.15.55:0/2913984058 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:02.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.839+0000 7f7f35506640 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 0x556c33901c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.839+0000 7f7f34d05640 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339029f0 0x556c33902e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.845+0000 7f7f35d07640 1 -- 172.21.15.55:0/2913984058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 msgr2=0x556c33901c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.845+0000 7f7f35d07640 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 0x556c33901c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.845+0000 7f7f35d07640 1 -- 172.21.15.55:0/2913984058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339029f0 msgr2=0x556c33902e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.845+0000 7f7f35d07640 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339029f0 0x556c33902e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.845+0000 7f7f35d07640 1 -- 172.21.15.55:0/2913984058 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3382ac30 con 0x556c339033b0 2024-04-26T22:00:02.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.845+0000 7f7f35d07640 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c33909c40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7f28099070 tx=0x7f7f2809d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728c8da0091b8dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.849+0000 7f7f1b7fe640 1 -- 172.21.15.55:0/2913984058 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280b5020 con 0x556c339033b0 2024-04-26T22:00:02.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.849+0000 7f7f1b7fe640 1 -- 172.21.15.55:0/2913984058 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f280a5880 con 0x556c339033b0 2024-04-26T22:00:02.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.851+0000 7f7f1b7fe640 1 -- 172.21.15.55:0/2913984058 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28098960 con 0x556c339033b0 2024-04-26T22:00:02.853 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:02 smithi055 ceph-mon[29221]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 833 MiB/s wr, 1.25k op/s 2024-04-26T22:00:02.853 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:02 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1914171005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f3a122740 1 -- 172.21.15.55:0/2913984058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 msgr2=0x556c33909c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c33909c40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7f28099070 tx=0x7f7f2809d440 comp rx=0 tx=0).stop 2024-04-26T22:00:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f34d05640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7f1c0a3e10 0x7f7f1c0a62d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f20004ba0 tx=0x7f7f2001a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f3a122740 1 -- 172.21.15.55:0/2913984058 shutdown_connections 2024-04-26T22:00:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c33909c40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339029f0 0x556c33902e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2913984058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 0x556c33901c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f3a122740 1 -- 172.21.15.55:0/2913984058 >> 172.21.15.55:0/2913984058 conn(0x556c33842910 msgr2=0x556c338fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.852+0000 7f7f3a122740 1 -- 172.21.15.55:0/2913984058 shutdown_connections 2024-04-26T22:00:02.855 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.853+0000 7f7f3a122740 1 -- 172.21.15.55:0/2913984058 wait complete. 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.853+0000 7f7f3a122740 1 -- start start 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.853+0000 7f7f3a122740 1 Processor -- start 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.854+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c33901820 0x556c3390bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.854+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 0x556c3390c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.854+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c339131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.854+0000 7f7f3a122740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556c3390a410 con 0x556c339033b0 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.854+0000 7f7f3a122740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556c337ecdc0 con 0x556c33901820 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.854+0000 7f7f3a122740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556c3390a590 con 0x556c339029f0 2024-04-26T22:00:02.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.854+0000 7f7f3a122740 10 client.0 _ll_register_callbacks cb 0x556c33808600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:02.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.856+0000 7f7f35d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c339131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.856+0000 7f7f35d07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c339131b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46120/0 (socket says 172.21.15.55:46120) 2024-04-26T22:00:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.856+0000 7f7f35d07640 1 -- 172.21.15.55:0/3708796903 learned_addr learned my addr 172.21.15.55:0/3708796903 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:02.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.859+0000 7f7f34d05640 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 0x556c3390c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.860+0000 7f7f35506640 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c33901820 0x556c3390bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.862+0000 7f7f35d07640 1 -- 172.21.15.55:0/3708796903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 msgr2=0x556c3390c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.862+0000 7f7f35d07640 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 0x556c3390c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.862+0000 7f7f35d07640 1 -- 172.21.15.55:0/3708796903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c33901820 msgr2=0x556c3390bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.862+0000 7f7f35d07640 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c33901820 0x556c3390bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.862+0000 7f7f35d07640 1 -- 172.21.15.55:0/3708796903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c335773f0 con 0x556c339033b0 2024-04-26T22:00:02.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.863+0000 7f7f35d07640 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c339131b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009fd0 tx=0x7f7f28005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4ee949ee33c2f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.864+0000 7f7f18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:02.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.867+0000 7f7f18ff9640 1 -- 172.21.15.55:0/3708796903 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f280b5070 con 0x556c339033b0 2024-04-26T22:00:02.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.867+0000 7f7f3a122740 1 -- 172.21.15.55:0/3708796903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f280a5880 con 0x556c339033b0 2024-04-26T22:00:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.868+0000 7f7f18ff9640 1 -- 172.21.15.55:0/3708796903 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7f2809c270 con 0x556c339033b0 2024-04-26T22:00:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.868+0000 7f7f18ff9640 1 -- 172.21.15.55:0/3708796903 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7f28098960 con 0x556c339033b0 2024-04-26T22:00:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.870+0000 7f7f3a122740 10 client.16524 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:02.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.870+0000 7f7f3a122740 1 -- 172.21.15.55:0/3708796903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556c336417d0 con 0x556c339033b0 2024-04-26T22:00:02.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.874+0000 7f7f18ff9640 1 -- 172.21.15.55:0/3708796903 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f280a5880 con 0x556c339033b0 2024-04-26T22:00:02.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.877+0000 7f7f18ff9640 1 -- 172.21.15.55:0/3708796903 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7f2800f040 con 0x556c339033b0 2024-04-26T22:00:02.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.877+0000 7f7f3a122740 10 client.16524 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:02.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.878+0000 7f7f3a122740 10 client.16524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:00:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.878+0000 7f7f3a122740 20 client.16524 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.878+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556c33917bf0 0x556c3391a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.878+0000 7f7f3a122740 4 client.16524 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:02.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.878+0000 7f7f3a122740 1 -- 172.21.15.55:0/3708796903 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c33873990 con 0x556c33917bf0 2024-04-26T22:00:02.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.882+0000 7f7f35506640 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556c33917bf0 0x556c3391a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.889+0000 7f7f18ff9640 10 client.16524 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:00:02.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.889+0000 7f7f35506640 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556c33917bf0 0x556c3391a030 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd0a5e503345e7fe server_cookie=af1480f119eb5971 in_seq=0 out_seq=0 2024-04-26T22:00:02.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.893+0000 7f7f18ff9640 1 -- 172.21.15.55:0/3708796903 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c33873990 con 0x556c33917bf0 2024-04-26T22:00:02.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.893+0000 7f7f18ff9640 10 client.16524 handle_command_reply: tid=0 2024-04-26T22:00:02.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.893+0000 7f7f3a122740 1 client.16524 shutdown 2024-04-26T22:00:02.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.898+0000 7f7f3a122740 1 -- 172.21.15.55:0/3708796903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 msgr2=0x556c339131b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.898+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c339131b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009fd0 tx=0x7f7f28005740 comp rx=0 tx=0).stop 2024-04-26T22:00:02.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.902+0000 7f7f3a122740 1 -- 172.21.15.55:0/3708796903 shutdown_connections 2024-04-26T22:00:02.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.902+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556c33917bf0 0x556c3391a030 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.902+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339033b0 0x556c339131b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.902+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 0x556c3390c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.902+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3708796903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c33901820 0x556c3390bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.902+0000 7f7f3a122740 1 -- 172.21.15.55:0/3708796903 >> 172.21.15.55:0/3708796903 conn(0x556c33842910 msgr2=0x556c338fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.905+0000 7f7f18ff9640 0 client.16524 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:00:02.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.905+0000 7f7f3a122740 1 -- 172.21.15.55:0/3708796903 shutdown_connections 2024-04-26T22:00:02.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.909+0000 7f7f3a122740 1 -- 172.21.15.55:0/3708796903 wait complete. 2024-04-26T22:00:02.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.909+0000 7f7f3a122740 20 client.16524 trim_cache size 0 max 16384 2024-04-26T22:00:02.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.978+0000 7f7f3a122740 1 Processor -- start 2024-04-26T22:00:02.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.979+0000 7f7f3a122740 1 -- start start 2024-04-26T22:00:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.979+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c33901820 0x556c339063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.979+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 0x556c33906930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.979+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339033b0 0x556c33913170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.980+0000 7f7f3a122740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7f301c3e00 con 0x556c33901820 2024-04-26T22:00:02.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.980+0000 7f7f3a122740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30084b10 con 0x556c339033b0 2024-04-26T22:00:02.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.980+0000 7f7f3a122740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30082a40 con 0x556c339029f0 2024-04-26T22:00:02.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.981+0000 7f7f35506640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c33901820 0x556c339063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:02.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.981+0000 7f7f35506640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c33901820 0x556c339063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46134/0 (socket says 172.21.15.55:46134) 2024-04-26T22:00:02.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.981+0000 7f7f35506640 1 -- 172.21.15.55:0/3738580262 learned_addr learned my addr 172.21.15.55:0/3738580262 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:02.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.982+0000 7f7f35506640 1 -- 172.21.15.55:0/3738580262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 msgr2=0x556c33906930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.982+0000 7f7f35506640 1 --2- 172.21.15.55:0/3738580262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 0x556c33906930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.982+0000 7f7f35506640 1 -- 172.21.15.55:0/3738580262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339033b0 msgr2=0x556c33913170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:02.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.982+0000 7f7f35506640 1 --2- 172.21.15.55:0/3738580262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339033b0 0x556c33913170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.982+0000 7f7f35506640 1 -- 172.21.15.55:0/3738580262 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3391a6b0 con 0x556c33901820 2024-04-26T22:00:02.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.982+0000 7f7f35506640 1 --2- 172.21.15.55:0/3738580262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c33901820 0x556c339063f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7f301a0730 tx=0x7f7f2c00b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d59cb634f2c268 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:02.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.988+0000 7f7f1affd640 1 -- 172.21.15.55:0/3738580262 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c013d10 con 0x556c33901820 2024-04-26T22:00:02.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.988+0000 7f7f1affd640 1 -- 172.21.15.55:0/3738580262 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f2c01b3b0 con 0x556c33901820 2024-04-26T22:00:02.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.988+0000 7f7f1affd640 1 -- 172.21.15.55:0/3738580262 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c003950 con 0x556c33901820 2024-04-26T22:00:02.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.989+0000 7f7f3a122740 1 -- 172.21.15.55:0/3738580262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c33901820 msgr2=0x556c339063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:02.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.989+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3738580262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c33901820 0x556c339063f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7f301a0730 tx=0x7f7f2c00b570 comp rx=0 tx=0).stop 2024-04-26T22:00:02.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.991+0000 7f7f3a122740 1 -- 172.21.15.55:0/3738580262 shutdown_connections 2024-04-26T22:00:02.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.991+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3738580262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339033b0 0x556c33913170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.991+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3738580262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c339029f0 0x556c33906930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.991+0000 7f7f3a122740 1 --2- 172.21.15.55:0/3738580262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c33901820 0x556c339063f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:02.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.991+0000 7f7f3a122740 1 -- 172.21.15.55:0/3738580262 >> 172.21.15.55:0/3738580262 conn(0x556c33842910 msgr2=0x556c3390bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:02.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.994+0000 7f7f3a122740 1 -- 172.21.15.55:0/3738580262 shutdown_connections 2024-04-26T22:00:02.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.994+0000 7f7f3a122740 1 -- 172.21.15.55:0/3738580262 wait complete. 2024-04-26T22:00:02.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.995+0000 7f7f3a122740 1 -- start start 2024-04-26T22:00:02.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.996+0000 7f7f3a122740 1 Processor -- start 2024-04-26T22:00:02.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.997+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 0x556c3390ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.997+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339029f0 0x556c33910470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.997+0000 7f7f3a122740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339033b0 0x556c3397c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:02.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.997+0000 7f7f3a122740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30082fd0 con 0x556c339029f0 2024-04-26T22:00:02.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.997+0000 7f7f3a122740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f30084440 con 0x556c339033b0 2024-04-26T22:00:02.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.997+0000 7f7f3a122740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f300836a0 con 0x556c33901820 2024-04-26T22:00:02.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.998+0000 7f7f3a122740 10 client.0 _ll_register_callbacks cb 0x7f7f2809d310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:03.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.998+0000 7f7f34d05640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339029f0 0x556c33910470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.998+0000 7f7f34d05640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339029f0 0x556c33910470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46150/0 (socket says 172.21.15.55:46150) 2024-04-26T22:00:03.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.998+0000 7f7f34d05640 1 -- 172.21.15.55:0/2857906863 learned_addr learned my addr 172.21.15.55:0/2857906863 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:03.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.999+0000 7f7f34d05640 1 -- 172.21.15.55:0/2857906863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 msgr2=0x556c3390ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.999+0000 7f7f34d05640 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 0x556c3390ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.999+0000 7f7f34d05640 1 -- 172.21.15.55:0/2857906863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339033b0 msgr2=0x556c3397c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.999+0000 7f7f34d05640 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339033b0 0x556c3397c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:02.999+0000 7f7f34d05640 1 -- 172.21.15.55:0/2857906863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c339134f0 con 0x556c339029f0 2024-04-26T22:00:03.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.000+0000 7f7f34d05640 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339029f0 0x556c33910470 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7f2001a910 tx=0x7f7f2003b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10ce19cc52ecdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.000+0000 7f7f18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:03.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.000+0000 7f7f18ff9640 1 -- 172.21.15.55:0/2857906863 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20005d50 con 0x556c339029f0 2024-04-26T22:00:03.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.001+0000 7f7f18ff9640 1 -- 172.21.15.55:0/2857906863 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f7f20004f00 con 0x556c339029f0 2024-04-26T22:00:03.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.001+0000 7f7f3a122740 1 -- 172.21.15.55:0/2857906863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f2c01b3b0 con 0x556c339029f0 2024-04-26T22:00:03.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.001+0000 7f7f18ff9640 1 -- 172.21.15.55:0/2857906863 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7f20005100 con 0x556c339029f0 2024-04-26T22:00:03.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.002+0000 7f7f3a122740 10 client.16530 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:03.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.002+0000 7f7f3a122740 1 -- 172.21.15.55:0/2857906863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556c3397d7e0 con 0x556c339029f0 2024-04-26T22:00:03.004 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.003+0000 7f7f18ff9640 1 -- 172.21.15.55:0/2857906863 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f20061070 con 0x556c339029f0 2024-04-26T22:00:03.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.003+0000 7f7f18ff9640 1 -- 172.21.15.55:0/2857906863 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7f2004e040 con 0x556c339029f0 2024-04-26T22:00:03.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.003+0000 7f7f3a122740 10 client.16530 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:03.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.003+0000 7f7f3a122740 10 client.16530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:00:03.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.003+0000 7f7f3a122740 20 client.16530 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:03.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.003+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556c3397e210 0x556c33980630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.004+0000 7f7f3a122740 4 client.16530 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:00:03.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.004+0000 7f7f3a122740 1 -- 172.21.15.55:0/2857906863 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7f2c01b3b0 con 0x556c3397e210 2024-04-26T22:00:03.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.006+0000 7f7f35506640 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556c3397e210 0x556c33980630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.014+0000 7f7f35506640 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556c3397e210 0x556c33980630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b57cd8ad591f611 server_cookie=c273881c540f040a in_seq=0 out_seq=0 2024-04-26T22:00:03.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.015+0000 7f7f18ff9640 10 client.16530 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:00:03.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.015+0000 7f7f18ff9640 1 -- 172.21.15.55:0/2857906863 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7f2c01b3b0 con 0x556c3397e210 2024-04-26T22:00:03.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.016+0000 7f7f18ff9640 10 client.16530 handle_command_reply: tid=0 2024-04-26T22:00:03.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.016+0000 7f7f3a122740 1 client.16530 shutdown 2024-04-26T22:00:03.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.017+0000 7f7f3a122740 1 -- 172.21.15.55:0/2857906863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339029f0 msgr2=0x556c33910470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.017+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339029f0 0x556c33910470 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7f2001a910 tx=0x7f7f2003b470 comp rx=0 tx=0).stop 2024-04-26T22:00:03.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.021+0000 7f7f3a122740 1 -- 172.21.15.55:0/2857906863 shutdown_connections 2024-04-26T22:00:03.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.021+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556c3397e210 0x556c33980630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.021+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c339033b0 0x556c3397c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.021+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556c339029f0 0x556c33910470 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.021+0000 7f7f3a122740 1 --2- 172.21.15.55:0/2857906863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c33901820 0x556c3390ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.021+0000 7f7f3a122740 1 -- 172.21.15.55:0/2857906863 >> 172.21.15.55:0/2857906863 conn(0x556c33842910 msgr2=0x556c3390bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.021+0000 7f7f18ff9640 0 client.16530 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:00:03.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.022+0000 7f7f3a122740 1 -- 172.21.15.55:0/2857906863 shutdown_connections 2024-04-26T22:00:03.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.022+0000 7f7f3a122740 1 -- 172.21.15.55:0/2857906863 wait complete. 2024-04-26T22:00:03.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.024+0000 7f7f3a122740 20 client.16530 trim_cache size 0 max 16384 2024-04-26T22:00:03.032 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.035+0000 7f7f3776a640 1 -- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7f1c0a3e10 msgr2=0x7f7f1c0a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.036+0000 7f7f3776a640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7f1c0a3e10 0x7f7f1c0a62d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f20004ba0 tx=0x7f7f2001a5c0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.036+0000 7f7f3776a640 1 -- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010ec70 msgr2=0x7f7f3007ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.036+0000 7f7f3776a640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010ec70 0x7f7f3007ab90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9d0 tx=0x7f7f2800ee90 comp rx=0 tx=0).stop 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.036+0000 7f7f3776a640 1 -- 172.21.15.55:0/3427230777 shutdown_connections 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.036+0000 7f7f3776a640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f7f1c0a3e10 0x7f7f1c0a62d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.036+0000 7f7f3776a640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f3010ec70 0x7f7f3007ab90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.036+0000 7f7f3776a640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f3010c2a0 0x7f7f3019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.036+0000 7f7f3776a640 1 --2- 172.21.15.55:0/3427230777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f30109930 0x7f7f3019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.037+0000 7f7f3776a640 1 -- 172.21.15.55:0/3427230777 >> 172.21.15.55:0/3427230777 conn(0x7f7f3006f7c0 msgr2=0x7f7f3010cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.040+0000 7f7f3776a640 1 -- 172.21.15.55:0/3427230777 shutdown_connections 2024-04-26T22:00:03.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.040+0000 7f7f3776a640 1 -- 172.21.15.55:0/3427230777 wait complete. 2024-04-26T22:00:03.082 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:00:03.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.252+0000 7fd837bc8640 1 -- 172.21.15.55:0/199347374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 msgr2=0x7fd8301081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.252+0000 7fd837bc8640 1 --2- 172.21.15.55:0/199347374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 0x7fd8301081b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fd824009900 tx=0x7fd82402a410 comp rx=0 tx=0).stop 2024-04-26T22:00:03.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.252+0000 7fd837bc8640 1 -- 172.21.15.55:0/199347374 shutdown_connections 2024-04-26T22:00:03.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.252+0000 7fd837bc8640 1 --2- 172.21.15.55:0/199347374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8301086f0 0x7fd83010aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.252+0000 7fd837bc8640 1 --2- 172.21.15.55:0/199347374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 0x7fd8301081b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.252+0000 7fd837bc8640 1 --2- 172.21.15.55:0/199347374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd830103490 0x7fd830105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.252+0000 7fd837bc8640 1 -- 172.21.15.55:0/199347374 >> 172.21.15.55:0/199347374 conn(0x7fd8300fcde0 msgr2=0x7fd8300ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.253+0000 7fd837bc8640 1 -- 172.21.15.55:0/199347374 shutdown_connections 2024-04-26T22:00:03.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.253+0000 7fd837bc8640 1 -- 172.21.15.55:0/199347374 wait complete. 2024-04-26T22:00:03.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.254+0000 7fd837bc8640 1 Processor -- start 2024-04-26T22:00:03.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.254+0000 7fd837bc8640 1 -- start start 2024-04-26T22:00:03.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.254+0000 7fd837bc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd830103490 0x7fd83019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.254+0000 7fd837bc8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 0x7fd83019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.254+0000 7fd837bc8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8301086f0 0x7fd8301a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.254+0000 7fd837bc8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010e4a0 con 0x7fd830105dc0 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.254+0000 7fd837bc8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010e320 con 0x7fd830103490 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.254+0000 7fd837bc8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010e620 con 0x7fd8301086f0 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 0x7fd83019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 0x7fd83019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46170/0 (socket says 172.21.15.55:46170) 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 -- 172.21.15.55:0/534279213 learned_addr learned my addr 172.21.15.55:0/534279213 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 -- 172.21.15.55:0/534279213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8301086f0 msgr2=0x7fd8301a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8301086f0 0x7fd8301a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 -- 172.21.15.55:0/534279213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd830103490 msgr2=0x7fd83019adf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:03.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd830103490 0x7fd83019adf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 -- 172.21.15.55:0/534279213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301a2a10 con 0x7fd830105dc0 2024-04-26T22:00:03.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.255+0000 7fd835163640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 0x7fd83019b330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd824006010 tx=0x7fd8240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.256+0000 7fd82effd640 1 -- 172.21.15.55:0/534279213 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82403f030 con 0x7fd830105dc0 2024-04-26T22:00:03.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.256+0000 7fd82effd640 1 -- 172.21.15.55:0/534279213 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd824003cb0 con 0x7fd830105dc0 2024-04-26T22:00:03.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.256+0000 7fd82effd640 1 -- 172.21.15.55:0/534279213 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824035070 con 0x7fd830105dc0 2024-04-26T22:00:03.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.256+0000 7fd837bc8640 1 -- 172.21.15.55:0/534279213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301a2ca0 con 0x7fd830105dc0 2024-04-26T22:00:03.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.257+0000 7fd837bc8640 1 -- 172.21.15.55:0/534279213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a3140 con 0x7fd830105dc0 2024-04-26T22:00:03.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:02 smithi064 ceph-mon[32765]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 833 MiB/s wr, 1.25k op/s 2024-04-26T22:00:03.261 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:02 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1914171005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:03.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.260+0000 7fd82effd640 1 -- 172.21.15.55:0/534279213 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd824042030 con 0x7fd830105dc0 2024-04-26T22:00:03.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.260+0000 7fd83a9bd740 1 Processor -- start 2024-04-26T22:00:03.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.261+0000 7fd83a9bd740 1 -- start start 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.261+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.261+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882aa9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.261+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 0x559882ab0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.261+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559882a0cd80 con 0x559882aa99f0 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.261+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5598829c6a20 con 0x559882aaa3b0 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.261+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559882a08d40 con 0x559882aa8820 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.262+0000 7fd82effd640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8180a3c40 0x7fd8180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.262+0000 7fd836165640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.262+0000 7fd836165640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48880/0 (socket says 172.21.15.55:48880) 2024-04-26T22:00:03.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.262+0000 7fd836165640 1 -- 172.21.15.55:0/1540654843 learned_addr learned my addr 172.21.15.55:0/1540654843 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:03.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.262+0000 7fd836165640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8180a3c40 0x7fd8180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.262+0000 7fd82effd640 1 -- 172.21.15.55:0/534279213 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd8240f0cd0 con 0x7fd830105dc0 2024-04-26T22:00:03.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.263+0000 7fd835964640 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882aa9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.263+0000 7fd836165640 1 -- 172.21.15.55:0/1540654843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 msgr2=0x559882ab0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.263+0000 7fd836165640 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 0x559882ab0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.263+0000 7fd836165640 1 -- 172.21.15.55:0/1540654843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 msgr2=0x559882aa9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.263+0000 7fd836165640 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882aa9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.263+0000 7fd836165640 1 -- 172.21.15.55:0/1540654843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598829d1c30 con 0x559882aa8820 2024-04-26T22:00:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.263+0000 7fd835964640 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882aa9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.263+0000 7fd836165640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8180a3c40 0x7fd8180a6100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd82000afd0 tx=0x7fd82000a630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.264+0000 7fd836165640 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa8c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd820005fd0 tx=0x7fd820019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ecc6e8203df50b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.264+0000 7fd817fff640 1 -- 172.21.15.55:0/1540654843 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820024070 con 0x559882aa8820 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.264+0000 7fd817fff640 1 -- 172.21.15.55:0/1540654843 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd82000fe40 con 0x559882aa8820 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.264+0000 7fd817fff640 1 -- 172.21.15.55:0/1540654843 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820003850 con 0x559882aa8820 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.266+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1540654843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 msgr2=0x559882aa8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.266+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa8c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd820005fd0 tx=0x7fd820019000 comp rx=0 tx=0).stop 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.266+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1540654843 shutdown_connections 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.266+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 0x559882ab0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.266+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882aa9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.266+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/1540654843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa8c40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.266+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1540654843 >> 172.21.15.55:0/1540654843 conn(0x5598829e9910 msgr2=0x559882aa4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.267+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1540654843 shutdown_connections 2024-04-26T22:00:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.267+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1540654843 wait complete. 2024-04-26T22:00:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.268+0000 7fd83a9bd740 1 -- start start 2024-04-26T22:00:03.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.268+0000 7fd83a9bd740 1 Processor -- start 2024-04-26T22:00:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.271+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882ab2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.271+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa99f0 0x559882ab2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.271+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.272+0000 7fd835964640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882ab2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.272+0000 7fd835163640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa99f0 0x559882ab2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.272+0000 7fd835163640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa99f0 0x559882ab2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48698/0 (socket says 172.21.15.55:48698) 2024-04-26T22:00:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.272+0000 7fd835163640 1 -- 172.21.15.55:0/3800052567 learned_addr learned my addr 172.21.15.55:0/3800052567 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.272+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559882ab1360 con 0x559882aaa3b0 2024-04-26T22:00:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.274+0000 7fd836165640 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.275+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559882993dc0 con 0x559882aa99f0 2024-04-26T22:00:03.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.275+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559882ab14e0 con 0x559882aa8820 2024-04-26T22:00:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.275+0000 7fd83a9bd740 10 client.0 _ll_register_callbacks cb 0x5598829af600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.276+0000 7fd836165640 1 -- 172.21.15.55:0/3800052567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 msgr2=0x559882ab2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.276+0000 7fd836165640 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882ab2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.276+0000 7fd836165640 1 -- 172.21.15.55:0/3800052567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa99f0 msgr2=0x559882ab2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.276+0000 7fd836165640 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa99f0 0x559882ab2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.276+0000 7fd836165640 1 -- 172.21.15.55:0/3800052567 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988271e3f0 con 0x559882aaa3b0 2024-04-26T22:00:03.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.276+0000 7fd836165640 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab9ec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd820005d00 tx=0x7fd820003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84ac42147fd4489 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.276+0000 7fd8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:03.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.277+0000 7fd8157fa640 1 -- 172.21.15.55:0/3800052567 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82002d070 con 0x559882aaa3b0 2024-04-26T22:00:03.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.277+0000 7fd8157fa640 1 -- 172.21.15.55:0/3800052567 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd82001c8e0 con 0x559882aaa3b0 2024-04-26T22:00:03.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.277+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559882a1a990 con 0x559882aaa3b0 2024-04-26T22:00:03.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.277+0000 7fd8157fa640 1 -- 172.21.15.55:0/3800052567 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd820036050 con 0x559882aaa3b0 2024-04-26T22:00:03.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.278+0000 7fd83a9bd740 10 client.16542 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:03.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.278+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5598827e87d0 con 0x559882aaa3b0 2024-04-26T22:00:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.279+0000 7fd8157fa640 1 -- 172.21.15.55:0/3800052567 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559882a1a990 con 0x559882aaa3b0 2024-04-26T22:00:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.279+0000 7fd8157fa640 1 -- 172.21.15.55:0/3800052567 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd820039020 con 0x559882aaa3b0 2024-04-26T22:00:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.279+0000 7fd83a9bd740 10 client.16542 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.279+0000 7fd83a9bd740 10 client.16542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:00:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.279+0000 7fd83a9bd740 20 client.16542 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.279+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x559882abe470 0x559882ac0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.279+0000 7fd83a9bd740 4 client.16542 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:03.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.280+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559882a14800 con 0x559882abe470 2024-04-26T22:00:03.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.281+0000 7fd835964640 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x559882abe470 0x559882ac0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.285+0000 7fd835964640 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x559882abe470 0x559882ac0860 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=491261630727212b server_cookie=a178b5e071f8e591 in_seq=0 out_seq=0 2024-04-26T22:00:03.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.285+0000 7fd8157fa640 10 client.16542 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.287+0000 7fd8157fa640 1 -- 172.21.15.55:0/3800052567 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559882a14800 con 0x559882abe470 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.287+0000 7fd8157fa640 10 client.16542 handle_command_reply: tid=0 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 client.16542 shutdown 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 msgr2=0x559882ab9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab9ec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd820005d00 tx=0x7fd820003dc0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 shutdown_connections 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x559882abe470 0x559882ac0860 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd8157fa640 0 client.16542 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab9ec0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa99f0 0x559882ab2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/3800052567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882ab2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 >> 172.21.15.55:0/3800052567 conn(0x5598829e9910 msgr2=0x559882aa3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 shutdown_connections 2024-04-26T22:00:03.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.288+0000 7fd83a9bd740 1 -- 172.21.15.55:0/3800052567 wait complete. 2024-04-26T22:00:03.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.289+0000 7fd83a9bd740 20 client.16542 trim_cache size 0 max 16384 2024-04-26T22:00:03.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:02 smithi165 ceph-mon[33296]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 833 MiB/s wr, 1.25k op/s 2024-04-26T22:00:03.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:02 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1914171005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:03.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.366+0000 7fd83a9bd740 1 Processor -- start 2024-04-26T22:00:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.374+0000 7fd83a9bd740 1 -- start start 2024-04-26T22:00:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.374+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.375+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882ab9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.375+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 0x559882ab0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.375+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301aa720 con 0x559882aa99f0 2024-04-26T22:00:03.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.375+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301a7ff0 con 0x559882aaa3b0 2024-04-26T22:00:03.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.375+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301a5e40 con 0x559882aa8820 2024-04-26T22:00:03.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.380+0000 7fd836165640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 0x559882ab0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.380+0000 7fd835964640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.380+0000 7fd836165640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 0x559882ab0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48702/0 (socket says 172.21.15.55:48702) 2024-04-26T22:00:03.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.380+0000 7fd836165640 1 -- 172.21.15.55:0/1607188016 learned_addr learned my addr 172.21.15.55:0/1607188016 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:03.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.385+0000 7fd835163640 1 --2- 172.21.15.55:0/1607188016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882ab9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.389+0000 7fd835964640 1 -- 172.21.15.55:0/1607188016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 msgr2=0x559882ab0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.389+0000 7fd835964640 1 --2- 172.21.15.55:0/1607188016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 0x559882ab0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.389+0000 7fd835964640 1 -- 172.21.15.55:0/1607188016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 msgr2=0x559882ab9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.389+0000 7fd835964640 1 --2- 172.21.15.55:0/1607188016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882ab9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.389+0000 7fd835964640 1 -- 172.21.15.55:0/1607188016 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559882ac0bd0 con 0x559882aa8820 2024-04-26T22:00:03.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.397+0000 7fd835964640 1 --2- 172.21.15.55:0/1607188016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa4310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x559882ac1000 tx=0x7fd828003b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=600d97ebb8ad1758 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.400+0000 7fd8177fe640 1 -- 172.21.15.55:0/1607188016 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82801e020 con 0x559882aa8820 2024-04-26T22:00:03.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.400+0000 7fd8177fe640 1 -- 172.21.15.55:0/1607188016 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd82801a770 con 0x559882aa8820 2024-04-26T22:00:03.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.401+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1607188016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 msgr2=0x559882aa4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.401+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/1607188016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa4310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x559882ac1000 tx=0x7fd828003b50 comp rx=0 tx=0).stop 2024-04-26T22:00:03.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.409+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1607188016 shutdown_connections 2024-04-26T22:00:03.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.409+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/1607188016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aaa3b0 0x559882ab0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.409+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/1607188016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aa99f0 0x559882ab9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.409+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/1607188016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa8820 0x559882aa4310 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.409+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1607188016 >> 172.21.15.55:0/1607188016 conn(0x5598829e9910 msgr2=0x559882aa3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.419+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1607188016 shutdown_connections 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.423+0000 7fd83a9bd740 1 -- 172.21.15.55:0/1607188016 wait complete. 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.423+0000 7fd83a9bd740 1 -- start start 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.424+0000 7fd83a9bd740 1 Processor -- start 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.424+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa8820 0x559882ab7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.424+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa99f0 0x559882ab7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.424+0000 7fd83a9bd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.424+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301a64b0 con 0x559882aaa3b0 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.424+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301a7920 con 0x559882aa8820 2024-04-26T22:00:03.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.424+0000 7fd83a9bd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301a6b80 con 0x559882aa99f0 2024-04-26T22:00:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.424+0000 7fd83a9bd740 10 client.0 _ll_register_callbacks cb 0x7fd820022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:03.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.428+0000 7fd835964640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa8820 0x559882ab7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.428+0000 7fd835163640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa99f0 0x559882ab7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.428+0000 7fd836165640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.428+0000 7fd836165640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46214/0 (socket says 172.21.15.55:46214) 2024-04-26T22:00:03.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.428+0000 7fd836165640 1 -- 172.21.15.55:0/773719339 learned_addr learned my addr 172.21.15.55:0/773719339 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:03.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.429+0000 7fd836165640 1 -- 172.21.15.55:0/773719339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa99f0 msgr2=0x559882ab7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.429+0000 7fd836165640 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa99f0 0x559882ab7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.429+0000 7fd836165640 1 -- 172.21.15.55:0/773719339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa8820 msgr2=0x559882ab7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:03.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.429+0000 7fd836165640 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa8820 0x559882ab7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.429+0000 7fd836165640 1 -- 172.21.15.55:0/773719339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559882ab8b00 con 0x559882aaa3b0 2024-04-26T22:00:03.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.429+0000 7fd836165640 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab8510 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd82000a470 tx=0x7fd8200039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9d20a3f720c207 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.430+0000 7fd8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:03.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.430+0000 7fd8157fa640 1 -- 172.21.15.55:0/773719339 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82002d040 con 0x559882aaa3b0 2024-04-26T22:00:03.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.431+0000 7fd83a9bd740 1 -- 172.21.15.55:0/773719339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55988298f610 con 0x559882aaa3b0 2024-04-26T22:00:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.431+0000 7fd8157fa640 1 -- 172.21.15.55:0/773719339 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd820004440 con 0x559882aaa3b0 2024-04-26T22:00:03.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.432+0000 7fd8157fa640 1 -- 172.21.15.55:0/773719339 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd820024040 con 0x559882aaa3b0 2024-04-26T22:00:03.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.436+0000 7fd83a9bd740 10 client.16548 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:03.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.436+0000 7fd83a9bd740 1 -- 172.21.15.55:0/773719339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559882ab9ec0 con 0x559882aaa3b0 2024-04-26T22:00:03.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.442+0000 7fd8157fa640 1 -- 172.21.15.55:0/773719339 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55988298f610 con 0x559882aaa3b0 2024-04-26T22:00:03.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.445+0000 7fd8157fa640 1 -- 172.21.15.55:0/773719339 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd820039070 con 0x559882aaa3b0 2024-04-26T22:00:03.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.445+0000 7fd83a9bd740 10 client.16548 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:03.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.445+0000 7fd83a9bd740 10 client.16548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:00:03.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.445+0000 7fd83a9bd740 20 client.16548 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:03.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.445+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x559882abe470 0x559882b37e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.445+0000 7fd83a9bd740 4 client.16548 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:00:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.445+0000 7fd83a9bd740 1 -- 172.21.15.55:0/773719339 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598829dd240 con 0x559882abe470 2024-04-26T22:00:03.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.452+0000 7fd835964640 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x559882abe470 0x559882b37e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.477+0000 7fd835964640 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x559882abe470 0x559882b37e60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f14fb7ce0c1d5535 server_cookie=fffd32160fd5d6b9 in_seq=0 out_seq=0 2024-04-26T22:00:03.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.477+0000 7fd8157fa640 10 client.16548 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:00:03.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.480+0000 7fd8157fa640 1 -- 172.21.15.55:0/773719339 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598829dd240 con 0x559882abe470 2024-04-26T22:00:03.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.480+0000 7fd8157fa640 10 client.16548 handle_command_reply: tid=0 2024-04-26T22:00:03.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.487+0000 7fd83a9bd740 1 client.16548 shutdown 2024-04-26T22:00:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd83a9bd740 1 -- 172.21.15.55:0/773719339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 msgr2=0x559882ab8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab8510 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd82000a470 tx=0x7fd8200039e0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd83a9bd740 1 -- 172.21.15.55:0/773719339 shutdown_connections 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x559882abe470 0x559882b37e60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559882aaa3b0 0x559882ab8510 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559882aa99f0 0x559882ab7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd83a9bd740 1 --2- 172.21.15.55:0/773719339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559882aa8820 0x559882ab7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd83a9bd740 1 -- 172.21.15.55:0/773719339 >> 172.21.15.55:0/773719339 conn(0x5598829e9910 msgr2=0x559882ab0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.490+0000 7fd8157fa640 0 client.16548 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.491+0000 7fd83a9bd740 1 -- 172.21.15.55:0/773719339 shutdown_connections 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.492+0000 7fd83a9bd740 1 -- 172.21.15.55:0/773719339 wait complete. 2024-04-26T22:00:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.492+0000 7fd83a9bd740 20 client.16548 trim_cache size 0 max 16384 2024-04-26T22:00:03.495 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:00:03.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.496+0000 7fd837bc8640 1 -- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8180a3c40 msgr2=0x7fd8180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.496+0000 7fd837bc8640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8180a3c40 0x7fd8180a6100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd82000afd0 tx=0x7fd82000a630 comp rx=0 tx=0).stop 2024-04-26T22:00:03.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.496+0000 7fd837bc8640 1 -- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 msgr2=0x7fd83019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.496+0000 7fd837bc8640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 0x7fd83019b330 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd824006010 tx=0x7fd8240042f0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.502+0000 7fd837bc8640 1 -- 172.21.15.55:0/534279213 shutdown_connections 2024-04-26T22:00:03.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.502+0000 7fd837bc8640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8180a3c40 0x7fd8180a6100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.502+0000 7fd837bc8640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8301086f0 0x7fd8301a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.502+0000 7fd837bc8640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd830105dc0 0x7fd83019b330 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.502+0000 7fd837bc8640 1 --2- 172.21.15.55:0/534279213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd830103490 0x7fd83019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.503+0000 7fd837bc8640 1 -- 172.21.15.55:0/534279213 >> 172.21.15.55:0/534279213 conn(0x7fd8300fcde0 msgr2=0x7fd830102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.506+0000 7fd837bc8640 1 -- 172.21.15.55:0/534279213 shutdown_connections 2024-04-26T22:00:03.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.506+0000 7fd837bc8640 1 -- 172.21.15.55:0/534279213 wait complete. 2024-04-26T22:00:03.559 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:00:03.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.943+0000 7f73acd90640 1 -- 172.21.15.55:0/1714545090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 msgr2=0x7f73a8108ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.943+0000 7f73acd90640 1 --2- 172.21.15.55:0/1714545090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 0x7f73a8108ec0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f739c01a910 tx=0x7f739c03b470 comp rx=0 tx=0).stop 2024-04-26T22:00:03.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.944+0000 7f73acd90640 1 -- 172.21.15.55:0/1714545090 shutdown_connections 2024-04-26T22:00:03.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.944+0000 7f73acd90640 1 --2- 172.21.15.55:0/1714545090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 0x7f73a8108ec0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.944+0000 7f73acd90640 1 --2- 172.21.15.55:0/1714545090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a8101c70 0x7f73a81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.944+0000 7f73acd90640 1 --2- 172.21.15.55:0/1714545090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73a8100a80 0x7f73a8100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.944+0000 7f73acd90640 1 -- 172.21.15.55:0/1714545090 >> 172.21.15.55:0/1714545090 conn(0x7f73a80fbff0 msgr2=0x7f73a80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:03.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.946+0000 7f73acd90640 1 -- 172.21.15.55:0/1714545090 shutdown_connections 2024-04-26T22:00:03.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.948+0000 7f73acd90640 1 -- 172.21.15.55:0/1714545090 wait complete. 2024-04-26T22:00:03.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.949+0000 7f73acd90640 1 Processor -- start 2024-04-26T22:00:03.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.955+0000 7f73acd90640 1 -- start start 2024-04-26T22:00:03.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.955+0000 7f73acd90640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a8100a80 0x7f73a8190a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.955+0000 7f73acd90640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73a8101c70 0x7f73a8192fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.955+0000 7f73acd90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 0x7f73a81954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.955+0000 7f73acd90640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f73a806ea20 con 0x7f73a8100a80 2024-04-26T22:00:03.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.955+0000 7f73acd90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73a806e8a0 con 0x7f73a8102630 2024-04-26T22:00:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.955+0000 7f73acd90640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73a806eba0 con 0x7f73a8101c70 2024-04-26T22:00:03.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.956+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 0x7f73a81954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.956+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 0x7f73a81954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48732/0 (socket says 172.21.15.55:48732) 2024-04-26T22:00:03.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.956+0000 7f73a7fff640 1 -- 172.21.15.55:0/1317975804 learned_addr learned my addr 172.21.15.55:0/1317975804 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:03.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.956+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73a8101c70 0x7f73a8192fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.958+0000 7f73a7fff640 1 -- 172.21.15.55:0/1317975804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73a8101c70 msgr2=0x7f73a8192fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.958+0000 7f73a7fff640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73a8101c70 0x7f73a8192fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.958+0000 7f73a7fff640 1 -- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a8100a80 msgr2=0x7f73a8190a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.958+0000 7f73a7fff640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a8100a80 0x7f73a8190a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.958+0000 7f73a7fff640 1 -- 172.21.15.55:0/1317975804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a8195db0 con 0x7f73a8102630 2024-04-26T22:00:03.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.959+0000 7f73a7fff640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 0x7f73a81954f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f739c004a30 tx=0x7f739c004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:03.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.967+0000 7f73a4ff9640 1 -- 172.21.15.55:0/1317975804 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c04f070 con 0x7f73a8102630 2024-04-26T22:00:03.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.967+0000 7f73a4ff9640 1 -- 172.21.15.55:0/1317975804 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c0409c0 con 0x7f73a8102630 2024-04-26T22:00:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.969+0000 7f73a4ff9640 1 -- 172.21.15.55:0/1317975804 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c046030 con 0x7f73a8102630 2024-04-26T22:00:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.970+0000 7f73acd90640 1 -- 172.21.15.55:0/1317975804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a8196040 con 0x7f73a8102630 2024-04-26T22:00:03.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.970+0000 7f73acd90640 1 -- 172.21.15.55:0/1317975804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a81964b0 con 0x7f73a8102630 2024-04-26T22:00:03.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.976+0000 7f73b0de0740 1 Processor -- start 2024-04-26T22:00:03.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.990+0000 7f73b0de0740 1 -- start start 2024-04-26T22:00:03.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.991+0000 7f73a4ff9640 1 -- 172.21.15.55:0/1317975804 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f739c052070 con 0x7f73a8102630 2024-04-26T22:00:03.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.991+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fca00 0x56109a6fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.991+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fdbd0 0x56109a6fe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.991+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fe590 0x56109a704e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:03.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.991+0000 7f73b0de0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56109a660d80 con 0x56109a6fca00 2024-04-26T22:00:03.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.991+0000 7f73b0de0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56109a61aa20 con 0x56109a6fdbd0 2024-04-26T22:00:03.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.991+0000 7f73b0de0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56109a65cd40 con 0x56109a6fe590 2024-04-26T22:00:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.992+0000 7f73a4ff9640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73900a3c20 0x7f73900a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.993+0000 7f73a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fdbd0 0x56109a6fe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.993+0000 7f73a6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fca00 0x56109a6fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.993+0000 7f73a6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fca00 0x56109a6fce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46248/0 (socket says 172.21.15.55:46248) 2024-04-26T22:00:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.993+0000 7f73a4ff9640 1 -- 172.21.15.55:0/1317975804 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f739c1016e0 con 0x7f73a8102630 2024-04-26T22:00:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.993+0000 7f73a6ffd640 1 -- 172.21.15.55:0/3874753429 learned_addr learned my addr 172.21.15.55:0/3874753429 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.995+0000 7f73a7fff640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73900a3c20 0x7f73900a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:03.996+0000 7f73a7fff640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73900a3c20 0x7f73900a60e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f739c005ab0 tx=0x7f739c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.005+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/3874753429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fe590 0x56109a704e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.006+0000 7f73a6ffd640 1 -- 172.21.15.55:0/3874753429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fe590 msgr2=0x56109a704e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.006+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/3874753429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fe590 0x56109a704e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.006+0000 7f73a6ffd640 1 -- 172.21.15.55:0/3874753429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fdbd0 msgr2=0x56109a6fe050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.006+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/3874753429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fdbd0 0x56109a6fe050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.006+0000 7f73a6ffd640 1 -- 172.21.15.55:0/3874753429 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109a625c30 con 0x56109a6fca00 2024-04-26T22:00:04.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.012+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/3874753429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fca00 0x56109a6fce20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7398009f90 tx=0x7f73980098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734b6aadeb659ece server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.019+0000 7f738dffb640 1 -- 172.21.15.55:0/3874753429 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398015150 con 0x56109a6fca00 2024-04-26T22:00:04.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.019+0000 7f738dffb640 1 -- 172.21.15.55:0/3874753429 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73980119d0 con 0x56109a6fca00 2024-04-26T22:00:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.020+0000 7f73b0de0740 1 -- 172.21.15.55:0/3874753429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fca00 msgr2=0x56109a6fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.020+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3874753429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fca00 0x56109a6fce20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7398009f90 tx=0x7f73980098d0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.024+0000 7f73b0de0740 1 -- 172.21.15.55:0/3874753429 shutdown_connections 2024-04-26T22:00:04.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.024+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3874753429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fe590 0x56109a704e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.024+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3874753429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fdbd0 0x56109a6fe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.024+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3874753429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fca00 0x56109a6fce20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.024+0000 7f73b0de0740 1 -- 172.21.15.55:0/3874753429 >> 172.21.15.55:0/3874753429 conn(0x56109a63d910 msgr2=0x56109a6f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.025+0000 7f73b0de0740 1 -- 172.21.15.55:0/3874753429 shutdown_connections 2024-04-26T22:00:04.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.025+0000 7f73b0de0740 1 -- 172.21.15.55:0/3874753429 wait complete. 2024-04-26T22:00:04.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.026+0000 7f73b0de0740 1 -- start start 2024-04-26T22:00:04.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.026+0000 7f73b0de0740 1 Processor -- start 2024-04-26T22:00:04.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.027+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fca00 0x56109a706cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.027+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fdbd0 0x56109a707230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.027+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fe590 0x56109a70e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.027+0000 7f73b0de0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56109a7055f0 con 0x56109a6fdbd0 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.027+0000 7f73b0de0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56109a5e7dc0 con 0x56109a6fe590 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.027+0000 7f73b0de0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56109a705770 con 0x56109a6fca00 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.027+0000 7f73b0de0740 10 client.0 _ll_register_callbacks cb 0x56109a603600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.028+0000 7f73a77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fdbd0 0x56109a707230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.028+0000 7f73a77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fdbd0 0x56109a707230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46258/0 (socket says 172.21.15.55:46258) 2024-04-26T22:00:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.028+0000 7f73a77fe640 1 -- 172.21.15.55:0/2764670188 learned_addr learned my addr 172.21.15.55:0/2764670188 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.029+0000 7f73a77fe640 1 -- 172.21.15.55:0/2764670188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fca00 msgr2=0x56109a706cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.029+0000 7f73a77fe640 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fca00 0x56109a706cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.029+0000 7f73a77fe640 1 -- 172.21.15.55:0/2764670188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fe590 msgr2=0x56109a70e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.029+0000 7f73a77fe640 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fe590 0x56109a70e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.029+0000 7f73a77fe640 1 -- 172.21.15.55:0/2764670188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109a3723f0 con 0x56109a6fdbd0 2024-04-26T22:00:04.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.030+0000 7f73a77fe640 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fdbd0 0x56109a707230 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f73a000a9f0 tx=0x7f73a000aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab86c1ba0a18eae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.035+0000 7f7386ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:04.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.035+0000 7f7386ffd640 1 -- 172.21.15.55:0/2764670188 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0018020 con 0x56109a6fdbd0 2024-04-26T22:00:04.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.035+0000 7f73b0de0740 1 -- 172.21.15.55:0/2764670188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73980119d0 con 0x56109a6fdbd0 2024-04-26T22:00:04.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.035+0000 7f7386ffd640 1 -- 172.21.15.55:0/2764670188 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f73a000eb70 con 0x56109a6fdbd0 2024-04-26T22:00:04.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.037+0000 7f7386ffd640 1 -- 172.21.15.55:0/2764670188 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f73a000ed70 con 0x56109a6fdbd0 2024-04-26T22:00:04.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.037+0000 7f73b0de0740 10 client.16557 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:04.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.037+0000 7f73b0de0740 1 -- 172.21.15.55:0/2764670188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56109a43c7d0 con 0x56109a6fdbd0 2024-04-26T22:00:04.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.048+0000 7f7386ffd640 1 -- 172.21.15.55:0/2764670188 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73980119d0 con 0x56109a6fdbd0 2024-04-26T22:00:04.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.050+0000 7f7386ffd640 1 -- 172.21.15.55:0/2764670188 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f73a0025020 con 0x56109a6fdbd0 2024-04-26T22:00:04.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.051+0000 7f73b0de0740 10 client.16557 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:04.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.051+0000 7f73b0de0740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:04.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.051+0000 7f73b0de0740 20 client.16557 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:04.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.051+0000 7f73b0de0740 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56109a713080 0x56109a7154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.051+0000 7f73b0de0740 4 client.16557 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:04.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.051+0000 7f73b0de0740 1 -- 172.21.15.55:0/2764670188 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56109a66e990 con 0x56109a713080 2024-04-26T22:00:04.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.052+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56109a713080 0x56109a7154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.056+0000 7f7386ffd640 10 client.16557 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:04.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.056+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56109a713080 0x56109a7154c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e5871d6080f14a server_cookie=a737b277d66f75c1 in_seq=0 out_seq=0 2024-04-26T22:00:04.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.065+0000 7f7386ffd640 1 -- 172.21.15.55:0/2764670188 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56109a66e990 con 0x56109a713080 2024-04-26T22:00:04.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.065+0000 7f7386ffd640 10 client.16557 handle_command_reply: tid=0 2024-04-26T22:00:04.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.065+0000 7f73b0de0740 1 client.16557 shutdown 2024-04-26T22:00:04.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.071+0000 7f73b0de0740 1 -- 172.21.15.55:0/2764670188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fdbd0 msgr2=0x56109a707230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.071+0000 7f73b0de0740 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fdbd0 0x56109a707230 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f73a000a9f0 tx=0x7f73a000aeb0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.073+0000 7f73b0de0740 1 -- 172.21.15.55:0/2764670188 shutdown_connections 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.073+0000 7f73b0de0740 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fe590 0x56109a70e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.073+0000 7f73b0de0740 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56109a713080 0x56109a7154c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.073+0000 7f73b0de0740 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fdbd0 0x56109a707230 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.073+0000 7f73b0de0740 1 --2- 172.21.15.55:0/2764670188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fca00 0x56109a706cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.073+0000 7f73b0de0740 1 -- 172.21.15.55:0/2764670188 >> 172.21.15.55:0/2764670188 conn(0x56109a63d910 msgr2=0x56109a6f7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.073+0000 7f7386ffd640 0 client.16557 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.075+0000 7f73b0de0740 1 -- 172.21.15.55:0/2764670188 shutdown_connections 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.076+0000 7f73b0de0740 1 -- 172.21.15.55:0/2764670188 wait complete. 2024-04-26T22:00:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.076+0000 7f73b0de0740 20 client.16557 trim_cache size 0 max 16384 2024-04-26T22:00:04.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.112+0000 7f73b0de0740 1 Processor -- start 2024-04-26T22:00:04.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.113+0000 7f73b0de0740 1 -- start start 2024-04-26T22:00:04.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.113+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a706770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.113+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 0x56109a6fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73b0de0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81b29c0 con 0x56109a6fe590 2024-04-26T22:00:04.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73b0de0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81b0290 con 0x56109a6fca00 2024-04-26T22:00:04.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a706770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a706770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48768/0 (socket says 172.21.15.55:48768) 2024-04-26T22:00:04.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73a6ffd640 1 -- 172.21.15.55:0/1263297156 learned_addr learned my addr 172.21.15.55:0/1263297156 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:04.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73a77fe640 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73b0de0740 1 -- 172.21.15.55:0/1263297156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81ae190 con 0x56109a6fdbd0 2024-04-26T22:00:04.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.114+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 0x56109a6fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.116+0000 7f73a6ffd640 1 -- 172.21.15.55:0/1263297156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 msgr2=0x56109a70e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.116+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.116+0000 7f73a6ffd640 1 -- 172.21.15.55:0/1263297156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 msgr2=0x56109a6fb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.116+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 0x56109a6fb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.116+0000 7f73a6ffd640 1 -- 172.21.15.55:0/1263297156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109a6f83c0 con 0x56109a6fca00 2024-04-26T22:00:04.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.120+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a706770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f739801d000 tx=0x7f7398015530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f6c950025a4c2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.123+0000 7f738d7fa640 1 -- 172.21.15.55:0/1263297156 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398015200 con 0x56109a6fca00 2024-04-26T22:00:04.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.123+0000 7f738d7fa640 1 -- 172.21.15.55:0/1263297156 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7398026070 con 0x56109a6fca00 2024-04-26T22:00:04.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.123+0000 7f738d7fa640 1 -- 172.21.15.55:0/1263297156 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739802a6d0 con 0x56109a6fca00 2024-04-26T22:00:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.127+0000 7f73b0de0740 1 -- 172.21.15.55:0/1263297156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 msgr2=0x56109a706770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.127+0000 7f73b0de0740 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a706770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f739801d000 tx=0x7f7398015530 comp rx=0 tx=0).stop 2024-04-26T22:00:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.127+0000 7f73b0de0740 1 -- 172.21.15.55:0/1263297156 shutdown_connections 2024-04-26T22:00:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.127+0000 7f73b0de0740 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 0x56109a6fb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.127+0000 7f73b0de0740 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.127+0000 7f73b0de0740 1 --2- 172.21.15.55:0/1263297156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a706770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.127+0000 7f73b0de0740 1 -- 172.21.15.55:0/1263297156 >> 172.21.15.55:0/1263297156 conn(0x56109a63d910 msgr2=0x56109a701430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.130+0000 7f73b0de0740 1 -- 172.21.15.55:0/1263297156 shutdown_connections 2024-04-26T22:00:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.131+0000 7f73b0de0740 1 -- 172.21.15.55:0/1263297156 wait complete. 2024-04-26T22:00:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.131+0000 7f73b0de0740 1 -- start start 2024-04-26T22:00:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.132+0000 7f73b0de0740 1 Processor -- start 2024-04-26T22:00:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.132+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a70aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.132+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.132+0000 7f73b0de0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 0x56109a777630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.132+0000 7f73b0de0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81ae750 con 0x56109a6fe590 2024-04-26T22:00:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.132+0000 7f73b0de0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81afbc0 con 0x56109a6fca00 2024-04-26T22:00:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.132+0000 7f73b0de0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73a81aee20 con 0x56109a6fdbd0 2024-04-26T22:00:04.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.132+0000 7f73b0de0740 10 client.0 _ll_register_callbacks cb 0x7f73a0009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:04.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.136+0000 7f73a77fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.136+0000 7f73a77fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48968/0 (socket says 172.21.15.55:48968) 2024-04-26T22:00:04.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.136+0000 7f73a77fe640 1 -- 172.21.15.55:0/3789406625 learned_addr learned my addr 172.21.15.55:0/3789406625 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:04.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.137+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 0x56109a777630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.137+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a70aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.141+0000 7f73a77fe640 1 -- 172.21.15.55:0/3789406625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 msgr2=0x56109a70aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.141+0000 7f73a77fe640 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a70aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.141+0000 7f73a77fe640 1 -- 172.21.15.55:0/3789406625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 msgr2=0x56109a777630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.141+0000 7f73a77fe640 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 0x56109a777630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.141+0000 7f73a77fe640 1 -- 172.21.15.55:0/3789406625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109a70ba70 con 0x56109a6fdbd0 2024-04-26T22:00:04.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.145+0000 7f73a77fe640 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70b430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x56109a7081d0 tx=0x7f73a001da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b5020ebe609927 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:04.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.145+0000 7f7386ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:00:04.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.149+0000 7f7386ffd640 1 -- 172.21.15.55:0/3789406625 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0018040 con 0x56109a6fdbd0 2024-04-26T22:00:04.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.149+0000 7f73b0de0740 1 -- 172.21.15.55:0/3789406625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7398026070 con 0x56109a6fdbd0 2024-04-26T22:00:04.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.149+0000 7f7386ffd640 1 -- 172.21.15.55:0/3789406625 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f73a00263f0 con 0x56109a6fdbd0 2024-04-26T22:00:04.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.154+0000 7f7386ffd640 1 -- 172.21.15.55:0/3789406625 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f73a0007000 con 0x56109a6fdbd0 2024-04-26T22:00:04.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.155+0000 7f73b0de0740 10 client.25780 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:04.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.155+0000 7f73b0de0740 1 -- 172.21.15.55:0/3789406625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56109a778ca0 con 0x56109a6fdbd0 2024-04-26T22:00:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.158+0000 7f7386ffd640 1 -- 172.21.15.55:0/3789406625 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7398026070 con 0x56109a6fdbd0 2024-04-26T22:00:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.158+0000 7f7386ffd640 1 -- 172.21.15.55:0/3789406625 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f73a0025070 con 0x56109a6fdbd0 2024-04-26T22:00:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.158+0000 7f73b0de0740 10 client.25780 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.158+0000 7f73b0de0740 10 client.25780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.158+0000 7f73b0de0740 20 client.25780 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.158+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56109a779630 0x56109a77ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.158+0000 7f73b0de0740 4 client.25780 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:00:04.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.158+0000 7f73b0de0740 1 -- 172.21.15.55:0/3789406625 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56109a668800 con 0x56109a779630 2024-04-26T22:00:04.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.160+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56109a779630 0x56109a77ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.166+0000 7f73a6ffd640 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56109a779630 0x56109a77ba20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557b34aa4191d711 server_cookie=ec37dfbcdfaafa4 in_seq=0 out_seq=0 2024-04-26T22:00:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.166+0000 7f7386ffd640 10 client.25780 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:04.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.170+0000 7f7386ffd640 1 -- 172.21.15.55:0/3789406625 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56109a668800 con 0x56109a779630 2024-04-26T22:00:04.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.170+0000 7f7386ffd640 10 client.25780 handle_command_reply: tid=0 2024-04-26T22:00:04.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.170+0000 7f73b0de0740 1 client.25780 shutdown 2024-04-26T22:00:04.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 -- 172.21.15.55:0/3789406625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 msgr2=0x56109a70b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70b430 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x56109a7081d0 tx=0x7f73a001da70 comp rx=0 tx=0).stop 2024-04-26T22:00:04.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 -- 172.21.15.55:0/3789406625 shutdown_connections 2024-04-26T22:00:04.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56109a779630 0x56109a77ba20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56109a6fe590 0x56109a777630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56109a6fdbd0 0x56109a70b430 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 --2- 172.21.15.55:0/3789406625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56109a6fca00 0x56109a70aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 -- 172.21.15.55:0/3789406625 >> 172.21.15.55:0/3789406625 conn(0x56109a63d910 msgr2=0x56109a6f6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f7386ffd640 0 client.25780 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:04.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 -- 172.21.15.55:0/3789406625 shutdown_connections 2024-04-26T22:00:04.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.171+0000 7f73b0de0740 1 -- 172.21.15.55:0/3789406625 wait complete. 2024-04-26T22:00:04.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.172+0000 7f73b0de0740 20 client.25780 trim_cache size 0 max 16384 2024-04-26T22:00:04.174 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:00:04.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 -- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73900a3c20 msgr2=0x7f73900a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73900a3c20 0x7f73900a60e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f739c005ab0 tx=0x7f739c005fd0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 -- 172.21.15.55:0/1317975804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 msgr2=0x7f73a81954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:04.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 0x7f73a81954f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f739c004a30 tx=0x7f739c004a60 comp rx=0 tx=0).stop 2024-04-26T22:00:04.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 -- 172.21.15.55:0/1317975804 shutdown_connections 2024-04-26T22:00:04.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73900a3c20 0x7f73900a60e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a8102630 0x7f73a81954f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73a8101c70 0x7f73a8192fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 --2- 172.21.15.55:0/1317975804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73a8100a80 0x7f73a8190a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:04.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 -- 172.21.15.55:0/1317975804 >> 172.21.15.55:0/1317975804 conn(0x7f73a80fbff0 msgr2=0x7f73a8071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:04.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 -- 172.21.15.55:0/1317975804 shutdown_connections 2024-04-26T22:00:04.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:04.174+0000 7f73acd90640 1 -- 172.21.15.55:0/1317975804 wait complete. 2024-04-26T22:00:05.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:04 smithi055 ceph-mon[29221]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 MiB/s wr, 1.32k op/s 2024-04-26T22:00:05.208 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:00:05.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:05.209 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:00:05.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:04 smithi064 ceph-mon[32765]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 MiB/s wr, 1.32k op/s 2024-04-26T22:00:05.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:04 smithi165 ceph-mon[33296]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 MiB/s wr, 1.32k op/s 2024-04-26T22:00:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3594188803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe224109930 msgr2=0x7fe224109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3594188803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe224109930 0x7fe224109d50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe210009930 tx=0x7fe21002a480 comp rx=0 tx=0).stop 2024-04-26T22:00:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3594188803 shutdown_connections 2024-04-26T22:00:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3594188803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe2241111b0 0x7fe2241135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3594188803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe22410a290 0x7fe224110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3594188803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe224109930 0x7fe224109d50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3594188803 >> 172.21.15.55:0/3594188803 conn(0x7fe22406f7c0 msgr2=0x7fe224071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3594188803 shutdown_connections 2024-04-26T22:00:05.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.514+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3594188803 wait complete. 2024-04-26T22:00:05.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.515+0000 7fe22ae8c640 1 Processor -- start 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.515+0000 7fe22ae8c640 1 -- start start 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe22ae8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22410a290 0x7fe2241a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe22ae8c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2241111b0 0x7fe2241a8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe22ae8c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe2241ad3a0 0x7fe2241af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe22ae8c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe224114bd0 con 0x7fe2241111b0 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe22ae8c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe224114a50 con 0x7fe22410a290 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe22ae8c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe224114d50 con 0x7fe2241ad3a0 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe223fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2241111b0 0x7fe2241a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe223fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2241111b0 0x7fe2241a8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46320/0 (socket says 172.21.15.55:46320) 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe223fff640 1 -- 172.21.15.55:0/3121006588 learned_addr learned my addr 172.21.15.55:0/3121006588 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe229429640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe2241ad3a0 0x7fe2241af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe223fff640 1 -- 172.21.15.55:0/3121006588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe2241ad3a0 msgr2=0x7fe2241af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.516+0000 7fe223fff640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe2241ad3a0 0x7fe2241af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.517+0000 7fe223fff640 1 -- 172.21.15.55:0/3121006588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22410a290 msgr2=0x7fe2241a7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.517+0000 7fe223fff640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22410a290 0x7fe2241a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.517+0000 7fe223fff640 1 -- 172.21.15.55:0/3121006588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22419f340 con 0x7fe2241111b0 2024-04-26T22:00:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.517+0000 7fe223fff640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2241111b0 0x7fe2241a8340 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe21800b470 tx=0x7fe21800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.518+0000 7fe221ffb640 1 -- 172.21.15.55:0/3121006588 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218015020 con 0x7fe2241111b0 2024-04-26T22:00:05.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.518+0000 7fe221ffb640 1 -- 172.21.15.55:0/3121006588 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe218011d40 con 0x7fe2241111b0 2024-04-26T22:00:05.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.518+0000 7fe221ffb640 1 -- 172.21.15.55:0/3121006588 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21800f340 con 0x7fe2241111b0 2024-04-26T22:00:05.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.518+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3121006588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22419f630 con 0x7fe2241111b0 2024-04-26T22:00:05.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.519+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3121006588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22419fbb0 con 0x7fe2241111b0 2024-04-26T22:00:05.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.522+0000 7fe22dba8740 1 Processor -- start 2024-04-26T22:00:05.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.523+0000 7fe22dba8740 1 -- start start 2024-04-26T22:00:05.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.523+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64959cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.523+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 0x55fa6495eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.523+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495f410 0x55fa649618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.523+0000 7fe22dba8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fa648beb00 con 0x55fa6495f410 2024-04-26T22:00:05.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.524+0000 7fe229429640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64959cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.524+0000 7fe229429640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64959cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48996/0 (socket says 172.21.15.55:48996) 2024-04-26T22:00:05.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.524+0000 7fe229429640 1 -- 172.21.15.55:0/76693611 learned_addr learned my addr 172.21.15.55:0/76693611 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:05.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.524+0000 7fe22dba8740 1 -- 172.21.15.55:0/76693611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa6485a920 con 0x55fa6495a2a0 2024-04-26T22:00:05.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.524+0000 7fe22dba8740 1 -- 172.21.15.55:0/76693611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa648baac0 con 0x55fa649598b0 2024-04-26T22:00:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.525+0000 7fe228c28640 1 --2- 172.21.15.55:0/76693611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 0x55fa6495eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.526+0000 7fe221ffb640 1 -- 172.21.15.55:0/3121006588 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe218018020 con 0x7fe2241111b0 2024-04-26T22:00:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.526+0000 7fe229429640 1 -- 172.21.15.55:0/76693611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 msgr2=0x55fa6495eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.526+0000 7fe229429640 1 --2- 172.21.15.55:0/76693611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 0x55fa6495eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.526+0000 7fe229429640 1 -- 172.21.15.55:0/76693611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495f410 msgr2=0x55fa649618a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:05.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.527+0000 7fe221ffb640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe2140a3cb0 0x7fe2140a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.528+0000 7fe221ffb640 1 -- 172.21.15.55:0/3121006588 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe2180cff60 con 0x7fe2241111b0 2024-04-26T22:00:05.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.530+0000 7fe229429640 1 --2- 172.21.15.55:0/76693611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495f410 0x55fa649618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.530+0000 7fe229429640 1 -- 172.21.15.55:0/76693611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa646987e0 con 0x55fa649598b0 2024-04-26T22:00:05.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.534+0000 7fe229429640 1 --2- 172.21.15.55:0/76693611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64959cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009f30 tx=0x7fe21c00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc9cebff5223cb9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.536+0000 7fe228c28640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe2140a3cb0 0x7fe2140a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.537+0000 7fe20effd640 1 -- 172.21.15.55:0/76693611 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c01a070 con 0x55fa649598b0 2024-04-26T22:00:05.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.537+0000 7fe20effd640 1 -- 172.21.15.55:0/76693611 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe21c0158c0 con 0x55fa649598b0 2024-04-26T22:00:05.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.538+0000 7fe22dba8740 1 -- 172.21.15.55:0/76693611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 msgr2=0x55fa64959cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.538+0000 7fe22dba8740 1 --2- 172.21.15.55:0/76693611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64959cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009f30 tx=0x7fe21c00c8b0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.540+0000 7fe228c28640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe2140a3cb0 0x7fe2140a6170 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe210009900 tx=0x7fe210000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.546+0000 7fe22dba8740 1 -- 172.21.15.55:0/76693611 shutdown_connections 2024-04-26T22:00:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.546+0000 7fe22dba8740 1 --2- 172.21.15.55:0/76693611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495f410 0x55fa649618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.546+0000 7fe22dba8740 1 --2- 172.21.15.55:0/76693611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 0x55fa6495eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.546+0000 7fe22dba8740 1 --2- 172.21.15.55:0/76693611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64959cd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.546+0000 7fe22dba8740 1 -- 172.21.15.55:0/76693611 >> 172.21.15.55:0/76693611 conn(0x55fa64954f20 msgr2=0x55fa64955320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.547+0000 7fe22dba8740 1 -- 172.21.15.55:0/76693611 shutdown_connections 2024-04-26T22:00:05.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.548+0000 7fe22dba8740 1 -- 172.21.15.55:0/76693611 wait complete. 2024-04-26T22:00:05.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.549+0000 7fe22dba8740 1 -- start start 2024-04-26T22:00:05.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.549+0000 7fe22dba8740 1 Processor -- start 2024-04-26T22:00:05.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.552+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64963dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.552+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495a2a0 0x55fa64964300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.553+0000 7fe223fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495a2a0 0x55fa64964300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.553+0000 7fe223fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495a2a0 0x55fa64964300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46338/0 (socket says 172.21.15.55:46338) 2024-04-26T22:00:05.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.553+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495f410 0x55fa6496b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.554+0000 7fe223fff640 1 -- 172.21.15.55:0/153504365 learned_addr learned my addr 172.21.15.55:0/153504365 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:05.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.554+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fa64962410 con 0x55fa6495a2a0 2024-04-26T22:00:05.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.554+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fa648585d0 con 0x55fa6495f410 2024-04-26T22:00:05.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.554+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fa64962590 con 0x55fa649598b0 2024-04-26T22:00:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.556+0000 7fe229429640 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495f410 0x55fa6496b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.559+0000 7fe229429640 1 -- 172.21.15.55:0/153504365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 msgr2=0x55fa64963dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:05.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.559+0000 7fe229429640 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64963dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.559+0000 7fe229429640 1 -- 172.21.15.55:0/153504365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495a2a0 msgr2=0x55fa64964300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:05.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.559+0000 7fe229429640 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495a2a0 0x55fa64964300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.559+0000 7fe229429640 1 -- 172.21.15.55:0/153504365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa646f8f70 con 0x55fa6495f410 2024-04-26T22:00:05.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.559+0000 7fe22dba8740 10 client.0 _ll_register_callbacks cb 0x55fa64846080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:05.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.560+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe21c0158c0 con 0x55fa6495f410 2024-04-26T22:00:05.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.569+0000 7fe229429640 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495f410 0x55fa6496b340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009f00 tx=0x7fe21c015ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6134a6778a68f0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.569+0000 7fe203fff640 10 client.25775 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:00:05.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.572+0000 7fe203fff640 1 -- 172.21.15.55:0/153504365 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c01a070 con 0x55fa6495f410 2024-04-26T22:00:05.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.572+0000 7fe203fff640 1 -- 172.21.15.55:0/153504365 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe21c00ac50 con 0x55fa6495f410 2024-04-26T22:00:05.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.573+0000 7fe203fff640 1 -- 172.21.15.55:0/153504365 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe21c009420 con 0x55fa6495f410 2024-04-26T22:00:05.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.576+0000 7fe22dba8740 10 client.25775 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:05.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.576+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55fa64590170 con 0x55fa6495f410 2024-04-26T22:00:05.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.580+0000 7fe203fff640 1 -- 172.21.15.55:0/153504365 <== mon.2 v2:172.21.15.64:3300/0 4 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe21c003a10 con 0x55fa6495f410 2024-04-26T22:00:05.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.582+0000 7fe22dba8740 10 client.25775 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:05.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.582+0000 7fe22dba8740 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:05.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.582+0000 7fe22dba8740 20 client.25775 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:05.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.582+0000 7fe22dba8740 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fa6496f2f0 0x55fa649716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.582+0000 7fe22dba8740 4 client.25775 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:05.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.582+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa648cc710 con 0x55fa6496f2f0 2024-04-26T22:00:05.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.584+0000 7fe228c28640 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fa6496f2f0 0x55fa649716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.599+0000 7fe228c28640 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fa6496f2f0 0x55fa649716e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4aca0104fd220e9 server_cookie=2702bef2ae81f4e9 in_seq=0 out_seq=0 2024-04-26T22:00:05.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.603+0000 7fe203fff640 10 client.25775 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:05.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.609+0000 7fe203fff640 1 -- 172.21.15.55:0/153504365 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa648cc710 con 0x55fa6496f2f0 2024-04-26T22:00:05.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.609+0000 7fe203fff640 10 client.25775 handle_command_reply: tid=0 2024-04-26T22:00:05.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.609+0000 7fe22dba8740 1 client.25775 shutdown 2024-04-26T22:00:05.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.609+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495f410 msgr2=0x55fa6496b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.609+0000 7fe22dba8740 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495f410 0x55fa6496b340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009f00 tx=0x7fe21c015ca0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.610+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 shutdown_connections 2024-04-26T22:00:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.610+0000 7fe22dba8740 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495f410 0x55fa6496b340 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.610+0000 7fe22dba8740 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fa6496f2f0 0x55fa649716e0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.610+0000 7fe22dba8740 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa6495a2a0 0x55fa64964300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.610+0000 7fe22dba8740 1 --2- 172.21.15.55:0/153504365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa649598b0 0x55fa64963dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.610+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 >> 172.21.15.55:0/153504365 conn(0x55fa64954f20 msgr2=0x55fa649540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.610+0000 7fe203fff640 0 client.25775 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:05.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.620+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 shutdown_connections 2024-04-26T22:00:05.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.620+0000 7fe22dba8740 1 -- 172.21.15.55:0/153504365 wait complete. 2024-04-26T22:00:05.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.620+0000 7fe22dba8740 20 client.25775 trim_cache size 0 max 16384 2024-04-26T22:00:05.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.697+0000 7fe22dba8740 1 Processor -- start 2024-04-26T22:00:05.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.700+0000 7fe22dba8740 1 -- start start 2024-04-26T22:00:05.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.700+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa64954f20 0x55fa64953f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.701+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa649598b0 0x55fa6496b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.701+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 0x55fa64975ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.701+0000 7fe22dba8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241c4610 con 0x55fa649598b0 2024-04-26T22:00:05.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.701+0000 7fe22dba8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241c1ee0 con 0x55fa6495a2a0 2024-04-26T22:00:05.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.701+0000 7fe22dba8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241a6a70 con 0x55fa64954f20 2024-04-26T22:00:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.703+0000 7fe223fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa649598b0 0x55fa6496b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.703+0000 7fe223fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa649598b0 0x55fa6496b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46352/0 (socket says 172.21.15.55:46352) 2024-04-26T22:00:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.703+0000 7fe223fff640 1 -- 172.21.15.55:0/2175439505 learned_addr learned my addr 172.21.15.55:0/2175439505 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.704+0000 7fe229429640 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 0x55fa64975ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.705+0000 7fe228c28640 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa64954f20 0x55fa64953f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.706+0000 7fe223fff640 1 -- 172.21.15.55:0/2175439505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa64954f20 msgr2=0x55fa64953f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:05.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.706+0000 7fe223fff640 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa64954f20 0x55fa64953f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.706+0000 7fe223fff640 1 -- 172.21.15.55:0/2175439505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 msgr2=0x55fa64975ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:05.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.706+0000 7fe223fff640 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 0x55fa64975ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.706+0000 7fe223fff640 1 -- 172.21.15.55:0/2175439505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa649644a0 con 0x55fa649598b0 2024-04-26T22:00:05.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.710+0000 7fe223fff640 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa649598b0 0x55fa6496b260 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe2180cde50 tx=0x7fe218098450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd8b1cd97a13876 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.711+0000 7fe20e7fc640 1 -- 172.21.15.55:0/2175439505 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180a0560 con 0x55fa649598b0 2024-04-26T22:00:05.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.711+0000 7fe20e7fc640 1 -- 172.21.15.55:0/2175439505 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2180a4e20 con 0x55fa649598b0 2024-04-26T22:00:05.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.711+0000 7fe20e7fc640 1 -- 172.21.15.55:0/2175439505 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218017b50 con 0x55fa649598b0 2024-04-26T22:00:05.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.713+0000 7fe22dba8740 1 -- 172.21.15.55:0/2175439505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa649598b0 msgr2=0x55fa6496b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:05.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.713+0000 7fe22dba8740 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa649598b0 0x55fa6496b260 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe2180cde50 tx=0x7fe218098450 comp rx=0 tx=0).stop 2024-04-26T22:00:05.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.713+0000 7fe22dba8740 1 -- 172.21.15.55:0/2175439505 shutdown_connections 2024-04-26T22:00:05.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.713+0000 7fe22dba8740 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa6495a2a0 0x55fa64975ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.713+0000 7fe22dba8740 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa649598b0 0x55fa6496b260 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.713+0000 7fe22dba8740 1 --2- 172.21.15.55:0/2175439505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa64954f20 0x55fa64953f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.713+0000 7fe22dba8740 1 -- 172.21.15.55:0/2175439505 >> 172.21.15.55:0/2175439505 conn(0x55fa64963980 msgr2=0x55fa64953850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.717+0000 7fe22dba8740 1 -- 172.21.15.55:0/2175439505 shutdown_connections 2024-04-26T22:00:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.719+0000 7fe22dba8740 1 -- 172.21.15.55:0/2175439505 wait complete. 2024-04-26T22:00:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.719+0000 7fe22dba8740 1 -- start start 2024-04-26T22:00:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.719+0000 7fe22dba8740 1 Processor -- start 2024-04-26T22:00:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.722+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa64954f20 0x55fa6496ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.722+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa649598b0 0x55fa64969470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.722+0000 7fe22dba8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa6495a2a0 0x55fa649699b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.722+0000 7fe22dba8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241a70e0 con 0x55fa64954f20 2024-04-26T22:00:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.722+0000 7fe22dba8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241c1810 con 0x55fa649598b0 2024-04-26T22:00:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.722+0000 7fe22dba8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe2241a77b0 con 0x55fa6495a2a0 2024-04-26T22:00:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.722+0000 7fe22dba8740 10 client.0 _ll_register_callbacks cb 0x7fe21c0157b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:05.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.724+0000 7fe229429640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa6495a2a0 0x55fa649699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.724+0000 7fe229429640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa6495a2a0 0x55fa649699b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49016/0 (socket says 172.21.15.55:49016) 2024-04-26T22:00:05.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.724+0000 7fe229429640 1 -- 172.21.15.55:0/3821812352 learned_addr learned my addr 172.21.15.55:0/3821812352 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:05.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.724+0000 7fe223fff640 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa649598b0 0x55fa64969470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.726+0000 7fe228c28640 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa64954f20 0x55fa6496ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.727+0000 7fe228c28640 1 -- 172.21.15.55:0/3821812352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa6495a2a0 msgr2=0x55fa649699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:05.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.727+0000 7fe228c28640 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa6495a2a0 0x55fa649699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.727+0000 7fe228c28640 1 -- 172.21.15.55:0/3821812352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa649598b0 msgr2=0x55fa64969470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:05.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.727+0000 7fe228c28640 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa649598b0 0x55fa64969470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:05.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.727+0000 7fe228c28640 1 -- 172.21.15.55:0/3821812352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa64969ef0 con 0x55fa64954f20 2024-04-26T22:00:05.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.727+0000 7fe228c28640 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa64954f20 0x55fa6496ae60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe210008000 tx=0x7fe210005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04186637ab98bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:05.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.728+0000 7fe203fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:05.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.729+0000 7fe203fff640 1 -- 172.21.15.55:0/3821812352 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210035020 con 0x55fa64954f20 2024-04-26T22:00:05.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.729+0000 7fe22dba8740 1 -- 172.21.15.55:0/3821812352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2180a4e20 con 0x55fa64954f20 2024-04-26T22:00:05.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.729+0000 7fe203fff640 1 -- 172.21.15.55:0/3821812352 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe2100316d0 con 0x55fa64954f20 2024-04-26T22:00:05.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.730+0000 7fe203fff640 1 -- 172.21.15.55:0/3821812352 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe21005b050 con 0x55fa64954f20 2024-04-26T22:00:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.735+0000 7fe22dba8740 10 client.16578 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.735+0000 7fe22dba8740 1 -- 172.21.15.55:0/3821812352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fa649d65b0 con 0x55fa64954f20 2024-04-26T22:00:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.739+0000 7fe203fff640 1 -- 172.21.15.55:0/3821812352 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2180a4e20 con 0x55fa64954f20 2024-04-26T22:00:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.741+0000 7fe203fff640 1 -- 172.21.15.55:0/3821812352 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe210048040 con 0x55fa64954f20 2024-04-26T22:00:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.745+0000 7fe22dba8740 10 client.16578 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.745+0000 7fe22dba8740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.745+0000 7fe22dba8740 20 client.16578 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.745+0000 7fe22dba8740 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fa6495f410 0x55fa649d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.745+0000 7fe22dba8740 4 client.16578 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:00:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.745+0000 7fe22dba8740 1 -- 172.21.15.55:0/3821812352 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa648c6580 con 0x55fa6495f410 2024-04-26T22:00:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.747+0000 7fe223fff640 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fa6495f410 0x55fa649d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:05.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.756+0000 7fe223fff640 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fa6495f410 0x55fa649d8f10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6892935dbbc92509 server_cookie=1549249294dbe155 in_seq=0 out_seq=0 2024-04-26T22:00:05.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:05.756+0000 7fe203fff640 10 client.16578 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.187+0000 7fe203fff640 1 -- 172.21.15.55:0/3821812352 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa648c6580 con 0x55fa6495f410 2024-04-26T22:00:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.187+0000 7fe203fff640 10 client.16578 handle_command_reply: tid=0 2024-04-26T22:00:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.188+0000 7fe22dba8740 1 client.16578 shutdown 2024-04-26T22:00:06.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 -- 172.21.15.55:0/3821812352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa64954f20 msgr2=0x55fa6496ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa64954f20 0x55fa6496ae60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe210008000 tx=0x7fe210005fd0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 -- 172.21.15.55:0/3821812352 shutdown_connections 2024-04-26T22:00:06.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fa6495a2a0 0x55fa649699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fa649598b0 0x55fa64969470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fa6495f410 0x55fa649d8f10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 --2- 172.21.15.55:0/3821812352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fa64954f20 0x55fa6496ae60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 -- 172.21.15.55:0/3821812352 >> 172.21.15.55:0/3821812352 conn(0x55fa64963980 msgr2=0x55fa649540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe203fff640 0 client.16578 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 -- 172.21.15.55:0/3821812352 shutdown_connections 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.190+0000 7fe22dba8740 1 -- 172.21.15.55:0/3821812352 wait complete. 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.191+0000 7fe22dba8740 20 client.16578 trim_cache size 0 max 16384 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:00:06.192 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "a3d39131-e6ff-463d-9ae1-1d0410d5843c", 2024-04-26T22:00:06.193 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:00:06.193 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:00:06.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.196+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe2140a3cb0 msgr2=0x7fe2140a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.196+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe2140a3cb0 0x7fe2140a6170 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe210009900 tx=0x7fe210000bc0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.196+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2241111b0 msgr2=0x7fe2241a8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.196+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2241111b0 0x7fe2241a8340 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe21800b470 tx=0x7fe21800b930 comp rx=0 tx=0).stop 2024-04-26T22:00:06.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.202+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3121006588 shutdown_connections 2024-04-26T22:00:06.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.202+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe2140a3cb0 0x7fe2140a6170 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.202+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe2241ad3a0 0x7fe2241af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.202+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2241111b0 0x7fe2241a8340 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.202+0000 7fe22ae8c640 1 --2- 172.21.15.55:0/3121006588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe22410a290 0x7fe2241a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.202+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3121006588 >> 172.21.15.55:0/3121006588 conn(0x7fe22406f7c0 msgr2=0x7fe22410ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.204+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3121006588 shutdown_connections 2024-04-26T22:00:06.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.205+0000 7fe22ae8c640 1 -- 172.21.15.55:0/3121006588 wait complete. 2024-04-26T22:00:06.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:00:06.255 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:00:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.683+0000 7fc502b5f640 1 -- 172.21.15.55:0/1016051763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4fc10ec70 msgr2=0x7fc4fc10f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.683+0000 7fc502b5f640 1 --2- 172.21.15.55:0/1016051763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4fc10ec70 0x7fc4fc10f120 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009900 tx=0x7fc4ec02a420 comp rx=0 tx=0).stop 2024-04-26T22:00:06.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.684+0000 7fc502b5f640 1 -- 172.21.15.55:0/1016051763 shutdown_connections 2024-04-26T22:00:06.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.684+0000 7fc502b5f640 1 --2- 172.21.15.55:0/1016051763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4fc10ec70 0x7fc4fc10f120 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.684+0000 7fc502b5f640 1 --2- 172.21.15.55:0/1016051763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc10c2a0 0x7fc4fc10e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.684+0000 7fc502b5f640 1 --2- 172.21.15.55:0/1016051763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc109930 0x7fc4fc10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.684+0000 7fc502b5f640 1 -- 172.21.15.55:0/1016051763 >> 172.21.15.55:0/1016051763 conn(0x7fc4fc06f7c0 msgr2=0x7fc4fc071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.694+0000 7fc502b5f640 1 -- 172.21.15.55:0/1016051763 shutdown_connections 2024-04-26T22:00:06.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.695+0000 7fc502b5f640 1 -- 172.21.15.55:0/1016051763 wait complete. 2024-04-26T22:00:06.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.697+0000 7fc502b5f640 1 Processor -- start 2024-04-26T22:00:06.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.697+0000 7fc502b5f640 1 -- start start 2024-04-26T22:00:06.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.697+0000 7fc502b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc109930 0x7fc4fc1a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.697+0000 7fc502b5f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc10c2a0 0x7fc4fc1a7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.697+0000 7fc502b5f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4fc10ec70 0x7fc4fc1aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.697+0000 7fc502b5f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc11d370 con 0x7fc4fc10ec70 2024-04-26T22:00:06.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.697+0000 7fc502b5f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc11d1f0 con 0x7fc4fc109930 2024-04-26T22:00:06.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.697+0000 7fc502b5f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc11d4f0 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.698+0000 7fc4fbfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc10c2a0 0x7fc4fc1a7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.698+0000 7fc4fbfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc10c2a0 0x7fc4fc1a7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49026/0 (socket says 172.21.15.55:49026) 2024-04-26T22:00:06.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.698+0000 7fc4fbfff640 1 -- 172.21.15.55:0/937995081 learned_addr learned my addr 172.21.15.55:0/937995081 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:06.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.699+0000 7fc4fbfff640 1 -- 172.21.15.55:0/937995081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc109930 msgr2=0x7fc4fc1a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.699+0000 7fc4fbfff640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc109930 0x7fc4fc1a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.699+0000 7fc4fbfff640 1 -- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4fc10ec70 msgr2=0x7fc4fc1aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.699+0000 7fc4fbfff640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4fc10ec70 0x7fc4fc1aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.699+0000 7fc4fbfff640 1 -- 172.21.15.55:0/937995081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc111190 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.700+0000 7fc4fbfff640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc10c2a0 0x7fc4fc1a7f80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000b470 tx=0x7fc4f000b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.704+0000 7fc4f9ffb640 1 -- 172.21.15.55:0/937995081 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0015020 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.704+0000 7fc4f9ffb640 1 -- 172.21.15.55:0/937995081 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f0011d40 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.704+0000 7fc4f9ffb640 1 -- 172.21.15.55:0/937995081 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000f340 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.706+0000 7fc502b5f640 1 -- 172.21.15.55:0/937995081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc111480 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.706+0000 7fc502b5f640 1 -- 172.21.15.55:0/937995081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc111a00 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.712+0000 7fc505874740 1 Processor -- start 2024-04-26T22:00:06.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.721+0000 7fc4f9ffb640 1 -- 172.21.15.55:0/937995081 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc4f0018020 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.722+0000 7fc505874740 1 -- start start 2024-04-26T22:00:06.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.723+0000 7fc505874740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9bee0 0x559221e9c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.723+0000 7fc505874740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221e9d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.723+0000 7fc505874740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221ea4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.723+0000 7fc505874740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559221e00d80 con 0x559221e9da70 2024-04-26T22:00:06.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.723+0000 7fc505874740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559221dba8b0 con 0x559221e9bee0 2024-04-26T22:00:06.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.723+0000 7fc505874740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559221dfcd40 con 0x559221e9d0b0 2024-04-26T22:00:06.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.723+0000 7fc4f9ffb640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4e40a3d90 0x7fc4e40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.724+0000 7fc4f9ffb640 1 -- 172.21.15.55:0/937995081 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc4f00cfea0 con 0x7fc4fc10c2a0 2024-04-26T22:00:06.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.725+0000 7fc5008fb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221e9d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.725+0000 7fc5008fb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221e9d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49038/0 (socket says 172.21.15.55:49038) 2024-04-26T22:00:06.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.725+0000 7fc5008fb640 1 -- 172.21.15.55:0/644578701 learned_addr learned my addr 172.21.15.55:0/644578701 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:06.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.727+0000 7fc5010fc640 1 --2- 172.21.15.55:0/644578701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9bee0 0x559221e9c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.728+0000 7fc5010fc640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4e40a3d90 0x7fc4e40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.729+0000 7fc5008fb640 1 -- 172.21.15.55:0/644578701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9bee0 msgr2=0x559221e9c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:06.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.729+0000 7fc5008fb640 1 --2- 172.21.15.55:0/644578701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9bee0 0x559221e9c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.729+0000 7fc5008fb640 1 -- 172.21.15.55:0/644578701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 msgr2=0x559221ea4300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:06.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.729+0000 7fc5008fb640 1 --2- 172.21.15.55:0/644578701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221ea4300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.729+0000 7fc5008fb640 1 -- 172.21.15.55:0/644578701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559221d8ba40 con 0x559221e9d0b0 2024-04-26T22:00:06.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.731+0000 7fc5008fb640 1 --2- 172.21.15.55:0/644578701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221e9d530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009e00 tx=0x7fc4f400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a75fed11780d346f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.733+0000 7fc4e2ffd640 1 -- 172.21.15.55:0/644578701 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4016070 con 0x559221e9d0b0 2024-04-26T22:00:06.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.733+0000 7fc4e2ffd640 1 -- 172.21.15.55:0/644578701 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f4004d20 con 0x559221e9d0b0 2024-04-26T22:00:06.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.733+0000 7fc4e2ffd640 1 -- 172.21.15.55:0/644578701 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f40116d0 con 0x559221e9d0b0 2024-04-26T22:00:06.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.736+0000 7fc505874740 1 -- 172.21.15.55:0/644578701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 msgr2=0x559221e9d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.736+0000 7fc505874740 1 --2- 172.21.15.55:0/644578701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221e9d530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009e00 tx=0x7fc4f400e6c0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.746+0000 7fc505874740 1 -- 172.21.15.55:0/644578701 shutdown_connections 2024-04-26T22:00:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.746+0000 7fc505874740 1 --2- 172.21.15.55:0/644578701 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221ea4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.746+0000 7fc505874740 1 --2- 172.21.15.55:0/644578701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221e9d530 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.746+0000 7fc505874740 1 --2- 172.21.15.55:0/644578701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9bee0 0x559221e9c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.746+0000 7fc505874740 1 -- 172.21.15.55:0/644578701 >> 172.21.15.55:0/644578701 conn(0x559221e973d0 msgr2=0x559221e977d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.747+0000 7fc5010fc640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4e40a3d90 0x7fc4e40a6250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec031500 tx=0x7fc4ec009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.748+0000 7fc505874740 1 -- 172.21.15.55:0/644578701 shutdown_connections 2024-04-26T22:00:06.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.753+0000 7fc505874740 1 -- 172.21.15.55:0/644578701 wait complete. 2024-04-26T22:00:06.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.753+0000 7fc505874740 1 -- start start 2024-04-26T22:00:06.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.754+0000 7fc505874740 1 Processor -- start 2024-04-26T22:00:06.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.754+0000 7fc505874740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221ea6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.754+0000 7fc505874740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221ea69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.755+0000 7fc505874740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9da70 0x559221eada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.755+0000 7fc505874740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559221ea4a80 con 0x559221e9bee0 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.755+0000 7fc505874740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559221d87d10 con 0x559221e9da70 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.755+0000 7fc505874740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559221ea4c00 con 0x559221e9d0b0 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.755+0000 7fc505874740 10 client.0 _ll_register_callbacks cb 0x559221db4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.756+0000 7fc5008fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221ea6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.756+0000 7fc5008fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221ea6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46396/0 (socket says 172.21.15.55:46396) 2024-04-26T22:00:06.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.756+0000 7fc5008fb640 1 -- 172.21.15.55:0/3827556791 learned_addr learned my addr 172.21.15.55:0/3827556791 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:06.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.757+0000 7fc5008fb640 1 -- 172.21.15.55:0/3827556791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 msgr2=0x559221ea69d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:06.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.757+0000 7fc5008fb640 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221ea69d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.757+0000 7fc5008fb640 1 -- 172.21.15.55:0/3827556791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9da70 msgr2=0x559221eada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:06.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.757+0000 7fc5008fb640 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9da70 0x559221eada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.757+0000 7fc5008fb640 1 -- 172.21.15.55:0/3827556791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559221bdc7d0 con 0x559221e9bee0 2024-04-26T22:00:06.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.760+0000 7fc5008fb640 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221ea6490 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4005b40 tx=0x7fc4f4009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffc7a3b31888ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.760+0000 7fc4dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:06.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.765+0000 7fc4dbfff640 1 -- 172.21.15.55:0/3827556791 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4016050 con 0x559221e9bee0 2024-04-26T22:00:06.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.766+0000 7fc505874740 1 -- 172.21.15.55:0/3827556791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f4004d20 con 0x559221e9bee0 2024-04-26T22:00:06.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.766+0000 7fc4dbfff640 1 -- 172.21.15.55:0/3827556791 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc4f4005230 con 0x559221e9bee0 2024-04-26T22:00:06.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.769+0000 7fc4dbfff640 1 -- 172.21.15.55:0/3827556791 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc4f400ee90 con 0x559221e9bee0 2024-04-26T22:00:06.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.769+0000 7fc505874740 10 client.16584 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:06.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.769+0000 7fc505874740 1 -- 172.21.15.55:0/3827556791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559221b123f0 con 0x559221e9bee0 2024-04-26T22:00:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.783+0000 7fc4dbfff640 1 -- 172.21.15.55:0/3827556791 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f4004d20 con 0x559221e9bee0 2024-04-26T22:00:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.783+0000 7fc4dbfff640 1 -- 172.21.15.55:0/3827556791 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc4f400f040 con 0x559221e9bee0 2024-04-26T22:00:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.783+0000 7fc505874740 10 client.16584 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.783+0000 7fc505874740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.783+0000 7fc505874740 20 client.16584 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.783+0000 7fc505874740 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559221eb1c50 0x559221eb4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.783+0000 7fc505874740 4 client.16584 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.783+0000 7fc505874740 1 -- 172.21.15.55:0/3827556791 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559221e0e990 con 0x559221eb1c50 2024-04-26T22:00:06.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.794+0000 7fc4fbfff640 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559221eb1c50 0x559221eb4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.805+0000 7fc4fbfff640 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559221eb1c50 0x559221eb4040 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab46506a7d8367b server_cookie=84bd2b55abdd9aea in_seq=0 out_seq=0 2024-04-26T22:00:06.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.805+0000 7fc4dbfff640 10 client.16584 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:06.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.810+0000 7fc4dbfff640 1 -- 172.21.15.55:0/3827556791 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559221e0e990 con 0x559221eb1c50 2024-04-26T22:00:06.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.810+0000 7fc4dbfff640 10 client.16584 handle_command_reply: tid=0 2024-04-26T22:00:06.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.810+0000 7fc505874740 1 client.16584 shutdown 2024-04-26T22:00:06.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.812+0000 7fc505874740 1 -- 172.21.15.55:0/3827556791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 msgr2=0x559221ea6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.812+0000 7fc505874740 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221ea6490 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4005b40 tx=0x7fc4f4009b70 comp rx=0 tx=0).stop 2024-04-26T22:00:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.813+0000 7fc505874740 1 -- 172.21.15.55:0/3827556791 shutdown_connections 2024-04-26T22:00:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.813+0000 7fc505874740 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9da70 0x559221eada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.813+0000 7fc505874740 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9d0b0 0x559221ea69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.813+0000 7fc505874740 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559221eb1c50 0x559221eb4040 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.813+0000 7fc505874740 1 --2- 172.21.15.55:0/3827556791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221ea6490 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.813+0000 7fc505874740 1 -- 172.21.15.55:0/3827556791 >> 172.21.15.55:0/3827556791 conn(0x559221e973d0 msgr2=0x559221e96550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.813+0000 7fc4dbfff640 0 client.16584 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.814+0000 7fc505874740 1 -- 172.21.15.55:0/3827556791 shutdown_connections 2024-04-26T22:00:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.814+0000 7fc505874740 1 -- 172.21.15.55:0/3827556791 wait complete. 2024-04-26T22:00:06.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.817+0000 7fc505874740 20 client.16584 trim_cache size 0 max 16384 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc505874740 1 Processor -- start 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc505874740 1 -- start start 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc505874740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221e95cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc505874740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 0x559221ead9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc505874740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9da70 0x559221e96200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc505874740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e8010f90 con 0x559221e9bee0 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc505874740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e800e860 con 0x559221e9d0b0 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc505874740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e800c6b0 con 0x559221e9da70 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc5008fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221e95cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc5008fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221e95cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46398/0 (socket says 172.21.15.55:46398) 2024-04-26T22:00:06.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.858+0000 7fc5008fb640 1 -- 172.21.15.55:0/512174589 learned_addr learned my addr 172.21.15.55:0/512174589 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:06.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc5008fb640 1 -- 172.21.15.55:0/512174589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9da70 msgr2=0x559221e96200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:06.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc5008fb640 1 --2- 172.21.15.55:0/512174589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9da70 0x559221e96200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc5008fb640 1 -- 172.21.15.55:0/512174589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 msgr2=0x559221ead9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:06.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc5008fb640 1 --2- 172.21.15.55:0/512174589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 0x559221ead9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc5008fb640 1 -- 172.21.15.55:0/512174589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559221ad20f0 con 0x559221e9bee0 2024-04-26T22:00:06.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc5008fb640 1 --2- 172.21.15.55:0/512174589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221e95cc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40224f0 tx=0x7fc4f4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c4ac3298f8ed49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc4e27fc640 1 -- 172.21.15.55:0/512174589 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f40226f0 con 0x559221e9bee0 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc4e27fc640 1 -- 172.21.15.55:0/512174589 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f40229f0 con 0x559221e9bee0 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.859+0000 7fc4e27fc640 1 -- 172.21.15.55:0/512174589 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f40056e0 con 0x559221e9bee0 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 -- 172.21.15.55:0/512174589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 msgr2=0x559221e95cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 --2- 172.21.15.55:0/512174589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221e95cc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40224f0 tx=0x7fc4f4000f40 comp rx=0 tx=0).stop 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 -- 172.21.15.55:0/512174589 shutdown_connections 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 --2- 172.21.15.55:0/512174589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9da70 0x559221e96200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 --2- 172.21.15.55:0/512174589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 0x559221ead9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 --2- 172.21.15.55:0/512174589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9bee0 0x559221e95cc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 -- 172.21.15.55:0/512174589 >> 172.21.15.55:0/512174589 conn(0x559221e973d0 msgr2=0x559221ea0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 -- 172.21.15.55:0/512174589 shutdown_connections 2024-04-26T22:00:06.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 -- 172.21.15.55:0/512174589 wait complete. 2024-04-26T22:00:06.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.861+0000 7fc505874740 1 -- start start 2024-04-26T22:00:06.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.862+0000 7fc505874740 1 Processor -- start 2024-04-26T22:00:06.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.862+0000 7fc505874740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9bee0 0x559221ea2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc505874740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 0x559221ea30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc505874740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221f16140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc505874740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e800cd20 con 0x559221e9da70 2024-04-26T22:00:06.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc505874740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e800e190 con 0x559221e9d0b0 2024-04-26T22:00:06.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc5010fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221f16140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc5010fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221f16140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46410/0 (socket says 172.21.15.55:46410) 2024-04-26T22:00:06.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc5010fc640 1 -- 172.21.15.55:0/1782933440 learned_addr learned my addr 172.21.15.55:0/1782933440 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:06.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc5008fb640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9bee0 0x559221ea2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc505874740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e800d3f0 con 0x559221e9bee0 2024-04-26T22:00:06.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.863+0000 7fc4fbfff640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 0x559221ea30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc505874740 10 client.0 _ll_register_callbacks cb 0x7fc4f4009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:06.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc5010fc640 1 -- 172.21.15.55:0/1782933440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9bee0 msgr2=0x559221ea2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc5010fc640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9bee0 0x559221ea2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc5010fc640 1 -- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 msgr2=0x559221ea30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc5010fc640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 0x559221ea30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc5010fc640 1 -- 172.21.15.55:0/1782933440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559221dc14b0 con 0x559221e9da70 2024-04-26T22:00:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc5008fb640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9bee0 0x559221ea2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc4fbfff640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 0x559221ea30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:00:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc5010fc640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221f16140 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc06c650 tx=0x7fc4ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d32923bb1740d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.864+0000 7fc4dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.866+0000 7fc4dbfff640 1 -- 172.21.15.55:0/1782933440 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec04b020 con 0x559221e9da70 2024-04-26T22:00:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.866+0000 7fc4dbfff640 1 -- 172.21.15.55:0/1782933440 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc4ec004320 con 0x559221e9da70 2024-04-26T22:00:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.866+0000 7fc505874740 1 -- 172.21.15.55:0/1782933440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f40229f0 con 0x559221e9da70 2024-04-26T22:00:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.866+0000 7fc4dbfff640 1 -- 172.21.15.55:0/1782933440 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc4ec056050 con 0x559221e9da70 2024-04-26T22:00:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.866+0000 7fc505874740 10 client.16590 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.866+0000 7fc505874740 1 -- 172.21.15.55:0/1782933440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559221f17730 con 0x559221e9da70 2024-04-26T22:00:06.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.867+0000 7fc4dbfff640 1 -- 172.21.15.55:0/1782933440 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f40229f0 con 0x559221e9da70 2024-04-26T22:00:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.867+0000 7fc4dbfff640 1 -- 172.21.15.55:0/1782933440 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc4ec03a040 con 0x559221e9da70 2024-04-26T22:00:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.868+0000 7fc505874740 10 client.16590 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.868+0000 7fc505874740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.868+0000 7fc505874740 20 client.16590 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.868+0000 7fc505874740 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559221f18110 0x559221f1a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.868+0000 7fc505874740 4 client.16590 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:00:06.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.868+0000 7fc505874740 1 -- 172.21.15.55:0/1782933440 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559221e08800 con 0x559221f18110 2024-04-26T22:00:06.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.870+0000 7fc5008fb640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559221f18110 0x559221f1a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:06.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.877+0000 7fc5008fb640 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559221f18110 0x559221f1a530 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b9725f63245234 server_cookie=de64e4a0ea2bee26 in_seq=0 out_seq=0 2024-04-26T22:00:06.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.877+0000 7fc4dbfff640 10 client.16590 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:06.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.884+0000 7fc4dbfff640 1 -- 172.21.15.55:0/1782933440 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x559221e08800 con 0x559221f18110 2024-04-26T22:00:06.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.884+0000 7fc4dbfff640 10 client.16590 handle_command_reply: tid=0 2024-04-26T22:00:06.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.884+0000 7fc505874740 1 client.16590 shutdown 2024-04-26T22:00:06.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.884+0000 7fc505874740 1 -- 172.21.15.55:0/1782933440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 msgr2=0x559221f16140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.884+0000 7fc505874740 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221f16140 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc06c650 tx=0x7fc4ec009260 comp rx=0 tx=0).stop 2024-04-26T22:00:06.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.884+0000 7fc505874740 1 -- 172.21.15.55:0/1782933440 shutdown_connections 2024-04-26T22:00:06.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.885+0000 7fc505874740 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x559221f18110 0x559221f1a530 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.885+0000 7fc505874740 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559221e9da70 0x559221f16140 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.885+0000 7fc4dbfff640 0 client.16590 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:06.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.885+0000 7fc505874740 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559221e9d0b0 0x559221ea30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.885+0000 7fc505874740 1 --2- 172.21.15.55:0/1782933440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559221e9bee0 0x559221ea2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.885+0000 7fc505874740 1 -- 172.21.15.55:0/1782933440 >> 172.21.15.55:0/1782933440 conn(0x559221e973d0 msgr2=0x559221e9a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.885+0000 7fc505874740 1 -- 172.21.15.55:0/1782933440 shutdown_connections 2024-04-26T22:00:06.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.885+0000 7fc505874740 1 -- 172.21.15.55:0/1782933440 wait complete. 2024-04-26T22:00:06.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.886+0000 7fc505874740 20 client.16590 trim_cache size 0 max 16384 2024-04-26T22:00:06.887 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:00:06.887 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (0 inodes in the stack)", 2024-04-26T22:00:06.887 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:00:06.887 INFO:teuthology.orchestra.run.smithi055.stdout: "a3d39131-e6ff-463d-9ae1-1d0410d5843c": { 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "a3d39131-e6ff-463d-9ae1-1d0410d5843c", 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.888+0000 7fc502b5f640 1 -- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4e40a3d90 msgr2=0x7fc4e40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.888+0000 7fc502b5f640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4e40a3d90 0x7fc4e40a6250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec031500 tx=0x7fc4ec009560 comp rx=0 tx=0).stop 2024-04-26T22:00:06.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.888+0000 7fc502b5f640 1 -- 172.21.15.55:0/937995081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc10c2a0 msgr2=0x7fc4fc1a7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:06.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.888+0000 7fc502b5f640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc10c2a0 0x7fc4fc1a7f80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000b470 tx=0x7fc4f000b930 comp rx=0 tx=0).stop 2024-04-26T22:00:06.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.889+0000 7fc502b5f640 1 -- 172.21.15.55:0/937995081 shutdown_connections 2024-04-26T22:00:06.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.889+0000 7fc502b5f640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc4e40a3d90 0x7fc4e40a6250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.889+0000 7fc502b5f640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4fc10ec70 0x7fc4fc1aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.889+0000 7fc502b5f640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4fc10c2a0 0x7fc4fc1a7f80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.889+0000 7fc502b5f640 1 --2- 172.21.15.55:0/937995081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4fc109930 0x7fc4fc1a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:06.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.889+0000 7fc502b5f640 1 -- 172.21.15.55:0/937995081 >> 172.21.15.55:0/937995081 conn(0x7fc4fc06f7c0 msgr2=0x7fc4fc10d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:06.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.889+0000 7fc502b5f640 1 -- 172.21.15.55:0/937995081 shutdown_connections 2024-04-26T22:00:06.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:06.889+0000 7fc502b5f640 1 -- 172.21.15.55:0/937995081 wait complete. 2024-04-26T22:00:06.916 INFO:tasks.cephfs.filesystem:scrub status for tag:a3d39131-e6ff-463d-9ae1-1d0410d5843c - {'path': '/', 'tag': 'a3d39131-e6ff-463d-9ae1-1d0410d5843c', 'options': 'recursive,force'} 2024-04-26T22:00:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:06 smithi165 ceph-mon[33296]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 MiB/s wr, 1.38k op/s 2024-04-26T22:00:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:06 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:00:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:06 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:07.129 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:06 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:00:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:06 smithi055 ceph-mon[29221]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 MiB/s wr, 1.38k op/s 2024-04-26T22:00:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:06 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:00:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:06 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:06 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:00:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:06 smithi064 ceph-mon[32765]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 MiB/s wr, 1.38k op/s 2024-04-26T22:00:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:06 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:00:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:06 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:06 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:00:08.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:07 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:00:08.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:07 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:07 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:00:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:07 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:08.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:07 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:00:08.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:07 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:09.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:08 smithi055 ceph-mon[29221]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 871 MiB/s wr, 1.45k op/s 2024-04-26T22:00:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:08 smithi064 ceph-mon[32765]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 871 MiB/s wr, 1.45k op/s 2024-04-26T22:00:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:08 smithi165 ceph-mon[33296]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 871 MiB/s wr, 1.45k op/s 2024-04-26T22:00:11.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:10 smithi064 ceph-mon[32765]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 905 MiB/s wr, 1.79k op/s 2024-04-26T22:00:11.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:10 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:11.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:10 smithi165 ceph-mon[33296]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 905 MiB/s wr, 1.79k op/s 2024-04-26T22:00:11.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:10 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:11.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:10 smithi055 ceph-mon[29221]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 905 MiB/s wr, 1.79k op/s 2024-04-26T22:00:11.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:10 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:13.239 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:12 smithi055 ceph-mon[29221]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 853 MiB/s wr, 1.94k op/s 2024-04-26T22:00:13.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:12 smithi064 ceph-mon[32765]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 853 MiB/s wr, 1.94k op/s 2024-04-26T22:00:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:12 smithi165 ceph-mon[33296]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 853 MiB/s wr, 1.94k op/s 2024-04-26T22:00:15.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:14 smithi064 ceph-mon[32765]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 861 MiB/s wr, 2.03k op/s 2024-04-26T22:00:15.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:14 smithi165 ceph-mon[33296]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 861 MiB/s wr, 2.03k op/s 2024-04-26T22:00:15.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:14 smithi055 ceph-mon[29221]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 861 MiB/s wr, 2.03k op/s 2024-04-26T22:00:17.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:16 smithi064 ceph-mon[32765]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 891 MiB/s wr, 2.06k op/s 2024-04-26T22:00:17.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:16 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:00:17.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:16 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:00:17.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:16 smithi165 ceph-mon[33296]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 891 MiB/s wr, 2.06k op/s 2024-04-26T22:00:17.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:16 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:00:17.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:16 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:00:17.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:16 smithi055 ceph-mon[29221]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 891 MiB/s wr, 2.06k op/s 2024-04-26T22:00:17.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:16 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:00:17.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:16 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:00:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:18 smithi064 ceph-mon[32765]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 2.02k op/s 2024-04-26T22:00:19.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:18 smithi165 ceph-mon[33296]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 2.02k op/s 2024-04-26T22:00:19.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:18 smithi055 ceph-mon[29221]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 2.02k op/s 2024-04-26T22:00:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:21 smithi165 ceph-mon[33296]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 897 MiB/s wr, 2.03k op/s 2024-04-26T22:00:21.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:21 smithi055 ceph-mon[29221]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 897 MiB/s wr, 2.03k op/s 2024-04-26T22:00:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:21 smithi064 ceph-mon[32765]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 897 MiB/s wr, 2.03k op/s 2024-04-26T22:00:23.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:23 smithi165 ceph-mon[33296]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 MiB/s wr, 1.73k op/s 2024-04-26T22:00:23.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:23 smithi055 ceph-mon[29221]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 MiB/s wr, 1.73k op/s 2024-04-26T22:00:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:23 smithi064 ceph-mon[32765]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 MiB/s wr, 1.73k op/s 2024-04-26T22:00:25.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:25 smithi165 ceph-mon[33296]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 1.45k op/s 2024-04-26T22:00:25.445 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:25 smithi055 ceph-mon[29221]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 1.45k op/s 2024-04-26T22:00:25.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:25 smithi064 ceph-mon[32765]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 848 MiB/s wr, 1.45k op/s 2024-04-26T22:00:27.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:27 smithi055 ceph-mon[29221]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 872 MiB/s wr, 1.32k op/s 2024-04-26T22:00:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:27 smithi064 ceph-mon[32765]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 872 MiB/s wr, 1.32k op/s 2024-04-26T22:00:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:27 smithi165 ceph-mon[33296]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 872 MiB/s wr, 1.32k op/s 2024-04-26T22:00:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:29 smithi055 ceph-mon[29221]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 872 MiB/s wr, 1.26k op/s 2024-04-26T22:00:29.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:29 smithi064 ceph-mon[32765]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 872 MiB/s wr, 1.26k op/s 2024-04-26T22:00:29.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:29 smithi165 ceph-mon[33296]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 872 MiB/s wr, 1.26k op/s 2024-04-26T22:00:31.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:31 smithi055 ceph-mon[29221]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 861 MiB/s wr, 1.30k op/s 2024-04-26T22:00:31.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:31 smithi064 ceph-mon[32765]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 861 MiB/s wr, 1.30k op/s 2024-04-26T22:00:31.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:31 smithi165 ceph-mon[33296]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 861 MiB/s wr, 1.30k op/s 2024-04-26T22:00:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:33 smithi055 ceph-mon[29221]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 861 MiB/s wr, 1.25k op/s 2024-04-26T22:00:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:33 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:33 smithi064 ceph-mon[32765]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 861 MiB/s wr, 1.25k op/s 2024-04-26T22:00:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:33 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:33.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:33 smithi165 ceph-mon[33296]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 861 MiB/s wr, 1.25k op/s 2024-04-26T22:00:33.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:33 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:34 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:34 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:34.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:34 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:35.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: Unable to set osd_memory_target on smithi165 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 867 MiB/s wr, 1.19k op/s 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:00:35.442 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:00:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T22:00:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: Unable to set osd_memory_target on smithi165 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-26T22:00:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 867 MiB/s wr, 1.19k op/s 2024-04-26T22:00:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-04-26T22:00:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-04-26T22:00:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:00:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:00:35.508 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi165 to 176.2M 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: Unable to set osd_memory_target on smithi165 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 867 MiB/s wr, 1.19k op/s 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:00:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:00:36.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:00:36.918 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:00:37.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.275+0000 7fc16ffff640 1 -- 172.21.15.55:0/2325515106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1680a4c40 msgr2=0x7fc1680a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.278 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.275+0000 7fc16ffff640 1 --2- 172.21.15.55:0/2325515106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1680a4c40 0x7fc1680a5060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc164009900 tx=0x7fc16402a420 comp rx=0 tx=0).stop 2024-04-26T22:00:37.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.282+0000 7fc16ffff640 1 -- 172.21.15.55:0/2325515106 shutdown_connections 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.282+0000 7fc16ffff640 1 --2- 172.21.15.55:0/2325515106 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1680a6710 0x7fc1680aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.282+0000 7fc16ffff640 1 --2- 172.21.15.55:0/2325515106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1680a5d70 0x7fc1680a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.282+0000 7fc16ffff640 1 --2- 172.21.15.55:0/2325515106 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1680a4c40 0x7fc1680a5060 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.282+0000 7fc16ffff640 1 -- 172.21.15.55:0/2325515106 >> 172.21.15.55:0/2325515106 conn(0x7fc1680a00d0 msgr2=0x7fc1680a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.284+0000 7fc16ffff640 1 -- 172.21.15.55:0/2325515106 shutdown_connections 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.286+0000 7fc16ffff640 1 -- 172.21.15.55:0/2325515106 wait complete. 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.287+0000 7fc16ffff640 1 Processor -- start 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.287+0000 7fc16ffff640 1 -- start start 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.288+0000 7fc16ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1680a4c40 0x7fc1680b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.288+0000 7fc16ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1680a5d70 0x7fc1680b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.288+0000 7fc16ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1680a6710 0x7fc1680b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.288+0000 7fc16ffff640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680ad100 con 0x7fc1680a6710 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.288+0000 7fc16ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680acf80 con 0x7fc1680a4c40 2024-04-26T22:00:37.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.288+0000 7fc16ffff640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680ad280 con 0x7fc1680a5d70 2024-04-26T22:00:37.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.290+0000 7fc16f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1680a6710 0x7fc1680b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.290+0000 7fc16f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1680a6710 0x7fc1680b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60568/0 (socket says 172.21.15.55:60568) 2024-04-26T22:00:37.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.290+0000 7fc16f7fe640 1 -- 172.21.15.55:0/917697523 learned_addr learned my addr 172.21.15.55:0/917697523 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:37.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.291+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1680a5d70 0x7fc1680b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.293+0000 7fc16effd640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1680a4c40 0x7fc1680b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.295+0000 7fc16e7fc640 1 -- 172.21.15.55:0/917697523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1680a4c40 msgr2=0x7fc1680b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.295+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1680a4c40 0x7fc1680b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.295+0000 7fc16e7fc640 1 -- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1680a6710 msgr2=0x7fc1680b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.295+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1680a6710 0x7fc1680b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.295+0000 7fc16e7fc640 1 -- 172.21.15.55:0/917697523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1680b66e0 con 0x7fc1680a5d70 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.296+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1680a6710 0x7fc1680b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.296+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1680a5d70 0x7fc1680b5030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc16000e9f0 tx=0x7fc16000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.297+0000 7fc157fff640 1 -- 172.21.15.55:0/917697523 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16000ccc0 con 0x7fc1680a5d70 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.297+0000 7fc157fff640 1 -- 172.21.15.55:0/917697523 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1600108e0 con 0x7fc1680a5d70 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.297+0000 7fc157fff640 1 -- 172.21.15.55:0/917697523 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160018840 con 0x7fc1680a5d70 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.298+0000 7fc16ffff640 1 -- 172.21.15.55:0/917697523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1680b69d0 con 0x7fc1680a5d70 2024-04-26T22:00:37.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.298+0000 7fc16ffff640 1 -- 172.21.15.55:0/917697523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1680b6ed0 con 0x7fc1680a5d70 2024-04-26T22:00:37.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.309+0000 7fc177fc6740 1 Processor -- start 2024-04-26T22:00:37.310 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.309+0000 7fc157fff640 1 -- 172.21.15.55:0/917697523 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc160010430 con 0x7fc1680a5d70 2024-04-26T22:00:37.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.312+0000 7fc157fff640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1580a3d80 0x7fc1580a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.312+0000 7fc177fc6740 1 -- start start 2024-04-26T22:00:37.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.313+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 0x55d8dd8694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.313+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd86a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.313+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd8714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.313+0000 7fc177fc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d8dd7cdd80 con 0x55d8dd86ac40 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.313+0000 7fc177fc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d8dd7878b0 con 0x55d8dd8690b0 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.313+0000 7fc177fc6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d8dd7c9d40 con 0x55d8dd86a280 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.314+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1580a3d80 0x7fc1580a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.314+0000 7fc16e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd86a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.315+0000 7fc157fff640 1 -- 172.21.15.55:0/917697523 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1600d0990 con 0x7fc1680a5d70 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.315+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1580a3d80 0x7fc1580a6240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc170000b60 tx=0x7fc1700738e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.315+0000 7fc16f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd8714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.315+0000 7fc16f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd8714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60580/0 (socket says 172.21.15.55:60580) 2024-04-26T22:00:37.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.315+0000 7fc16f7fe640 1 -- 172.21.15.55:0/165282271 learned_addr learned my addr 172.21.15.55:0/165282271 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:37.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.317+0000 7fc16effd640 1 --2- 172.21.15.55:0/165282271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 0x55d8dd8694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.317+0000 7fc16f7fe640 1 -- 172.21.15.55:0/165282271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 msgr2=0x55d8dd86a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.317+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/165282271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd86a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.317+0000 7fc16f7fe640 1 -- 172.21.15.55:0/165282271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 msgr2=0x55d8dd8694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.317+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/165282271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 0x55d8dd8694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.317+0000 7fc16f7fe640 1 -- 172.21.15.55:0/165282271 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8dd758a40 con 0x55d8dd86ac40 2024-04-26T22:00:37.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.319+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/165282271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd8714d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc170052070 tx=0x7fc17007d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8a1431dc43128 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.324+0000 7fc1557fa640 1 -- 172.21.15.55:0/165282271 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17008e070 con 0x55d8dd86ac40 2024-04-26T22:00:37.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.324+0000 7fc1557fa640 1 -- 172.21.15.55:0/165282271 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc170081e10 con 0x55d8dd86ac40 2024-04-26T22:00:37.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.325+0000 7fc177fc6740 1 -- 172.21.15.55:0/165282271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 msgr2=0x55d8dd8714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.326+0000 7fc177fc6740 1 --2- 172.21.15.55:0/165282271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd8714d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc170052070 tx=0x7fc17007d000 comp rx=0 tx=0).stop 2024-04-26T22:00:37.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.326+0000 7fc177fc6740 1 -- 172.21.15.55:0/165282271 shutdown_connections 2024-04-26T22:00:37.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.326+0000 7fc177fc6740 1 --2- 172.21.15.55:0/165282271 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd8714d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.326+0000 7fc177fc6740 1 --2- 172.21.15.55:0/165282271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd86a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.326+0000 7fc177fc6740 1 --2- 172.21.15.55:0/165282271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 0x55d8dd8694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.326+0000 7fc177fc6740 1 -- 172.21.15.55:0/165282271 >> 172.21.15.55:0/165282271 conn(0x55d8dd8645a0 msgr2=0x55d8dd8649a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.331+0000 7fc177fc6740 1 -- 172.21.15.55:0/165282271 shutdown_connections 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.331+0000 7fc177fc6740 1 -- 172.21.15.55:0/165282271 wait complete. 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.332+0000 7fc177fc6740 1 -- start start 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.332+0000 7fc177fc6740 1 Processor -- start 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 0x55d8dd873570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd873ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd87aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc177fc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d8dd871cf0 con 0x55d8dd86ac40 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc177fc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d8dd754d10 con 0x55d8dd8690b0 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc177fc6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d8dd871e70 con 0x55d8dd86a280 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc177fc6740 10 client.0 _ll_register_callbacks cb 0x55d8dd781210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc16e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd873ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.336 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc16e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd873ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:39500/0 (socket says 172.21.15.55:39500) 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.333+0000 7fc16e7fc640 1 -- 172.21.15.55:0/784402410 learned_addr learned my addr 172.21.15.55:0/784402410 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.334+0000 7fc16e7fc640 1 -- 172.21.15.55:0/784402410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 msgr2=0x55d8dd873570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.334+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 0x55d8dd873570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.334+0000 7fc16e7fc640 1 -- 172.21.15.55:0/784402410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 msgr2=0x55d8dd87aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.334+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd87aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.334+0000 7fc16e7fc640 1 -- 172.21.15.55:0/784402410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8dd5a97d0 con 0x55d8dd86a280 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.335+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd873ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc160098ed0 tx=0x7fc160010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84829aaf231fac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.335+0000 7fc152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:00:37.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.335+0000 7fc152ff5640 1 -- 172.21.15.55:0/784402410 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1600a0e20 con 0x55d8dd86a280 2024-04-26T22:00:37.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.336+0000 7fc152ff5640 1 -- 172.21.15.55:0/784402410 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1600a5250 con 0x55d8dd86a280 2024-04-26T22:00:37.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.336+0000 7fc177fc6740 1 -- 172.21.15.55:0/784402410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc170081e10 con 0x55d8dd86a280 2024-04-26T22:00:37.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.337+0000 7fc152ff5640 1 -- 172.21.15.55:0/784402410 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1600a56e0 con 0x55d8dd86a280 2024-04-26T22:00:37.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.338+0000 7fc177fc6740 10 client.25810 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:37.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.338+0000 7fc177fc6740 1 -- 172.21.15.55:0/784402410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d8dd4df3f0 con 0x55d8dd86a280 2024-04-26T22:00:37.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.338+0000 7fc152ff5640 1 -- 172.21.15.55:0/784402410 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc170081e10 con 0x55d8dd86a280 2024-04-26T22:00:37.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.338+0000 7fc152ff5640 1 -- 172.21.15.55:0/784402410 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc1600987c0 con 0x55d8dd86a280 2024-04-26T22:00:37.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.339+0000 7fc177fc6740 10 client.25810 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:37.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.339+0000 7fc177fc6740 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:37.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.339+0000 7fc177fc6740 20 client.25810 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:37.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.339+0000 7fc177fc6740 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d8dd8a15b0 0x55d8dd8a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.339+0000 7fc177fc6740 4 client.25810 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:37.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.339+0000 7fc177fc6740 1 -- 172.21.15.55:0/784402410 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8dd7db990 con 0x55d8dd8a15b0 2024-04-26T22:00:37.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.339+0000 7fc16effd640 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d8dd8a15b0 0x55d8dd8a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.340+0000 7fc16effd640 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d8dd8a15b0 0x55d8dd8a3a30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a70e9276bf2f8c server_cookie=67da24c025de9245 in_seq=0 out_seq=0 2024-04-26T22:00:37.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.341+0000 7fc152ff5640 10 client.25810 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:37.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc152ff5640 1 -- 172.21.15.55:0/784402410 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8dd7db990 con 0x55d8dd8a15b0 2024-04-26T22:00:37.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc152ff5640 10 client.25810 handle_command_reply: tid=0 2024-04-26T22:00:37.344 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 client.25810 shutdown 2024-04-26T22:00:37.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 -- 172.21.15.55:0/784402410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 msgr2=0x55d8dd873ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd873ab0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc160098ed0 tx=0x7fc160010720 comp rx=0 tx=0).stop 2024-04-26T22:00:37.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 -- 172.21.15.55:0/784402410 shutdown_connections 2024-04-26T22:00:37.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d8dd8a15b0 0x55d8dd8a3a30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd86ac40 0x55d8dd87aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd873ab0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 --2- 172.21.15.55:0/784402410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8690b0 0x55d8dd873570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.344+0000 7fc177fc6740 1 -- 172.21.15.55:0/784402410 >> 172.21.15.55:0/784402410 conn(0x55d8dd8645a0 msgr2=0x55d8dd8636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.345+0000 7fc152ff5640 0 client.25810 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:37.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.345+0000 7fc177fc6740 1 -- 172.21.15.55:0/784402410 shutdown_connections 2024-04-26T22:00:37.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.345+0000 7fc177fc6740 1 -- 172.21.15.55:0/784402410 wait complete. 2024-04-26T22:00:37.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.345+0000 7fc177fc6740 20 client.25810 trim_cache size 0 max 16384 2024-04-26T22:00:37.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.376+0000 7fc177fc6740 1 Processor -- start 2024-04-26T22:00:37.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.376+0000 7fc177fc6740 1 -- start start 2024-04-26T22:00:37.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.376+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8645a0 0x55d8dd8722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.376+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd8690b0 0x55d8dd87aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.376+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd86a280 0x55d8dd872830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.376+0000 7fc16effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8645a0 0x55d8dd8722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8645a0 0x55d8dd8722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60592/0 (socket says 172.21.15.55:60592) 2024-04-26T22:00:37.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16effd640 1 -- 172.21.15.55:0/2296032858 learned_addr learned my addr 172.21.15.55:0/2296032858 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:37.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.376+0000 7fc177fc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c010e90 con 0x55d8dd8645a0 2024-04-26T22:00:37.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc177fc6740 1 -- 172.21.15.55:0/2296032858 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00e760 con 0x55d8dd86a280 2024-04-26T22:00:37.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc177fc6740 1 -- 172.21.15.55:0/2296032858 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00c5b0 con 0x55d8dd8690b0 2024-04-26T22:00:37.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd8690b0 0x55d8dd87aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16effd640 1 -- 172.21.15.55:0/2296032858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd8690b0 msgr2=0x55d8dd87aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16effd640 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd8690b0 0x55d8dd87aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16effd640 1 -- 172.21.15.55:0/2296032858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd86a280 msgr2=0x55d8dd872830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd86a280 0x55d8dd872830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16effd640 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd86a280 0x55d8dd872830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16effd640 1 -- 172.21.15.55:0/2296032858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8dd49f0f0 con 0x55d8dd8645a0 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd86a280 0x55d8dd872830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.377+0000 7fc16effd640 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8645a0 0x55d8dd8722f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc164033040 tx=0x7fc164005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f4ea63d6bf42f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.378+0000 7fc154ff9640 1 -- 172.21.15.55:0/2296032858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1640048c0 con 0x55d8dd8645a0 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.378+0000 7fc154ff9640 1 -- 172.21.15.55:0/2296032858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc16404b070 con 0x55d8dd8645a0 2024-04-26T22:00:37.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.378+0000 7fc154ff9640 1 -- 172.21.15.55:0/2296032858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1640054a0 con 0x55d8dd8645a0 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 -- 172.21.15.55:0/2296032858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8645a0 msgr2=0x55d8dd8722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8645a0 0x55d8dd8722f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc164033040 tx=0x7fc164005fd0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 -- 172.21.15.55:0/2296032858 shutdown_connections 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd86a280 0x55d8dd872830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd8690b0 0x55d8dd87aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 --2- 172.21.15.55:0/2296032858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8645a0 0x55d8dd8722f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 -- 172.21.15.55:0/2296032858 >> 172.21.15.55:0/2296032858 conn(0x55d8dd8715f0 msgr2=0x55d8dd8636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 -- 172.21.15.55:0/2296032858 shutdown_connections 2024-04-26T22:00:37.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 -- 172.21.15.55:0/2296032858 wait complete. 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.379+0000 7fc177fc6740 1 -- start start 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc177fc6740 1 Processor -- start 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8645a0 0x55d8dd877f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8690b0 0x55d8dd878450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc177fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd879410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc177fc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00cc20 con 0x55d8dd8690b0 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc177fc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00e090 con 0x55d8dd8645a0 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc177fc6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c00d2f0 con 0x55d8dd86a280 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc177fc6740 10 client.0 _ll_register_callbacks cb 0x7fc160098d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc16e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8690b0 0x55d8dd878450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc16e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8690b0 0x55d8dd878450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60602/0 (socket says 172.21.15.55:60602) 2024-04-26T22:00:37.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc16e7fc640 1 -- 172.21.15.55:0/322628762 learned_addr learned my addr 172.21.15.55:0/322628762 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc16f7fe640 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd879410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.380+0000 7fc16effd640 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8645a0 0x55d8dd877f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc16e7fc640 1 -- 172.21.15.55:0/322628762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 msgr2=0x55d8dd879410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd879410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc16e7fc640 1 -- 172.21.15.55:0/322628762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8645a0 msgr2=0x55d8dd877f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8645a0 0x55d8dd877f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc16e7fc640 1 -- 172.21.15.55:0/322628762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8dd78e4b0 con 0x55d8dd8690b0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc16e7fc640 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8690b0 0x55d8dd878450 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc16009b950 tx=0x7fc16009af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555bcfab27cfe695 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc152ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc152ff5640 1 -- 172.21.15.55:0/322628762 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16009c5b0 con 0x55d8dd8690b0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc152ff5640 1 -- 172.21.15.55:0/322628762 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1600ce4f0 con 0x55d8dd8690b0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc177fc6740 1 -- 172.21.15.55:0/322628762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16404b070 con 0x55d8dd8690b0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc152ff5640 1 -- 172.21.15.55:0/322628762 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1600a0e20 con 0x55d8dd8690b0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc177fc6740 10 client.16605 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.381+0000 7fc177fc6740 1 -- 172.21.15.55:0/322628762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d8dd87ab20 con 0x55d8dd8690b0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.382+0000 7fc152ff5640 1 -- 172.21.15.55:0/322628762 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc16404b070 con 0x55d8dd8690b0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.382+0000 7fc152ff5640 1 -- 172.21.15.55:0/322628762 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc1600a5320 con 0x55d8dd8690b0 2024-04-26T22:00:37.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.382+0000 7fc177fc6740 10 client.16605 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.382+0000 7fc177fc6740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.382+0000 7fc177fc6740 20 client.16605 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.382+0000 7fc177fc6740 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d8dd86ac40 0x55d8dd90a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.382+0000 7fc177fc6740 4 client.16605 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.382+0000 7fc177fc6740 1 -- 172.21.15.55:0/322628762 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8dd7d5800 con 0x55d8dd86ac40 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.383+0000 7fc16effd640 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d8dd86ac40 0x55d8dd90a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.384+0000 7fc152ff5640 10 client.16605 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.384+0000 7fc16effd640 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d8dd86ac40 0x55d8dd90a140 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3790e13d33245093 server_cookie=9e1df8898adae3a0 in_seq=0 out_seq=0 2024-04-26T22:00:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.386+0000 7fc152ff5640 1 -- 172.21.15.55:0/322628762 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d8dd7d5800 con 0x55d8dd86ac40 2024-04-26T22:00:37.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.386+0000 7fc152ff5640 10 client.16605 handle_command_reply: tid=0 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.386+0000 7fc177fc6740 1 client.16605 shutdown 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.386+0000 7fc177fc6740 1 -- 172.21.15.55:0/322628762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8690b0 msgr2=0x55d8dd878450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.386+0000 7fc177fc6740 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8690b0 0x55d8dd878450 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc16009b950 tx=0x7fc16009af80 comp rx=0 tx=0).stop 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 1 -- 172.21.15.55:0/322628762 shutdown_connections 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d8dd86a280 0x55d8dd879410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d8dd86ac40 0x55d8dd90a140 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d8dd8690b0 0x55d8dd878450 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 1 --2- 172.21.15.55:0/322628762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d8dd8645a0 0x55d8dd877f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc152ff5640 0 client.16605 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 1 -- 172.21.15.55:0/322628762 >> 172.21.15.55:0/322628762 conn(0x55d8dd8715f0 msgr2=0x55d8dd8674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 1 -- 172.21.15.55:0/322628762 shutdown_connections 2024-04-26T22:00:37.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 1 -- 172.21.15.55:0/322628762 wait complete. 2024-04-26T22:00:37.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.387+0000 7fc177fc6740 20 client.16605 trim_cache size 0 max 16384 2024-04-26T22:00:37.389 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:00:37.389 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:00:37.389 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:00:37.389 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:00:37.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 -- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1580a3d80 msgr2=0x7fc1580a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1580a3d80 0x7fc1580a6240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc170000b60 tx=0x7fc1700738e0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 -- 172.21.15.55:0/917697523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1680a5d70 msgr2=0x7fc1680b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1680a5d70 0x7fc1680b5030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc16000e9f0 tx=0x7fc16000edf0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 -- 172.21.15.55:0/917697523 shutdown_connections 2024-04-26T22:00:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1580a3d80 0x7fc1580a6240 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1680a6710 0x7fc1680b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1680a5d70 0x7fc1680b5030 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 --2- 172.21.15.55:0/917697523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1680a4c40 0x7fc1680b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 -- 172.21.15.55:0/917697523 >> 172.21.15.55:0/917697523 conn(0x7fc1680a00d0 msgr2=0x7fc1680a9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 -- 172.21.15.55:0/917697523 shutdown_connections 2024-04-26T22:00:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.388+0000 7fc16ffff640 1 -- 172.21.15.55:0/917697523 wait complete. 2024-04-26T22:00:37.414 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:37 smithi055 ceph-mon[29221]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 810 MiB/s wr, 1.08k op/s 2024-04-26T22:00:37.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:00:37.416 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:00:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:37 smithi064 ceph-mon[32765]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 810 MiB/s wr, 1.08k op/s 2024-04-26T22:00:37.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:37 smithi165 ceph-mon[33296]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 810 MiB/s wr, 1.08k op/s 2024-04-26T22:00:37.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.737+0000 7ffac71f0640 1 -- 172.21.15.55:0/1681953554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac0109930 msgr2=0x7ffac010bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.737+0000 7ffac71f0640 1 --2- 172.21.15.55:0/1681953554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac0109930 0x7ffac010bd60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009860 tx=0x7ffab402a380 comp rx=0 tx=0).stop 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.741+0000 7ffac71f0640 1 -- 172.21.15.55:0/1681953554 shutdown_connections 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.741+0000 7ffac71f0640 1 --2- 172.21.15.55:0/1681953554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac010ec50 0x7ffac010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.741+0000 7ffac71f0640 1 --2- 172.21.15.55:0/1681953554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac010c2a0 0x7ffac010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.741+0000 7ffac71f0640 1 --2- 172.21.15.55:0/1681953554 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac0109930 0x7ffac010bd60 secure :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009860 tx=0x7ffab402a380 comp rx=0 tx=0).stop 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.741+0000 7ffac71f0640 1 -- 172.21.15.55:0/1681953554 >> 172.21.15.55:0/1681953554 conn(0x7ffac006f6c0 msgr2=0x7ffac0071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.764+0000 7ffac71f0640 1 -- 172.21.15.55:0/1681953554 shutdown_connections 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.767+0000 7ffac71f0640 1 -- 172.21.15.55:0/1681953554 wait complete. 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.769+0000 7ffac71f0640 1 Processor -- start 2024-04-26T22:00:37.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.769+0000 7ffac71f0640 1 -- start start 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.769+0000 7ffac71f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac010c2a0 0x7ffac01162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.770+0000 7ffac71f0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac010ec50 0x7ffac0116830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.770+0000 7ffac71f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac01b5810 0x7ffac01b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.770+0000 7ffac71f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffac011d110 con 0x7ffac01b5810 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.770+0000 7ffac71f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffac011cf90 con 0x7ffac010c2a0 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.770+0000 7ffac71f0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffac011d290 con 0x7ffac010ec50 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.770+0000 7ffac4f8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac010c2a0 0x7ffac01162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.771+0000 7ffac4f8c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac010c2a0 0x7ffac01162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:44738/0 (socket says 172.21.15.55:44738) 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.771+0000 7ffac4f8c640 1 -- 172.21.15.55:0/3909354939 learned_addr learned my addr 172.21.15.55:0/3909354939 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.772+0000 7ffabffff640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac010ec50 0x7ffac0116830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.772+0000 7ffac4f8c640 1 -- 172.21.15.55:0/3909354939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac010ec50 msgr2=0x7ffac0116830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.772+0000 7ffac4f8c640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac010ec50 0x7ffac0116830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.772+0000 7ffac4f8c640 1 -- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac01b5810 msgr2=0x7ffac01b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.772+0000 7ffac4f8c640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac01b5810 0x7ffac01b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:37.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.772+0000 7ffac4f8c640 1 -- 172.21.15.55:0/3909354939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac01b8240 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.779+0000 7ffac4f8c640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac010c2a0 0x7ffac01162f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffab402a970 tx=0x7ffab4030d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.782+0000 7ffabdffb640 1 -- 172.21.15.55:0/3909354939 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4037050 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.783+0000 7ffac71f0640 1 -- 172.21.15.55:0/3909354939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac01b84d0 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.783+0000 7ffac71f0640 1 -- 172.21.15.55:0/3909354939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac01b8970 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.783+0000 7ffabdffb640 1 -- 172.21.15.55:0/3909354939 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab4043070 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.787+0000 7ffabdffb640 1 -- 172.21.15.55:0/3909354939 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab403e6b0 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.790+0000 7ffac71f0640 1 -- 172.21.15.55:0/3909354939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaac0052c0 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.791+0000 7ffabdffb640 1 -- 172.21.15.55:0/3909354939 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffab40035f0 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.793+0000 7ffabdffb640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffaa80a3d30 0x7ffaa80a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.794+0000 7ffabffff640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffaa80a3d30 0x7ffaa80a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.796+0000 7ffabdffb640 1 -- 172.21.15.55:0/3909354939 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ffab40f0b80 con 0x7ffac010c2a0 2024-04-26T22:00:37.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.798+0000 7ffabdffb640 1 -- 172.21.15.55:0/3909354939 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ffab40bd4f0 con 0x7ffac010c2a0 2024-04-26T22:00:37.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:37.803+0000 7ffabffff640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffaa80a3d30 0x7ffaa80a61f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffac01177b0 tx=0x7ffab000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.257+0000 7ffac71f0640 1 -- 172.21.15.55:0/3909354939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaac005880 con 0x7ffac010c2a0 2024-04-26T22:00:38.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.259+0000 7ffabdffb640 1 -- 172.21.15.55:0/3909354939 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7ffab4004450 con 0x7ffac010c2a0 2024-04-26T22:00:38.262 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:00:38.262 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:00:38.263 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:00:38.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.275+0000 7ffaa77fe640 1 -- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffaa80a3d30 msgr2=0x7ffaa80a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.275+0000 7ffaa77fe640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffaa80a3d30 0x7ffaa80a61f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffac01177b0 tx=0x7ffab000a4c0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.275+0000 7ffaa77fe640 1 -- 172.21.15.55:0/3909354939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac010c2a0 msgr2=0x7ffac01162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.275+0000 7ffaa77fe640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac010c2a0 0x7ffac01162f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffab402a970 tx=0x7ffab4030d50 comp rx=0 tx=0).stop 2024-04-26T22:00:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.279+0000 7ffaa77fe640 1 -- 172.21.15.55:0/3909354939 shutdown_connections 2024-04-26T22:00:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.279+0000 7ffaa77fe640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffaa80a3d30 0x7ffaa80a61f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.279+0000 7ffaa77fe640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffac01b5810 0x7ffac01b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.279+0000 7ffaa77fe640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac010ec50 0x7ffac0116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.279+0000 7ffaa77fe640 1 --2- 172.21.15.55:0/3909354939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffac010c2a0 0x7ffac01162f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.279+0000 7ffaa77fe640 1 -- 172.21.15.55:0/3909354939 >> 172.21.15.55:0/3909354939 conn(0x7ffac006f6c0 msgr2=0x7ffac010d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.282+0000 7ffaa77fe640 1 -- 172.21.15.55:0/3909354939 shutdown_connections 2024-04-26T22:00:38.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.282+0000 7ffaa77fe640 1 -- 172.21.15.55:0/3909354939 wait complete. 2024-04-26T22:00:38.351 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:00:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 -- 172.21.15.55:0/3650981165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 msgr2=0x7f9614101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 --2- 172.21.15.55:0/3650981165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 0x7f9614101e20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9608009900 tx=0x7f960802a410 comp rx=0 tx=0).stop 2024-04-26T22:00:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 -- 172.21.15.55:0/3650981165 shutdown_connections 2024-04-26T22:00:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 --2- 172.21.15.55:0/3650981165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9614102360 0x7f9614102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 --2- 172.21.15.55:0/3650981165 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 0x7f9614101e20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 --2- 172.21.15.55:0/3650981165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9614100c40 0x7f9614101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 -- 172.21.15.55:0/3650981165 >> 172.21.15.55:0/3650981165 conn(0x7f96140fc570 msgr2=0x7f96140fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 -- 172.21.15.55:0/3650981165 shutdown_connections 2024-04-26T22:00:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.636+0000 7f96193b4640 1 -- 172.21.15.55:0/3650981165 wait complete. 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.637+0000 7f96193b4640 1 Processor -- start 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.637+0000 7f96193b4640 1 -- start start 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.637+0000 7f96193b4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9614100c40 0x7f961419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f96193b4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 0x7f961419f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f96193b4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9614102360 0x7f96141a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f96193b4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f96141107c0 con 0x7f96141019a0 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f96193b4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9614110640 con 0x7f9614102360 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f96193b4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9614110940 con 0x7f9614100c40 2024-04-26T22:00:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f96127fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 0x7f961419f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f96127fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 0x7f961419f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60630/0 (socket says 172.21.15.55:60630) 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f96127fc640 1 -- 172.21.15.55:0/4143243845 learned_addr learned my addr 172.21.15.55:0/4143243845 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.638+0000 7f9612ffd640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9614100c40 0x7f961419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f96127fc640 1 -- 172.21.15.55:0/4143243845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9614100c40 msgr2=0x7f961419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f96127fc640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9614100c40 0x7f961419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f96127fc640 1 -- 172.21.15.55:0/4143243845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9614102360 msgr2=0x7f96141a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f96127fc640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9614102360 0x7f96141a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f96127fc640 1 -- 172.21.15.55:0/4143243845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96141a7030 con 0x7f96141019a0 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f96127fc640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 0x7f961419f950 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9608006010 tx=0x7f9608004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f95fbfff640 1 -- 172.21.15.55:0/4143243845 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960803f030 con 0x7f96141019a0 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f95fbfff640 1 -- 172.21.15.55:0/4143243845 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9608003cb0 con 0x7f96141019a0 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.639+0000 7f95fbfff640 1 -- 172.21.15.55:0/4143243845 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608035070 con 0x7f96141019a0 2024-04-26T22:00:38.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.640+0000 7f96193b4640 1 -- 172.21.15.55:0/4143243845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96141a72c0 con 0x7f96141019a0 2024-04-26T22:00:38.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.640+0000 7f96193b4640 1 -- 172.21.15.55:0/4143243845 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96141a77e0 con 0x7f96141019a0 2024-04-26T22:00:38.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.642+0000 7f95fbfff640 1 -- 172.21.15.55:0/4143243845 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9608042030 con 0x7f96141019a0 2024-04-26T22:00:38.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.643+0000 7f95fbfff640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95fc0a3c40 0x7f95fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.643+0000 7f9612ffd640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95fc0a3c40 0x7f95fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.644+0000 7f95fbfff640 1 -- 172.21.15.55:0/4143243845 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f96080f1010 con 0x7f96141019a0 2024-04-26T22:00:38.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.644+0000 7f9612ffd640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95fc0a3c40 0x7f95fc0a6100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9604006fd0 tx=0x7f96040090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.645+0000 7f961c1ad740 1 Processor -- start 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f961c1ad740 1 -- start start 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e85c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e869f0 0x561fe1e86e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e873b0 0x561fe1e8dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f961c1ad740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561fe1de9d80 con 0x561fe1e873b0 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f961c1ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561fe1da3a20 con 0x561fe1e85820 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f961c1ad740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561fe1de5d40 con 0x561fe1e869f0 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f96137fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e873b0 0x561fe1e8dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f96137fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e873b0 0x561fe1e8dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60642/0 (socket says 172.21.15.55:60642) 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f96137fe640 1 -- 172.21.15.55:0/1094509942 learned_addr learned my addr 172.21.15.55:0/1094509942 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.646+0000 7f96127fc640 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e869f0 0x561fe1e86e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.647+0000 7f9612ffd640 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e85c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.647+0000 7f96137fe640 1 -- 172.21.15.55:0/1094509942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e869f0 msgr2=0x561fe1e86e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.647+0000 7f96137fe640 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e869f0 0x561fe1e86e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.647+0000 7f96137fe640 1 -- 172.21.15.55:0/1094509942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 msgr2=0x561fe1e85c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.647+0000 7f96137fe640 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e85c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.647+0000 7f96137fe640 1 -- 172.21.15.55:0/1094509942 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe1daec30 con 0x561fe1e873b0 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.647+0000 7f96137fe640 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e873b0 0x561fe1e8dc40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f960c009d00 tx=0x7f960c00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4de4f988b3a439 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.648+0000 7f95f97fa640 1 -- 172.21.15.55:0/1094509942 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c00bd40 con 0x561fe1e873b0 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.648+0000 7f95f97fa640 1 -- 172.21.15.55:0/1094509942 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960c017070 con 0x561fe1e873b0 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.648+0000 7f95f97fa640 1 -- 172.21.15.55:0/1094509942 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c0115f0 con 0x561fe1e873b0 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.648+0000 7f961c1ad740 1 -- 172.21.15.55:0/1094509942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e873b0 msgr2=0x561fe1e8dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.648+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e873b0 0x561fe1e8dc40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f960c009d00 tx=0x7f960c00b420 comp rx=0 tx=0).stop 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.648+0000 7f961c1ad740 1 -- 172.21.15.55:0/1094509942 shutdown_connections 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.649+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e873b0 0x561fe1e8dc40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.649+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e869f0 0x561fe1e86e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.649+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1094509942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e85c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.649+0000 7f961c1ad740 1 -- 172.21.15.55:0/1094509942 >> 172.21.15.55:0/1094509942 conn(0x561fe1dc6910 msgr2=0x561fe1e81120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.649+0000 7f961c1ad740 1 -- 172.21.15.55:0/1094509942 shutdown_connections 2024-04-26T22:00:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.649+0000 7f961c1ad740 1 -- 172.21.15.55:0/1094509942 wait complete. 2024-04-26T22:00:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.649+0000 7f961c1ad740 1 -- start start 2024-04-26T22:00:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.650+0000 7f961c1ad740 1 Processor -- start 2024-04-26T22:00:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e85820 0x561fe1e8faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e869f0 0x561fe1e8ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e97020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f961c1ad740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561fe1e8e3c0 con 0x561fe1e85820 2024-04-26T22:00:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f961c1ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561fe1d70dc0 con 0x561fe1e869f0 2024-04-26T22:00:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f961c1ad740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561fe1e8e540 con 0x561fe1e873b0 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f9612ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e85820 0x561fe1e8faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f9612ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e85820 0x561fe1e8faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60650/0 (socket says 172.21.15.55:60650) 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f9612ffd640 1 -- 172.21.15.55:0/1845475489 learned_addr learned my addr 172.21.15.55:0/1845475489 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f961c1ad740 10 client.0 _ll_register_callbacks cb 0x561fe1d8c600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f96137fe640 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e97020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f9612ffd640 1 -- 172.21.15.55:0/1845475489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 msgr2=0x561fe1e97020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f9612ffd640 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e97020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f9612ffd640 1 -- 172.21.15.55:0/1845475489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e869f0 msgr2=0x561fe1e8ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f9612ffd640 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e869f0 0x561fe1e8ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.651+0000 7f9612ffd640 1 -- 172.21.15.55:0/1845475489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe1afb3f0 con 0x561fe1e85820 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.652+0000 7f9612ffd640 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e85820 0x561fe1e8faa0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9604009fd0 tx=0x7f9604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a45506f904d987 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.652+0000 7f95f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.652+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1845475489 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604024020 con 0x561fe1e85820 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.652+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1845475489 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f960400f040 con 0x561fe1e85820 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 1 -- 172.21.15.55:0/1845475489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f960c017070 con 0x561fe1e85820 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1845475489 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9604020670 con 0x561fe1e85820 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 10 client.16620 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 1 -- 172.21.15.55:0/1845475489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561fe1bc57d0 con 0x561fe1e85820 2024-04-26T22:00:38.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1845475489 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9604020cc0 con 0x561fe1e85820 2024-04-26T22:00:38.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1845475489 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9604031020 con 0x561fe1e85820 2024-04-26T22:00:38.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 10 client.16620 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:38.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 10 client.16620 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:00:38.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 20 client.16620 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:38.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561fe1e9ba50 0x561fe1e9dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 4 client.16620 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:38.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.653+0000 7f961c1ad740 1 -- 172.21.15.55:0/1845475489 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f960c017070 con 0x561fe1e9ba50 2024-04-26T22:00:38.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.654+0000 7f96127fc640 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561fe1e9ba50 0x561fe1e9dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.655+0000 7f96127fc640 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561fe1e9ba50 0x561fe1e9dea0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4998f13e89aa09a8 server_cookie=f5e93dccf00f92a3 in_seq=0 out_seq=0 2024-04-26T22:00:38.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.655+0000 7f95f6ff5640 10 client.16620 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:00:38.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1845475489 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f960c017070 con 0x561fe1e9ba50 2024-04-26T22:00:38.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f95f6ff5640 10 client.16620 handle_command_reply: tid=0 2024-04-26T22:00:38.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 client.16620 shutdown 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 -- 172.21.15.55:0/1845475489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e85820 msgr2=0x561fe1e8faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e85820 0x561fe1e8faa0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9604009fd0 tx=0x7f9604004750 comp rx=0 tx=0).stop 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 -- 172.21.15.55:0/1845475489 shutdown_connections 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561fe1e9ba50 0x561fe1e9dea0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f95f6ff5640 0 client.16620 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e97020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e869f0 0x561fe1e8ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1845475489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e85820 0x561fe1e8faa0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 -- 172.21.15.55:0/1845475489 >> 172.21.15.55:0/1845475489 conn(0x561fe1dc6910 msgr2=0x561fe1e80280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.657+0000 7f961c1ad740 1 -- 172.21.15.55:0/1845475489 shutdown_connections 2024-04-26T22:00:38.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.658+0000 7f961c1ad740 1 -- 172.21.15.55:0/1845475489 wait complete. 2024-04-26T22:00:38.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.658+0000 7f961c1ad740 20 client.16620 trim_cache size 0 max 16384 2024-04-26T22:00:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.675+0000 7f961c1ad740 1 Processor -- start 2024-04-26T22:00:38.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.675+0000 7f961c1ad740 1 -- start start 2024-04-26T22:00:38.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e80c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e96fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e8fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f961c1ad740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f96141b2ff0 con 0x561fe1e869f0 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f961c1ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96141b08c0 con 0x561fe1e85820 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f961c1ad740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96141ae710 con 0x561fe1e873b0 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f9612ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e80c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f96137fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e8fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f96127fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e96fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f96137fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e8fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:39586/0 (socket says 172.21.15.55:39586) 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f96137fe640 1 -- 172.21.15.55:0/1912134459 learned_addr learned my addr 172.21.15.55:0/1912134459 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f96127fc640 1 -- 172.21.15.55:0/1912134459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 msgr2=0x561fe1e8fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f96127fc640 1 --2- 172.21.15.55:0/1912134459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e8fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.676+0000 7f96127fc640 1 -- 172.21.15.55:0/1912134459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 msgr2=0x561fe1e80c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.677+0000 7f96127fc640 1 --2- 172.21.15.55:0/1912134459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e80c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.677+0000 7f96127fc640 1 -- 172.21.15.55:0/1912134459 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe1e81dd0 con 0x561fe1e869f0 2024-04-26T22:00:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.677+0000 7f9612ffd640 1 --2- 172.21.15.55:0/1912134459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e80c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:00:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.677+0000 7f96127fc640 1 --2- 172.21.15.55:0/1912134459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e96fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f96080ee980 tx=0x7f96080eef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28dcd1accf53136 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.677+0000 7f95f8ff9640 1 -- 172.21.15.55:0/1912134459 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608049040 con 0x561fe1e869f0 2024-04-26T22:00:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.677+0000 7f95f8ff9640 1 -- 172.21.15.55:0/1912134459 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96080c5dc0 con 0x561fe1e869f0 2024-04-26T22:00:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.677+0000 7f95f8ff9640 1 -- 172.21.15.55:0/1912134459 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960803b460 con 0x561fe1e869f0 2024-04-26T22:00:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.679+0000 7f961c1ad740 1 -- 172.21.15.55:0/1912134459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 msgr2=0x561fe1e96fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.679+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1912134459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e96fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f96080ee980 tx=0x7f96080eef90 comp rx=0 tx=0).stop 2024-04-26T22:00:38.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.679+0000 7f961c1ad740 1 -- 172.21.15.55:0/1912134459 shutdown_connections 2024-04-26T22:00:38.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.679+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1912134459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e873b0 0x561fe1e8fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.679+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1912134459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e96fe0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.679+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1912134459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e85820 0x561fe1e80c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.680+0000 7f961c1ad740 1 -- 172.21.15.55:0/1912134459 >> 172.21.15.55:0/1912134459 conn(0x561fe1dc6910 msgr2=0x561fe1e8f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.680+0000 7f961c1ad740 1 -- 172.21.15.55:0/1912134459 shutdown_connections 2024-04-26T22:00:38.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.680+0000 7f961c1ad740 1 -- 172.21.15.55:0/1912134459 wait complete. 2024-04-26T22:00:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.680+0000 7f961c1ad740 1 -- start start 2024-04-26T22:00:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f961c1ad740 1 Processor -- start 2024-04-26T22:00:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e85820 0x561fe1e94410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e94950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f961c1ad740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e873b0 0x561fe1e95960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f961c1ad740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f96141aed80 con 0x561fe1e869f0 2024-04-26T22:00:38.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f961c1ad740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96141b01f0 con 0x561fe1e873b0 2024-04-26T22:00:38.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f961c1ad740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f96141af450 con 0x561fe1e85820 2024-04-26T22:00:38.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f961c1ad740 10 client.0 _ll_register_callbacks cb 0x7f9604006d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:38.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f96127fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e94950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f96127fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e94950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60670/0 (socket says 172.21.15.55:60670) 2024-04-26T22:00:38.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f96127fc640 1 -- 172.21.15.55:0/1768026361 learned_addr learned my addr 172.21.15.55:0/1768026361 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:38.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.681+0000 7f9612ffd640 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e85820 0x561fe1e94410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f96127fc640 1 -- 172.21.15.55:0/1768026361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e85820 msgr2=0x561fe1e94410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f96127fc640 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e85820 0x561fe1e94410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f96127fc640 1 -- 172.21.15.55:0/1768026361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e873b0 msgr2=0x561fe1e95960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f96127fc640 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e873b0 0x561fe1e95960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f96127fc640 1 -- 172.21.15.55:0/1768026361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe1e95f50 con 0x561fe1e869f0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f96127fc640 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e94950 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f96080ee9b0 tx=0x7f96080b97c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacb83a303343918 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f95f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1768026361 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96080eec90 con 0x561fe1e869f0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1768026361 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f960803bc40 con 0x561fe1e869f0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f961c1ad740 1 -- 172.21.15.55:0/1768026361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96080c5dc0 con 0x561fe1e869f0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1768026361 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f960803be40 con 0x561fe1e869f0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f961c1ad740 10 client.16626 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.682+0000 7f961c1ad740 1 -- 172.21.15.55:0/1768026361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561fe1e972c0 con 0x561fe1e869f0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1768026361 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96080c5dc0 con 0x561fe1e869f0 2024-04-26T22:00:38.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1768026361 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f96080c59c0 con 0x561fe1e869f0 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f961c1ad740 10 client.16626 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f961c1ad740 10 client.16626 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f961c1ad740 20 client.16626 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561fe1e9ba50 0x561fe1f04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f961c1ad740 4 client.16626 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f961c1ad740 1 -- 172.21.15.55:0/1768026361 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fe1df7990 con 0x561fe1e9ba50 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.683+0000 7f9612ffd640 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561fe1e9ba50 0x561fe1f04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f9612ffd640 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561fe1e9ba50 0x561fe1f04760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3af0498fff56b52 server_cookie=438823542f6f7e26 in_seq=0 out_seq=0 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f95f6ff5640 10 client.16626 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f95f6ff5640 1 -- 172.21.15.55:0/1768026361 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fe1df7990 con 0x561fe1e9ba50 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f95f6ff5640 10 client.16626 handle_command_reply: tid=0 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f961c1ad740 1 client.16626 shutdown 2024-04-26T22:00:38.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f961c1ad740 1 -- 172.21.15.55:0/1768026361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 msgr2=0x561fe1e94950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e94950 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f96080ee9b0 tx=0x7f96080b97c0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f961c1ad740 1 -- 172.21.15.55:0/1768026361 shutdown_connections 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.684+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561fe1e9ba50 0x561fe1f04760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561fe1e873b0 0x561fe1e95960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561fe1e869f0 0x561fe1e94950 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f95f6ff5640 0 client.16626 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f961c1ad740 1 --2- 172.21.15.55:0/1768026361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561fe1e85820 0x561fe1e94410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f961c1ad740 1 -- 172.21.15.55:0/1768026361 >> 172.21.15.55:0/1768026361 conn(0x561fe1dc6910 msgr2=0x561fe1e7f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f961c1ad740 1 -- 172.21.15.55:0/1768026361 shutdown_connections 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f961c1ad740 1 -- 172.21.15.55:0/1768026361 wait complete. 2024-04-26T22:00:38.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f961c1ad740 20 client.16626 trim_cache size 0 max 16384 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f96193b4640 1 -- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95fc0a3c40 msgr2=0x7f95fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f96193b4640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95fc0a3c40 0x7f95fc0a6100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9604006fd0 tx=0x7f96040090e0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f96193b4640 1 -- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 msgr2=0x7f961419f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.685+0000 7f96193b4640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 0x7f961419f950 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9608006010 tx=0x7f9608004430 comp rx=0 tx=0).stop 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.686+0000 7f96193b4640 1 -- 172.21.15.55:0/4143243845 shutdown_connections 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.686+0000 7f96193b4640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f95fc0a3c40 0x7f95fc0a6100 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.686+0000 7f96193b4640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9614102360 0x7f96141a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.686+0000 7f96193b4640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96141019a0 0x7f961419f950 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.686+0000 7f96193b4640 1 --2- 172.21.15.55:0/4143243845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9614100c40 0x7f961419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:38.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.686+0000 7f96193b4640 1 -- 172.21.15.55:0/4143243845 >> 172.21.15.55:0/4143243845 conn(0x7f96140fc570 msgr2=0x7f9614105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:38.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.686+0000 7f96193b4640 1 -- 172.21.15.55:0/4143243845 shutdown_connections 2024-04-26T22:00:38.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:38.686+0000 7f96193b4640 1 -- 172.21.15.55:0/4143243845 wait complete. 2024-04-26T22:00:38.711 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:00:39.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.106+0000 7fab9bbbf640 1 -- 172.21.15.55:0/3855296621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 msgr2=0x7fab94102980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.106+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/3855296621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 0x7fab94102980 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009ca0 tx=0x7fab8c02a8f0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 -- 172.21.15.55:0/3855296621 shutdown_connections 2024-04-26T22:00:39.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/3855296621 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 0x7fab94102980 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/3855296621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab94101b60 0x7fab94101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/3855296621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab94101200 0x7fab94101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 -- 172.21.15.55:0/3855296621 >> 172.21.15.55:0/3855296621 conn(0x7fab940fce20 msgr2=0x7fab940ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 -- 172.21.15.55:0/3855296621 shutdown_connections 2024-04-26T22:00:39.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 -- 172.21.15.55:0/3855296621 wait complete. 2024-04-26T22:00:39.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 Processor -- start 2024-04-26T22:00:39.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.107+0000 7fab9bbbf640 1 -- start start 2024-04-26T22:00:39.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9bbbf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab94101200 0x7fab9419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9bbbf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab94101b60 0x7fab9419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9bbbf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 0x7fab941a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9bbbf640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fab94110810 con 0x7fab94102500 2024-04-26T22:00:39.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9bbbf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab94110690 con 0x7fab94101b60 2024-04-26T22:00:39.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9bbbf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab94110990 con 0x7fab94101200 2024-04-26T22:00:39.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9a15c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 0x7fab941a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9a15c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 0x7fab941a2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60682/0 (socket says 172.21.15.55:60682) 2024-04-26T22:00:39.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.108+0000 7fab9a15c640 1 -- 172.21.15.55:0/2579797746 learned_addr learned my addr 172.21.15.55:0/2579797746 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.109+0000 7fab9a15c640 1 -- 172.21.15.55:0/2579797746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab94101200 msgr2=0x7fab9419af70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:39.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.109+0000 7fab9a15c640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab94101200 0x7fab9419af70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.109+0000 7fab9a15c640 1 -- 172.21.15.55:0/2579797746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab94101b60 msgr2=0x7fab9419b4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:39.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.109+0000 7fab9a15c640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab94101b60 0x7fab9419b4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.109+0000 7fab9a15c640 1 -- 172.21.15.55:0/2579797746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab941a2b90 con 0x7fab94102500 2024-04-26T22:00:39.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.109+0000 7fab9a15c640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 0x7fab941a2520 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fab8c003ba0 tx=0x7fab8c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.110+0000 7fab8affd640 1 -- 172.21.15.55:0/2579797746 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c031860 con 0x7fab94102500 2024-04-26T22:00:39.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.110+0000 7fab9bbbf640 1 -- 172.21.15.55:0/2579797746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab941a2e20 con 0x7fab94102500 2024-04-26T22:00:39.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.111+0000 7fab9bbbf640 1 -- 172.21.15.55:0/2579797746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab941a33b0 con 0x7fab94102500 2024-04-26T22:00:39.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.111+0000 7fab8affd640 1 -- 172.21.15.55:0/2579797746 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab8c02d070 con 0x7fab94102500 2024-04-26T22:00:39.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.111+0000 7fab8affd640 1 -- 172.21.15.55:0/2579797746 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c02fce0 con 0x7fab94102500 2024-04-26T22:00:39.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.118+0000 7fab9e9bb740 1 Processor -- start 2024-04-26T22:00:39.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.118+0000 7fab9e9bb740 1 -- start start 2024-04-26T22:00:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.119+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925c820 0x5607e925cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.119+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e925de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.119+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e9264c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.119+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5607e91c0d80 con 0x5607e925e3b0 2024-04-26T22:00:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.119+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5607e917aa20 con 0x5607e925d9f0 2024-04-26T22:00:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.119+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5607e91bcd40 con 0x5607e925c820 2024-04-26T22:00:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.120+0000 7fab8affd640 1 -- 172.21.15.55:0/2579797746 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fab8c004040 con 0x7fab94102500 2024-04-26T22:00:39.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.120+0000 7fab9995b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e925de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.120+0000 7fab9995b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e925de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:44822/0 (socket says 172.21.15.55:44822) 2024-04-26T22:00:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.120+0000 7fab9995b640 1 -- 172.21.15.55:0/1689602755 learned_addr learned my addr 172.21.15.55:0/1689602755 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.121+0000 7fab9995b640 1 -- 172.21.15.55:0/1689602755 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925c820 msgr2=0x5607e925cc40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.121+0000 7fab9995b640 1 --2- 172.21.15.55:0/1689602755 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925c820 0x5607e925cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.121+0000 7fab9995b640 1 -- 172.21.15.55:0/1689602755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 msgr2=0x5607e9264c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.121+0000 7fab9995b640 1 --2- 172.21.15.55:0/1689602755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e9264c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.121+0000 7fab9995b640 1 -- 172.21.15.55:0/1689602755 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e9185c30 con 0x5607e925d9f0 2024-04-26T22:00:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.122+0000 7fab8affd640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab800a3d40 0x7fab800a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.122+0000 7fab9995b640 1 --2- 172.21.15.55:0/1689602755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e925de70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab90009770 tx=0x7fab90009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5be5ef723a003376 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.123+0000 7fab8affd640 1 -- 172.21.15.55:0/2579797746 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fab8c0f1c80 con 0x7fab94102500 2024-04-26T22:00:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.124+0000 7fab9915a640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab800a3d40 0x7fab800a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.125+0000 7fab9915a640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab800a3d40 0x7fab800a6200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5607e925d8f0 tx=0x7fab84006d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.125+0000 7fab7ffff640 1 -- 172.21.15.55:0/1689602755 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90015020 con 0x5607e925d9f0 2024-04-26T22:00:39.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.125+0000 7fab7ffff640 1 -- 172.21.15.55:0/1689602755 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab90011420 con 0x5607e925d9f0 2024-04-26T22:00:39.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.125+0000 7fab7ffff640 1 -- 172.21.15.55:0/1689602755 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab900105e0 con 0x5607e925d9f0 2024-04-26T22:00:39.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.126+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1689602755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 msgr2=0x5607e925de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.126+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1689602755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e925de70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab90009770 tx=0x7fab90009c30 comp rx=0 tx=0).stop 2024-04-26T22:00:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.126+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1689602755 shutdown_connections 2024-04-26T22:00:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.126+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1689602755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e9264c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.126+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1689602755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e925de70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.126+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1689602755 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925c820 0x5607e925cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.126+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1689602755 >> 172.21.15.55:0/1689602755 conn(0x5607e919d910 msgr2=0x5607e9258120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.127+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1689602755 shutdown_connections 2024-04-26T22:00:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.127+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1689602755 wait complete. 2024-04-26T22:00:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.127+0000 7fab9e9bb740 1 -- start start 2024-04-26T22:00:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.127+0000 7fab9e9bb740 1 Processor -- start 2024-04-26T22:00:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e926f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 0x5607e926f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9995b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e926f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9995b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e926f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60700/0 (socket says 172.21.15.55:60700) 2024-04-26T22:00:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9995b640 1 -- 172.21.15.55:0/1683956903 learned_addr learned my addr 172.21.15.55:0/1683956903 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 0x5607e92766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5607e92653c0 con 0x5607e925c820 2024-04-26T22:00:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5607e9147dc0 con 0x5607e925e3b0 2024-04-26T22:00:39.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5607e9265540 con 0x5607e925d9f0 2024-04-26T22:00:39.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.129+0000 7fab9e9bb740 10 client.0 _ll_register_callbacks cb 0x5607e9163600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:39.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.130+0000 7fab9995b640 1 -- 172.21.15.55:0/1683956903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 msgr2=0x5607e926f690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:39.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.130+0000 7fab9995b640 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 0x5607e926f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.130+0000 7fab9995b640 1 -- 172.21.15.55:0/1683956903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 msgr2=0x5607e92766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.130+0000 7fab9995b640 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 0x5607e92766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.130+0000 7fab9995b640 1 -- 172.21.15.55:0/1683956903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e8ed23f0 con 0x5607e925c820 2024-04-26T22:00:39.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.130+0000 7fab9995b640 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e926f150 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fab90011830 tx=0x7fab90011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866803f6141db20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.130+0000 7fab7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:39.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.131+0000 7fab7d7fa640 1 -- 172.21.15.55:0/1683956903 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90004460 con 0x5607e925c820 2024-04-26T22:00:39.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.131+0000 7fab7d7fa640 1 -- 172.21.15.55:0/1683956903 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fab9000b8d0 con 0x5607e925c820 2024-04-26T22:00:39.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.131+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab90011420 con 0x5607e925c820 2024-04-26T22:00:39.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.131+0000 7fab7d7fa640 1 -- 172.21.15.55:0/1683956903 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fab90004600 con 0x5607e925c820 2024-04-26T22:00:39.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab9e9bb740 10 client.16635 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5607e8f9c7d0 con 0x5607e925c820 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab7d7fa640 1 -- 172.21.15.55:0/1683956903 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab90011420 con 0x5607e925c820 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab7d7fa640 1 -- 172.21.15.55:0/1683956903 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fab90023470 con 0x5607e925c820 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab9e9bb740 10 client.16635 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab9e9bb740 10 client.16635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab9e9bb740 20 client.16635 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5607e927ac70 0x5607e927d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab9e9bb740 4 client.16635 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.132+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e91ce990 con 0x5607e927ac70 2024-04-26T22:00:39.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.133+0000 7fab9915a640 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5607e927ac70 0x5607e927d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.134+0000 7fab7d7fa640 10 client.16635 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:00:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.134+0000 7fab9915a640 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5607e927ac70 0x5607e927d110 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b26cc70ebd50fb8c server_cookie=aebb84f8f7429b66 in_seq=0 out_seq=0 2024-04-26T22:00:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.135+0000 7fab7d7fa640 1 -- 172.21.15.55:0/1683956903 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607e91ce990 con 0x5607e927ac70 2024-04-26T22:00:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.135+0000 7fab7d7fa640 10 client.16635 handle_command_reply: tid=0 2024-04-26T22:00:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.135+0000 7fab9e9bb740 1 client.16635 shutdown 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 msgr2=0x5607e926f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e926f150 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fab90011830 tx=0x7fab90011860 comp rx=0 tx=0).stop 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 shutdown_connections 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5607e927ac70 0x5607e927d110 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 0x5607e92766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab7d7fa640 0 client.16635 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 0x5607e926f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1683956903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e926f150 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 >> 172.21.15.55:0/1683956903 conn(0x5607e919d910 msgr2=0x5607e9257280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 shutdown_connections 2024-04-26T22:00:39.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.136+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1683956903 wait complete. 2024-04-26T22:00:39.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.137+0000 7fab9e9bb740 20 client.16635 trim_cache size 0 max 16384 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9e9bb740 1 Processor -- start 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9e9bb740 1 -- start start 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e92582b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 0x5607e9276690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 0x5607e926ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fab78010e50 con 0x5607e925c820 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab7800e720 con 0x5607e925e3b0 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab7800c570 con 0x5607e925d9f0 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9995b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e92582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9995b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e92582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60712/0 (socket says 172.21.15.55:60712) 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9995b640 1 -- 172.21.15.55:0/1975224297 learned_addr learned my addr 172.21.15.55:0/1975224297 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9a15c640 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 0x5607e926ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.155+0000 7fab9915a640 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 0x5607e9276690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab9995b640 1 -- 172.21.15.55:0/1975224297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 msgr2=0x5607e9276690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab9995b640 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 0x5607e9276690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab9995b640 1 -- 172.21.15.55:0/1975224297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 msgr2=0x5607e926ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab9995b640 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 0x5607e926ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab9995b640 1 -- 172.21.15.55:0/1975224297 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e925d470 con 0x5607e925c820 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab9995b640 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e92582b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fab90023b60 tx=0x7fab90009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca19132f4037df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab7f7fe640 1 -- 172.21.15.55:0/1975224297 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90015070 con 0x5607e925c820 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab7f7fe640 1 -- 172.21.15.55:0/1975224297 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab90032070 con 0x5607e925c820 2024-04-26T22:00:39.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.156+0000 7fab7f7fe640 1 -- 172.21.15.55:0/1975224297 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90004420 con 0x5607e925c820 2024-04-26T22:00:39.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.157+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1975224297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 msgr2=0x5607e92582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.157+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e92582b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fab90023b60 tx=0x7fab90009e80 comp rx=0 tx=0).stop 2024-04-26T22:00:39.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.157+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1975224297 shutdown_connections 2024-04-26T22:00:39.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.157+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925e3b0 0x5607e926ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.157+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925d9f0 0x5607e9276690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.157+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/1975224297 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925c820 0x5607e92582b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.157+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1975224297 >> 172.21.15.55:0/1975224297 conn(0x5607e919d910 msgr2=0x5607e9257ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.158+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1975224297 shutdown_connections 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.158+0000 7fab9e9bb740 1 -- 172.21.15.55:0/1975224297 wait complete. 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.158+0000 7fab9e9bb740 1 -- start start 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.158+0000 7fab9e9bb740 1 Processor -- start 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925c820 0x5607e9274680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e92dd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9e9bb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e92df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fab7800cbe0 con 0x5607e925e3b0 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab7800e050 con 0x5607e925d9f0 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9e9bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fab7800d2b0 con 0x5607e925c820 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9e9bb740 10 client.0 _ll_register_callbacks cb 0x7fab900095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:39.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9a15c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e92df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.159+0000 7fab9a15c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e92df7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60720/0 (socket says 172.21.15.55:60720) 2024-04-26T22:00:39.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.160+0000 7fab9a15c640 1 -- 172.21.15.55:0/17315540 learned_addr learned my addr 172.21.15.55:0/17315540 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.160+0000 7fab9915a640 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e92dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.160+0000 7fab9a15c640 1 -- 172.21.15.55:0/17315540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925c820 msgr2=0x5607e9274680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.160+0000 7fab9a15c640 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925c820 0x5607e9274680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.160+0000 7fab9a15c640 1 -- 172.21.15.55:0/17315540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 msgr2=0x5607e92dd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.160+0000 7fab9a15c640 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e92dd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.160+0000 7fab9a15c640 1 -- 172.21.15.55:0/17315540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e92dfea0 con 0x5607e925e3b0 2024-04-26T22:00:39.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.161+0000 7fab9a15c640 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e92df7c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0bdb70 tx=0x7fab8c02f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b410a2fc904f67d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.161+0000 7fab7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.161+0000 7fab7d7fa640 1 -- 172.21.15.55:0/17315540 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c02fe80 con 0x5607e925e3b0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.161+0000 7fab7d7fa640 1 -- 172.21.15.55:0/17315540 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fab8c0ef7e0 con 0x5607e925e3b0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.161+0000 7fab9e9bb740 1 -- 172.21.15.55:0/17315540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab90032070 con 0x5607e925e3b0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.161+0000 7fab7d7fa640 1 -- 172.21.15.55:0/17315540 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fab8c0d7220 con 0x5607e925e3b0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.161+0000 7fab9e9bb740 10 client.16641 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.161+0000 7fab9e9bb740 1 -- 172.21.15.55:0/17315540 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5607e92e1270 con 0x5607e925e3b0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.162+0000 7fab7d7fa640 1 -- 172.21.15.55:0/17315540 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab90032070 con 0x5607e925e3b0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.162+0000 7fab7d7fa640 1 -- 172.21.15.55:0/17315540 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fab8c09cce0 con 0x5607e925e3b0 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.162+0000 7fab9e9bb740 10 client.16641 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.162+0000 7fab9e9bb740 10 client.16641 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.162+0000 7fab9e9bb740 20 client.16641 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:39.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.162+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5607e927ac70 0x5607e92e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.162+0000 7fab9e9bb740 4 client.16641 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:00:39.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.162+0000 7fab9e9bb740 1 -- 172.21.15.55:0/17315540 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607e91c8800 con 0x5607e927ac70 2024-04-26T22:00:39.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.163+0000 7fab9995b640 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5607e927ac70 0x5607e92e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.165+0000 7fab9995b640 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5607e927ac70 0x5607e92e3a80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6d42496080eb1cc server_cookie=4f30163d7f493e1a in_seq=0 out_seq=0 2024-04-26T22:00:39.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.165+0000 7fab7d7fa640 10 client.16641 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:00:39.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.166+0000 7fab7d7fa640 1 -- 172.21.15.55:0/17315540 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607e91c8800 con 0x5607e927ac70 2024-04-26T22:00:39.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.166+0000 7fab7d7fa640 10 client.16641 handle_command_reply: tid=0 2024-04-26T22:00:39.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.166+0000 7fab9e9bb740 1 client.16641 shutdown 2024-04-26T22:00:39.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.166+0000 7fab9e9bb740 1 -- 172.21.15.55:0/17315540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 msgr2=0x5607e92df7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.166+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e92df7c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0bdb70 tx=0x7fab8c02f700 comp rx=0 tx=0).stop 2024-04-26T22:00:39.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 1 -- 172.21.15.55:0/17315540 shutdown_connections 2024-04-26T22:00:39.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5607e927ac70 0x5607e92e3a80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab7d7fa640 0 client.16641 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:00:39.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5607e925e3b0 0x5607e92df7c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5607e925d9f0 0x5607e92dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 1 --2- 172.21.15.55:0/17315540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5607e925c820 0x5607e9274680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 1 -- 172.21.15.55:0/17315540 >> 172.21.15.55:0/17315540 conn(0x5607e919d910 msgr2=0x5607e926ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 1 -- 172.21.15.55:0/17315540 shutdown_connections 2024-04-26T22:00:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 1 -- 172.21.15.55:0/17315540 wait complete. 2024-04-26T22:00:39.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.167+0000 7fab9e9bb740 20 client.16641 trim_cache size 0 max 16384 2024-04-26T22:00:39.171 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.168+0000 7fab9bbbf640 1 -- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab800a3d40 msgr2=0x7fab800a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.168+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab800a3d40 0x7fab800a6200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5607e925d8f0 tx=0x7fab84006d10 comp rx=0 tx=0).stop 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.168+0000 7fab9bbbf640 1 -- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 msgr2=0x7fab941a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.168+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 0x7fab941a2520 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fab8c003ba0 tx=0x7fab8c003bd0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.169+0000 7fab9bbbf640 1 -- 172.21.15.55:0/2579797746 shutdown_connections 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.169+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fab800a3d40 0x7fab800a6200 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.169+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94102500 0x7fab941a2520 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.169+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab94101b60 0x7fab9419b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.169+0000 7fab9bbbf640 1 --2- 172.21.15.55:0/2579797746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab94101200 0x7fab9419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.169+0000 7fab9bbbf640 1 -- 172.21.15.55:0/2579797746 >> 172.21.15.55:0/2579797746 conn(0x7fab940fce20 msgr2=0x7fab940fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.169+0000 7fab9bbbf640 1 -- 172.21.15.55:0/2579797746 shutdown_connections 2024-04-26T22:00:39.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.169+0000 7fab9bbbf640 1 -- 172.21.15.55:0/2579797746 wait complete. 2024-04-26T22:00:39.196 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:00:39.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:39 smithi055 ceph-mon[29221]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 868 MiB/s wr, 1.42k op/s 2024-04-26T22:00:39.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:39 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3909354939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:39.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:39 smithi064 ceph-mon[32765]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 868 MiB/s wr, 1.42k op/s 2024-04-26T22:00:39.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:39 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3909354939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:39.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:39 smithi165 ceph-mon[33296]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 868 MiB/s wr, 1.42k op/s 2024-04-26T22:00:39.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:39 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3909354939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:00:39.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.584+0000 7fc1f9e20640 1 -- 172.21.15.55:0/599694325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 msgr2=0x7fc1f4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.584+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/599694325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 0x7fc1f4109cd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00f0a0 tx=0x7fc1ec02e6f0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.585+0000 7fc1f9e20640 1 -- 172.21.15.55:0/599694325 shutdown_connections 2024-04-26T22:00:39.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.585+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/599694325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 0x7fc1f4109cd0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.585+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/599694325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1f4102a80 0x7fc1f4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.585+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/599694325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1f4101890 0x7fc1f4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.585+0000 7fc1f9e20640 1 -- 172.21.15.55:0/599694325 >> 172.21.15.55:0/599694325 conn(0x7fc1f40fce20 msgr2=0x7fc1f40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.585+0000 7fc1f9e20640 1 -- 172.21.15.55:0/599694325 shutdown_connections 2024-04-26T22:00:39.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.585+0000 7fc1f9e20640 1 -- 172.21.15.55:0/599694325 wait complete. 2024-04-26T22:00:39.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.587+0000 7fc1f9e20640 1 Processor -- start 2024-04-26T22:00:39.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.587+0000 7fc1f9e20640 1 -- start start 2024-04-26T22:00:39.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.588+0000 7fc1f9e20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1f4101890 0x7fc1f419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.588+0000 7fc1f9e20640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1f4102a80 0x7fc1f419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.588+0000 7fc1f9e20640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 0x7fc1f41a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.588+0000 7fc1f9e20640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f410c310 con 0x7fc1f4103440 2024-04-26T22:00:39.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.588+0000 7fc1f9e20640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f410c190 con 0x7fc1f4101890 2024-04-26T22:00:39.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.588+0000 7fc1f9e20640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f410c490 con 0x7fc1f4102a80 2024-04-26T22:00:39.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.589+0000 7fc1f3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 0x7fc1f41a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.589+0000 7fc1f3fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 0x7fc1f41a6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60736/0 (socket says 172.21.15.55:60736) 2024-04-26T22:00:39.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.589+0000 7fc1f3fff640 1 -- 172.21.15.55:0/1798063972 learned_addr learned my addr 172.21.15.55:0/1798063972 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.589+0000 7fc1f3fff640 1 -- 172.21.15.55:0/1798063972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1f4102a80 msgr2=0x7fc1f419f6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:39.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.589+0000 7fc1f3fff640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1f4102a80 0x7fc1f419f6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.589+0000 7fc1f3fff640 1 -- 172.21.15.55:0/1798063972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1f4101890 msgr2=0x7fc1f419f1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:39.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.589+0000 7fc1f3fff640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1f4101890 0x7fc1f419f1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.589+0000 7fc1f3fff640 1 -- 172.21.15.55:0/1798063972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f41a6dc0 con 0x7fc1f4103440 2024-04-26T22:00:39.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.590+0000 7fc1f3fff640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 0x7fc1f41a6750 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec02ece0 tx=0x7fc1ec036eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.591+0000 7fc1f0ff9640 1 -- 172.21.15.55:0/1798063972 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec03a030 con 0x7fc1f4103440 2024-04-26T22:00:39.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.591+0000 7fc1f0ff9640 1 -- 172.21.15.55:0/1798063972 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1ec03c070 con 0x7fc1f4103440 2024-04-26T22:00:39.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.591+0000 7fc1f9e20640 1 -- 172.21.15.55:0/1798063972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f41a7050 con 0x7fc1f4103440 2024-04-26T22:00:39.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.591+0000 7fc1f0ff9640 1 -- 172.21.15.55:0/1798063972 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec004770 con 0x7fc1f4103440 2024-04-26T22:00:39.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.591+0000 7fc1f9e20640 1 -- 172.21.15.55:0/1798063972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f41a7610 con 0x7fc1f4103440 2024-04-26T22:00:39.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.602+0000 7fc1fcc15740 1 Processor -- start 2024-04-26T22:00:39.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.602+0000 7fc1f0ff9640 1 -- 172.21.15.55:0/1798063972 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc1ec03e030 con 0x7fc1f4103440 2024-04-26T22:00:39.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.602+0000 7fc1fcc15740 1 -- start start 2024-04-26T22:00:39.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.603+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba7820 0x562c2fba7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.603+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fba8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.603+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba93b0 0x562c2fbafc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.603+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562c2fb0bd80 con 0x562c2fba89f0 2024-04-26T22:00:39.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.603+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562c2fac5a20 con 0x562c2fba93b0 2024-04-26T22:00:39.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.603+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c2fb07d40 con 0x562c2fba7820 2024-04-26T22:00:39.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.604+0000 7fc1f0ff9640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1dc0a3cc0 0x7fc1dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.604+0000 7fc1f0ff9640 1 -- 172.21.15.55:0/1798063972 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1ec0f0ac0 con 0x7fc1f4103440 2024-04-26T22:00:39.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.604+0000 7fc1f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba93b0 0x562c2fbafc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.605+0000 7fc1f2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba93b0 0x562c2fbafc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:44886/0 (socket says 172.21.15.55:44886) 2024-04-26T22:00:39.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.605+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/1802714039 learned_addr learned my addr 172.21.15.55:0/1802714039 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.605+0000 7fc1f3fff640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1dc0a3cc0 0x7fc1dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.605+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/1802714039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba7820 0x562c2fba7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.607+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/1802714039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba93b0 msgr2=0x562c2fbafc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.607+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/1802714039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba93b0 0x562c2fbafc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.607+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/1802714039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 msgr2=0x562c2fba8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.607+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/1802714039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fba8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.607+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/1802714039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2fad0c30 con 0x562c2fba7820 2024-04-26T22:00:39.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.608+0000 7fc1f3fff640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1dc0a3cc0 0x7fc1dc0a6180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0b95d0 tx=0x7fc1ec00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.608+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/1802714039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba7820 0x562c2fba7c40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8015370 tx=0x7fc1e8000fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ffb7dead0610e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.608+0000 7fc1d9ffb640 1 -- 172.21.15.55:0/1802714039 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8012040 con 0x562c2fba7820 2024-04-26T22:00:39.609 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.609+0000 7fc1d9ffb640 1 -- 172.21.15.55:0/1802714039 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e8008e30 con 0x562c2fba7820 2024-04-26T22:00:39.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.609+0000 7fc1d9ffb640 1 -- 172.21.15.55:0/1802714039 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8004a30 con 0x562c2fba7820 2024-04-26T22:00:39.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.610+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1802714039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba7820 msgr2=0x562c2fba7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.610+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1802714039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba7820 0x562c2fba7c40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8015370 tx=0x7fc1e8000fd0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.611+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1802714039 shutdown_connections 2024-04-26T22:00:39.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.611+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1802714039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba93b0 0x562c2fbafc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.611+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1802714039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fba8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.611+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1802714039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba7820 0x562c2fba7c40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.611+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1802714039 >> 172.21.15.55:0/1802714039 conn(0x562c2fae8910 msgr2=0x562c2fba3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.611+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1802714039 shutdown_connections 2024-04-26T22:00:39.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.611+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1802714039 wait complete. 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.611+0000 7fc1fcc15740 1 -- start start 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.612+0000 7fc1fcc15740 1 Processor -- start 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.612+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba7820 0x562c2fbb1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.612+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fbb2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.612+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.612+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562c2fbb0360 con 0x562c2fba89f0 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.612+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562c2fa92dc0 con 0x562c2fba7820 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.612+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562c2fbb04e0 con 0x562c2fba93b0 2024-04-26T22:00:39.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.612+0000 7fc1fcc15740 10 client.0 _ll_register_callbacks cb 0x562c2faae600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fbb2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fbb2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60764/0 (socket says 172.21.15.55:60764) 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 -- 172.21.15.55:0/3619863259 learned_addr learned my addr 172.21.15.55:0/3619863259 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 -- 172.21.15.55:0/3619863259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 msgr2=0x562c2fbb9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 -- 172.21.15.55:0/3619863259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba7820 msgr2=0x562c2fbb1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba7820 0x562c2fbb1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 -- 172.21.15.55:0/3619863259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2f81d3f0 con 0x562c2fba89f0 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.613+0000 7fc1f37fe640 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fbb2000 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400afd0 tx=0x7fc1e400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44e09cf8d96ca9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.614+0000 7fc1d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.614+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/3619863259 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4004220 con 0x562c2fba89f0 2024-04-26T22:00:39.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.614+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/3619863259 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1e40043c0 con 0x562c2fba89f0 2024-04-26T22:00:39.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.614+0000 7fc1fcc15740 1 -- 172.21.15.55:0/3619863259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e8008e30 con 0x562c2fba89f0 2024-04-26T22:00:39.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.614+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/3619863259 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1e400f660 con 0x562c2fba89f0 2024-04-26T22:00:39.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.614+0000 7fc1fcc15740 10 client.16650 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:39.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.614+0000 7fc1fcc15740 1 -- 172.21.15.55:0/3619863259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562c2f8e77d0 con 0x562c2fba89f0 2024-04-26T22:00:39.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.615+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/3619863259 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e8008e30 con 0x562c2fba89f0 2024-04-26T22:00:39.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.615+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/3619863259 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc1e4003b40 con 0x562c2fba89f0 2024-04-26T22:00:39.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.615+0000 7fc1fcc15740 10 client.16650 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:39.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.615+0000 7fc1fcc15740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:39.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.615+0000 7fc1fcc15740 20 client.16650 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:39.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.615+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2fbbd9d0 0x562c2fbbfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.615+0000 7fc1fcc15740 4 client.16650 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:39.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.615+0000 7fc1fcc15740 1 -- 172.21.15.55:0/3619863259 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c2fb19990 con 0x562c2fbbd9d0 2024-04-26T22:00:39.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.616+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2fbbd9d0 0x562c2fbbfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.618+0000 7fc1d2ffd640 10 client.16650 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:39.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.618+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2fbbd9d0 0x562c2fbbfe20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd933c576473ab90 server_cookie=5f06a0b75c29c39d in_seq=0 out_seq=0 2024-04-26T22:00:39.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/3619863259 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c2fb19990 con 0x562c2fbbd9d0 2024-04-26T22:00:39.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1d2ffd640 10 client.16650 handle_command_reply: tid=0 2024-04-26T22:00:39.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 client.16650 shutdown 2024-04-26T22:00:39.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 -- 172.21.15.55:0/3619863259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 msgr2=0x562c2fbb2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fbb2000 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400afd0 tx=0x7fc1e400c7e0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 -- 172.21.15.55:0/3619863259 shutdown_connections 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2fbbd9d0 0x562c2fbbfe20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba89f0 0x562c2fbb2000 secure :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400afd0 tx=0x7fc1e400c7e0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/3619863259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba7820 0x562c2fbb1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 -- 172.21.15.55:0/3619863259 >> 172.21.15.55:0/3619863259 conn(0x562c2fae8910 msgr2=0x562c2fba2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1d2ffd640 0 client.16650 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 -- 172.21.15.55:0/3619863259 shutdown_connections 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.623+0000 7fc1fcc15740 1 -- 172.21.15.55:0/3619863259 wait complete. 2024-04-26T22:00:39.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.624+0000 7fc1fcc15740 20 client.16650 trim_cache size 0 max 16384 2024-04-26T22:00:39.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.654+0000 7fc1fcc15740 1 Processor -- start 2024-04-26T22:00:39.655 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.654+0000 7fc1fcc15740 1 -- start start 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.654+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba7820 0x562c2fba5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.654+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.654+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fbaca60 0x562c2fba6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.654+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41b2e80 con 0x562c2fbaca60 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.654+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41b0750 con 0x562c2fba7820 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.654+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41ae5a0 con 0x562c2fba93b0 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fbaca60 0x562c2fba6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fbaca60 0x562c2fba6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60780/0 (socket says 172.21.15.55:60780) 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/2354150446 learned_addr learned my addr 172.21.15.55:0/2354150446 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/2354150446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 msgr2=0x562c2fbb9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/2354150446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/2354150446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba7820 msgr2=0x562c2fba5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/2354150446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba7820 0x562c2fba5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/2354150446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2fbbc2f0 con 0x562c2fbaca60 2024-04-26T22:00:39.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.655+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/2354150446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fbaca60 0x562c2fba6320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8015370 tx=0x7fc1e8009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51d0012b16e8576 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.656+0000 7fc1d97fa640 1 -- 172.21.15.55:0/2354150446 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8004020 con 0x562c2fbaca60 2024-04-26T22:00:39.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.656+0000 7fc1d97fa640 1 -- 172.21.15.55:0/2354150446 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e8008e30 con 0x562c2fbaca60 2024-04-26T22:00:39.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.656+0000 7fc1d97fa640 1 -- 172.21.15.55:0/2354150446 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80073a0 con 0x562c2fbaca60 2024-04-26T22:00:39.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 -- 172.21.15.55:0/2354150446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fbaca60 msgr2=0x562c2fba6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/2354150446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fbaca60 0x562c2fba6320 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8015370 tx=0x7fc1e8009e00 comp rx=0 tx=0).stop 2024-04-26T22:00:39.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 -- 172.21.15.55:0/2354150446 shutdown_connections 2024-04-26T22:00:39.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/2354150446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fbaca60 0x562c2fba6320 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/2354150446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/2354150446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fba7820 0x562c2fba5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 -- 172.21.15.55:0/2354150446 >> 172.21.15.55:0/2354150446 conn(0x562c2fae8910 msgr2=0x562c2fbac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 -- 172.21.15.55:0/2354150446 shutdown_connections 2024-04-26T22:00:39.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.657+0000 7fc1fcc15740 1 -- 172.21.15.55:0/2354150446 wait complete. 2024-04-26T22:00:39.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.658+0000 7fc1fcc15740 1 -- start start 2024-04-26T22:00:39.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.658+0000 7fc1fcc15740 1 Processor -- start 2024-04-26T22:00:39.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.658+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba7820 0x562c2fbb6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.658+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.658+0000 7fc1fcc15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fbb7a80 0x562c2fbb7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41aec10 con 0x562c2fba7820 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41b0080 con 0x562c2fbb7a80 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1fcc15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41af2e0 con 0x562c2fba93b0 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1fcc15740 10 client.0 _ll_register_callbacks cb 0x7fc1e4005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1f2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba7820 0x562c2fbb6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1f2ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba7820 0x562c2fbb6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60786/0 (socket says 172.21.15.55:60786) 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/1558516012 learned_addr learned my addr 172.21.15.55:0/1558516012 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/1558516012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 msgr2=0x562c2fbb6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/1558516012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fbb7a80 msgr2=0x562c2fbb7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fbb7a80 0x562c2fbb7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.659+0000 7fc1f2ffd640 1 -- 172.21.15.55:0/1558516012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2fbb8440 con 0x562c2fba7820 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.660+0000 7fc1f2ffd640 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba7820 0x562c2fbb6530 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8001630 tx=0x7fc1e80044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16be6c3f046fe031 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:39.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.660+0000 7fc1d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:39.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.660+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/1558516012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8007120 con 0x562c2fba7820 2024-04-26T22:00:39.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.660+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/1558516012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1e80072c0 con 0x562c2fba7820 2024-04-26T22:00:39.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.660+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1558516012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1e8008e30 con 0x562c2fba7820 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.661+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/1558516012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1e8007500 con 0x562c2fba7820 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.661+0000 7fc1fcc15740 10 client.16656 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.661+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1558516012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562c2fc24170 con 0x562c2fba7820 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.661+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/1558516012 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1e8008e30 con 0x562c2fba7820 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.662+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/1558516012 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc1e8022070 con 0x562c2fba7820 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.662+0000 7fc1fcc15740 10 client.16656 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.662+0000 7fc1fcc15740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.662+0000 7fc1fcc15740 20 client.16656 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.662+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2fc24980 0x562c2fc26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:39.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.662+0000 7fc1fcc15740 4 client.16656 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:00:39.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.662+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1558516012 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c2fb13800 con 0x562c2fc24980 2024-04-26T22:00:39.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.663+0000 7fc1f37fe640 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2fc24980 0x562c2fc26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:39.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.664+0000 7fc1d2ffd640 10 client.16656 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:39.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.664+0000 7fc1f37fe640 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2fc24980 0x562c2fc26da0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e32e30acce461287 server_cookie=1bd67159c7fe3039 in_seq=0 out_seq=0 2024-04-26T22:00:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.669+0000 7fc1d2ffd640 1 -- 172.21.15.55:0/1558516012 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c2fb13800 con 0x562c2fc24980 2024-04-26T22:00:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.669+0000 7fc1d2ffd640 10 client.16656 handle_command_reply: tid=0 2024-04-26T22:00:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.669+0000 7fc1fcc15740 1 client.16656 shutdown 2024-04-26T22:00:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.670+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1558516012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba7820 msgr2=0x562c2fbb6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.670+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba7820 0x562c2fbb6530 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8001630 tx=0x7fc1e80044b0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.671+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1558516012 shutdown_connections 2024-04-26T22:00:39.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.671+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c2fbb7a80 0x562c2fbb7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.671+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562c2fba93b0 0x562c2fbb6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.671+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562c2fc24980 0x562c2fc26da0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.671+0000 7fc1fcc15740 1 --2- 172.21.15.55:0/1558516012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c2fba7820 0x562c2fbb6530 secure :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8001630 tx=0x7fc1e80044b0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.671+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1558516012 >> 172.21.15.55:0/1558516012 conn(0x562c2fae8910 msgr2=0x562c2fbaf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.671+0000 7fc1d2ffd640 0 client.16656 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:39.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.673+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1558516012 shutdown_connections 2024-04-26T22:00:39.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.673+0000 7fc1fcc15740 1 -- 172.21.15.55:0/1558516012 wait complete. 2024-04-26T22:00:39.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.674+0000 7fc1fcc15740 20 client.16656 trim_cache size 0 max 16384 2024-04-26T22:00:39.674 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:00:39.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 -- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1dc0a3cc0 msgr2=0x7fc1dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1dc0a3cc0 0x7fc1dc0a6180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0b95d0 tx=0x7fc1ec00d040 comp rx=0 tx=0).stop 2024-04-26T22:00:39.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 -- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 msgr2=0x7fc1f41a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:39.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 0x7fc1f41a6750 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec02ece0 tx=0x7fc1ec036eb0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 -- 172.21.15.55:0/1798063972 shutdown_connections 2024-04-26T22:00:39.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1dc0a3cc0 0x7fc1dc0a6180 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1f4103440 0x7fc1f41a6750 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1f4102a80 0x7fc1f419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 --2- 172.21.15.55:0/1798063972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1f4101890 0x7fc1f419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:39.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.675+0000 7fc1f9e20640 1 -- 172.21.15.55:0/1798063972 >> 172.21.15.55:0/1798063972 conn(0x7fc1f40fce20 msgr2=0x7fc1f4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:39.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.676+0000 7fc1f9e20640 1 -- 172.21.15.55:0/1798063972 shutdown_connections 2024-04-26T22:00:39.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:39.676+0000 7fc1f9e20640 1 -- 172.21.15.55:0/1798063972 wait complete. 2024-04-26T22:00:40.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:00:40.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:00:40.702 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:00:40.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.940+0000 7f2aeb47e640 1 -- 172.21.15.55:0/2774567956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4105d80 msgr2=0x7f2ae4106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.940+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/2774567956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4105d80 0x7f2ae4106200 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009900 tx=0x7f2ad802a410 comp rx=0 tx=0).stop 2024-04-26T22:00:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.940+0000 7f2aeb47e640 1 -- 172.21.15.55:0/2774567956 shutdown_connections 2024-04-26T22:00:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.940+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/2774567956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae4106740 0x7f2ae410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.940+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/2774567956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4105d80 0x7f2ae4106200 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.940+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/2774567956 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ae4104b90 0x7f2ae4104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.940+0000 7f2aeb47e640 1 -- 172.21.15.55:0/2774567956 >> 172.21.15.55:0/2774567956 conn(0x7f2ae4100120 msgr2=0x7f2ae4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.941+0000 7f2aeb47e640 1 -- 172.21.15.55:0/2774567956 shutdown_connections 2024-04-26T22:00:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.941+0000 7f2aeb47e640 1 -- 172.21.15.55:0/2774567956 wait complete. 2024-04-26T22:00:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.942+0000 7f2aeb47e640 1 Processor -- start 2024-04-26T22:00:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.942+0000 7f2aeb47e640 1 -- start start 2024-04-26T22:00:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.942+0000 7f2aeb47e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4104b90 0x7f2ae419b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.942+0000 7f2aeb47e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae4106740 0x7f2ae419b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.942+0000 7f2aeb47e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ae41a0640 0x7f2ae41a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.942+0000 7f2aeb47e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4110790 con 0x7f2ae4104b90 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.942+0000 7f2aeb47e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4110610 con 0x7f2ae4106740 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.942+0000 7f2aeb47e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4110910 con 0x7f2ae41a0640 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae921a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4104b90 0x7f2ae419b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae921a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4104b90 0x7f2ae419b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58590/0 (socket says 172.21.15.55:58590) 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae921a640 1 -- 172.21.15.55:0/3839647186 learned_addr learned my addr 172.21.15.55:0/3839647186 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ae41a0640 0x7f2ae41a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae921a640 1 -- 172.21.15.55:0/3839647186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ae41a0640 msgr2=0x7f2ae41a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae921a640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ae41a0640 0x7f2ae41a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae921a640 1 -- 172.21.15.55:0/3839647186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae4106740 msgr2=0x7f2ae419b5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae921a640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae4106740 0x7f2ae419b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae921a640 1 -- 172.21.15.55:0/3839647186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae41a30a0 con 0x7f2ae4104b90 2024-04-26T22:00:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.943+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ae41a0640 0x7f2ae41a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:00:40.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.944+0000 7f2ae921a640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4104b90 0x7f2ae419b0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400eaa0 tx=0x7f2ad400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.944+0000 7f2ae27fc640 1 -- 172.21.15.55:0/3839647186 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4016070 con 0x7f2ae4104b90 2024-04-26T22:00:40.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.944+0000 7f2aeb47e640 1 -- 172.21.15.55:0/3839647186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae41a3390 con 0x7f2ae4104b90 2024-04-26T22:00:40.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.944+0000 7f2ae27fc640 1 -- 172.21.15.55:0/3839647186 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad4003e20 con 0x7f2ae4104b90 2024-04-26T22:00:40.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.944+0000 7f2ae27fc640 1 -- 172.21.15.55:0/3839647186 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4020060 con 0x7f2ae4104b90 2024-04-26T22:00:40.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.944+0000 7f2aeb47e640 1 -- 172.21.15.55:0/3839647186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41a38b0 con 0x7f2ae4104b90 2024-04-26T22:00:40.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.949+0000 7f2ae27fc640 1 -- 172.21.15.55:0/3839647186 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2ad4019070 con 0x7f2ae4104b90 2024-04-26T22:00:40.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.951+0000 7f2ae27fc640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2acc0a3cc0 0x7f2acc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.951+0000 7f2ae27fc640 1 -- 172.21.15.55:0/3839647186 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2ad40d0730 con 0x7f2ae4104b90 2024-04-26T22:00:40.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.952+0000 7f2ae8a19640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2acc0a3cc0 0x7f2acc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.953+0000 7f2aee198740 1 Processor -- start 2024-04-26T22:00:40.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.953+0000 7f2aee198740 1 -- start start 2024-04-26T22:00:40.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.953+0000 7f2aee198740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.955 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.954+0000 7f2aee198740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912a52a0 0x5648912a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.954+0000 7f2aee198740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912aa410 0x5648912ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.954+0000 7f2ae9a1b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912aa410 0x5648912ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.954+0000 7f2ae9a1b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912aa410 0x5648912ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58598/0 (socket says 172.21.15.55:58598) 2024-04-26T22:00:40.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.954+0000 7f2ae9a1b640 1 -- 172.21.15.55:0/1116863884 learned_addr learned my addr 172.21.15.55:0/1116863884 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:40.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.954+0000 7f2aee198740 1 -- 172.21.15.55:0/1116863884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564891209b00 con 0x5648912aa410 2024-04-26T22:00:40.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.954+0000 7f2aee198740 1 -- 172.21.15.55:0/1116863884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5648911a5920 con 0x5648912a52a0 2024-04-26T22:00:40.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.954+0000 7f2aee198740 1 -- 172.21.15.55:0/1116863884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564891205ac0 con 0x5648912a48b0 2024-04-26T22:00:40.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.955+0000 7f2ae9a1b640 1 -- 172.21.15.55:0/1116863884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 msgr2=0x5648912a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.955+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/1116863884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.955+0000 7f2ae9a1b640 1 -- 172.21.15.55:0/1116863884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912a52a0 msgr2=0x5648912a9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.955+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/1116863884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912a52a0 0x5648912a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.955+0000 7f2ae9a1b640 1 -- 172.21.15.55:0/1116863884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564890fe37e0 con 0x5648912aa410 2024-04-26T22:00:40.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.956+0000 7f2ae8a19640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2acc0a3cc0 0x7f2acc0a6180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8006010 tx=0x7f2ad8005060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.956+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/1116863884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912aa410 0x5648912ac8a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00abf0 tx=0x7f2adc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676111ca8e2b7652 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.958+0000 7f2acb7fe640 1 -- 172.21.15.55:0/1116863884 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc015070 con 0x5648912aa410 2024-04-26T22:00:40.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.958+0000 7f2acb7fe640 1 -- 172.21.15.55:0/1116863884 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2adc00cc30 con 0x5648912aa410 2024-04-26T22:00:40.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.958+0000 7f2acb7fe640 1 -- 172.21.15.55:0/1116863884 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0185c0 con 0x5648912aa410 2024-04-26T22:00:40.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.960+0000 7f2aee198740 1 -- 172.21.15.55:0/1116863884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912aa410 msgr2=0x5648912ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:40.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.960+0000 7f2aee198740 1 --2- 172.21.15.55:0/1116863884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912aa410 0x5648912ac8a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00abf0 tx=0x7f2adc00c520 comp rx=0 tx=0).stop 2024-04-26T22:00:40.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.960+0000 7f2aee198740 1 -- 172.21.15.55:0/1116863884 shutdown_connections 2024-04-26T22:00:40.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.960+0000 7f2aee198740 1 --2- 172.21.15.55:0/1116863884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912aa410 0x5648912ac8a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.960+0000 7f2aee198740 1 --2- 172.21.15.55:0/1116863884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912a52a0 0x5648912a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.960+0000 7f2aee198740 1 --2- 172.21.15.55:0/1116863884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.960+0000 7f2aee198740 1 -- 172.21.15.55:0/1116863884 >> 172.21.15.55:0/1116863884 conn(0x56489129ff20 msgr2=0x5648912a0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:40.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.961+0000 7f2aee198740 1 -- 172.21.15.55:0/1116863884 shutdown_connections 2024-04-26T22:00:40.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.961+0000 7f2aee198740 1 -- 172.21.15.55:0/1116863884 wait complete. 2024-04-26T22:00:40.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.962+0000 7f2aee198740 1 -- start start 2024-04-26T22:00:40.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.962+0000 7f2aee198740 1 Processor -- start 2024-04-26T22:00:40.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.963+0000 7f2aee198740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a48b0 0x5648912aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.963+0000 7f2aee198740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a52a0 0x5648912af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.963+0000 7f2aee198740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 0x5648912b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.963+0000 7f2ae921a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a48b0 0x5648912aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.963+0000 7f2ae921a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a48b0 0x5648912aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58614/0 (socket says 172.21.15.55:58614) 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.963+0000 7f2aee198740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5648912ad3c0 con 0x5648912a48b0 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.963+0000 7f2aee198740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5648911a35d0 con 0x5648912aa410 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.963+0000 7f2aee198740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5648912ad540 con 0x5648912a52a0 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2aee198740 10 client.0 _ll_register_callbacks cb 0x564891191080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2ae921a640 1 -- 172.21.15.55:0/375853329 learned_addr learned my addr 172.21.15.55:0/375853329 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:40.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2ae8a19640 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a52a0 0x5648912af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:40.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2ae921a640 1 -- 172.21.15.55:0/375853329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a52a0 msgr2=0x5648912af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:40.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2ae921a640 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a52a0 0x5648912af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2ae921a640 1 -- 172.21.15.55:0/375853329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 msgr2=0x5648912b6190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:40.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2ae921a640 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 0x5648912b6190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:40.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2ae921a640 1 -- 172.21.15.55:0/375853329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564891043f70 con 0x5648912a48b0 2024-04-26T22:00:40.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.964+0000 7f2aee198740 1 -- 172.21.15.55:0/375853329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2adc00cc30 con 0x5648912a48b0 2024-04-26T22:00:40.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.965+0000 7f2ae921a640 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a48b0 0x5648912aec10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009270 tx=0x7f2ad40ce4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd83b2b14c85df1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:40.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.965+0000 7f2ac8ff9640 10 client.16668 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:40.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.966+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/375853329 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad40a0bc0 con 0x5648912a48b0 2024-04-26T22:00:40.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.966+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/375853329 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2ad40040c0 con 0x5648912a48b0 2024-04-26T22:00:40.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.966+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/375853329 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2ad40a5480 con 0x5648912a48b0 2024-04-26T22:00:40.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.966+0000 7f2aee198740 10 client.16668 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:40.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.966+0000 7f2aee198740 1 -- 172.21.15.55:0/375853329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564890edb170 con 0x5648912a48b0 2024-04-26T22:00:40.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.968+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/375853329 <== mon.0 v2:172.21.15.55:3300/0 4 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2ad400f040 con 0x5648912a48b0 2024-04-26T22:00:40.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.968+0000 7f2aee198740 10 client.16668 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:40.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.968+0000 7f2aee198740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:40.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.968+0000 7f2aee198740 20 client.16668 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:40.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.968+0000 7f2aee198740 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5648912ba410 0x5648912bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:40.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.968+0000 7f2aee198740 4 client.16668 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:40.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.968+0000 7f2aee198740 1 -- 172.21.15.55:0/375853329 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564891217710 con 0x5648912ba410 2024-04-26T22:00:40.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:40.986+0000 7f2ae8a19640 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5648912ba410 0x5648912bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.005+0000 7f2ac8ff9640 10 client.16668 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:41.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.005+0000 7f2ae8a19640 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5648912ba410 0x5648912bc890 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34b9761a253ff7f server_cookie=79147da9fc3c1595 in_seq=0 out_seq=0 2024-04-26T22:00:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.017+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/375853329 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564891217710 con 0x5648912ba410 2024-04-26T22:00:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.017+0000 7f2ac8ff9640 10 client.16668 handle_command_reply: tid=0 2024-04-26T22:00:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.017+0000 7f2aee198740 1 client.16668 shutdown 2024-04-26T22:00:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.023+0000 7f2aee198740 1 -- 172.21.15.55:0/375853329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a48b0 msgr2=0x5648912aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.023+0000 7f2aee198740 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a48b0 0x5648912aec10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009270 tx=0x7f2ad40ce4c0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.024+0000 7f2aee198740 1 -- 172.21.15.55:0/375853329 shutdown_connections 2024-04-26T22:00:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.024+0000 7f2aee198740 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 0x5648912b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.024+0000 7f2aee198740 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a52a0 0x5648912af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.024+0000 7f2aee198740 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5648912ba410 0x5648912bc890 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.024+0000 7f2aee198740 1 --2- 172.21.15.55:0/375853329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a48b0 0x5648912aec10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.024+0000 7f2aee198740 1 -- 172.21.15.55:0/375853329 >> 172.21.15.55:0/375853329 conn(0x56489129ff20 msgr2=0x56489129f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.024+0000 7f2ac8ff9640 0 client.16668 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.028+0000 7f2aee198740 1 -- 172.21.15.55:0/375853329 shutdown_connections 2024-04-26T22:00:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.028+0000 7f2aee198740 1 -- 172.21.15.55:0/375853329 wait complete. 2024-04-26T22:00:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.028+0000 7f2aee198740 20 client.16668 trim_cache size 0 max 16384 2024-04-26T22:00:41.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.127+0000 7f2aee198740 1 Processor -- start 2024-04-26T22:00:41.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.131+0000 7f2aee198740 1 -- start start 2024-04-26T22:00:41.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.132+0000 7f2aee198740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912a48b0 0x5648912aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.132+0000 7f2aee198740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a52a0 0x5648912b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.132+0000 7f2aee198740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912aa410 0x5648912c1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.132+0000 7f2aee198740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41af110 con 0x5648912a52a0 2024-04-26T22:00:41.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.132+0000 7f2aee198740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41ac9e0 con 0x5648912a48b0 2024-04-26T22:00:41.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.132+0000 7f2aee198740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41aa830 con 0x5648912aa410 2024-04-26T22:00:41.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.133+0000 7f2ae9a1b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912aa410 0x5648912c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.133+0000 7f2ae9a1b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912aa410 0x5648912c1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42950/0 (socket says 172.21.15.55:42950) 2024-04-26T22:00:41.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.133+0000 7f2ae9a1b640 1 -- 172.21.15.55:0/4201923118 learned_addr learned my addr 172.21.15.55:0/4201923118 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:41.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.135+0000 7f2ae9a1b640 1 -- 172.21.15.55:0/4201923118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912a48b0 msgr2=0x5648912aef30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:41.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.135+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/4201923118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912a48b0 0x5648912aef30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.135+0000 7f2ae9a1b640 1 -- 172.21.15.55:0/4201923118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a52a0 msgr2=0x5648912b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.135+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/4201923118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a52a0 0x5648912b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.135+0000 7f2ae9a1b640 1 -- 172.21.15.55:0/4201923118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56489129fd50 con 0x5648912aa410 2024-04-26T22:00:41.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.139+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/4201923118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912aa410 0x5648912c1150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006fd0 tx=0x7f2adc00a6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef6e4507233650c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.141+0000 7f2acaffd640 1 -- 172.21.15.55:0/4201923118 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc01e070 con 0x5648912aa410 2024-04-26T22:00:41.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.141+0000 7f2acaffd640 1 -- 172.21.15.55:0/4201923118 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2adc023b10 con 0x5648912aa410 2024-04-26T22:00:41.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.141+0000 7f2acaffd640 1 -- 172.21.15.55:0/4201923118 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc015050 con 0x5648912aa410 2024-04-26T22:00:41.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.147+0000 7f2aee198740 1 -- 172.21.15.55:0/4201923118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912aa410 msgr2=0x5648912c1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.147+0000 7f2aee198740 1 --2- 172.21.15.55:0/4201923118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912aa410 0x5648912c1150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006fd0 tx=0x7f2adc00a6d0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.151+0000 7f2aee198740 1 -- 172.21.15.55:0/4201923118 shutdown_connections 2024-04-26T22:00:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.151+0000 7f2aee198740 1 --2- 172.21.15.55:0/4201923118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912aa410 0x5648912c1150 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.151+0000 7f2aee198740 1 --2- 172.21.15.55:0/4201923118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a52a0 0x5648912b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.151+0000 7f2aee198740 1 --2- 172.21.15.55:0/4201923118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912a48b0 0x5648912aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.151+0000 7f2aee198740 1 -- 172.21.15.55:0/4201923118 >> 172.21.15.55:0/4201923118 conn(0x56489129ff20 msgr2=0x5648912ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.155+0000 7f2aee198740 1 -- 172.21.15.55:0/4201923118 shutdown_connections 2024-04-26T22:00:41.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.155+0000 7f2aee198740 1 -- 172.21.15.55:0/4201923118 wait complete. 2024-04-26T22:00:41.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.156+0000 7f2aee198740 1 -- start start 2024-04-26T22:00:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.157+0000 7f2aee198740 1 Processor -- start 2024-04-26T22:00:41.160 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:41 smithi055 ceph-mon[29221]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 849 MiB/s wr, 1.52k op/s 2024-04-26T22:00:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.159+0000 7f2aee198740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.159+0000 7f2aee198740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a52a0 0x5648912b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.160+0000 7f2aee198740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 0x5648912b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.160+0000 7f2aee198740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41aaea0 con 0x5648912a52a0 2024-04-26T22:00:41.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.160+0000 7f2aee198740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41ac310 con 0x5648912aa410 2024-04-26T22:00:41.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.160+0000 7f2aee198740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41ab570 con 0x5648912a48b0 2024-04-26T22:00:41.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.160+0000 7f2aee198740 10 client.0 _ll_register_callbacks cb 0x7f2ad409c3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:41.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.161+0000 7f2ae921a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.161+0000 7f2ae921a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42956/0 (socket says 172.21.15.55:42956) 2024-04-26T22:00:41.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.161+0000 7f2ae921a640 1 -- 172.21.15.55:0/1429832591 learned_addr learned my addr 172.21.15.55:0/1429832591 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:41.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.161+0000 7f2ae9a1b640 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 0x5648912b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.162+0000 7f2ae921a640 1 -- 172.21.15.55:0/1429832591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 msgr2=0x5648912b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.162+0000 7f2ae921a640 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 0x5648912b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.162+0000 7f2ae921a640 1 -- 172.21.15.55:0/1429832591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a52a0 msgr2=0x5648912b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.162+0000 7f2ae921a640 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a52a0 0x5648912b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.162+0000 7f2ae921a640 1 -- 172.21.15.55:0/1429832591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648912b4a40 con 0x5648912a48b0 2024-04-26T22:00:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.164+0000 7f2ae921a640 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912b5bb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40a4f50 tx=0x7f2ad40d0070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c27d2f92492bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.164+0000 7f2ac8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:00:41.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.166+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/1429832591 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad407f070 con 0x5648912a48b0 2024-04-26T22:00:41.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.167+0000 7f2aee198740 1 -- 172.21.15.55:0/1429832591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2adc023b10 con 0x5648912a48b0 2024-04-26T22:00:41.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.167+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/1429832591 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2ad409dd40 con 0x5648912a48b0 2024-04-26T22:00:41.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.169+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/1429832591 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2ad4098ab0 con 0x5648912a48b0 2024-04-26T22:00:41.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.173+0000 7f2aee198740 10 client.25858 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:41.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.173+0000 7f2aee198740 1 -- 172.21.15.55:0/1429832591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564891329ce0 con 0x5648912a48b0 2024-04-26T22:00:41.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.174+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/1429832591 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2adc023b10 con 0x5648912a48b0 2024-04-26T22:00:41.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.174+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/1429832591 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2ad40986b0 con 0x5648912a48b0 2024-04-26T22:00:41.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.175+0000 7f2aee198740 10 client.25858 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:41.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.175+0000 7f2aee198740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:41.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.175+0000 7f2aee198740 20 client.25858 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:41.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.175+0000 7f2aee198740 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56489132a630 0x56489132ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.175+0000 7f2aee198740 4 client.25858 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:00:41.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.175+0000 7f2aee198740 1 -- 172.21.15.55:0/1429832591 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564891211580 con 0x56489132a630 2024-04-26T22:00:41.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.180+0000 7f2ae8a19640 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56489132a630 0x56489132ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.200+0000 7f2ae8a19640 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56489132a630 0x56489132ca20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a2a33b1c8ab4d2 server_cookie=89b0a46d313565b in_seq=0 out_seq=0 2024-04-26T22:00:41.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.200+0000 7f2ac8ff9640 10 client.25858 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:41.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.220+0000 7f2ac8ff9640 1 -- 172.21.15.55:0/1429832591 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564891211580 con 0x56489132a630 2024-04-26T22:00:41.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.220+0000 7f2ac8ff9640 10 client.25858 handle_command_reply: tid=0 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.223+0000 7f2aee198740 1 client.25858 shutdown 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.225+0000 7f2aee198740 1 -- 172.21.15.55:0/1429832591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 msgr2=0x5648912b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.225+0000 7f2aee198740 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912b5bb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40a4f50 tx=0x7f2ad40d0070 comp rx=0 tx=0).stop 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.226+0000 7f2aee198740 1 -- 172.21.15.55:0/1429832591 shutdown_connections 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.226+0000 7f2aee198740 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5648912aa410 0x5648912b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.226+0000 7f2aee198740 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56489132a630 0x56489132ca20 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.226+0000 7f2aee198740 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5648912a52a0 0x5648912b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.226+0000 7f2aee198740 1 --2- 172.21.15.55:0/1429832591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648912a48b0 0x5648912b5bb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.226+0000 7f2aee198740 1 -- 172.21.15.55:0/1429832591 >> 172.21.15.55:0/1429832591 conn(0x56489129ff20 msgr2=0x56489129e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.226+0000 7f2ac8ff9640 0 client.25858 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:41.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.226+0000 7f2aee198740 1 -- 172.21.15.55:0/1429832591 shutdown_connections 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.232+0000 7f2aee198740 1 -- 172.21.15.55:0/1429832591 wait complete. 2024-04-26T22:00:41.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.232+0000 7f2aee198740 20 client.25858 trim_cache size 0 max 16384 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "bc101095-6a11-4e1b-bd01-cb37d7bc2319", 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:00:41.235 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.240+0000 7f2aeb47e640 1 -- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2acc0a3cc0 msgr2=0x7f2acc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.240+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2acc0a3cc0 0x7f2acc0a6180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8006010 tx=0x7f2ad8005060 comp rx=0 tx=0).stop 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.240+0000 7f2aeb47e640 1 -- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4104b90 msgr2=0x7f2ae419b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.240+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4104b90 0x7f2ae419b0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2ad400eaa0 tx=0x7f2ad400ef60 comp rx=0 tx=0).stop 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.241+0000 7f2aeb47e640 1 -- 172.21.15.55:0/3839647186 shutdown_connections 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.241+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2acc0a3cc0 0x7f2acc0a6180 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.241+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ae41a0640 0x7f2ae41a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.241+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ae4106740 0x7f2ae419b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.241+0000 7f2aeb47e640 1 --2- 172.21.15.55:0/3839647186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ae4104b90 0x7f2ae419b0a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.241+0000 7f2aeb47e640 1 -- 172.21.15.55:0/3839647186 >> 172.21.15.55:0/3839647186 conn(0x7f2ae4100120 msgr2=0x7f2ae41091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.244+0000 7f2aeb47e640 1 -- 172.21.15.55:0/3839647186 shutdown_connections 2024-04-26T22:00:41.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.246+0000 7f2aeb47e640 1 -- 172.21.15.55:0/3839647186 wait complete. 2024-04-26T22:00:41.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:00:41.302 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:00:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:41 smithi064 ceph-mon[32765]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 849 MiB/s wr, 1.52k op/s 2024-04-26T22:00:41.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:41 smithi165 ceph-mon[33296]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 849 MiB/s wr, 1.52k op/s 2024-04-26T22:00:41.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.686+0000 7f9393578640 1 -- 172.21.15.55:0/4122130308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c10c2a0 msgr2=0x7f938c10e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.686+0000 7f9393578640 1 --2- 172.21.15.55:0/4122130308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c10c2a0 0x7f938c10e730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f937c009860 tx=0x7f937c02a380 comp rx=0 tx=0).stop 2024-04-26T22:00:41.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.689+0000 7f9393578640 1 -- 172.21.15.55:0/4122130308 shutdown_connections 2024-04-26T22:00:41.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.689+0000 7f9393578640 1 --2- 172.21.15.55:0/4122130308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c10ec70 0x7f938c10f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.689+0000 7f9393578640 1 --2- 172.21.15.55:0/4122130308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c10c2a0 0x7f938c10e730 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.689+0000 7f9393578640 1 --2- 172.21.15.55:0/4122130308 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938c109930 0x7f938c10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.689+0000 7f9393578640 1 -- 172.21.15.55:0/4122130308 >> 172.21.15.55:0/4122130308 conn(0x7f938c06f7c0 msgr2=0x7f938c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.694+0000 7f9393578640 1 -- 172.21.15.55:0/4122130308 shutdown_connections 2024-04-26T22:00:41.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.694+0000 7f9393578640 1 -- 172.21.15.55:0/4122130308 wait complete. 2024-04-26T22:00:41.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.695+0000 7f9393578640 1 Processor -- start 2024-04-26T22:00:41.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.696+0000 7f9393578640 1 -- start start 2024-04-26T22:00:41.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.696+0000 7f9393578640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c109930 0x7f938c116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.696+0000 7f9393578640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938c10c2a0 0x7f938c1166b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.696+0000 7f9393578640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c10ec70 0x7f938c1a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.696+0000 7f9393578640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f938c11d2d0 con 0x7f938c10c2a0 2024-04-26T22:00:41.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.696+0000 7f9393578640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f938c11d150 con 0x7f938c10ec70 2024-04-26T22:00:41.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.696+0000 7f9393578640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f938c11d450 con 0x7f938c109930 2024-04-26T22:00:41.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.697+0000 7f9390b13640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938c10c2a0 0x7f938c1166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.698+0000 7f9390b13640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938c10c2a0 0x7f938c1166b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58656/0 (socket says 172.21.15.55:58656) 2024-04-26T22:00:41.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.698+0000 7f9390b13640 1 -- 172.21.15.55:0/676597552 learned_addr learned my addr 172.21.15.55:0/676597552 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:41.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.699+0000 7f9391b15640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c10ec70 0x7f938c1a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.700+0000 7f9390b13640 1 -- 172.21.15.55:0/676597552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c109930 msgr2=0x7f938c116170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:00:41.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.700+0000 7f9390b13640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c109930 0x7f938c116170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.700+0000 7f9390b13640 1 -- 172.21.15.55:0/676597552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c10ec70 msgr2=0x7f938c1a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.700+0000 7f9390b13640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c10ec70 0x7f938c1a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.700+0000 7f9390b13640 1 -- 172.21.15.55:0/676597552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c1a6f10 con 0x7f938c10c2a0 2024-04-26T22:00:41.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.704+0000 7f9390b13640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938c10c2a0 0x7f938c1166b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f937c035040 tx=0x7f937c032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.715+0000 7f93827fc640 1 -- 172.21.15.55:0/676597552 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c037050 con 0x7f938c10c2a0 2024-04-26T22:00:41.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.715+0000 7f93827fc640 1 -- 172.21.15.55:0/676597552 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f937c043070 con 0x7f938c10c2a0 2024-04-26T22:00:41.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.716+0000 7f93827fc640 1 -- 172.21.15.55:0/676597552 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c003d40 con 0x7f938c10c2a0 2024-04-26T22:00:41.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.716+0000 7f9393578640 1 -- 172.21.15.55:0/676597552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c1a71a0 con 0x7f938c10c2a0 2024-04-26T22:00:41.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.716+0000 7f9393578640 1 -- 172.21.15.55:0/676597552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c1a7640 con 0x7f938c10c2a0 2024-04-26T22:00:41.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.725+0000 7f9396369740 1 Processor -- start 2024-04-26T22:00:41.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.725+0000 7f9396369740 1 -- start start 2024-04-26T22:00:41.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.726+0000 7f9396369740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd13ee0 0x56244bd14300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.726+0000 7f9396369740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd150b0 0x56244bd15530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.726+0000 7f9396369740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd15a70 0x56244bd1c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.726+0000 7f9396369740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56244bc78d80 con 0x56244bd150b0 2024-04-26T22:00:41.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.726+0000 7f9396369740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56244bc328b0 con 0x56244bd15a70 2024-04-26T22:00:41.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.726+0000 7f9396369740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56244bc74d40 con 0x56244bd13ee0 2024-04-26T22:00:41.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.727+0000 7f9391314640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd150b0 0x56244bd15530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.728+0000 7f9391314640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd150b0 0x56244bd15530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58660/0 (socket says 172.21.15.55:58660) 2024-04-26T22:00:41.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.728+0000 7f9391314640 1 -- 172.21.15.55:0/571840926 learned_addr learned my addr 172.21.15.55:0/571840926 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:41.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.728+0000 7f9391b15640 1 --2- 172.21.15.55:0/571840926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd13ee0 0x56244bd14300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.729+0000 7f9391b15640 1 -- 172.21.15.55:0/571840926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd15a70 msgr2=0x56244bd1c300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:00:41.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.729+0000 7f9391b15640 1 --2- 172.21.15.55:0/571840926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd15a70 0x56244bd1c300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.729+0000 7f9391b15640 1 -- 172.21.15.55:0/571840926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd150b0 msgr2=0x56244bd15530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.729+0000 7f9391b15640 1 --2- 172.21.15.55:0/571840926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd150b0 0x56244bd15530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.729+0000 7f9391b15640 1 -- 172.21.15.55:0/571840926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56244bc03a40 con 0x56244bd13ee0 2024-04-26T22:00:41.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.729+0000 7f9391b15640 1 --2- 172.21.15.55:0/571840926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd13ee0 0x56244bd14300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9384005fd0 tx=0x7f9384010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c3ebd00e4ee535 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.736+0000 7f93777fe640 1 -- 172.21.15.55:0/571840926 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384016070 con 0x56244bd13ee0 2024-04-26T22:00:41.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.737+0000 7f93777fe640 1 -- 172.21.15.55:0/571840926 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9384010e50 con 0x56244bd13ee0 2024-04-26T22:00:41.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.737+0000 7f93777fe640 1 -- 172.21.15.55:0/571840926 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93840126b0 con 0x56244bd13ee0 2024-04-26T22:00:41.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.742+0000 7f9396369740 1 -- 172.21.15.55:0/571840926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd13ee0 msgr2=0x56244bd14300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.742+0000 7f9396369740 1 --2- 172.21.15.55:0/571840926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd13ee0 0x56244bd14300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9384005fd0 tx=0x7f9384010740 comp rx=0 tx=0).stop 2024-04-26T22:00:41.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.745+0000 7f93827fc640 1 -- 172.21.15.55:0/676597552 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f937c004040 con 0x7f938c10c2a0 2024-04-26T22:00:41.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.753+0000 7f93827fc640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93780a3dc0 0x7f93780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.754+0000 7f93827fc640 1 -- 172.21.15.55:0/676597552 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f937c0f0a20 con 0x7f938c10c2a0 2024-04-26T22:00:41.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.754+0000 7f9396369740 1 -- 172.21.15.55:0/571840926 shutdown_connections 2024-04-26T22:00:41.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.754+0000 7f9396369740 1 --2- 172.21.15.55:0/571840926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd15a70 0x56244bd1c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.754+0000 7f9396369740 1 --2- 172.21.15.55:0/571840926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd150b0 0x56244bd15530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.754+0000 7f9396369740 1 --2- 172.21.15.55:0/571840926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd13ee0 0x56244bd14300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.754+0000 7f9396369740 1 -- 172.21.15.55:0/571840926 >> 172.21.15.55:0/571840926 conn(0x56244bd0f3d0 msgr2=0x56244bd0f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.757+0000 7f9396369740 1 -- 172.21.15.55:0/571840926 shutdown_connections 2024-04-26T22:00:41.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.758+0000 7f9396369740 1 -- 172.21.15.55:0/571840926 wait complete. 2024-04-26T22:00:41.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.758+0000 7f9396369740 1 -- start start 2024-04-26T22:00:41.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.759+0000 7f9396369740 1 Processor -- start 2024-04-26T22:00:41.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.760+0000 7f9391b15640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93780a3dc0 0x7f93780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.765+0000 7f9396369740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd13ee0 0x56244bd1e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.765+0000 7f9396369740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd1e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.765+0000 7f9396369740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd15a70 0x56244bd259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.765+0000 7f9396369740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56244bd1ca80 con 0x56244bd13ee0 2024-04-26T22:00:41.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.765+0000 7f9396369740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56244bbffd10 con 0x56244bd150b0 2024-04-26T22:00:41.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.765+0000 7f9396369740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56244bd1cc00 con 0x56244bd15a70 2024-04-26T22:00:41.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.765+0000 7f9396369740 10 client.0 _ll_register_callbacks cb 0x56244bc2c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:41.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.767+0000 7f9390b13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd1e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.768+0000 7f9390b13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd1e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:57312/0 (socket says 172.21.15.55:57312) 2024-04-26T22:00:41.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.768+0000 7f9391314640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd13ee0 0x56244bd1e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.768+0000 7f9391314640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd13ee0 0x56244bd1e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58672/0 (socket says 172.21.15.55:58672) 2024-04-26T22:00:41.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.768+0000 7f9391314640 1 -- 172.21.15.55:0/3341616945 learned_addr learned my addr 172.21.15.55:0/3341616945 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:41.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.774+0000 7f9391314640 1 -- 172.21.15.55:0/3341616945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd15a70 msgr2=0x56244bd259e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:00:41.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.774+0000 7f9391314640 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd15a70 0x56244bd259e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.774+0000 7f9391314640 1 -- 172.21.15.55:0/3341616945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 msgr2=0x56244bd1e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.774+0000 7f9391314640 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd1e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.774+0000 7f9391314640 1 -- 172.21.15.55:0/3341616945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56244ba547d0 con 0x56244bd13ee0 2024-04-26T22:00:41.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.775+0000 7f9391b15640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93780a3dc0 0x7f93780a6280 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9384009be0 tx=0x7f9384000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.778+0000 7f9391314640 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd13ee0 0x56244bd1e460 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f938800bda0 tx=0x7f93880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c325ac82b039881 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.778+0000 7f9374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:41.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.779+0000 7f9374ff9640 1 -- 172.21.15.55:0/3341616945 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388011850 con 0x56244bd13ee0 2024-04-26T22:00:41.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.779+0000 7f9374ff9640 1 -- 172.21.15.55:0/3341616945 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f93880119f0 con 0x56244bd13ee0 2024-04-26T22:00:41.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.779+0000 7f9396369740 1 -- 172.21.15.55:0/3341616945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384010e50 con 0x56244bd13ee0 2024-04-26T22:00:41.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.782+0000 7f9374ff9640 1 -- 172.21.15.55:0/3341616945 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9388010e50 con 0x56244bd13ee0 2024-04-26T22:00:41.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.784+0000 7f9396369740 10 client.16680 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:41.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.784+0000 7f9396369740 1 -- 172.21.15.55:0/3341616945 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56244b98a3f0 con 0x56244bd13ee0 2024-04-26T22:00:41.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.786+0000 7f9374ff9640 1 -- 172.21.15.55:0/3341616945 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9384010e50 con 0x56244bd13ee0 2024-04-26T22:00:41.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.786+0000 7f9374ff9640 1 -- 172.21.15.55:0/3341616945 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9388026420 con 0x56244bd13ee0 2024-04-26T22:00:41.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.786+0000 7f9396369740 10 client.16680 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:41.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.786+0000 7f9396369740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:41.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.786+0000 7f9396369740 20 client.16680 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:41.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.786+0000 7f9396369740 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56244bd2a010 0x56244bd2c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.786+0000 7f9396369740 4 client.16680 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:00:41.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.786+0000 7f9396369740 1 -- 172.21.15.55:0/3341616945 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56244bc86990 con 0x56244bd2a010 2024-04-26T22:00:41.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.793+0000 7f9390b13640 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56244bd2a010 0x56244bd2c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.809+0000 7f9374ff9640 10 client.16680 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:41.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.809+0000 7f9390b13640 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56244bd2a010 0x56244bd2c450 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3141039c81a4d68f server_cookie=35950b2a41ccb40c in_seq=0 out_seq=0 2024-04-26T22:00:41.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.814+0000 7f9374ff9640 1 -- 172.21.15.55:0/3341616945 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56244bc86990 con 0x56244bd2a010 2024-04-26T22:00:41.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.814+0000 7f9374ff9640 10 client.16680 handle_command_reply: tid=0 2024-04-26T22:00:41.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.815+0000 7f9396369740 1 client.16680 shutdown 2024-04-26T22:00:41.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.824+0000 7f9396369740 1 -- 172.21.15.55:0/3341616945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd13ee0 msgr2=0x56244bd1e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.824+0000 7f9396369740 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd13ee0 0x56244bd1e460 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f938800bda0 tx=0x7f93880107a0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.825+0000 7f9396369740 1 -- 172.21.15.55:0/3341616945 shutdown_connections 2024-04-26T22:00:41.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.825+0000 7f9396369740 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd15a70 0x56244bd259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.825+0000 7f9396369740 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd1e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.825+0000 7f9396369740 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56244bd2a010 0x56244bd2c450 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.825+0000 7f9396369740 1 --2- 172.21.15.55:0/3341616945 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd13ee0 0x56244bd1e460 secure :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f938800bda0 tx=0x7f93880107a0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.825+0000 7f9396369740 1 -- 172.21.15.55:0/3341616945 >> 172.21.15.55:0/3341616945 conn(0x56244bd0f3d0 msgr2=0x56244bd0e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.825+0000 7f9374ff9640 0 client.16680 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:41.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.835+0000 7f9396369740 1 -- 172.21.15.55:0/3341616945 shutdown_connections 2024-04-26T22:00:41.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.836+0000 7f9396369740 1 -- 172.21.15.55:0/3341616945 wait complete. 2024-04-26T22:00:41.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.836+0000 7f9396369740 20 client.16680 trim_cache size 0 max 16384 2024-04-26T22:00:41.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.918+0000 7f9396369740 1 Processor -- start 2024-04-26T22:00:41.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.919+0000 7f9396369740 1 -- start start 2024-04-26T22:00:41.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.920+0000 7f9396369740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd18150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.920+0000 7f9396369740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd259a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.920+0000 7f9396369740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd12760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.920+0000 7f9396369740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1b2da0 con 0x56244bd15a70 2024-04-26T22:00:41.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.920+0000 7f9396369740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1b0670 con 0x56244bd150b0 2024-04-26T22:00:41.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.920+0000 7f9396369740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1ae4c0 con 0x56244bd122b0 2024-04-26T22:00:41.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.921+0000 7f9391b15640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd12760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.921+0000 7f9391b15640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd12760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42984/0 (socket says 172.21.15.55:42984) 2024-04-26T22:00:41.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.921+0000 7f9391b15640 1 -- 172.21.15.55:0/3866830178 learned_addr learned my addr 172.21.15.55:0/3866830178 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:41.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.921+0000 7f9391314640 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd18150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.922+0000 7f9390b13640 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd259a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.922+0000 7f9391b15640 1 -- 172.21.15.55:0/3866830178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 msgr2=0x56244bd18150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.922+0000 7f9391b15640 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd18150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.922+0000 7f9391b15640 1 -- 172.21.15.55:0/3866830178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 msgr2=0x56244bd259a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.922+0000 7f9391b15640 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd259a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.922+0000 7f9391b15640 1 -- 172.21.15.55:0/3866830178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56244bd1d540 con 0x56244bd122b0 2024-04-26T22:00:41.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.923+0000 7f9391b15640 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd12760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9384009d70 tx=0x7f9384020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6b7ef263091904 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.926+0000 7f9376ffd640 1 -- 172.21.15.55:0/3866830178 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384016040 con 0x56244bd122b0 2024-04-26T22:00:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.926+0000 7f9376ffd640 1 -- 172.21.15.55:0/3866830178 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93840278e0 con 0x56244bd122b0 2024-04-26T22:00:41.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.927+0000 7f9396369740 1 -- 172.21.15.55:0/3866830178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 msgr2=0x56244bd12760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.927+0000 7f9396369740 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd12760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9384009d70 tx=0x7f9384020000 comp rx=0 tx=0).stop 2024-04-26T22:00:41.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.930+0000 7f9396369740 1 -- 172.21.15.55:0/3866830178 shutdown_connections 2024-04-26T22:00:41.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.930+0000 7f9396369740 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd12760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.930+0000 7f9396369740 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd259a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.930+0000 7f9396369740 1 --2- 172.21.15.55:0/3866830178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd18150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.930+0000 7f9396369740 1 -- 172.21.15.55:0/3866830178 >> 172.21.15.55:0/3866830178 conn(0x56244bd0f3d0 msgr2=0x56244bd0a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.933+0000 7f9396369740 1 -- 172.21.15.55:0/3866830178 shutdown_connections 2024-04-26T22:00:41.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.933+0000 7f9396369740 1 -- 172.21.15.55:0/3866830178 wait complete. 2024-04-26T22:00:41.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.933+0000 7f9396369740 1 -- start start 2024-04-26T22:00:41.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.934+0000 7f9396369740 1 Processor -- start 2024-04-26T22:00:41.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.935+0000 7f9396369740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd20a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.935+0000 7f9396369740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd20fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.935+0000 7f9396369740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd23f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.936 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.935+0000 7f9396369740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1aeb30 con 0x56244bd15a70 2024-04-26T22:00:41.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.935+0000 7f9396369740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1affa0 con 0x56244bd150b0 2024-04-26T22:00:41.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.935+0000 7f9396369740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f938c1af200 con 0x56244bd122b0 2024-04-26T22:00:41.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.935+0000 7f9396369740 10 client.0 _ll_register_callbacks cb 0x7f938800b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:00:41.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.938+0000 7f9391314640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd20a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.938+0000 7f9391b15640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd23f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.938+0000 7f9391b15640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd23f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58698/0 (socket says 172.21.15.55:58698) 2024-04-26T22:00:41.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.938+0000 7f9391b15640 1 -- 172.21.15.55:0/2645550465 learned_addr learned my addr 172.21.15.55:0/2645550465 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:00:41.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.939+0000 7f9391b15640 1 -- 172.21.15.55:0/2645550465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 msgr2=0x56244bd20a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.939+0000 7f9391b15640 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd20a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.939+0000 7f9391b15640 1 -- 172.21.15.55:0/2645550465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 msgr2=0x56244bd20fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:00:41.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.939+0000 7f9391b15640 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd20fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.939+0000 7f9391b15640 1 -- 172.21.15.55:0/2645550465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56244b94a0f0 con 0x56244bd15a70 2024-04-26T22:00:41.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.940+0000 7f9391b15640 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd23f60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f938401a7c0 tx=0x7f9384010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b85796661badf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:00:41.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.941+0000 7f9374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:00:41.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.942+0000 7f9374ff9640 1 -- 172.21.15.55:0/2645550465 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384039020 con 0x56244bd15a70 2024-04-26T22:00:41.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.942+0000 7f9396369740 1 -- 172.21.15.55:0/2645550465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93840278e0 con 0x56244bd15a70 2024-04-26T22:00:41.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.942+0000 7f9374ff9640 1 -- 172.21.15.55:0/2645550465 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9384027c90 con 0x56244bd15a70 2024-04-26T22:00:41.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.943+0000 7f9374ff9640 1 -- 172.21.15.55:0/2645550465 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9384044050 con 0x56244bd15a70 2024-04-26T22:00:41.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.943+0000 7f9396369740 10 client.16686 fetch_fsmap learned FSMap version 23 2024-04-26T22:00:41.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.943+0000 7f9396369740 1 -- 172.21.15.55:0/2645550465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56244bc394b0 con 0x56244bd15a70 2024-04-26T22:00:41.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.944+0000 7f9374ff9640 1 -- 172.21.15.55:0/2645550465 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93840278e0 con 0x56244bd15a70 2024-04-26T22:00:41.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.944+0000 7f9374ff9640 1 -- 172.21.15.55:0/2645550465 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f938400f040 con 0x56244bd15a70 2024-04-26T22:00:41.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.944+0000 7f9396369740 10 client.16686 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:00:41.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.944+0000 7f9396369740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:00:41.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.944+0000 7f9396369740 20 client.16686 populate_metadata read hostname 'smithi055' 2024-04-26T22:00:41.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.944+0000 7f9396369740 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56244bd90590 0x56244bd92980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:00:41.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.944+0000 7f9396369740 4 client.16686 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:00:41.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.944+0000 7f9396369740 1 -- 172.21.15.55:0/2645550465 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56244bc80800 con 0x56244bd90590 2024-04-26T22:00:41.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.945+0000 7f9391314640 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56244bd90590 0x56244bd92980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:00:41.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.953+0000 7f9391314640 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56244bd90590 0x56244bd92980 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608dc1857302aaba server_cookie=36cddb132e9ed54 in_seq=0 out_seq=0 2024-04-26T22:00:41.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.953+0000 7f9374ff9640 10 client.16686 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:41.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.956+0000 7f9374ff9640 1 -- 172.21.15.55:0/2645550465 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x56244bc80800 con 0x56244bd90590 2024-04-26T22:00:41.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.956+0000 7f9374ff9640 10 client.16686 handle_command_reply: tid=0 2024-04-26T22:00:41.956 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.956+0000 7f9396369740 1 client.16686 shutdown 2024-04-26T22:00:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.959+0000 7f9396369740 1 -- 172.21.15.55:0/2645550465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 msgr2=0x56244bd23f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.959+0000 7f9396369740 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd23f60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f938401a7c0 tx=0x7f9384010ec0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.959+0000 7f9396369740 1 -- 172.21.15.55:0/2645550465 shutdown_connections 2024-04-26T22:00:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.959+0000 7f9396369740 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56244bd90590 0x56244bd92980 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.959+0000 7f9396369740 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56244bd15a70 0x56244bd23f60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.959+0000 7f9396369740 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56244bd150b0 0x56244bd20fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.959+0000 7f9396369740 1 --2- 172.21.15.55:0/2645550465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56244bd122b0 0x56244bd20a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.959+0000 7f9396369740 1 -- 172.21.15.55:0/2645550465 >> 172.21.15.55:0/2645550465 conn(0x56244bd0f3d0 msgr2=0x56244bd12700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.961+0000 7f9374ff9640 0 client.16686 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:00:41.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.961+0000 7f9396369740 1 -- 172.21.15.55:0/2645550465 shutdown_connections 2024-04-26T22:00:41.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.961+0000 7f9396369740 1 -- 172.21.15.55:0/2645550465 wait complete. 2024-04-26T22:00:41.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.961+0000 7f9396369740 20 client.16686 trim_cache size 0 max 16384 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (2 inodes in the stack)", 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout: "bc101095-6a11-4e1b-bd01-cb37d7bc2319": { 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "bc101095-6a11-4e1b-bd01-cb37d7bc2319", 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:00:41.963 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:00:41.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.965+0000 7f9393578640 1 -- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93780a3dc0 msgr2=0x7f93780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.965+0000 7f9393578640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93780a3dc0 0x7f93780a6280 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9384009be0 tx=0x7f9384000f40 comp rx=0 tx=0).stop 2024-04-26T22:00:41.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.965+0000 7f9393578640 1 -- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938c10c2a0 msgr2=0x7f938c1166b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:00:41.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.965+0000 7f9393578640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938c10c2a0 0x7f938c1166b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f937c035040 tx=0x7f937c032cd0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.966+0000 7f9393578640 1 -- 172.21.15.55:0/676597552 shutdown_connections 2024-04-26T22:00:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.966+0000 7f9393578640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93780a3dc0 0x7f93780a6280 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.966+0000 7f9393578640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f938c10ec70 0x7f938c1a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.966+0000 7f9393578640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f938c10c2a0 0x7f938c1166b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.967+0000 7f9393578640 1 --2- 172.21.15.55:0/676597552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c109930 0x7f938c116170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:00:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.967+0000 7f9393578640 1 -- 172.21.15.55:0/676597552 >> 172.21.15.55:0/676597552 conn(0x7f938c06f7c0 msgr2=0x7f938c109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:00:41.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.967+0000 7f9393578640 1 -- 172.21.15.55:0/676597552 shutdown_connections 2024-04-26T22:00:41.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:00:41.967+0000 7f9393578640 1 -- 172.21.15.55:0/676597552 wait complete. 2024-04-26T22:00:42.005 INFO:tasks.cephfs.filesystem:scrub status for tag:bc101095-6a11-4e1b-bd01-cb37d7bc2319 - {'path': '/', 'tag': 'bc101095-6a11-4e1b-bd01-cb37d7bc2319', 'options': 'recursive,force'} 2024-04-26T22:00:42.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:42 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:00:42.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:42 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:42.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:42 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:00:42.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:42 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:00:42.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:42 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:42.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:42 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:00:42.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:42 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:00:42.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:42 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:42.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:42 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:00:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:43 smithi064 ceph-mon[32765]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 887 MiB/s wr, 1.54k op/s 2024-04-26T22:00:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:43 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:00:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:43 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:43 smithi165 ceph-mon[33296]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 887 MiB/s wr, 1.54k op/s 2024-04-26T22:00:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:43 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:00:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:43 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:43.696 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:43 smithi055 ceph-mon[29221]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 887 MiB/s wr, 1.54k op/s 2024-04-26T22:00:43.696 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:43 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:00:43.696 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:43 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:45.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:45 smithi064 ceph-mon[32765]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.49k op/s 2024-04-26T22:00:45.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:45 smithi165 ceph-mon[33296]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.49k op/s 2024-04-26T22:00:45.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:45 smithi055 ceph-mon[29221]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.49k op/s 2024-04-26T22:00:47.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:47 smithi064 ceph-mon[32765]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 785 MiB/s wr, 1.45k op/s 2024-04-26T22:00:47.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:47 smithi165 ceph-mon[33296]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 785 MiB/s wr, 1.45k op/s 2024-04-26T22:00:47.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:47 smithi055 ceph-mon[29221]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 785 MiB/s wr, 1.45k op/s 2024-04-26T22:00:48.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:48 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:48.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:48 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:48.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:48 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:00:49.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:49 smithi165 ceph-mon[33296]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.70k op/s 2024-04-26T22:00:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:49 smithi055 ceph-mon[29221]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.70k op/s 2024-04-26T22:00:49.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:49 smithi064 ceph-mon[32765]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.70k op/s 2024-04-26T22:00:51.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:51 smithi165 ceph-mon[33296]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 856 MiB/s wr, 1.47k op/s 2024-04-26T22:00:51.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:51 smithi055 ceph-mon[29221]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 856 MiB/s wr, 1.47k op/s 2024-04-26T22:00:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:51 smithi064 ceph-mon[32765]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 856 MiB/s wr, 1.47k op/s 2024-04-26T22:00:52.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:52 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:00:52.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:52 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:00:52.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:52 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:00:52.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:52 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:00:52.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:52 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:00:52.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:52 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:00:53.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:53 smithi165 ceph-mon[33296]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 887 MiB/s wr, 1.39k op/s 2024-04-26T22:00:53.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:53 smithi055 ceph-mon[29221]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 887 MiB/s wr, 1.39k op/s 2024-04-26T22:00:53.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:53 smithi064 ceph-mon[32765]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 887 MiB/s wr, 1.39k op/s 2024-04-26T22:00:55.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:55 smithi165 ceph-mon[33296]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 837 MiB/s wr, 1.35k op/s 2024-04-26T22:00:55.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:55 smithi055 ceph-mon[29221]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 837 MiB/s wr, 1.35k op/s 2024-04-26T22:00:55.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:55 smithi064 ceph-mon[32765]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 837 MiB/s wr, 1.35k op/s 2024-04-26T22:00:57.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:57 smithi165 ceph-mon[33296]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 791 MiB/s wr, 1.28k op/s 2024-04-26T22:00:57.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:57 smithi055 ceph-mon[29221]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 791 MiB/s wr, 1.28k op/s 2024-04-26T22:00:57.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:57 smithi064 ceph-mon[32765]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 791 MiB/s wr, 1.28k op/s 2024-04-26T22:00:59.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:00:59 smithi055 ceph-mon[29221]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 898 MiB/s wr, 1.42k op/s 2024-04-26T22:00:59.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:00:59 smithi064 ceph-mon[32765]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 898 MiB/s wr, 1.42k op/s 2024-04-26T22:00:59.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:00:59 smithi165 ceph-mon[33296]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 898 MiB/s wr, 1.42k op/s 2024-04-26T22:01:01.625 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:01 smithi165 ceph-mon[33296]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 888 MiB/s wr, 1.32k op/s 2024-04-26T22:01:01.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:01 smithi055 ceph-mon[29221]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 888 MiB/s wr, 1.32k op/s 2024-04-26T22:01:01.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:01 smithi064 ceph-mon[32765]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 888 MiB/s wr, 1.32k op/s 2024-04-26T22:01:03.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:03 smithi055 ceph-mon[29221]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 881 MiB/s wr, 1.36k op/s 2024-04-26T22:01:03.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:03 smithi064 ceph-mon[32765]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 881 MiB/s wr, 1.36k op/s 2024-04-26T22:01:03.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:03 smithi165 ceph-mon[33296]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 881 MiB/s wr, 1.36k op/s 2024-04-26T22:01:05.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:05 smithi165 ceph-mon[33296]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 827 MiB/s wr, 1.49k op/s 2024-04-26T22:01:05.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:05 smithi055 ceph-mon[29221]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 827 MiB/s wr, 1.49k op/s 2024-04-26T22:01:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:05 smithi064 ceph-mon[32765]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 827 MiB/s wr, 1.49k op/s 2024-04-26T22:01:07.809 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:07 smithi064 ceph-mon[32765]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 795 MiB/s wr, 1.64k op/s 2024-04-26T22:01:07.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:07 smithi165 ceph-mon[33296]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 795 MiB/s wr, 1.64k op/s 2024-04-26T22:01:07.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:07 smithi055 ceph-mon[29221]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 795 MiB/s wr, 1.64k op/s 2024-04-26T22:01:09.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:09 smithi165 ceph-mon[33296]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 910 MiB/s wr, 1.98k op/s 2024-04-26T22:01:09.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:09 smithi055 ceph-mon[29221]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 910 MiB/s wr, 1.98k op/s 2024-04-26T22:01:10.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:09 smithi064 ceph-mon[32765]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 910 MiB/s wr, 1.98k op/s 2024-04-26T22:01:11.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:11 smithi165 ceph-mon[33296]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 895 MiB/s wr, 1.88k op/s 2024-04-26T22:01:11.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:11 smithi055 ceph-mon[29221]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 895 MiB/s wr, 1.88k op/s 2024-04-26T22:01:12.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:01:12.007 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:01:12.008 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:11 smithi064 ceph-mon[32765]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 895 MiB/s wr, 1.88k op/s 2024-04-26T22:01:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.426+0000 7fbcedc09640 1 -- 172.21.15.55:0/655140720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce810c2a0 msgr2=0x7fbce810e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.426+0000 7fbcedc09640 1 --2- 172.21.15.55:0/655140720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce810c2a0 0x7fbce810e730 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbce001acb0 tx=0x7fbce003b920 comp rx=0 tx=0).stop 2024-04-26T22:01:12.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.433+0000 7fbcedc09640 1 -- 172.21.15.55:0/655140720 shutdown_connections 2024-04-26T22:01:12.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.433+0000 7fbcedc09640 1 --2- 172.21.15.55:0/655140720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce810ec70 0x7fbce810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.433+0000 7fbcedc09640 1 --2- 172.21.15.55:0/655140720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce810c2a0 0x7fbce810e730 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.433+0000 7fbcedc09640 1 --2- 172.21.15.55:0/655140720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce8109930 0x7fbce810bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.433+0000 7fbcedc09640 1 -- 172.21.15.55:0/655140720 >> 172.21.15.55:0/655140720 conn(0x7fbce806f7c0 msgr2=0x7fbce8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:12.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.434+0000 7fbcedc09640 1 -- 172.21.15.55:0/655140720 shutdown_connections 2024-04-26T22:01:12.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.436+0000 7fbcedc09640 1 -- 172.21.15.55:0/655140720 wait complete. 2024-04-26T22:01:12.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.437+0000 7fbcedc09640 1 Processor -- start 2024-04-26T22:01:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.445+0000 7fbcedc09640 1 -- start start 2024-04-26T22:01:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.446+0000 7fbcedc09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce8109930 0x7fbce819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.446+0000 7fbcedc09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce810c2a0 0x7fbce819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.446+0000 7fbcedc09640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce810ec70 0x7fbce81a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.446+0000 7fbcedc09640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbce8112bd0 con 0x7fbce810c2a0 2024-04-26T22:01:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.446+0000 7fbcedc09640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbce8112a50 con 0x7fbce8109930 2024-04-26T22:01:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.446+0000 7fbcedc09640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbce8112d50 con 0x7fbce810ec70 2024-04-26T22:01:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.447+0000 7fbce77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce8109930 0x7fbce819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.448+0000 7fbce77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce8109930 0x7fbce819f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46144/0 (socket says 172.21.15.55:46144) 2024-04-26T22:01:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.448+0000 7fbce77fe640 1 -- 172.21.15.55:0/1088833286 learned_addr learned my addr 172.21.15.55:0/1088833286 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:12.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.449+0000 7fbce77fe640 1 -- 172.21.15.55:0/1088833286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce810ec70 msgr2=0x7fbce81a6840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:01:12.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.449+0000 7fbce77fe640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce810ec70 0x7fbce81a6840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.449+0000 7fbce77fe640 1 -- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce810c2a0 msgr2=0x7fbce819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:12.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.449+0000 7fbce77fe640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce810c2a0 0x7fbce819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.449+0000 7fbce77fe640 1 -- 172.21.15.55:0/1088833286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce81a6eb0 con 0x7fbce8109930 2024-04-26T22:01:12.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.451+0000 7fbce77fe640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce8109930 0x7fbce819f290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00b450 tx=0x7fbcdc00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:12.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.460+0000 7fbce4ff9640 1 -- 172.21.15.55:0/1088833286 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc015020 con 0x7fbce8109930 2024-04-26T22:01:12.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.462+0000 7fbce4ff9640 1 -- 172.21.15.55:0/1088833286 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc011d40 con 0x7fbce8109930 2024-04-26T22:01:12.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.462+0000 7fbcedc09640 1 -- 172.21.15.55:0/1088833286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce81a71a0 con 0x7fbce8109930 2024-04-26T22:01:12.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.462+0000 7fbcedc09640 1 -- 172.21.15.55:0/1088833286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce81a76f0 con 0x7fbce8109930 2024-04-26T22:01:12.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.462+0000 7fbce4ff9640 1 -- 172.21.15.55:0/1088833286 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc00f340 con 0x7fbce8109930 2024-04-26T22:01:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.467+0000 7fbce4ff9640 1 -- 172.21.15.55:0/1088833286 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbcdc018020 con 0x7fbce8109930 2024-04-26T22:01:12.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.469+0000 7fbce4ff9640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbcd00a3dc0 0x7fbcd00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.469+0000 7fbce4ff9640 1 -- 172.21.15.55:0/1088833286 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbcdc0d0ef0 con 0x7fbce8109930 2024-04-26T22:01:12.471 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.471+0000 7fbcf0a00740 1 Processor -- start 2024-04-26T22:01:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.474+0000 7fbcf0a00740 1 -- start start 2024-04-26T22:01:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.474+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.475+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.475+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b6a70 0x564af57bd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.475+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564af5719d80 con 0x564af57b6a70 2024-04-26T22:01:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.475+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564af56d38b0 con 0x564af57b4ee0 2024-04-26T22:01:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.475+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564af5715d40 con 0x564af57b60b0 2024-04-26T22:01:12.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.475+0000 7fbce77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.475+0000 7fbce77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46158/0 (socket says 172.21.15.55:46158) 2024-04-26T22:01:12.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.475+0000 7fbce77fe640 1 -- 172.21.15.55:0/2932243443 learned_addr learned my addr 172.21.15.55:0/2932243443 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:12.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.479+0000 7fbce77fe640 1 -- 172.21.15.55:0/2932243443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 msgr2=0x564af57b6530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:12.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.479+0000 7fbce77fe640 1 --2- 172.21.15.55:0/2932243443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57b6530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.479+0000 7fbce77fe640 1 -- 172.21.15.55:0/2932243443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b6a70 msgr2=0x564af57bd300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:01:12.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.480+0000 7fbce7fff640 1 --2- 172.21.15.55:0/2932243443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b6a70 0x564af57bd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.480+0000 7fbce6ffd640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbcd00a3dc0 0x7fbcd00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.480+0000 7fbce77fe640 1 --2- 172.21.15.55:0/2932243443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b6a70 0x564af57bd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.480+0000 7fbce77fe640 1 -- 172.21.15.55:0/2932243443 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af56a4a40 con 0x564af57b4ee0 2024-04-26T22:01:12.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.483+0000 7fbce77fe640 1 --2- 172.21.15.55:0/2932243443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57b5300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00be10 tx=0x7fbcdc09cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab47517f0156d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:12.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.484+0000 7fbce6ffd640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbcd00a3dc0 0x7fbcd00a6280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbce003bf10 tx=0x7fbce0005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:12.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.484+0000 7fbccdffb640 1 -- 172.21.15.55:0/2932243443 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0b6070 con 0x564af57b4ee0 2024-04-26T22:01:12.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.484+0000 7fbccdffb640 1 -- 172.21.15.55:0/2932243443 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc0ceab0 con 0x564af57b4ee0 2024-04-26T22:01:12.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.484+0000 7fbccdffb640 1 -- 172.21.15.55:0/2932243443 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0b1e40 con 0x564af57b4ee0 2024-04-26T22:01:12.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.486+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2932243443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 msgr2=0x564af57b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:12.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.486+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2932243443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57b5300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00be10 tx=0x7fbcdc09cdb0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.489+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2932243443 shutdown_connections 2024-04-26T22:01:12.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.489+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2932243443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b6a70 0x564af57bd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.489+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2932243443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.489+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2932243443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57b5300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.489+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2932243443 >> 172.21.15.55:0/2932243443 conn(0x564af57b03d0 msgr2=0x564af57b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:12.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.496+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2932243443 shutdown_connections 2024-04-26T22:01:12.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.496+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2932243443 wait complete. 2024-04-26T22:01:12.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.497+0000 7fbcf0a00740 1 -- start start 2024-04-26T22:01:12.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.498+0000 7fbcf0a00740 1 Processor -- start 2024-04-26T22:01:12.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.499+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57bf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.500+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.500+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b6a70 0x564af57c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.500+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564af57bda80 con 0x564af57b4ee0 2024-04-26T22:01:12.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.500+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564af56a0d10 con 0x564af57b6a70 2024-04-26T22:01:12.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.500+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564af57bdc00 con 0x564af57b60b0 2024-04-26T22:01:12.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.500+0000 7fbcf0a00740 10 client.0 _ll_register_callbacks cb 0x564af56cd210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:12.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.501+0000 7fbce77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57bf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.501+0000 7fbce6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.501+0000 7fbce77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57bf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50470/0 (socket says 172.21.15.55:50470) 2024-04-26T22:01:12.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.502+0000 7fbce77fe640 1 -- 172.21.15.55:0/2128323593 learned_addr learned my addr 172.21.15.55:0/2128323593 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:12.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.502+0000 7fbce7fff640 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b6a70 0x564af57c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.502+0000 7fbce77fe640 1 -- 172.21.15.55:0/2128323593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 msgr2=0x564af57bf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:12.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.502+0000 7fbce77fe640 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57bf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.503+0000 7fbce77fe640 1 -- 172.21.15.55:0/2128323593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b6a70 msgr2=0x564af57c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:12.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.503+0000 7fbce77fe640 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b6a70 0x564af57c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.503+0000 7fbce77fe640 1 -- 172.21.15.55:0/2128323593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af54f57d0 con 0x564af57b4ee0 2024-04-26T22:01:12.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.505+0000 7fbce77fe640 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57bf3d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0a5880 tx=0x7fbcdc09d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79d647d9dee0e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.508+0000 7fbcc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.508+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/2128323593 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0ba070 con 0x564af57b4ee0 2024-04-26T22:01:12.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.508+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2128323593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcdc0ceab0 con 0x564af57b4ee0 2024-04-26T22:01:12.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.508+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/2128323593 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbcdc011aa0 con 0x564af57b4ee0 2024-04-26T22:01:12.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.514+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/2128323593 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbcdc0ad050 con 0x564af57b4ee0 2024-04-26T22:01:12.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.515+0000 7fbcf0a00740 10 client.16692 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:12.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.515+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2128323593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564af542b3f0 con 0x564af57b4ee0 2024-04-26T22:01:12.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.517+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/2128323593 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc0ceab0 con 0x564af57b4ee0 2024-04-26T22:01:12.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.517+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/2128323593 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fbcdc0a2680 con 0x564af57b4ee0 2024-04-26T22:01:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.517+0000 7fbcf0a00740 10 client.16692 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.517+0000 7fbcf0a00740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.517+0000 7fbcf0a00740 20 client.16692 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.517+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564af57caab0 0x564af57ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.517+0000 7fbcf0a00740 4 client.16692 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:12.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.517+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2128323593 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564af5727990 con 0x564af57caab0 2024-04-26T22:01:12.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.522+0000 7fbce6ffd640 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564af57caab0 0x564af57ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.527+0000 7fbce6ffd640 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564af57caab0 0x564af57ccea0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a1ec963ee1c8f5 server_cookie=58b43dfcdfd3c3b8 in_seq=0 out_seq=0 2024-04-26T22:01:12.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.527+0000 7fbcc6ffd640 10 client.16692 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:12.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.532+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/2128323593 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564af5727990 con 0x564af57caab0 2024-04-26T22:01:12.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.532+0000 7fbcc6ffd640 10 client.16692 handle_command_reply: tid=0 2024-04-26T22:01:12.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.532+0000 7fbcf0a00740 1 client.16692 shutdown 2024-04-26T22:01:12.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.533+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2128323593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 msgr2=0x564af57bf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:12.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.533+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57bf3d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0a5880 tx=0x7fbcdc09d040 comp rx=0 tx=0).stop 2024-04-26T22:01:12.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.534+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2128323593 shutdown_connections 2024-04-26T22:01:12.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.534+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b6a70 0x564af57c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.534+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.534+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564af57caab0 0x564af57ccea0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.534+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/2128323593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57bf3d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.534+0000 7fbcc6ffd640 0 client.16692 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:12.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.534+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2128323593 >> 172.21.15.55:0/2128323593 conn(0x564af57b03d0 msgr2=0x564af57af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:12.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.536+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2128323593 shutdown_connections 2024-04-26T22:01:12.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.537+0000 7fbcf0a00740 1 -- 172.21.15.55:0/2128323593 wait complete. 2024-04-26T22:01:12.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.537+0000 7fbcf0a00740 20 client.16692 trim_cache size 0 max 16384 2024-04-26T22:01:12.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.585+0000 7fbcf0a00740 1 Processor -- start 2024-04-26T22:01:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.585+0000 7fbcf0a00740 1 -- start start 2024-04-26T22:01:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.586+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b03d0 0x564af57afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.586+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.586+0000 7fbce6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.586+0000 7fbce6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50480/0 (socket says 172.21.15.55:50480) 2024-04-26T22:01:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.586+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.587+0000 7fbce6ffd640 1 -- 172.21.15.55:0/4203140466 learned_addr learned my addr 172.21.15.55:0/4203140466 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.587+0000 7fbcf0a00740 1 -- 172.21.15.55:0/4203140466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4010ee0 con 0x564af57b4ee0 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.587+0000 7fbce77fe640 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b03d0 0x564af57afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.587+0000 7fbcf0a00740 1 -- 172.21.15.55:0/4203140466 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd400e7b0 con 0x564af57b03d0 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.587+0000 7fbcf0a00740 1 -- 172.21.15.55:0/4203140466 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd400c600 con 0x564af57b60b0 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.587+0000 7fbce6ffd640 1 -- 172.21.15.55:0/4203140466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 msgr2=0x564af57b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.587+0000 7fbce6ffd640 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.588+0000 7fbce6ffd640 1 -- 172.21.15.55:0/4203140466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b03d0 msgr2=0x564af57afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.588+0000 7fbce6ffd640 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b03d0 0x564af57afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.588+0000 7fbce6ffd640 1 -- 172.21.15.55:0/4203140466 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af53eb0f0 con 0x564af57b4ee0 2024-04-26T22:01:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.588+0000 7fbce77fe640 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b03d0 0x564af57afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.588+0000 7fbce6ffd640 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57c6910 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbce001acb0 tx=0x7fbce004d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334756e65417ca07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.589+0000 7fbccd7fa640 1 -- 172.21.15.55:0/4203140466 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0051070 con 0x564af57b4ee0 2024-04-26T22:01:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.589+0000 7fbccd7fa640 1 -- 172.21.15.55:0/4203140466 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce00419f0 con 0x564af57b4ee0 2024-04-26T22:01:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.589+0000 7fbccd7fa640 1 -- 172.21.15.55:0/4203140466 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0045040 con 0x564af57b4ee0 2024-04-26T22:01:12.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.590+0000 7fbcf0a00740 1 -- 172.21.15.55:0/4203140466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 msgr2=0x564af57c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:12.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.591+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57c6910 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbce001acb0 tx=0x7fbce004d000 comp rx=0 tx=0).stop 2024-04-26T22:01:12.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.591+0000 7fbcf0a00740 1 -- 172.21.15.55:0/4203140466 shutdown_connections 2024-04-26T22:01:12.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.591+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b60b0 0x564af57b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.591+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b4ee0 0x564af57c6910 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.591+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/4203140466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b03d0 0x564af57afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.591+0000 7fbcf0a00740 1 -- 172.21.15.55:0/4203140466 >> 172.21.15.55:0/4203140466 conn(0x564af57ab7f0 msgr2=0x564af57b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:12.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.591+0000 7fbcf0a00740 1 -- 172.21.15.55:0/4203140466 shutdown_connections 2024-04-26T22:01:12.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.592+0000 7fbcf0a00740 1 -- 172.21.15.55:0/4203140466 wait complete. 2024-04-26T22:01:12.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.592+0000 7fbcf0a00740 1 -- start start 2024-04-26T22:01:12.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.596+0000 7fbcf0a00740 1 Processor -- start 2024-04-26T22:01:12.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.596+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b03d0 0x564af57c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.597+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.597+0000 7fbcf0a00740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b60b0 0x564af57c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.598+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd400cc70 con 0x564af57b60b0 2024-04-26T22:01:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.598+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd400e0e0 con 0x564af57b4ee0 2024-04-26T22:01:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.598+0000 7fbcf0a00740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd400d340 con 0x564af57b03d0 2024-04-26T22:01:12.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.598+0000 7fbcf0a00740 10 client.0 _ll_register_callbacks cb 0x7fbcdc0b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.599+0000 7fbce7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b60b0 0x564af57c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.599+0000 7fbce7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b60b0 0x564af57c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50488/0 (socket says 172.21.15.55:50488) 2024-04-26T22:01:12.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.599+0000 7fbce7fff640 1 -- 172.21.15.55:0/1093670840 learned_addr learned my addr 172.21.15.55:0/1093670840 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:12.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.601+0000 7fbce7fff640 1 -- 172.21.15.55:0/1093670840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b03d0 msgr2=0x564af57c4850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:12.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.601+0000 7fbce7fff640 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b03d0 0x564af57c4850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.601+0000 7fbce7fff640 1 -- 172.21.15.55:0/1093670840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 msgr2=0x564af57c4d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:12.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.601+0000 7fbce7fff640 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57c4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.601+0000 7fbce7fff640 1 -- 172.21.15.55:0/1093670840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af56da4b0 con 0x564af57b60b0 2024-04-26T22:01:12.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.602+0000 7fbce7fff640 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b60b0 0x564af57c5da0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8004700 tx=0x7fbcd800e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638b7f05a8543671 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.604+0000 7fbcc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:12.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.604+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/1093670840 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8018020 con 0x564af57b60b0 2024-04-26T22:01:12.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.604+0000 7fbcf0a00740 1 -- 172.21.15.55:0/1093670840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce00419f0 con 0x564af57b60b0 2024-04-26T22:01:12.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.604+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/1093670840 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fbcd800ede0 con 0x564af57b60b0 2024-04-26T22:01:12.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.610+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/1093670840 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbcd8023050 con 0x564af57b60b0 2024-04-26T22:01:12.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.610+0000 7fbcf0a00740 10 client.16698 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:12.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.610+0000 7fbcf0a00740 1 -- 172.21.15.55:0/1093670840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564af58314d0 con 0x564af57b60b0 2024-04-26T22:01:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.612+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/1093670840 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd8013900 con 0x564af57b60b0 2024-04-26T22:01:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.612+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/1093670840 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fbcd80147d0 con 0x564af57b60b0 2024-04-26T22:01:12.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.612+0000 7fbcf0a00740 10 client.16698 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:12.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.612+0000 7fbcf0a00740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:12.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.612+0000 7fbcf0a00740 20 client.16698 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:12.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.613+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564af57b6a70 0x564af5833f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:12.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.613+0000 7fbcf0a00740 4 client.16698 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:01:12.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.613+0000 7fbcf0a00740 1 -- 172.21.15.55:0/1093670840 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbce00419f0 con 0x564af57b6a70 2024-04-26T22:01:12.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.634+0000 7fbce77fe640 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564af57b6a70 0x564af5833f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.640+0000 7fbce77fe640 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564af57b6a70 0x564af5833f70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b77c022d8236cc server_cookie=febf59180deb2207 in_seq=0 out_seq=0 2024-04-26T22:01:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.640+0000 7fbcc6ffd640 10 client.16698 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:12.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.644+0000 7fbcc6ffd640 1 -- 172.21.15.55:0/1093670840 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fbce00419f0 con 0x564af57b6a70 2024-04-26T22:01:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.644+0000 7fbcc6ffd640 10 client.16698 handle_command_reply: tid=0 2024-04-26T22:01:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.644+0000 7fbcf0a00740 1 client.16698 shutdown 2024-04-26T22:01:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcf0a00740 1 -- 172.21.15.55:0/1093670840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b60b0 msgr2=0x564af57c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:12.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b60b0 0x564af57c5da0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8004700 tx=0x7fbcd800e5a0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcf0a00740 1 -- 172.21.15.55:0/1093670840 shutdown_connections 2024-04-26T22:01:12.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564af57b6a70 0x564af5833f70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564af57b60b0 0x564af57c5da0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564af57b4ee0 0x564af57c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcf0a00740 1 --2- 172.21.15.55:0/1093670840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564af57b03d0 0x564af57c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcf0a00740 1 -- 172.21.15.55:0/1093670840 >> 172.21.15.55:0/1093670840 conn(0x564af57ab7f0 msgr2=0x564af57b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:12.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.645+0000 7fbcc6ffd640 0 client.16698 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:12.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.646+0000 7fbcf0a00740 1 -- 172.21.15.55:0/1093670840 shutdown_connections 2024-04-26T22:01:12.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.646+0000 7fbcf0a00740 1 -- 172.21.15.55:0/1093670840 wait complete. 2024-04-26T22:01:12.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.646+0000 7fbcf0a00740 20 client.16698 trim_cache size 0 max 16384 2024-04-26T22:01:12.650 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:01:12.651 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:01:12.651 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:01:12.651 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:01:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.651+0000 7fbcedc09640 1 -- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbcd00a3dc0 msgr2=0x7fbcd00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.651+0000 7fbcedc09640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbcd00a3dc0 0x7fbcd00a6280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbce003bf10 tx=0x7fbce0005f80 comp rx=0 tx=0).stop 2024-04-26T22:01:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.651+0000 7fbcedc09640 1 -- 172.21.15.55:0/1088833286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce8109930 msgr2=0x7fbce819f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:12.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.651+0000 7fbcedc09640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce8109930 0x7fbce819f290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00b450 tx=0x7fbcdc00b910 comp rx=0 tx=0).stop 2024-04-26T22:01:12.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.653+0000 7fbcedc09640 1 -- 172.21.15.55:0/1088833286 shutdown_connections 2024-04-26T22:01:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.653+0000 7fbcedc09640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fbcd00a3dc0 0x7fbcd00a6280 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.653+0000 7fbcedc09640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbce810ec70 0x7fbce81a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.653+0000 7fbcedc09640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbce810c2a0 0x7fbce819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.653+0000 7fbcedc09640 1 --2- 172.21.15.55:0/1088833286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbce8109930 0x7fbce819f290 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:12.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.653+0000 7fbcedc09640 1 -- 172.21.15.55:0/1088833286 >> 172.21.15.55:0/1088833286 conn(0x7fbce806f7c0 msgr2=0x7fbce810cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:12.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.657+0000 7fbcedc09640 1 -- 172.21.15.55:0/1088833286 shutdown_connections 2024-04-26T22:01:12.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:12.657+0000 7fbcedc09640 1 -- 172.21.15.55:0/1088833286 wait complete. 2024-04-26T22:01:12.708 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:01:12.708 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:13.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.089+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/2372676736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 msgr2=0x7fc6e0104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.089+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/2372676736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 0x7fc6e0104fd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009ca0 tx=0x7fc6dc02a8f0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.089+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/2372676736 shutdown_connections 2024-04-26T22:01:13.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.089+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/2372676736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e0106760 0x7fc6e010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.089+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/2372676736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6e0105da0 0x7fc6e0106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.089+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/2372676736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 0x7fc6e0104fd0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.089+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/2372676736 >> 172.21.15.55:0/2372676736 conn(0x7fc6e0100120 msgr2=0x7fc6e0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:13.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.089+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/2372676736 shutdown_connections 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.090+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/2372676736 wait complete. 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.090+0000 7fc6e6fe7640 1 Processor -- start 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e6fe7640 1 -- start start 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e6fe7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 0x7fc6e019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e6fe7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e0105da0 0x7fc6e019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e6fe7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6e0106760 0x7fc6e01a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e4d83640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 0x7fc6e019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e6fe7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01107f0 con 0x7fc6e0104bb0 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e6fe7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0110670 con 0x7fc6e0105da0 2024-04-26T22:01:13.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e6fe7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0110970 con 0x7fc6e0106760 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e4d83640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 0x7fc6e019b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50518/0 (socket says 172.21.15.55:50518) 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.091+0000 7fc6e4d83640 1 -- 172.21.15.55:0/3226971990 learned_addr learned my addr 172.21.15.55:0/3226971990 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6e4d83640 1 -- 172.21.15.55:0/3226971990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6e0106760 msgr2=0x7fc6e01a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6e4d83640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6e0106760 0x7fc6e01a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6e4d83640 1 -- 172.21.15.55:0/3226971990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e0105da0 msgr2=0x7fc6e019b570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6e4d83640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e0105da0 0x7fc6e019b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6e4d83640 1 -- 172.21.15.55:0/3226971990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e01a2c00 con 0x7fc6e0104bb0 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6e4d83640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 0x7fc6e019b030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc033040 tx=0x7fc6dc003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6d5ffb640 1 -- 172.21.15.55:0/3226971990 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc004240 con 0x7fc6e0104bb0 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6d5ffb640 1 -- 172.21.15.55:0/3226971990 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6dc02d070 con 0x7fc6e0104bb0 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.092+0000 7fc6d5ffb640 1 -- 172.21.15.55:0/3226971990 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc03c410 con 0x7fc6e0104bb0 2024-04-26T22:01:13.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.093+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e01a2e90 con 0x7fc6e0104bb0 2024-04-26T22:01:13.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.093+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e01a3350 con 0x7fc6e0104bb0 2024-04-26T22:01:13.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.095+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e010c060 con 0x7fc6e0104bb0 2024-04-26T22:01:13.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.098+0000 7fc6d5ffb640 1 -- 172.21.15.55:0/3226971990 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc6dc030710 con 0x7fc6e0104bb0 2024-04-26T22:01:13.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.099+0000 7fc6d5ffb640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6c80a3d40 0x7fc6c80a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:13.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.100+0000 7fc6d5ffb640 1 -- 172.21.15.55:0/3226971990 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc6dc0f21c0 con 0x7fc6e0104bb0 2024-04-26T22:01:13.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.100+0000 7fc6d7fff640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6c80a3d40 0x7fc6c80a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:13.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.101+0000 7fc6d7fff640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6c80a3d40 0x7fc6c80a6200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc6e019c4f0 tx=0x7fc6d0006bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:13.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.107+0000 7fc6d5ffb640 1 -- 172.21.15.55:0/3226971990 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc6dc081060 con 0x7fc6e0104bb0 2024-04-26T22:01:13.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.728+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6e002dad0 con 0x7fc6e0104bb0 2024-04-26T22:01:13.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.729+0000 7fc6d5ffb640 1 -- 172.21.15.55:0/3226971990 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fc6dc03a390 con 0x7fc6e0104bb0 2024-04-26T22:01:13.730 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:01:13.730 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:01:13.730 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:01:13.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6c80a3d40 msgr2=0x7fc6c80a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6c80a3d40 0x7fc6c80a6200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc6e019c4f0 tx=0x7fc6d0006bd0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 msgr2=0x7fc6e019b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 0x7fc6e019b030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc033040 tx=0x7fc6dc003d60 comp rx=0 tx=0).stop 2024-04-26T22:01:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 shutdown_connections 2024-04-26T22:01:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc6c80a3d40 0x7fc6c80a6200 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6e0106760 0x7fc6e01a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6e0105da0 0x7fc6e019b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 --2- 172.21.15.55:0/3226971990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6e0104bb0 0x7fc6e019b030 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:13.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.735+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 >> 172.21.15.55:0/3226971990 conn(0x7fc6e0100120 msgr2=0x7fc6e0109230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:13.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.736+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 shutdown_connections 2024-04-26T22:01:13.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:13.736+0000 7fc6e6fe7640 1 -- 172.21.15.55:0/3226971990 wait complete. 2024-04-26T22:01:13.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:01:13.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:13 smithi165 ceph-mon[33296]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 2.07k op/s 2024-04-26T22:01:13.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:13 smithi055 ceph-mon[29221]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 2.07k op/s 2024-04-26T22:01:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:13 smithi064 ceph-mon[32765]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 2.07k op/s 2024-04-26T22:01:14.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.225+0000 7fc2b82c4640 1 -- 172.21.15.55:0/4074017535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01027c0 msgr2=0x7fc2b0102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.225+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/4074017535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01027c0 0x7fc2b0102be0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009900 tx=0x7fc2a002a400 comp rx=0 tx=0).stop 2024-04-26T22:01:14.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.226+0000 7fc2b82c4640 1 -- 172.21.15.55:0/4074017535 shutdown_connections 2024-04-26T22:01:14.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.226+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/4074017535 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b0104370 0x7fc2b010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.226+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/4074017535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b01039b0 0x7fc2b0103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.226+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/4074017535 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01027c0 0x7fc2b0102be0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.226+0000 7fc2b82c4640 1 -- 172.21.15.55:0/4074017535 >> 172.21.15.55:0/4074017535 conn(0x7fc2b00fdd30 msgr2=0x7fc2b0100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.226+0000 7fc2b82c4640 1 -- 172.21.15.55:0/4074017535 shutdown_connections 2024-04-26T22:01:14.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.227+0000 7fc2b82c4640 1 -- 172.21.15.55:0/4074017535 wait complete. 2024-04-26T22:01:14.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.228+0000 7fc2b82c4640 1 Processor -- start 2024-04-26T22:01:14.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.228+0000 7fc2b82c4640 1 -- start start 2024-04-26T22:01:14.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.228+0000 7fc2b82c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b01027c0 0x7fc2b0071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.228+0000 7fc2b82c4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01039b0 0x7fc2b0071870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.228+0000 7fc2b82c4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b0104370 0x7fc2b01a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.228+0000 7fc2b82c4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b010e390 con 0x7fc2b01039b0 2024-04-26T22:01:14.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.228+0000 7fc2b82c4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b010e210 con 0x7fc2b01027c0 2024-04-26T22:01:14.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.228+0000 7fc2b82c4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b010e510 con 0x7fc2b0104370 2024-04-26T22:01:14.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b6861640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b0104370 0x7fc2b01a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b6861640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b0104370 0x7fc2b01a8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:44922/0 (socket says 172.21.15.55:44922) 2024-04-26T22:01:14.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b6861640 1 -- 172.21.15.55:0/203208203 learned_addr learned my addr 172.21.15.55:0/203208203 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:14.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b585f640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01039b0 0x7fc2b0071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b585f640 1 -- 172.21.15.55:0/203208203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b0104370 msgr2=0x7fc2b01a8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b585f640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b0104370 0x7fc2b01a8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b585f640 1 -- 172.21.15.55:0/203208203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b01027c0 msgr2=0x7fc2b0071330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:01:14.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b585f640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b01027c0 0x7fc2b0071330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.229+0000 7fc2b585f640 1 -- 172.21.15.55:0/203208203 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b01a8fe0 con 0x7fc2b01039b0 2024-04-26T22:01:14.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.230+0000 7fc2b585f640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01039b0 0x7fc2b0071870 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005fd0 tx=0x7fc2ac00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.230+0000 7fc2a77fe640 1 -- 172.21.15.55:0/203208203 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac009c30 con 0x7fc2b01039b0 2024-04-26T22:01:14.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.230+0000 7fc2b82c4640 1 -- 172.21.15.55:0/203208203 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b01a91b0 con 0x7fc2b01039b0 2024-04-26T22:01:14.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.230+0000 7fc2b82c4640 1 -- 172.21.15.55:0/203208203 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b01a9720 con 0x7fc2b01039b0 2024-04-26T22:01:14.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.230+0000 7fc2a77fe640 1 -- 172.21.15.55:0/203208203 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac009dd0 con 0x7fc2b01039b0 2024-04-26T22:01:14.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.230+0000 7fc2a77fe640 1 -- 172.21.15.55:0/203208203 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac009c30 con 0x7fc2b01039b0 2024-04-26T22:01:14.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.235+0000 7fc2bafdc740 1 Processor -- start 2024-04-26T22:01:14.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.235+0000 7fc2bafdc740 1 -- start start 2024-04-26T22:01:14.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 0x55be21cb9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 0x55be21cbb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbb590 0x55be21cc6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2a77fe640 1 -- 172.21.15.55:0/203208203 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc2ac0035f0 con 0x7fc2b01039b0 2024-04-26T22:01:14.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55be21c1dd80 con 0x55be21cbb590 2024-04-26T22:01:14.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55be21bd7a20 con 0x55be21cbabd0 2024-04-26T22:01:14.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55be21c19d40 con 0x55be21cb9a00 2024-04-26T22:01:14.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2b585f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbb590 0x55be21cc6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2b585f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbb590 0x55be21cc6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50534/0 (socket says 172.21.15.55:50534) 2024-04-26T22:01:14.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2b585f640 1 -- 172.21.15.55:0/3787453583 learned_addr learned my addr 172.21.15.55:0/3787453583 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.236+0000 7fc2b6861640 1 --2- 172.21.15.55:0/3787453583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 0x55be21cb9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.237+0000 7fc2b585f640 1 -- 172.21.15.55:0/3787453583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 msgr2=0x55be21cb9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.237+0000 7fc2b585f640 1 --2- 172.21.15.55:0/3787453583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 0x55be21cb9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.237+0000 7fc2b585f640 1 -- 172.21.15.55:0/3787453583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 msgr2=0x55be21cbb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.237+0000 7fc2b585f640 1 --2- 172.21.15.55:0/3787453583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 0x55be21cbb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.237+0000 7fc2b585f640 1 -- 172.21.15.55:0/3787453583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be21be2c30 con 0x55be21cbb590 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.237+0000 7fc2b585f640 1 --2- 172.21.15.55:0/3787453583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbb590 0x55be21cc6230 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0cd3f0 tx=0x7fc2ac0d7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a08ae71f7c92c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.237+0000 7fc2a77fe640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc29c0a4070 0x7fc29c0a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.238 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.238+0000 7fc2b6861640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc29c0a4070 0x7fc29c0a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.238+0000 7fc2a77fe640 1 -- 172.21.15.55:0/203208203 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2ac009a30 con 0x7fc2b01039b0 2024-04-26T22:01:14.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.238+0000 7fc2a4bfe640 1 -- 172.21.15.55:0/3787453583 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0de070 con 0x55be21cbb590 2024-04-26T22:01:14.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.238+0000 7fc2b6861640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc29c0a4070 0x7fc29c0a6530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55be21cbaad0 tx=0x7fc2a800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.239+0000 7fc2a4bfe640 1 -- 172.21.15.55:0/3787453583 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac0d7e40 con 0x55be21cbb590 2024-04-26T22:01:14.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.240+0000 7fc2a4bfe640 1 -- 172.21.15.55:0/3787453583 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac09b7c0 con 0x55be21cbb590 2024-04-26T22:01:14.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.240+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3787453583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbb590 msgr2=0x55be21cc6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.240+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3787453583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbb590 0x55be21cc6230 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0cd3f0 tx=0x7fc2ac0d7670 comp rx=0 tx=0).stop 2024-04-26T22:01:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.241+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3787453583 shutdown_connections 2024-04-26T22:01:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.241+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3787453583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbb590 0x55be21cc6230 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.241+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3787453583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 0x55be21cbb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.241+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3787453583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 0x55be21cb9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.241+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3787453583 >> 172.21.15.55:0/3787453583 conn(0x55be21bfa910 msgr2=0x55be21cb52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.241+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3787453583 shutdown_connections 2024-04-26T22:01:14.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.241+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3787453583 wait complete. 2024-04-26T22:01:14.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.242+0000 7fc2bafdc740 1 -- start start 2024-04-26T22:01:14.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.242+0000 7fc2bafdc740 1 Processor -- start 2024-04-26T22:01:14.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.248+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cc3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.248+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 0x55be21cc41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.248+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cbb590 0x55be21cc51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.248+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55be21cc00a0 con 0x55be21cb9a00 2024-04-26T22:01:14.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.248+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55be21ba4dc0 con 0x55be21cbabd0 2024-04-26T22:01:14.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.248+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55be21cc0220 con 0x55be21cbb590 2024-04-26T22:01:14.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.248+0000 7fc2bafdc740 10 client.0 _ll_register_callbacks cb 0x55be21bc0600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:14.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.250+0000 7fc2b6060640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cc3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.250+0000 7fc2b6060640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cc3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50542/0 (socket says 172.21.15.55:50542) 2024-04-26T22:01:14.251 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.250+0000 7fc2b6060640 1 -- 172.21.15.55:0/3419246597 learned_addr learned my addr 172.21.15.55:0/3419246597 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.252+0000 7fc2b6861640 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cbb590 0x55be21cc51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.252+0000 7fc2b585f640 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 0x55be21cc41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.257+0000 7fc2b585f640 1 -- 172.21.15.55:0/3419246597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cbb590 msgr2=0x55be21cc51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.257+0000 7fc2b585f640 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cbb590 0x55be21cc51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.257+0000 7fc2b585f640 1 -- 172.21.15.55:0/3419246597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 msgr2=0x55be21cc3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.257+0000 7fc2b585f640 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cc3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.257+0000 7fc2b585f640 1 -- 172.21.15.55:0/3419246597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be2192f3f0 con 0x55be21cbabd0 2024-04-26T22:01:14.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.261+0000 7fc2b585f640 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 0x55be21cc41e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac003990 tx=0x7fc2ac0d5370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a915c1d1027ab40b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.264+0000 7fc299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:01:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.270+0000 7fc299ffb640 1 -- 172.21.15.55:0/3419246597 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0de070 con 0x55be21cbabd0 2024-04-26T22:01:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.270+0000 7fc299ffb640 1 -- 172.21.15.55:0/3419246597 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2ac0d56d0 con 0x55be21cbabd0 2024-04-26T22:01:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.270+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3419246597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be21c2b990 con 0x55be21cbabd0 2024-04-26T22:01:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.271+0000 7fc299ffb640 1 -- 172.21.15.55:0/3419246597 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac009c30 con 0x55be21cbabd0 2024-04-26T22:01:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.271+0000 7fc2bafdc740 10 client.25847 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.271+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3419246597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55be219f97d0 con 0x55be21cbabd0 2024-04-26T22:01:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.272+0000 7fc299ffb640 1 -- 172.21.15.55:0/3419246597 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55be21c2b990 con 0x55be21cbabd0 2024-04-26T22:01:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.272+0000 7fc299ffb640 1 -- 172.21.15.55:0/3419246597 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc2ac0982f0 con 0x55be21cbabd0 2024-04-26T22:01:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.272+0000 7fc2bafdc740 10 client.25847 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.272+0000 7fc2bafdc740 10 client.25847 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:01:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.272+0000 7fc2bafdc740 20 client.25847 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.272+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55be21cd84d0 0x55be21cda930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.272+0000 7fc2bafdc740 4 client.25847 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.272+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3419246597 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be21c25800 con 0x55be21cd84d0 2024-04-26T22:01:14.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.274+0000 7fc2b6060640 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55be21cd84d0 0x55be21cda930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.279+0000 7fc2b6060640 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55be21cd84d0 0x55be21cda930 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3af123886ee3deb7 server_cookie=aba4be356af71938 in_seq=0 out_seq=0 2024-04-26T22:01:14.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.280+0000 7fc299ffb640 10 client.25847 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:01:14.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.283+0000 7fc299ffb640 1 -- 172.21.15.55:0/3419246597 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be21c25800 con 0x55be21cd84d0 2024-04-26T22:01:14.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.284+0000 7fc299ffb640 10 client.25847 handle_command_reply: tid=0 2024-04-26T22:01:14.286 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.284+0000 7fc2bafdc740 1 client.25847 shutdown 2024-04-26T22:01:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3419246597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 msgr2=0x55be21cc41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 0x55be21cc41e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac003990 tx=0x7fc2ac0d5370 comp rx=0 tx=0).stop 2024-04-26T22:01:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3419246597 shutdown_connections 2024-04-26T22:01:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55be21cd84d0 0x55be21cda930 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cbb590 0x55be21cc51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbabd0 0x55be21cc41e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3419246597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cc3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3419246597 >> 172.21.15.55:0/3419246597 conn(0x55be21bfa910 msgr2=0x55be21cb4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc299ffb640 0 client.25847 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:01:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.289+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3419246597 shutdown_connections 2024-04-26T22:01:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.290+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3419246597 wait complete. 2024-04-26T22:01:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.290+0000 7fc2bafdc740 20 client.25847 trim_cache size 0 max 16384 2024-04-26T22:01:14.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.358+0000 7fc2bafdc740 1 Processor -- start 2024-04-26T22:01:14.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.358+0000 7fc2bafdc740 1 -- start start 2024-04-26T22:01:14.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.359+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 0x55be21cb0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.359+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbb590 0x55be21cb4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.359+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbeaa0 0x55be21cb5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.360+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc294010e20 con 0x55be21cbeaa0 2024-04-26T22:01:14.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.360+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc29400e6f0 con 0x55be21cbb590 2024-04-26T22:01:14.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.360+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc29400c540 con 0x55be21cb9a00 2024-04-26T22:01:14.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.361+0000 7fc2b585f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbb590 0x55be21cb4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.362+0000 7fc2b585f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbb590 0x55be21cb4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46216/0 (socket says 172.21.15.55:46216) 2024-04-26T22:01:14.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.362+0000 7fc2b585f640 1 -- 172.21.15.55:0/550563496 learned_addr learned my addr 172.21.15.55:0/550563496 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:14.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.362+0000 7fc2b6060640 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 0x55be21cb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.362+0000 7fc2b6861640 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbeaa0 0x55be21cb5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.363+0000 7fc2b585f640 1 -- 172.21.15.55:0/550563496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 msgr2=0x55be21cb0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.363+0000 7fc2b585f640 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 0x55be21cb0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.363+0000 7fc2b585f640 1 -- 172.21.15.55:0/550563496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbeaa0 msgr2=0x55be21cb5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.363+0000 7fc2b585f640 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbeaa0 0x55be21cb5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.363+0000 7fc2b585f640 1 -- 172.21.15.55:0/550563496 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be21cbff30 con 0x55be21cbb590 2024-04-26T22:01:14.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.371+0000 7fc2b585f640 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbb590 0x55be21cb4d30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac09bc10 tx=0x7fc2ac09a190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc12a0ce25af996c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.376+0000 7fc29bfff640 1 -- 172.21.15.55:0/550563496 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0de070 con 0x55be21cbb590 2024-04-26T22:01:14.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.376+0000 7fc29bfff640 1 -- 172.21.15.55:0/550563496 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac09c380 con 0x55be21cbb590 2024-04-26T22:01:14.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.376+0000 7fc29bfff640 1 -- 172.21.15.55:0/550563496 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac09b5e0 con 0x55be21cbb590 2024-04-26T22:01:14.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.380+0000 7fc2bafdc740 1 -- 172.21.15.55:0/550563496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbb590 msgr2=0x55be21cb4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.380+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbb590 0x55be21cb4d30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac09bc10 tx=0x7fc2ac09a190 comp rx=0 tx=0).stop 2024-04-26T22:01:14.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.381+0000 7fc2bafdc740 1 -- 172.21.15.55:0/550563496 shutdown_connections 2024-04-26T22:01:14.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.381+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cbeaa0 0x55be21cb5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.381+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbb590 0x55be21cb4d30 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac09bc10 tx=0x7fc2ac09a190 comp rx=0 tx=0).stop 2024-04-26T22:01:14.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.381+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/550563496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cb9a00 0x55be21cb0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.381+0000 7fc2bafdc740 1 -- 172.21.15.55:0/550563496 >> 172.21.15.55:0/550563496 conn(0x55be21bfa910 msgr2=0x55be21cbe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.385+0000 7fc2bafdc740 1 -- 172.21.15.55:0/550563496 shutdown_connections 2024-04-26T22:01:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.388+0000 7fc2bafdc740 1 -- 172.21.15.55:0/550563496 wait complete. 2024-04-26T22:01:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.389+0000 7fc2bafdc740 1 -- start start 2024-04-26T22:01:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.390+0000 7fc2bafdc740 1 Processor -- start 2024-04-26T22:01:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.390+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cd17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbeaa0 0x55be21cd1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2bafdc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cd4b60 0x55be21d3ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc29400cbb0 con 0x55be21cb9a00 2024-04-26T22:01:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc29400e020 con 0x55be21cbeaa0 2024-04-26T22:01:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2bafdc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc29400d280 con 0x55be21cd4b60 2024-04-26T22:01:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2bafdc740 10 client.0 _ll_register_callbacks cb 0x7fc2ac0d5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:14.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2b6861640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cd4b60 0x55be21d3ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2b6861640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cd4b60 0x55be21d3ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:44972/0 (socket says 172.21.15.55:44972) 2024-04-26T22:01:14.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.391+0000 7fc2b6861640 1 -- 172.21.15.55:0/3113473451 learned_addr learned my addr 172.21.15.55:0/3113473451 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:14.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.392+0000 7fc2b585f640 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbeaa0 0x55be21cd1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.392+0000 7fc2b6060640 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cd17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.392+0000 7fc2b6861640 1 -- 172.21.15.55:0/3113473451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbeaa0 msgr2=0x55be21cd1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.392+0000 7fc2b6861640 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbeaa0 0x55be21cd1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.392+0000 7fc2b6861640 1 -- 172.21.15.55:0/3113473451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 msgr2=0x55be21cd17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.392+0000 7fc2b6861640 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cd17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.392+0000 7fc2b6861640 1 -- 172.21.15.55:0/3113473451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be21d3cff0 con 0x55be21cd4b60 2024-04-26T22:01:14.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.393+0000 7fc2b6861640 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cd4b60 0x55be21d3ca60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8016b00 tx=0x7fc2a8004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd5ea5e2ac5422 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.394+0000 7fc299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:01:14.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.394+0000 7fc299ffb640 1 -- 172.21.15.55:0/3113473451 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8028020 con 0x55be21cd4b60 2024-04-26T22:01:14.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.394+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3113473451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac09c380 con 0x55be21cd4b60 2024-04-26T22:01:14.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.394+0000 7fc299ffb640 1 -- 172.21.15.55:0/3113473451 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2a8014ce0 con 0x55be21cd4b60 2024-04-26T22:01:14.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.395+0000 7fc299ffb640 1 -- 172.21.15.55:0/3113473451 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc2a8033050 con 0x55be21cd4b60 2024-04-26T22:01:14.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.395+0000 7fc2bafdc740 10 client.25882 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:14.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.395+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3113473451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55be21d3e3f0 con 0x55be21cd4b60 2024-04-26T22:01:14.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.397+0000 7fc299ffb640 1 -- 172.21.15.55:0/3113473451 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac09c380 con 0x55be21cd4b60 2024-04-26T22:01:14.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.398+0000 7fc299ffb640 1 -- 172.21.15.55:0/3113473451 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc2a80234e0 con 0x55be21cd4b60 2024-04-26T22:01:14.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.398+0000 7fc2bafdc740 10 client.25882 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:14.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.398+0000 7fc2bafdc740 10 client.25882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:01:14.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.398+0000 7fc2bafdc740 20 client.25882 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:14.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.398+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55be21d3edd0 0x55be21d411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.398+0000 7fc2bafdc740 4 client.25882 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:01:14.399 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.398+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3113473451 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be21ba0610 con 0x55be21d3edd0 2024-04-26T22:01:14.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.399+0000 7fc2b6060640 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55be21d3edd0 0x55be21d411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.402+0000 7fc2b6060640 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55be21d3edd0 0x55be21d411f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8193d6001b402c75 server_cookie=bb403cb47fa4b6f0 in_seq=0 out_seq=0 2024-04-26T22:01:14.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.404+0000 7fc299ffb640 10 client.25882 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:01:14.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.406+0000 7fc299ffb640 1 -- 172.21.15.55:0/3113473451 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be21ba0610 con 0x55be21d3edd0 2024-04-26T22:01:14.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.406+0000 7fc299ffb640 10 client.25882 handle_command_reply: tid=0 2024-04-26T22:01:14.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.406+0000 7fc2bafdc740 1 client.25882 shutdown 2024-04-26T22:01:14.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3113473451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cd4b60 msgr2=0x55be21d3ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cd4b60 0x55be21d3ca60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8016b00 tx=0x7fc2a8004b50 comp rx=0 tx=0).stop 2024-04-26T22:01:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3113473451 shutdown_connections 2024-04-26T22:01:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55be21d3edd0 0x55be21d411f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be21cd4b60 0x55be21d3ca60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55be21cbeaa0 0x55be21cd1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc2bafdc740 1 --2- 172.21.15.55:0/3113473451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55be21cb9a00 0x55be21cd17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3113473451 >> 172.21.15.55:0/3113473451 conn(0x55be21bfa910 msgr2=0x55be21cb04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.410+0000 7fc299ffb640 0 client.25882 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:01:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.416+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3113473451 shutdown_connections 2024-04-26T22:01:14.419 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.417+0000 7fc2bafdc740 1 -- 172.21.15.55:0/3113473451 wait complete. 2024-04-26T22:01:14.420 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:01:14.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.417+0000 7fc2bafdc740 20 client.25882 trim_cache size 0 max 16384 2024-04-26T22:01:14.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 -- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc29c0a4070 msgr2=0x7fc29c0a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc29c0a4070 0x7fc29c0a6530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55be21cbaad0 tx=0x7fc2a800b040 comp rx=0 tx=0).stop 2024-04-26T22:01:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 -- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01039b0 msgr2=0x7fc2b0071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01039b0 0x7fc2b0071870 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005fd0 tx=0x7fc2ac00dde0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 -- 172.21.15.55:0/203208203 shutdown_connections 2024-04-26T22:01:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc29c0a4070 0x7fc29c0a6530 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2b0104370 0x7fc2b01a8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2b01039b0 0x7fc2b0071870 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 --2- 172.21.15.55:0/203208203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2b01027c0 0x7fc2b0071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.423+0000 7fc2b82c4640 1 -- 172.21.15.55:0/203208203 >> 172.21.15.55:0/203208203 conn(0x7fc2b00fdd30 msgr2=0x7fc2b0106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.424+0000 7fc2b82c4640 1 -- 172.21.15.55:0/203208203 shutdown_connections 2024-04-26T22:01:14.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.424+0000 7fc2b82c4640 1 -- 172.21.15.55:0/203208203 wait complete. 2024-04-26T22:01:14.470 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:01:14.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:14 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3226971990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:14.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.841+0000 7f992f3b4640 1 -- 172.21.15.55:0/205115016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f992810ec70 msgr2=0x7f992810f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.841+0000 7f992f3b4640 1 --2- 172.21.15.55:0/205115016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f992810ec70 0x7f992810f120 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9920009ca0 tx=0x7f992002a8e0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.842+0000 7f992f3b4640 1 -- 172.21.15.55:0/205115016 shutdown_connections 2024-04-26T22:01:14.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.842+0000 7f992f3b4640 1 --2- 172.21.15.55:0/205115016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f992810ec70 0x7f992810f120 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.842+0000 7f992f3b4640 1 --2- 172.21.15.55:0/205115016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992810c2a0 0x7f992810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.842+0000 7f992f3b4640 1 --2- 172.21.15.55:0/205115016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9928109930 0x7f992810bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.842+0000 7f992f3b4640 1 -- 172.21.15.55:0/205115016 >> 172.21.15.55:0/205115016 conn(0x7f992806f7c0 msgr2=0x7f9928071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.842+0000 7f992f3b4640 1 -- 172.21.15.55:0/205115016 shutdown_connections 2024-04-26T22:01:14.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.842+0000 7f992f3b4640 1 -- 172.21.15.55:0/205115016 wait complete. 2024-04-26T22:01:14.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.844+0000 7f992f3b4640 1 Processor -- start 2024-04-26T22:01:14.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.844+0000 7f992f3b4640 1 -- start start 2024-04-26T22:01:14.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.844+0000 7f992f3b4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9928109930 0x7f99281a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.844+0000 7f992f3b4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f992810c2a0 0x7f99281a8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.844+0000 7f992f3b4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992810ec70 0x7f99281af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.844+0000 7f992f3b4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f992811d420 con 0x7f9928109930 2024-04-26T22:01:14.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.844+0000 7f992f3b4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f992811d2a0 con 0x7f992810c2a0 2024-04-26T22:01:14.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.844+0000 7f992f3b4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f992811d5a0 con 0x7f992810ec70 2024-04-26T22:01:14.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.848+0000 7f992c94f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f992810c2a0 0x7f99281a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.846+0000 7f992d150640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9928109930 0x7f99281a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.848+0000 7f992c94f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f992810c2a0 0x7f99281a8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46250/0 (socket says 172.21.15.55:46250) 2024-04-26T22:01:14.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.848+0000 7f992c94f640 1 -- 172.21.15.55:0/2139138698 learned_addr learned my addr 172.21.15.55:0/2139138698 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:14.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.848+0000 7f992d951640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992810ec70 0x7f99281af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.852+0000 7f992d150640 1 -- 172.21.15.55:0/2139138698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992810ec70 msgr2=0x7f99281af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.852+0000 7f992d150640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992810ec70 0x7f99281af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.852+0000 7f992d150640 1 -- 172.21.15.55:0/2139138698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f992810c2a0 msgr2=0x7f99281a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.852+0000 7f992d150640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f992810c2a0 0x7f99281a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.852+0000 7f992d150640 1 -- 172.21.15.55:0/2139138698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9928111190 con 0x7f9928109930 2024-04-26T22:01:14.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.867+0000 7f992d150640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9928109930 0x7f99281a7b50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9924009fd0 tx=0x7f992400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.867+0000 7f991e7fc640 1 -- 172.21.15.55:0/2139138698 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924015020 con 0x7f9928109930 2024-04-26T22:01:14.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.867+0000 7f991e7fc640 1 -- 172.21.15.55:0/2139138698 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9924011ce0 con 0x7f9928109930 2024-04-26T22:01:14.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.867+0000 7f991e7fc640 1 -- 172.21.15.55:0/2139138698 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924010de0 con 0x7f9928109930 2024-04-26T22:01:14.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.870+0000 7f992f3b4640 1 -- 172.21.15.55:0/2139138698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9928111480 con 0x7f9928109930 2024-04-26T22:01:14.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.870+0000 7f992f3b4640 1 -- 172.21.15.55:0/2139138698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9928111a00 con 0x7f9928109930 2024-04-26T22:01:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.881+0000 7f99321af740 1 Processor -- start 2024-04-26T22:01:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.881+0000 7f99321af740 1 -- start start 2024-04-26T22:01:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.881+0000 7f99321af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.881+0000 7f99321af740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7d9f0 0x55679ee7de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.881+0000 7f99321af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee84c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.881+0000 7f99321af740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55679ede0d80 con 0x55679ee7d9f0 2024-04-26T22:01:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.881+0000 7f99321af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55679ed9aa20 con 0x55679ee7e3b0 2024-04-26T22:01:14.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.881+0000 7f99321af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55679eddcd40 con 0x55679ee7c820 2024-04-26T22:01:14.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.882+0000 7f992d951640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.882+0000 7f992d951640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45006/0 (socket says 172.21.15.55:45006) 2024-04-26T22:01:14.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.882+0000 7f992d951640 1 -- 172.21.15.55:0/4166105765 learned_addr learned my addr 172.21.15.55:0/4166105765 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.883+0000 7f992d951640 1 -- 172.21.15.55:0/4166105765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 msgr2=0x55679ee84c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:01:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.883+0000 7f992d951640 1 --2- 172.21.15.55:0/4166105765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee84c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.883+0000 7f992d951640 1 -- 172.21.15.55:0/4166105765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7d9f0 msgr2=0x55679ee7de70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:01:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.883+0000 7f992d951640 1 --2- 172.21.15.55:0/4166105765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7d9f0 0x55679ee7de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.883+0000 7f992d951640 1 -- 172.21.15.55:0/4166105765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55679eda5c30 con 0x55679ee7c820 2024-04-26T22:01:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.883+0000 7f991e7fc640 1 -- 172.21.15.55:0/2139138698 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9924018020 con 0x7f9928109930 2024-04-26T22:01:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.885+0000 7f991e7fc640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99140a3cc0 0x7f99140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.885+0000 7f991e7fc640 1 -- 172.21.15.55:0/2139138698 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f99240cfc00 con 0x7f9928109930 2024-04-26T22:01:14.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.887+0000 7f992d951640 1 --2- 172.21.15.55:0/4166105765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7cc40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9920009ca0 tx=0x7f9920005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeb31b60a6cb0bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.893+0000 7f992d150640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99140a3cc0 0x7f99140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.894+0000 7f99137fe640 1 -- 172.21.15.55:0/4166105765 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920031bd0 con 0x55679ee7c820 2024-04-26T22:01:14.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.894+0000 7f99137fe640 1 -- 172.21.15.55:0/4166105765 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9920036040 con 0x55679ee7c820 2024-04-26T22:01:14.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.894+0000 7f99137fe640 1 -- 172.21.15.55:0/4166105765 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99200305e0 con 0x55679ee7c820 2024-04-26T22:01:14.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.896+0000 7f99321af740 1 -- 172.21.15.55:0/4166105765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 msgr2=0x55679ee7cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.896+0000 7f99321af740 1 --2- 172.21.15.55:0/4166105765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7cc40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9920009ca0 tx=0x7f9920005fa0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.897+0000 7f99321af740 1 -- 172.21.15.55:0/4166105765 shutdown_connections 2024-04-26T22:01:14.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.897+0000 7f99321af740 1 --2- 172.21.15.55:0/4166105765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee84c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.897+0000 7f99321af740 1 --2- 172.21.15.55:0/4166105765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7d9f0 0x55679ee7de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.897+0000 7f99321af740 1 --2- 172.21.15.55:0/4166105765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7cc40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.897+0000 7f99321af740 1 -- 172.21.15.55:0/4166105765 >> 172.21.15.55:0/4166105765 conn(0x55679edbd910 msgr2=0x55679ee78120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.899+0000 7f99321af740 1 -- 172.21.15.55:0/4166105765 shutdown_connections 2024-04-26T22:01:14.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.899+0000 7f99321af740 1 -- 172.21.15.55:0/4166105765 wait complete. 2024-04-26T22:01:14.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.900+0000 7f99321af740 1 -- start start 2024-04-26T22:01:14.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.900+0000 7f99321af740 1 Processor -- start 2024-04-26T22:01:14.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.901+0000 7f99321af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7c820 0x55679ee86ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.901+0000 7f99321af740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7d9f0 0x55679ee87010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.901+0000 7f99321af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7e3b0 0x55679ee8e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.901+0000 7f99321af740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55679ee85360 con 0x55679ee7d9f0 2024-04-26T22:01:14.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.901+0000 7f99321af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55679ed67dc0 con 0x55679ee7c820 2024-04-26T22:01:14.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.901+0000 7f99321af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55679ee854e0 con 0x55679ee7e3b0 2024-04-26T22:01:14.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.901+0000 7f99321af740 10 client.0 _ll_register_callbacks cb 0x55679ed83600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:14.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.902+0000 7f992d951640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7e3b0 0x55679ee8e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.902+0000 7f992d951640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7e3b0 0x55679ee8e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45010/0 (socket says 172.21.15.55:45010) 2024-04-26T22:01:14.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.902+0000 7f992d951640 1 -- 172.21.15.55:0/3852900307 learned_addr learned my addr 172.21.15.55:0/3852900307 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:14.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.903+0000 7f992d951640 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7c820 0x55679ee86ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.903+0000 7f992d951640 1 -- 172.21.15.55:0/3852900307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7c820 msgr2=0x55679ee86ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:14.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.903+0000 7f992d951640 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7c820 0x55679ee86ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.903+0000 7f992d951640 1 -- 172.21.15.55:0/3852900307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7d9f0 msgr2=0x55679ee87010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:01:14.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.903+0000 7f992d951640 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7d9f0 0x55679ee87010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.903+0000 7f992d951640 1 -- 172.21.15.55:0/3852900307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55679eaf23f0 con 0x55679ee7e3b0 2024-04-26T22:01:14.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.904+0000 7f992d951640 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7e3b0 0x55679ee8e050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f992002aed0 tx=0x7f99200095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49dc847f318c9c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.904+0000 7f992d150640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99140a3cc0 0x7f99140a6180 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f992409baf0 tx=0x7f9924004750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.913+0000 7f9910ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:01:14.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.913+0000 7f9910ff9640 1 -- 172.21.15.55:0/3852900307 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920056060 con 0x55679ee7e3b0 2024-04-26T22:01:14.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.914+0000 7f9910ff9640 1 -- 172.21.15.55:0/3852900307 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9920031c60 con 0x55679ee7e3b0 2024-04-26T22:01:14.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.917+0000 7f99321af740 1 -- 172.21.15.55:0/3852900307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9920036040 con 0x55679ee7e3b0 2024-04-26T22:01:14.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.921+0000 7f9910ff9640 1 -- 172.21.15.55:0/3852900307 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9920003970 con 0x55679ee7e3b0 2024-04-26T22:01:14.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.922+0000 7f99321af740 10 client.25891 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:14.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.922+0000 7f99321af740 1 -- 172.21.15.55:0/3852900307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55679ebbc7d0 con 0x55679ee7e3b0 2024-04-26T22:01:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.927+0000 7f9910ff9640 1 -- 172.21.15.55:0/3852900307 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9920036040 con 0x55679ee7e3b0 2024-04-26T22:01:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.927+0000 7f9910ff9640 1 -- 172.21.15.55:0/3852900307 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9920059050 con 0x55679ee7e3b0 2024-04-26T22:01:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.928+0000 7f99321af740 10 client.25891 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.928+0000 7f99321af740 10 client.25891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:01:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.928+0000 7f99321af740 20 client.25891 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.928+0000 7f99321af740 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55679ee92540 0x55679ee94930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.928+0000 7f99321af740 4 client.25891 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:14.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.928+0000 7f99321af740 1 -- 172.21.15.55:0/3852900307 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55679edee990 con 0x55679ee92540 2024-04-26T22:01:14.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.930+0000 7f992c94f640 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55679ee92540 0x55679ee94930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:14.940 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:14 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3226971990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:14.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.946+0000 7f992c94f640 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55679ee92540 0x55679ee94930 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3461f88b46c0447 server_cookie=61efb82efa8cf74a in_seq=0 out_seq=0 2024-04-26T22:01:14.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.946+0000 7f9910ff9640 10 client.25891 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:01:14.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.948+0000 7f9910ff9640 1 -- 172.21.15.55:0/3852900307 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55679edee990 con 0x55679ee92540 2024-04-26T22:01:14.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.948+0000 7f9910ff9640 10 client.25891 handle_command_reply: tid=0 2024-04-26T22:01:14.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.949+0000 7f99321af740 1 client.25891 shutdown 2024-04-26T22:01:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.954+0000 7f99321af740 1 -- 172.21.15.55:0/3852900307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7e3b0 msgr2=0x55679ee8e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:14.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.954+0000 7f99321af740 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7e3b0 0x55679ee8e050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f992002aed0 tx=0x7f99200095c0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.958+0000 7f99321af740 1 -- 172.21.15.55:0/3852900307 shutdown_connections 2024-04-26T22:01:14.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.958+0000 7f99321af740 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55679ee92540 0x55679ee94930 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.958+0000 7f99321af740 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7e3b0 0x55679ee8e050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.958+0000 7f99321af740 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7d9f0 0x55679ee87010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.958+0000 7f99321af740 1 --2- 172.21.15.55:0/3852900307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7c820 0x55679ee86ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:14.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.958+0000 7f99321af740 1 -- 172.21.15.55:0/3852900307 >> 172.21.15.55:0/3852900307 conn(0x55679edbd910 msgr2=0x55679ee77280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:14.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.958+0000 7f9910ff9640 0 client.25891 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:01:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.963+0000 7f99321af740 1 -- 172.21.15.55:0/3852900307 shutdown_connections 2024-04-26T22:01:14.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.965+0000 7f99321af740 1 -- 172.21.15.55:0/3852900307 wait complete. 2024-04-26T22:01:14.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:14.968+0000 7f99321af740 20 client.25891 trim_cache size 0 max 16384 2024-04-26T22:01:15.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:14 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3226971990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.009+0000 7f99321af740 1 Processor -- start 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.009+0000 7f99321af740 1 -- start start 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.010+0000 7f99321af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.010+0000 7f99321af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7d9f0 0x55679ee8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.010+0000 7f99321af740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7e3b0 0x55679ee7b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.010+0000 7f99321af740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f990c010f40 con 0x55679ee7e3b0 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.010+0000 7f99321af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f990c00e810 con 0x55679ee7d9f0 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.010+0000 7f99321af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f990c00c660 con 0x55679ee7c820 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.011+0000 7f992d951640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.011+0000 7f992d951640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45022/0 (socket says 172.21.15.55:45022) 2024-04-26T22:01:15.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.011+0000 7f992d951640 1 -- 172.21.15.55:0/1790262955 learned_addr learned my addr 172.21.15.55:0/1790262955 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.011+0000 7f992d951640 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7e3b0 0x55679ee7b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.011+0000 7f992c94f640 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7d9f0 0x55679ee8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.012+0000 7f992d951640 1 -- 172.21.15.55:0/1790262955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 msgr2=0x55679ee7b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.012+0000 7f992d951640 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.012+0000 7f992d951640 1 -- 172.21.15.55:0/1790262955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7d9f0 msgr2=0x55679ee8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.012+0000 7f992d951640 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7d9f0 0x55679ee8dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.012+0000 7f992d951640 1 -- 172.21.15.55:0/1790262955 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55679ee84fb0 con 0x55679ee7e3b0 2024-04-26T22:01:15.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.013+0000 7f992d951640 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7e3b0 0x55679ee7b550 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f992005b600 tx=0x7f992002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa9d9b0117fa73f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.014+0000 7f9912ffd640 1 -- 172.21.15.55:0/1790262955 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920056070 con 0x55679ee7e3b0 2024-04-26T22:01:15.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.015+0000 7f9912ffd640 1 -- 172.21.15.55:0/1790262955 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9920036070 con 0x55679ee7e3b0 2024-04-26T22:01:15.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.016+0000 7f99321af740 1 -- 172.21.15.55:0/1790262955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7e3b0 msgr2=0x55679ee7b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.016+0000 7f99321af740 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7e3b0 0x55679ee7b550 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f992005b600 tx=0x7f992002a8e0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.025+0000 7f99321af740 1 -- 172.21.15.55:0/1790262955 shutdown_connections 2024-04-26T22:01:15.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.025+0000 7f99321af740 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7e3b0 0x55679ee7b550 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.025+0000 7f99321af740 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7d9f0 0x55679ee8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.025+0000 7f99321af740 1 --2- 172.21.15.55:0/1790262955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7c820 0x55679ee7b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.025+0000 7f99321af740 1 -- 172.21.15.55:0/1790262955 >> 172.21.15.55:0/1790262955 conn(0x55679edbd910 msgr2=0x55679ee812b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.027+0000 7f99321af740 1 -- 172.21.15.55:0/1790262955 shutdown_connections 2024-04-26T22:01:15.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.027+0000 7f99321af740 1 -- 172.21.15.55:0/1790262955 wait complete. 2024-04-26T22:01:15.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.027+0000 7f99321af740 1 -- start start 2024-04-26T22:01:15.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.027+0000 7f99321af740 1 Processor -- start 2024-04-26T22:01:15.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.028+0000 7f99321af740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7c820 0x55679ee8b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.028+0000 7f99321af740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7d9f0 0x55679ee8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.028+0000 7f99321af740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee8c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.028+0000 7f99321af740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f990c00ccd0 con 0x55679ee7c820 2024-04-26T22:01:15.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.028+0000 7f99321af740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f990c00e140 con 0x55679ee7e3b0 2024-04-26T22:01:15.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.028+0000 7f99321af740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f990c00d3a0 con 0x55679ee7d9f0 2024-04-26T22:01:15.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.028+0000 7f99321af740 10 client.0 _ll_register_callbacks cb 0x7f9920031470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:15.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.029+0000 7f992d951640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee8c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.029+0000 7f992d951640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee8c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46268/0 (socket says 172.21.15.55:46268) 2024-04-26T22:01:15.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.029+0000 7f992d951640 1 -- 172.21.15.55:0/747609413 learned_addr learned my addr 172.21.15.55:0/747609413 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:15.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.030+0000 7f992d951640 1 -- 172.21.15.55:0/747609413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7d9f0 msgr2=0x55679ee8b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.030+0000 7f992d951640 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7d9f0 0x55679ee8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.030+0000 7f992d951640 1 -- 172.21.15.55:0/747609413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7c820 msgr2=0x55679ee8b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.030+0000 7f992d951640 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7c820 0x55679ee8b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.030+0000 7f992d951640 1 -- 172.21.15.55:0/747609413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55679ee8cbb0 con 0x55679ee7e3b0 2024-04-26T22:01:15.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.032+0000 7f992d951640 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee8c5c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f992002b010 tx=0x7f99200095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97cb1131ca81f5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.033+0000 7f9910ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:01:15.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.033+0000 7f9910ff9640 1 -- 172.21.15.55:0/747609413 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920056070 con 0x55679ee7e3b0 2024-04-26T22:01:15.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.033+0000 7f99321af740 1 -- 172.21.15.55:0/747609413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9920036070 con 0x55679ee7e3b0 2024-04-26T22:01:15.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.033+0000 7f9910ff9640 1 -- 172.21.15.55:0/747609413 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9920004330 con 0x55679ee7e3b0 2024-04-26T22:01:15.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.035+0000 7f9910ff9640 1 -- 172.21.15.55:0/747609413 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f992004a040 con 0x55679ee7e3b0 2024-04-26T22:01:15.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.035+0000 7f99321af740 10 client.25862 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:15.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.035+0000 7f99321af740 1 -- 172.21.15.55:0/747609413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55679ee8deb0 con 0x55679ee7e3b0 2024-04-26T22:01:15.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.039+0000 7f9910ff9640 1 -- 172.21.15.55:0/747609413 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9920036070 con 0x55679ee7e3b0 2024-04-26T22:01:15.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.039+0000 7f9910ff9640 1 -- 172.21.15.55:0/747609413 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9920059040 con 0x55679ee7e3b0 2024-04-26T22:01:15.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.039+0000 7f99321af740 10 client.25862 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:15.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.039+0000 7f99321af740 10 client.25862 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:01:15.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.039+0000 7f99321af740 20 client.25862 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:15.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.039+0000 7f99321af740 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55679eef8c10 0x55679eefb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.039+0000 7f99321af740 4 client.25862 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:01:15.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.039+0000 7f99321af740 1 -- 172.21.15.55:0/747609413 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55679ede8800 con 0x55679eef8c10 2024-04-26T22:01:15.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.044+0000 7f992c94f640 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55679eef8c10 0x55679eefb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.055+0000 7f992c94f640 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55679eef8c10 0x55679eefb030 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e435e5218be9094 server_cookie=9957370ff4658375 in_seq=0 out_seq=0 2024-04-26T22:01:15.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.055+0000 7f9910ff9640 10 client.25862 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:01:15.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.057+0000 7f9910ff9640 1 -- 172.21.15.55:0/747609413 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55679ede8800 con 0x55679eef8c10 2024-04-26T22:01:15.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.057+0000 7f9910ff9640 10 client.25862 handle_command_reply: tid=0 2024-04-26T22:01:15.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.057+0000 7f99321af740 1 client.25862 shutdown 2024-04-26T22:01:15.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.057+0000 7f99321af740 1 -- 172.21.15.55:0/747609413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 msgr2=0x55679ee8c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.057+0000 7f99321af740 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee8c5c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f992002b010 tx=0x7f99200095c0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.058+0000 7f99321af740 1 -- 172.21.15.55:0/747609413 shutdown_connections 2024-04-26T22:01:15.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.058+0000 7f99321af740 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55679eef8c10 0x55679eefb030 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.058+0000 7f99321af740 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55679ee7e3b0 0x55679ee8c5c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.058+0000 7f99321af740 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55679ee7d9f0 0x55679ee8b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.058+0000 7f99321af740 1 --2- 172.21.15.55:0/747609413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55679ee7c820 0x55679ee8b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.058+0000 7f99321af740 1 -- 172.21.15.55:0/747609413 >> 172.21.15.55:0/747609413 conn(0x55679edbd910 msgr2=0x55679ee769f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.058+0000 7f9910ff9640 0 client.25862 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:01:15.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.063+0000 7f99321af740 1 -- 172.21.15.55:0/747609413 shutdown_connections 2024-04-26T22:01:15.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.067+0000 7f99321af740 1 -- 172.21.15.55:0/747609413 wait complete. 2024-04-26T22:01:15.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.068+0000 7f99321af740 20 client.25862 trim_cache size 0 max 16384 2024-04-26T22:01:15.069 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:01:15.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.088+0000 7f992f3b4640 1 -- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99140a3cc0 msgr2=0x7f99140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99140a3cc0 0x7f99140a6180 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f992409baf0 tx=0x7f9924004750 comp rx=0 tx=0).stop 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 -- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9928109930 msgr2=0x7f99281a7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9928109930 0x7f99281a7b50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9924009fd0 tx=0x7f992400b870 comp rx=0 tx=0).stop 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 -- 172.21.15.55:0/2139138698 shutdown_connections 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f99140a3cc0 0x7f99140a6180 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f992810ec70 0x7f99281af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f992810c2a0 0x7f99281a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 --2- 172.21.15.55:0/2139138698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9928109930 0x7f99281a7b50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.089+0000 7f992f3b4640 1 -- 172.21.15.55:0/2139138698 >> 172.21.15.55:0/2139138698 conn(0x7f992806f7c0 msgr2=0x7f9928109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.090+0000 7f992f3b4640 1 -- 172.21.15.55:0/2139138698 shutdown_connections 2024-04-26T22:01:15.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.090+0000 7f992f3b4640 1 -- 172.21.15.55:0/2139138698 wait complete. 2024-04-26T22:01:15.140 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:01:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.509+0000 7f390f59e640 1 -- 172.21.15.55:0/467882197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 msgr2=0x7f391010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.509+0000 7f390f59e640 1 --2- 172.21.15.55:0/467882197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 0x7f391010a470 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3900009900 tx=0x7f390002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.510+0000 7f390f59e640 1 -- 172.21.15.55:0/467882197 shutdown_connections 2024-04-26T22:01:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.510+0000 7f390f59e640 1 --2- 172.21.15.55:0/467882197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f391010a9b0 0x7f391010f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.510+0000 7f390f59e640 1 --2- 172.21.15.55:0/467882197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 0x7f391010a470 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.510+0000 7f390f59e640 1 --2- 172.21.15.55:0/467882197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3910073d70 0x7f3910074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.510+0000 7f390f59e640 1 -- 172.21.15.55:0/467882197 >> 172.21.15.55:0/467882197 conn(0x7f391006f840 msgr2=0x7f3910071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.513+0000 7f390f59e640 1 -- 172.21.15.55:0/467882197 shutdown_connections 2024-04-26T22:01:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.513+0000 7f390f59e640 1 -- 172.21.15.55:0/467882197 wait complete. 2024-04-26T22:01:15.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.515+0000 7f390f59e640 1 Processor -- start 2024-04-26T22:01:15.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.515+0000 7f390f59e640 1 -- start start 2024-04-26T22:01:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.516+0000 7f390f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3910073d70 0x7f39101a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.516+0000 7f390f59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 0x7f39101a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.516+0000 7f390f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391010a9b0 0x7f39101aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.516+0000 7f390f59e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f391011d1c0 con 0x7f391010a010 2024-04-26T22:01:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.516+0000 7f390f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f391011d040 con 0x7f391010a9b0 2024-04-26T22:01:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.516+0000 7f390f59e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f391011d340 con 0x7f3910073d70 2024-04-26T22:01:15.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.518+0000 7f390e59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3910073d70 0x7f39101a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.518+0000 7f390e59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3910073d70 0x7f39101a7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45060/0 (socket says 172.21.15.55:45060) 2024-04-26T22:01:15.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.518+0000 7f390e59c640 1 -- 172.21.15.55:0/1411866975 learned_addr learned my addr 172.21.15.55:0/1411866975 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:15.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.519+0000 7f390dd9b640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 0x7f39101a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.519+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391010a9b0 0x7f39101aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.519+0000 7f390dd9b640 1 -- 172.21.15.55:0/1411866975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3910073d70 msgr2=0x7f39101a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.519+0000 7f390dd9b640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3910073d70 0x7f39101a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.519+0000 7f390dd9b640 1 -- 172.21.15.55:0/1411866975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391010a9b0 msgr2=0x7f39101aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.519+0000 7f390dd9b640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391010a9b0 0x7f39101aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.519+0000 7f390dd9b640 1 -- 172.21.15.55:0/1411866975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39101af470 con 0x7f391010a010 2024-04-26T22:01:15.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.520+0000 7f390e59c640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3910073d70 0x7f39101a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:15.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.521+0000 7f390dd9b640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 0x7f39101a7d90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3900009850 tx=0x7f3900003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.522+0000 7f38ff7fe640 1 -- 172.21.15.55:0/1411866975 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390003f070 con 0x7f391010a010 2024-04-26T22:01:15.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.522+0000 7f390f59e640 1 -- 172.21.15.55:0/1411866975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39101af700 con 0x7f391010a010 2024-04-26T22:01:15.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.522+0000 7f390f59e640 1 -- 172.21.15.55:0/1411866975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39101afbf0 con 0x7f391010a010 2024-04-26T22:01:15.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.522+0000 7f38ff7fe640 1 -- 172.21.15.55:0/1411866975 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3900031c50 con 0x7f391010a010 2024-04-26T22:01:15.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.524+0000 7f38ff7fe640 1 -- 172.21.15.55:0/1411866975 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900035030 con 0x7f391010a010 2024-04-26T22:01:15.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.527+0000 7f3917c7d740 1 Processor -- start 2024-04-26T22:01:15.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.528+0000 7f3917c7d740 1 -- start start 2024-04-26T22:01:15.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.529+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x56417584ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.529+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x56417584c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.529+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x564175852e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.529+0000 7f3917c7d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5641757aed80 con 0x56417584bbd0 2024-04-26T22:01:15.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.529+0000 7f3917c7d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564175768a20 con 0x56417584c590 2024-04-26T22:01:15.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.529+0000 7f3917c7d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641757aad40 con 0x56417584aa00 2024-04-26T22:01:15.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.529+0000 7f38ff7fe640 1 -- 172.21.15.55:0/1411866975 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3900042070 con 0x7f391010a010 2024-04-26T22:01:15.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.531+0000 7f390dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x564175852e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.531+0000 7f38ff7fe640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38f80a3dc0 0x7f38f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.531+0000 7f390dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x564175852e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46302/0 (socket says 172.21.15.55:46302) 2024-04-26T22:01:15.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.531+0000 7f390dd9b640 1 -- 172.21.15.55:0/4138883191 learned_addr learned my addr 172.21.15.55:0/4138883191 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:15.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.531+0000 7f38ff7fe640 1 -- 172.21.15.55:0/1411866975 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f39000f0d40 con 0x7f391010a010 2024-04-26T22:01:15.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.531+0000 7f390e59c640 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x56417584c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.531+0000 7f390ed9d640 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x56417584ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.532+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38f80a3dc0 0x7f38f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.532+0000 7f390dd9b640 1 -- 172.21.15.55:0/4138883191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 msgr2=0x56417584ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.532+0000 7f390dd9b640 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x56417584ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.532+0000 7f390dd9b640 1 -- 172.21.15.55:0/4138883191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 msgr2=0x56417584c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.532+0000 7f390dd9b640 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x56417584c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.532+0000 7f390dd9b640 1 -- 172.21.15.55:0/4138883191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564175773c30 con 0x56417584c590 2024-04-26T22:01:15.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.533+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38f80a3dc0 0x7f38f80a6280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56417584bad0 tx=0x7f3908004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.533+0000 7f390dd9b640 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x564175852e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39000ee680 tx=0x7f390002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cefa436d64dbeaf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.535+0000 7f38fcff9640 1 -- 172.21.15.55:0/4138883191 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900049030 con 0x56417584c590 2024-04-26T22:01:15.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.535+0000 7f38fcff9640 1 -- 172.21.15.55:0/4138883191 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39000c11d0 con 0x56417584c590 2024-04-26T22:01:15.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.535+0000 7f38fcff9640 1 -- 172.21.15.55:0/4138883191 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39000c5600 con 0x56417584c590 2024-04-26T22:01:15.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.538+0000 7f3917c7d740 1 -- 172.21.15.55:0/4138883191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 msgr2=0x564175852e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.538+0000 7f3917c7d740 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x564175852e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39000ee680 tx=0x7f390002a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.538+0000 7f3917c7d740 1 -- 172.21.15.55:0/4138883191 shutdown_connections 2024-04-26T22:01:15.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.538+0000 7f3917c7d740 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x564175852e20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.538+0000 7f3917c7d740 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x56417584c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.538+0000 7f3917c7d740 1 --2- 172.21.15.55:0/4138883191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x56417584ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.538+0000 7f3917c7d740 1 -- 172.21.15.55:0/4138883191 >> 172.21.15.55:0/4138883191 conn(0x56417578b910 msgr2=0x5641758462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.540+0000 7f3917c7d740 1 -- 172.21.15.55:0/4138883191 shutdown_connections 2024-04-26T22:01:15.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.541+0000 7f3917c7d740 1 -- 172.21.15.55:0/4138883191 wait complete. 2024-04-26T22:01:15.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.542+0000 7f3917c7d740 1 -- start start 2024-04-26T22:01:15.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.542+0000 7f3917c7d740 1 Processor -- start 2024-04-26T22:01:15.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.542+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175854a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.543+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584bbd0 0x564175854fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.543+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584c590 0x56417585bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.543+0000 7f3917c7d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564175853360 con 0x56417584c590 2024-04-26T22:01:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.543+0000 7f3917c7d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564175735dc0 con 0x56417584bbd0 2024-04-26T22:01:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.543+0000 7f3917c7d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641758534e0 con 0x56417584aa00 2024-04-26T22:01:15.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.543+0000 7f3917c7d740 10 client.0 _ll_register_callbacks cb 0x564175751600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:15.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.545+0000 7f390e59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175854a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.545+0000 7f390e59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175854a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45088/0 (socket says 172.21.15.55:45088) 2024-04-26T22:01:15.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.545+0000 7f390e59c640 1 -- 172.21.15.55:0/3809474918 learned_addr learned my addr 172.21.15.55:0/3809474918 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:15.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.545+0000 7f390ed9d640 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584c590 0x56417585bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.546+0000 7f390dd9b640 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584bbd0 0x564175854fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.548+0000 7f390dd9b640 1 -- 172.21.15.55:0/3809474918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 msgr2=0x564175854a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.548+0000 7f390dd9b640 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175854a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.548+0000 7f390dd9b640 1 -- 172.21.15.55:0/3809474918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584c590 msgr2=0x56417585bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.548+0000 7f390dd9b640 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584c590 0x56417585bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.548+0000 7f390dd9b640 1 -- 172.21.15.55:0/3809474918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641754c03f0 con 0x56417584bbd0 2024-04-26T22:01:15.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.551+0000 7f390dd9b640 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584bbd0 0x564175854fa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39000b97c0 tx=0x7f39000bd020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f506b599432c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.551+0000 7f38f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:01:15.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.553+0000 7f38f5ffb640 1 -- 172.21.15.55:0/3809474918 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39000d5070 con 0x56417584bbd0 2024-04-26T22:01:15.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.553+0000 7f38f5ffb640 1 -- 172.21.15.55:0/3809474918 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f39000d6260 con 0x56417584bbd0 2024-04-26T22:01:15.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.554+0000 7f3917c7d740 1 -- 172.21.15.55:0/3809474918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39000c11d0 con 0x56417584bbd0 2024-04-26T22:01:15.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.557+0000 7f38f5ffb640 1 -- 172.21.15.55:0/3809474918 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3900049050 con 0x56417584bbd0 2024-04-26T22:01:15.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.557+0000 7f3917c7d740 10 client.25874 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:15.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.557+0000 7f3917c7d740 1 -- 172.21.15.55:0/3809474918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56417558a7d0 con 0x56417584bbd0 2024-04-26T22:01:15.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.558+0000 7f38f5ffb640 1 -- 172.21.15.55:0/3809474918 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39000c11d0 con 0x56417584bbd0 2024-04-26T22:01:15.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.558+0000 7f38f5ffb640 1 -- 172.21.15.55:0/3809474918 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f39000c5ce0 con 0x56417584bbd0 2024-04-26T22:01:15.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.561+0000 7f3917c7d740 10 client.25874 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:15.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.561+0000 7f3917c7d740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:15.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.561+0000 7f3917c7d740 20 client.25874 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:15.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.561+0000 7f3917c7d740 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641758604d0 0x564175862900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.561+0000 7f3917c7d740 4 client.25874 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:15.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.561+0000 7f3917c7d740 1 -- 172.21.15.55:0/3809474918 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641757bc990 con 0x5641758604d0 2024-04-26T22:01:15.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.562+0000 7f390e59c640 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641758604d0 0x564175862900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.565+0000 7f390e59c640 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641758604d0 0x564175862900 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad4e6bcf3a03677 server_cookie=f501c2d0b93710de in_seq=0 out_seq=0 2024-04-26T22:01:15.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.566+0000 7f38f5ffb640 10 client.25874 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.568+0000 7f38f5ffb640 1 -- 172.21.15.55:0/3809474918 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641757bc990 con 0x5641758604d0 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.568+0000 7f38f5ffb640 10 client.25874 handle_command_reply: tid=0 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.568+0000 7f3917c7d740 1 client.25874 shutdown 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.569+0000 7f3917c7d740 1 -- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584bbd0 msgr2=0x564175854fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.569+0000 7f3917c7d740 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584bbd0 0x564175854fa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39000b97c0 tx=0x7f39000bd020 comp rx=0 tx=0).stop 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.569+0000 7f3917c7d740 1 -- 172.21.15.55:0/3809474918 shutdown_connections 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.569+0000 7f3917c7d740 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641758604d0 0x564175862900 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.569+0000 7f3917c7d740 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584c590 0x56417585bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.569+0000 7f3917c7d740 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584bbd0 0x564175854fa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.569+0000 7f3917c7d740 1 --2- 172.21.15.55:0/3809474918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175854a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.569+0000 7f3917c7d740 1 -- 172.21.15.55:0/3809474918 >> 172.21.15.55:0/3809474918 conn(0x56417578b910 msgr2=0x564175845450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.570+0000 7f38f5ffb640 0 client.25874 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:15.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.570+0000 7f3917c7d740 1 -- 172.21.15.55:0/3809474918 shutdown_connections 2024-04-26T22:01:15.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.570+0000 7f3917c7d740 1 -- 172.21.15.55:0/3809474918 wait complete. 2024-04-26T22:01:15.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.571+0000 7f3917c7d740 20 client.25874 trim_cache size 0 max 16384 2024-04-26T22:01:15.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.674+0000 7f3917c7d740 1 Processor -- start 2024-04-26T22:01:15.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.678+0000 7f3917c7d740 1 -- start start 2024-04-26T22:01:15.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.679+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x5641758624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.679+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x56417585bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.679+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417584f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.679+0000 7f3917c7d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f38f0010f80 con 0x56417584bbd0 2024-04-26T22:01:15.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.679+0000 7f3917c7d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000e850 con 0x56417584c590 2024-04-26T22:01:15.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.679+0000 7f3917c7d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000c6a0 con 0x56417584aa00 2024-04-26T22:01:15.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.680+0000 7f390ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417584f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.680+0000 7f390ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417584f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46312/0 (socket says 172.21.15.55:46312) 2024-04-26T22:01:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.680+0000 7f390ed9d640 1 -- 172.21.15.55:0/1840836846 learned_addr learned my addr 172.21.15.55:0/1840836846 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:15.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.683+0000 7f390e59c640 1 --2- 172.21.15.55:0/1840836846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x5641758624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.684+0000 7f390ed9d640 1 -- 172.21.15.55:0/1840836846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 msgr2=0x5641758624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.684+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1840836846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x5641758624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.684+0000 7f390ed9d640 1 -- 172.21.15.55:0/1840836846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 msgr2=0x56417585bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.684+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1840836846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x56417585bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.684+0000 7f390ed9d640 1 -- 172.21.15.55:0/1840836846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417584b650 con 0x56417584c590 2024-04-26T22:01:15.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.690+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1840836846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417584f4d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39080118d0 tx=0x7f390800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e992cdcba3d7684 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.694+0000 7f38f7fff640 1 -- 172.21.15.55:0/1840836846 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908004d20 con 0x56417584c590 2024-04-26T22:01:15.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.694+0000 7f38f7fff640 1 -- 172.21.15.55:0/1840836846 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390800fe80 con 0x56417584c590 2024-04-26T22:01:15.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.694+0000 7f38f7fff640 1 -- 172.21.15.55:0/1840836846 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908005900 con 0x56417584c590 2024-04-26T22:01:15.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.696+0000 7f3917c7d740 1 -- 172.21.15.55:0/1840836846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 msgr2=0x56417584f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.696+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1840836846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417584f4d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39080118d0 tx=0x7f390800f000 comp rx=0 tx=0).stop 2024-04-26T22:01:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.701+0000 7f3917c7d740 1 -- 172.21.15.55:0/1840836846 shutdown_connections 2024-04-26T22:01:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.701+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1840836846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417584f4d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.701+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1840836846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x56417585bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.701+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1840836846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x5641758624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.701+0000 7f3917c7d740 1 -- 172.21.15.55:0/1840836846 >> 172.21.15.55:0/1840836846 conn(0x56417578b910 msgr2=0x564175854a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.702+0000 7f3917c7d740 1 -- 172.21.15.55:0/1840836846 shutdown_connections 2024-04-26T22:01:15.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.702+0000 7f3917c7d740 1 -- 172.21.15.55:0/1840836846 wait complete. 2024-04-26T22:01:15.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.703+0000 7f3917c7d740 1 -- start start 2024-04-26T22:01:15.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.703+0000 7f3917c7d740 1 Processor -- start 2024-04-26T22:01:15.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.705+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175859600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.705+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x564175859b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.705+0000 7f3917c7d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417585ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.705+0000 7f3917c7d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000cd10 con 0x56417584bbd0 2024-04-26T22:01:15.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.705+0000 7f3917c7d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000e180 con 0x56417584c590 2024-04-26T22:01:15.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.705+0000 7f3917c7d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f38f000d3e0 con 0x56417584aa00 2024-04-26T22:01:15.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.706+0000 7f3917c7d740 10 client.0 _ll_register_callbacks cb 0x7f39000bca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.707+0000 7f390ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417585ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.707+0000 7f390ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417585ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46324/0 (socket says 172.21.15.55:46324) 2024-04-26T22:01:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.707+0000 7f390ed9d640 1 -- 172.21.15.55:0/1185076517 learned_addr learned my addr 172.21.15.55:0/1185076517 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:15.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.708+0000 7f390dd9b640 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x564175859b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.708+0000 7f390e59c640 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175859600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.709+0000 7f390ed9d640 1 -- 172.21.15.55:0/1185076517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 msgr2=0x564175859600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.709+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175859600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.710+0000 7f390ed9d640 1 -- 172.21.15.55:0/1185076517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 msgr2=0x564175859b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:15.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.710+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x564175859b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.710+0000 7f390ed9d640 1 -- 172.21.15.55:0/1185076517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417585b140 con 0x56417584c590 2024-04-26T22:01:15.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.716+0000 7f390ed9d640 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417585ab50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f390800a2a0 tx=0x7f390800a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dd186e4d6999407 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:15.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.716+0000 7f38f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:01:15.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.721+0000 7f38f5ffb640 1 -- 172.21.15.55:0/1185076517 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908037070 con 0x56417584c590 2024-04-26T22:01:15.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.721+0000 7f38f5ffb640 1 -- 172.21.15.55:0/1185076517 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3908003970 con 0x56417584c590 2024-04-26T22:01:15.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.721+0000 7f3917c7d740 1 -- 172.21.15.55:0/1185076517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641757b6800 con 0x56417584c590 2024-04-26T22:01:15.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.725+0000 7f38f5ffb640 1 -- 172.21.15.55:0/1185076517 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3908040050 con 0x56417584c590 2024-04-26T22:01:15.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.725+0000 7f3917c7d740 10 client.25880 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:15.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.725+0000 7f3917c7d740 1 -- 172.21.15.55:0/1185076517 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5641758c6a80 con 0x56417584c590 2024-04-26T22:01:15.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.730+0000 7f38f5ffb640 1 -- 172.21.15.55:0/1185076517 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641757b6800 con 0x56417584c590 2024-04-26T22:01:15.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.730+0000 7f38f5ffb640 1 -- 172.21.15.55:0/1185076517 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f39080285b0 con 0x56417584c590 2024-04-26T22:01:15.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.730+0000 7f3917c7d740 10 client.25880 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:15.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.730+0000 7f3917c7d740 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:15.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.730+0000 7f3917c7d740 20 client.25880 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:15.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.730+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641758c7380 0x5641758c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:15.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.730+0000 7f3917c7d740 4 client.25880 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:01:15.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.730+0000 7f3917c7d740 1 -- 172.21.15.55:0/1185076517 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564175731610 con 0x5641758c7380 2024-04-26T22:01:15.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.734+0000 7f390e59c640 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641758c7380 0x5641758c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:15.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.737+0000 7f390e59c640 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641758c7380 0x5641758c97a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b043348e5754deaf server_cookie=74c2dfda78e344ac in_seq=0 out_seq=0 2024-04-26T22:01:15.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.738+0000 7f38f5ffb640 10 client.25880 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.743+0000 7f38f5ffb640 1 -- 172.21.15.55:0/1185076517 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564175731610 con 0x5641758c7380 2024-04-26T22:01:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.743+0000 7f38f5ffb640 10 client.25880 handle_command_reply: tid=0 2024-04-26T22:01:15.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.743+0000 7f3917c7d740 1 client.25880 shutdown 2024-04-26T22:01:15.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.748+0000 7f3917c7d740 1 -- 172.21.15.55:0/1185076517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 msgr2=0x56417585ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.748+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417585ab50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f390800a2a0 tx=0x7f390800a4d0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.752+0000 7f3917c7d740 1 -- 172.21.15.55:0/1185076517 shutdown_connections 2024-04-26T22:01:15.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.752+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56417584c590 0x56417585ab50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.752+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641758c7380 0x5641758c97a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.752+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56417584bbd0 0x564175859b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.752+0000 7f3917c7d740 1 --2- 172.21.15.55:0/1185076517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56417584aa00 0x564175859600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.752+0000 7f3917c7d740 1 -- 172.21.15.55:0/1185076517 >> 172.21.15.55:0/1185076517 conn(0x56417578b910 msgr2=0x5641758549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.754+0000 7f38f5ffb640 0 client.25880 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:15.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.754+0000 7f3917c7d740 1 -- 172.21.15.55:0/1185076517 shutdown_connections 2024-04-26T22:01:15.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.756+0000 7f3917c7d740 1 -- 172.21.15.55:0/1185076517 wait complete. 2024-04-26T22:01:15.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.756+0000 7f3917c7d740 20 client.25880 trim_cache size 0 max 16384 2024-04-26T22:01:15.759 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:01:15.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.763+0000 7f390f59e640 1 -- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38f80a3dc0 msgr2=0x7f38f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.763+0000 7f390f59e640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38f80a3dc0 0x7f38f80a6280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x56417584bad0 tx=0x7f3908004ab0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.763+0000 7f390f59e640 1 -- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 msgr2=0x7f39101a7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:15.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.763+0000 7f390f59e640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 0x7f39101a7d90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3900009850 tx=0x7f3900003c00 comp rx=0 tx=0).stop 2024-04-26T22:01:15.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.766+0000 7f390f59e640 1 -- 172.21.15.55:0/1411866975 shutdown_connections 2024-04-26T22:01:15.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.766+0000 7f390f59e640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f38f80a3dc0 0x7f38f80a6280 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.766+0000 7f390f59e640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f391010a9b0 0x7f39101aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.766+0000 7f390f59e640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f391010a010 0x7f39101a7d90 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.766+0000 7f390f59e640 1 --2- 172.21.15.55:0/1411866975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3910073d70 0x7f39101a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:15.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.766+0000 7f390f59e640 1 -- 172.21.15.55:0/1411866975 >> 172.21.15.55:0/1411866975 conn(0x7f391006f840 msgr2=0x7f3910071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:15.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.767+0000 7f390f59e640 1 -- 172.21.15.55:0/1411866975 shutdown_connections 2024-04-26T22:01:15.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:15.767+0000 7f390f59e640 1 -- 172.21.15.55:0/1411866975 wait complete. 2024-04-26T22:01:15.823 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:15 smithi055 ceph-mon[29221]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 839 MiB/s wr, 2.05k op/s 2024-04-26T22:01:16.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:15 smithi064 ceph-mon[32765]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 839 MiB/s wr, 2.05k op/s 2024-04-26T22:01:16.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:15 smithi165 ceph-mon[33296]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 839 MiB/s wr, 2.05k op/s 2024-04-26T22:01:16.823 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:01:16.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:01:16.824 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:01:16.929 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:16 smithi055 ceph-mon[29221]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 811 MiB/s wr, 1.91k op/s 2024-04-26T22:01:17.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:16 smithi064 ceph-mon[32765]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 811 MiB/s wr, 1.91k op/s 2024-04-26T22:01:17.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:16 smithi165 ceph-mon[33296]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 811 MiB/s wr, 1.91k op/s 2024-04-26T22:01:17.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.147+0000 7f5693322640 1 -- 172.21.15.55:0/3000876083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f568c074540 msgr2=0x7f568c109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.147+0000 7f5693322640 1 --2- 172.21.15.55:0/3000876083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f568c074540 0x7f568c109f80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5688009900 tx=0x7f568802a410 comp rx=0 tx=0).stop 2024-04-26T22:01:17.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.148+0000 7f5693322640 1 -- 172.21.15.55:0/3000876083 shutdown_connections 2024-04-26T22:01:17.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.148+0000 7f5693322640 1 --2- 172.21.15.55:0/3000876083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c10a640 0x7f568c10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.148+0000 7f5693322640 1 --2- 172.21.15.55:0/3000876083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f568c074540 0x7f568c109f80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.148+0000 7f5693322640 1 --2- 172.21.15.55:0/3000876083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f568c073be0 0x7f568c074000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.148+0000 7f5693322640 1 -- 172.21.15.55:0/3000876083 >> 172.21.15.55:0/3000876083 conn(0x7f568c06f780 msgr2=0x7f568c071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.151+0000 7f5693322640 1 -- 172.21.15.55:0/3000876083 shutdown_connections 2024-04-26T22:01:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.152+0000 7f5693322640 1 -- 172.21.15.55:0/3000876083 wait complete. 2024-04-26T22:01:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.154+0000 7f5693322640 1 Processor -- start 2024-04-26T22:01:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.154+0000 7f5693322640 1 -- start start 2024-04-26T22:01:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.155+0000 7f5693322640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c073be0 0x7f568c119290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.155+0000 7f5693322640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f568c074540 0x7f568c1197d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.155+0000 7f5693322640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f568c10a640 0x7f568c11a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.155+0000 7f5693322640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f568c111920 con 0x7f568c074540 2024-04-26T22:01:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.155+0000 7f5693322640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1117a0 con 0x7f568c073be0 2024-04-26T22:01:17.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.155+0000 7f5693322640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f568c111aa0 con 0x7f568c10a640 2024-04-26T22:01:17.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.157+0000 7f56910be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c073be0 0x7f568c119290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.157+0000 7f56910be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c073be0 0x7f568c119290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46354/0 (socket says 172.21.15.55:46354) 2024-04-26T22:01:17.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.157+0000 7f56910be640 1 -- 172.21.15.55:0/3259325486 learned_addr learned my addr 172.21.15.55:0/3259325486 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:17.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.159+0000 7f56918bf640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f568c10a640 0x7f568c11a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.160+0000 7f56908bd640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f568c074540 0x7f568c1197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.161+0000 7f56910be640 1 -- 172.21.15.55:0/3259325486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f568c10a640 msgr2=0x7f568c11a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.161+0000 7f56910be640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f568c10a640 0x7f568c11a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.161+0000 7f56910be640 1 -- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f568c074540 msgr2=0x7f568c1197d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.161+0000 7f56910be640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f568c074540 0x7f568c1197d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.166 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.161+0000 7f56910be640 1 -- 172.21.15.55:0/3259325486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c11aec0 con 0x7f568c073be0 2024-04-26T22:01:17.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.168+0000 7f56910be640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c073be0 0x7f568c119290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f568400eab0 tx=0x7f568400ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.174+0000 7f56827fc640 1 -- 172.21.15.55:0/3259325486 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568400ce40 con 0x7f568c073be0 2024-04-26T22:01:17.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.175+0000 7f5693322640 1 -- 172.21.15.55:0/3259325486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c11b1b0 con 0x7f568c073be0 2024-04-26T22:01:17.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.175+0000 7f5693322640 1 -- 172.21.15.55:0/3259325486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c07ab90 con 0x7f568c073be0 2024-04-26T22:01:17.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.175+0000 7f56827fc640 1 -- 172.21.15.55:0/3259325486 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5684011920 con 0x7f568c073be0 2024-04-26T22:01:17.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.175+0000 7f56827fc640 1 -- 172.21.15.55:0/3259325486 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568401aa60 con 0x7f568c073be0 2024-04-26T22:01:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.190+0000 7f56827fc640 1 -- 172.21.15.55:0/3259325486 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f568401ac00 con 0x7f568c073be0 2024-04-26T22:01:17.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.192+0000 7f56827fc640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f56780a3dc0 0x7f56780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.192+0000 7f56827fc640 1 -- 172.21.15.55:0/3259325486 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f56840d0ef0 con 0x7f568c073be0 2024-04-26T22:01:17.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.192+0000 7f56908bd640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f56780a3dc0 0x7f56780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.193+0000 7f56908bd640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f56780a3dc0 0x7f56780a6280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5688006010 tx=0x7f56880314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.202+0000 7f5696035740 1 Processor -- start 2024-04-26T22:01:17.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.210+0000 7f5696035740 1 -- start start 2024-04-26T22:01:17.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.210+0000 7f5696035740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293697fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.211+0000 7f5696035740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x562936984f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.211+0000 7f5696035740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x562936987a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.211+0000 7f5696035740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5629368e4b00 con 0x56293697fa80 2024-04-26T22:01:17.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.211+0000 7f5696035740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562936880920 con 0x5629369855e0 2024-04-26T22:01:17.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.211+0000 7f5696035740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5629368e0ac0 con 0x562936980470 2024-04-26T22:01:17.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.211+0000 7f56908bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x562936984f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.211+0000 7f56908bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x562936984f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45138/0 (socket says 172.21.15.55:45138) 2024-04-26T22:01:17.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.211+0000 7f56908bd640 1 -- 172.21.15.55:0/440497948 learned_addr learned my addr 172.21.15.55:0/440497948 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:17.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.213+0000 7f56910be640 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293697fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.213+0000 7f56918bf640 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x562936987a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.214+0000 7f56908bd640 1 -- 172.21.15.55:0/440497948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 msgr2=0x562936987a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:17.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.214+0000 7f56908bd640 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x562936987a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.214+0000 7f56908bd640 1 -- 172.21.15.55:0/440497948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 msgr2=0x56293697fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:17.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.214+0000 7f56908bd640 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293697fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.214+0000 7f56908bd640 1 -- 172.21.15.55:0/440497948 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629366be7e0 con 0x562936980470 2024-04-26T22:01:17.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.215+0000 7f56910be640 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293697fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:01:17.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.215+0000 7f56918bf640 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x562936987a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:01:17.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.215+0000 7f56908bd640 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x562936984f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5688000c00 tx=0x7f568803c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e91e78cdfab6cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.216+0000 7f56777fe640 1 -- 172.21.15.55:0/440497948 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688035050 con 0x562936980470 2024-04-26T22:01:17.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.216+0000 7f56777fe640 1 -- 172.21.15.55:0/440497948 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f568804d070 con 0x562936980470 2024-04-26T22:01:17.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.216+0000 7f56777fe640 1 -- 172.21.15.55:0/440497948 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880039b0 con 0x562936980470 2024-04-26T22:01:17.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.217+0000 7f5696035740 1 -- 172.21.15.55:0/440497948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 msgr2=0x562936984f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.217+0000 7f5696035740 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x562936984f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5688000c00 tx=0x7f568803c000 comp rx=0 tx=0).stop 2024-04-26T22:01:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.218+0000 7f5696035740 1 -- 172.21.15.55:0/440497948 shutdown_connections 2024-04-26T22:01:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.218+0000 7f5696035740 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x562936987a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.218+0000 7f5696035740 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x562936984f70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.218+0000 7f5696035740 1 --2- 172.21.15.55:0/440497948 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293697fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.218+0000 7f5696035740 1 -- 172.21.15.55:0/440497948 >> 172.21.15.55:0/440497948 conn(0x56293697b0f0 msgr2=0x56293697b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.218+0000 7f5696035740 1 -- 172.21.15.55:0/440497948 shutdown_connections 2024-04-26T22:01:17.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.218+0000 7f5696035740 1 -- 172.21.15.55:0/440497948 wait complete. 2024-04-26T22:01:17.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.219+0000 7f5696035740 1 -- start start 2024-04-26T22:01:17.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.219+0000 7f5696035740 1 Processor -- start 2024-04-26T22:01:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f5696035740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56293697fa80 0x562936989ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f5696035740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f5696035740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 0x562936991420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f5696035740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562936988580 con 0x5629369855e0 2024-04-26T22:01:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f5696035740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56293687e5d0 con 0x56293697fa80 2024-04-26T22:01:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f5696035740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562936988700 con 0x562936980470 2024-04-26T22:01:17.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f56908bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f56908bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45144/0 (socket says 172.21.15.55:45144) 2024-04-26T22:01:17.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.221+0000 7f56908bd640 1 -- 172.21.15.55:0/1103843100 learned_addr learned my addr 172.21.15.55:0/1103843100 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:17.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.222+0000 7f56908bd640 1 -- 172.21.15.55:0/1103843100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56293697fa80 msgr2=0x562936989ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:01:17.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.222+0000 7f56908bd640 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56293697fa80 0x562936989ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.222+0000 7f56908bd640 1 -- 172.21.15.55:0/1103843100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 msgr2=0x562936991420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:01:17.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.222+0000 7f56908bd640 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 0x562936991420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.222+0000 7f56908bd640 1 -- 172.21.15.55:0/1103843100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56293671ef70 con 0x562936980470 2024-04-26T22:01:17.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.223+0000 7f5696035740 10 client.0 _ll_register_callbacks cb 0x56293686c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:17.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.223+0000 7f56908bd640 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698a3e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f568802a450 tx=0x7f5688005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3bacd7b2b28afd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.223+0000 7f5696035740 1 -- 172.21.15.55:0/1103843100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568804d070 con 0x562936980470 2024-04-26T22:01:17.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.223+0000 7f5674ff9640 10 client.25918 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:01:17.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.223+0000 7f5674ff9640 1 -- 172.21.15.55:0/1103843100 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688035070 con 0x562936980470 2024-04-26T22:01:17.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.223+0000 7f5674ff9640 1 -- 172.21.15.55:0/1103843100 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5688004980 con 0x562936980470 2024-04-26T22:01:17.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.224+0000 7f5674ff9640 1 -- 172.21.15.55:0/1103843100 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5688004ba0 con 0x562936980470 2024-04-26T22:01:17.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.224+0000 7f5696035740 10 client.25918 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:17.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.224+0000 7f5696035740 1 -- 172.21.15.55:0/1103843100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5629365b6170 con 0x562936980470 2024-04-26T22:01:17.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.226+0000 7f5674ff9640 1 -- 172.21.15.55:0/1103843100 <== mon.1 v2:172.21.15.165:3300/0 4 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f56880036f0 con 0x562936980470 2024-04-26T22:01:17.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.226+0000 7f5696035740 10 client.25918 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:17.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.226+0000 7f5696035740 10 client.25918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:17.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.226+0000 7f5696035740 20 client.25918 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:17.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.226+0000 7f5696035740 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5629369951b0 0x562936997640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.226+0000 7f5696035740 4 client.25918 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:17.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.226+0000 7f5696035740 1 -- 172.21.15.55:0/1103843100 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629368f2710 con 0x5629369951b0 2024-04-26T22:01:17.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.228+0000 7f56910be640 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5629369951b0 0x562936997640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.235+0000 7f56910be640 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5629369951b0 0x562936997640 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a78b82fcd5170e server_cookie=66ccf28017574ab1 in_seq=0 out_seq=0 2024-04-26T22:01:17.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.236+0000 7f5674ff9640 10 client.25918 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:17.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.239+0000 7f5674ff9640 1 -- 172.21.15.55:0/1103843100 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629368f2710 con 0x5629369951b0 2024-04-26T22:01:17.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.239+0000 7f5674ff9640 10 client.25918 handle_command_reply: tid=0 2024-04-26T22:01:17.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.239+0000 7f5696035740 1 client.25918 shutdown 2024-04-26T22:01:17.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.243+0000 7f5696035740 1 -- 172.21.15.55:0/1103843100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 msgr2=0x56293698a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.243+0000 7f5696035740 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698a3e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f568802a450 tx=0x7f5688005640 comp rx=0 tx=0).stop 2024-04-26T22:01:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5696035740 1 -- 172.21.15.55:0/1103843100 shutdown_connections 2024-04-26T22:01:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5696035740 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5629369951b0 0x562936997640 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5696035740 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 0x562936991420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5696035740 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698a3e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5696035740 1 --2- 172.21.15.55:0/1103843100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56293697fa80 0x562936989ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5696035740 1 -- 172.21.15.55:0/1103843100 >> 172.21.15.55:0/1103843100 conn(0x56293697b0f0 msgr2=0x56293697a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5674ff9640 0 client.25918 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:17.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5696035740 1 -- 172.21.15.55:0/1103843100 shutdown_connections 2024-04-26T22:01:17.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.270+0000 7f5696035740 1 -- 172.21.15.55:0/1103843100 wait complete. 2024-04-26T22:01:17.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.271+0000 7f5696035740 20 client.25918 trim_cache size 0 max 16384 2024-04-26T22:01:17.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.372+0000 7f5696035740 1 Processor -- start 2024-04-26T22:01:17.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.372+0000 7f5696035740 1 -- start start 2024-04-26T22:01:17.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.373+0000 7f5696035740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293697fa80 0x5629369898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.374+0000 7f5696035740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562936980470 0x5629369913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.374+0000 7f5696035740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 0x56293697e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.375+0000 7f56908bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562936980470 0x5629369913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.376+0000 7f56918bf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 0x56293697e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.376+0000 7f56908bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562936980470 0x5629369913e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46372/0 (socket says 172.21.15.55:46372) 2024-04-26T22:01:17.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.376+0000 7f56908bd640 1 -- 172.21.15.55:0/4212178186 learned_addr learned my addr 172.21.15.55:0/4212178186 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:17.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.376+0000 7f5696035740 1 -- 172.21.15.55:0/4212178186 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5670010f20 con 0x5629369855e0 2024-04-26T22:01:17.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.376+0000 7f5696035740 1 -- 172.21.15.55:0/4212178186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f567000e7f0 con 0x562936980470 2024-04-26T22:01:17.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.376+0000 7f5696035740 1 -- 172.21.15.55:0/4212178186 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f567000c640 con 0x56293697fa80 2024-04-26T22:01:17.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.376+0000 7f56910be640 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293697fa80 0x5629369898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.378+0000 7f56908bd640 1 -- 172.21.15.55:0/4212178186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293697fa80 msgr2=0x5629369898a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:17.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.378+0000 7f56908bd640 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293697fa80 0x5629369898a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.378+0000 7f56908bd640 1 -- 172.21.15.55:0/4212178186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 msgr2=0x56293697e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:17.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.378+0000 7f56908bd640 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 0x56293697e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.378+0000 7f56908bd640 1 -- 172.21.15.55:0/4212178186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562936980850 con 0x562936980470 2024-04-26T22:01:17.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.379+0000 7f56908bd640 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562936980470 0x5629369913e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f568802aa00 tx=0x7f5688054000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf1228b4262c4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.380+0000 7f5676ffd640 1 -- 172.21.15.55:0/4212178186 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688004980 con 0x562936980470 2024-04-26T22:01:17.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.387+0000 7f5676ffd640 1 -- 172.21.15.55:0/4212178186 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f568804d070 con 0x562936980470 2024-04-26T22:01:17.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.387+0000 7f5676ffd640 1 -- 172.21.15.55:0/4212178186 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688032cc0 con 0x562936980470 2024-04-26T22:01:17.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.389+0000 7f5696035740 1 -- 172.21.15.55:0/4212178186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562936980470 msgr2=0x5629369913e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.390+0000 7f5696035740 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562936980470 0x5629369913e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f568802aa00 tx=0x7f5688054000 comp rx=0 tx=0).stop 2024-04-26T22:01:17.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.396+0000 7f56918bf640 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 0x56293697e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:01:17.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.396+0000 7f5696035740 1 -- 172.21.15.55:0/4212178186 shutdown_connections 2024-04-26T22:01:17.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.396+0000 7f5696035740 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5629369855e0 0x56293697e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.396+0000 7f5696035740 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562936980470 0x5629369913e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.396+0000 7f5696035740 1 --2- 172.21.15.55:0/4212178186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56293697fa80 0x5629369898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.396+0000 7f5696035740 1 -- 172.21.15.55:0/4212178186 >> 172.21.15.55:0/4212178186 conn(0x56293697b0f0 msgr2=0x562936989190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.402+0000 7f5696035740 1 -- 172.21.15.55:0/4212178186 shutdown_connections 2024-04-26T22:01:17.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.403+0000 7f5696035740 1 -- 172.21.15.55:0/4212178186 wait complete. 2024-04-26T22:01:17.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.404+0000 7f5696035740 1 -- start start 2024-04-26T22:01:17.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.405+0000 7f5696035740 1 Processor -- start 2024-04-26T22:01:17.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.407+0000 7f5696035740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293698e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.407+0000 7f5696035740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.408+0000 7f56910be640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293698e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.408+0000 7f5696035740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x56293698fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.408+0000 7f5696035740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f567000ccb0 con 0x56293697fa80 2024-04-26T22:01:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.408+0000 7f5696035740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f567000e120 con 0x5629369855e0 2024-04-26T22:01:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.408+0000 7f5696035740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f567000d380 con 0x562936980470 2024-04-26T22:01:17.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.408+0000 7f5696035740 10 client.0 _ll_register_callbacks cb 0x7f5688004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:17.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.409+0000 7f56908bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.410+0000 7f56908bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45148/0 (socket says 172.21.15.55:45148) 2024-04-26T22:01:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.410+0000 7f56908bd640 1 -- 172.21.15.55:0/758060256 learned_addr learned my addr 172.21.15.55:0/758060256 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.412+0000 7f56918bf640 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x56293698fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.413+0000 7f56908bd640 1 -- 172.21.15.55:0/758060256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 msgr2=0x56293698fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.413+0000 7f56908bd640 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x56293698fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.413 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.413+0000 7f56908bd640 1 -- 172.21.15.55:0/758060256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 msgr2=0x56293698e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.413+0000 7f56908bd640 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293698e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.413+0000 7f56908bd640 1 -- 172.21.15.55:0/758060256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629369901a0 con 0x562936980470 2024-04-26T22:01:17.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.417+0000 7f56908bd640 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698eba0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5688009850 tx=0x7f56880327e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22313bed141eaa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:17.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.417+0000 7f5674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:01:17.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.417+0000 7f5674ff9640 1 -- 172.21.15.55:0/758060256 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568803f040 con 0x562936980470 2024-04-26T22:01:17.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.419+0000 7f5696035740 1 -- 172.21.15.55:0/758060256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568804d070 con 0x562936980470 2024-04-26T22:01:17.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.419+0000 7f5674ff9640 1 -- 172.21.15.55:0/758060256 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5688032de0 con 0x562936980470 2024-04-26T22:01:17.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.421+0000 7f5674ff9640 1 -- 172.21.15.55:0/758060256 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f568807e050 con 0x562936980470 2024-04-26T22:01:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.421+0000 7f5696035740 10 client.25924 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:17.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.421+0000 7f5696035740 1 -- 172.21.15.55:0/758060256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562936991480 con 0x562936980470 2024-04-26T22:01:17.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.424+0000 7f5674ff9640 1 -- 172.21.15.55:0/758060256 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f568804d070 con 0x562936980470 2024-04-26T22:01:17.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.424+0000 7f5674ff9640 1 -- 172.21.15.55:0/758060256 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f5688048930 con 0x562936980470 2024-04-26T22:01:17.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.424+0000 7f5696035740 10 client.25924 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:17.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.424+0000 7f5696035740 10 client.25924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:17.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.424+0000 7f5696035740 20 client.25924 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:17.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.424+0000 7f5696035740 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5629369fbcf0 0x5629369fe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:17.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.424+0000 7f5696035740 4 client.25924 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:01:17.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.424+0000 7f5696035740 1 -- 172.21.15.55:0/758060256 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5629368ec580 con 0x5629369fbcf0 2024-04-26T22:01:17.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.427+0000 7f56910be640 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5629369fbcf0 0x5629369fe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:17.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.429+0000 7f56910be640 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5629369fbcf0 0x5629369fe110 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b4a5588cd6cb38 server_cookie=af9ff85020517876 in_seq=0 out_seq=0 2024-04-26T22:01:17.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.429+0000 7f5674ff9640 10 client.25924 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.551+0000 7f5674ff9640 1 -- 172.21.15.55:0/758060256 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5629368ec580 con 0x5629369fbcf0 2024-04-26T22:01:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.551+0000 7f5674ff9640 10 client.25924 handle_command_reply: tid=0 2024-04-26T22:01:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.551+0000 7f5696035740 1 client.25924 shutdown 2024-04-26T22:01:17.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.555+0000 7f5696035740 1 -- 172.21.15.55:0/758060256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 msgr2=0x56293698eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.555+0000 7f5696035740 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698eba0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5688009850 tx=0x7f56880327e0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.557+0000 7f5696035740 1 -- 172.21.15.55:0/758060256 shutdown_connections 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.557+0000 7f5696035740 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5629369855e0 0x56293698fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.557+0000 7f5696035740 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562936980470 0x56293698eba0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.557+0000 7f5696035740 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5629369fbcf0 0x5629369fe110 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.557+0000 7f5696035740 1 --2- 172.21.15.55:0/758060256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56293697fa80 0x56293698e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.557+0000 7f5696035740 1 -- 172.21.15.55:0/758060256 >> 172.21.15.55:0/758060256 conn(0x56293697b0f0 msgr2=0x56293697e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.557+0000 7f5674ff9640 0 client.25924 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.558+0000 7f5696035740 1 -- 172.21.15.55:0/758060256 shutdown_connections 2024-04-26T22:01:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.558+0000 7f5696035740 1 -- 172.21.15.55:0/758060256 wait complete. 2024-04-26T22:01:17.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.559+0000 7f5696035740 20 client.25924 trim_cache size 0 max 16384 2024-04-26T22:01:17.561 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:01:17.561 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:01:17.561 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "ffb70e35-f8e5-4fa3-a89d-334a383f3451", 2024-04-26T22:01:17.561 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:01:17.562 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:01:17.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.566+0000 7f5693322640 1 -- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f56780a3dc0 msgr2=0x7f56780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.566+0000 7f5693322640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f56780a3dc0 0x7f56780a6280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5688006010 tx=0x7f56880314c0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.566+0000 7f5693322640 1 -- 172.21.15.55:0/3259325486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c073be0 msgr2=0x7f568c119290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:17.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.566+0000 7f5693322640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c073be0 0x7f568c119290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f568400eab0 tx=0x7f568400ef70 comp rx=0 tx=0).stop 2024-04-26T22:01:17.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.573+0000 7f5693322640 1 -- 172.21.15.55:0/3259325486 shutdown_connections 2024-04-26T22:01:17.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.573+0000 7f5693322640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f56780a3dc0 0x7f56780a6280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.573+0000 7f5693322640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f568c10a640 0x7f568c11a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.573+0000 7f5693322640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f568c074540 0x7f568c1197d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.573+0000 7f5693322640 1 --2- 172.21.15.55:0/3259325486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f568c073be0 0x7f568c119290 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:17.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.573+0000 7f5693322640 1 -- 172.21.15.55:0/3259325486 >> 172.21.15.55:0/3259325486 conn(0x7f568c06f780 msgr2=0x7f568c10d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:17.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.577+0000 7f5693322640 1 -- 172.21.15.55:0/3259325486 shutdown_connections 2024-04-26T22:01:17.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:17.578+0000 7f5693322640 1 -- 172.21.15.55:0/3259325486 wait complete. 2024-04-26T22:01:17.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:01:17.618 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:01:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.031+0000 7f726bfff640 1 -- 172.21.15.55:0/2664950805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a9b0 msgr2=0x7f726c10f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.031+0000 7f726bfff640 1 --2- 172.21.15.55:0/2664950805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a9b0 0x7f726c10f270 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7264009900 tx=0x7f726402a440 comp rx=0 tx=0).stop 2024-04-26T22:01:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.032+0000 7f726bfff640 1 -- 172.21.15.55:0/2664950805 shutdown_connections 2024-04-26T22:01:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.032+0000 7f726bfff640 1 --2- 172.21.15.55:0/2664950805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a9b0 0x7f726c10f270 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.032+0000 7f726bfff640 1 --2- 172.21.15.55:0/2664950805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f726c10a010 0x7f726c10a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.032+0000 7f726bfff640 1 --2- 172.21.15.55:0/2664950805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726c073d70 0x7f726c074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.032+0000 7f726bfff640 1 -- 172.21.15.55:0/2664950805 >> 172.21.15.55:0/2664950805 conn(0x7f726c06f840 msgr2=0x7f726c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:18.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.036+0000 7f726bfff640 1 -- 172.21.15.55:0/2664950805 shutdown_connections 2024-04-26T22:01:18.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.036+0000 7f726bfff640 1 -- 172.21.15.55:0/2664950805 wait complete. 2024-04-26T22:01:18.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.037+0000 7f726bfff640 1 Processor -- start 2024-04-26T22:01:18.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.038+0000 7f726bfff640 1 -- start start 2024-04-26T22:01:18.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.039+0000 7f726bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f726c073d70 0x7f726c116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.039+0000 7f726bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a010 0x7f726c1165a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.039+0000 7f726bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726c10a9b0 0x7f726c1b7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.039+0000 7f726bfff640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f726c11d170 con 0x7f726c10a010 2024-04-26T22:01:18.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.039+0000 7f726bfff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f726c11cff0 con 0x7f726c10a9b0 2024-04-26T22:01:18.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.039+0000 7f726bfff640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f726c11d2f0 con 0x7f726c073d70 2024-04-26T22:01:18.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.042+0000 7f726b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726c10a9b0 0x7f726c1b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.042+0000 7f726b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726c10a9b0 0x7f726c1b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46398/0 (socket says 172.21.15.55:46398) 2024-04-26T22:01:18.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.042+0000 7f726b7fe640 1 -- 172.21.15.55:0/144877109 learned_addr learned my addr 172.21.15.55:0/144877109 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:18.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.042+0000 7f726a7fc640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a010 0x7f726c1165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.044+0000 7f726affd640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f726c073d70 0x7f726c116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.046+0000 7f726a7fc640 1 -- 172.21.15.55:0/144877109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f726c073d70 msgr2=0x7f726c116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.046+0000 7f726a7fc640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f726c073d70 0x7f726c116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.046+0000 7f726a7fc640 1 -- 172.21.15.55:0/144877109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726c10a9b0 msgr2=0x7f726c1b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.046+0000 7f726a7fc640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726c10a9b0 0x7f726c1b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.046+0000 7f726a7fc640 1 -- 172.21.15.55:0/144877109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726c1b7d80 con 0x7f726c10a010 2024-04-26T22:01:18.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.047+0000 7f726a7fc640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a010 0x7f726c1165a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f725400c970 tx=0x7f725400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.051+0000 7f7270902640 1 -- 172.21.15.55:0/144877109 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254007bb0 con 0x7f726c10a010 2024-04-26T22:01:18.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.051+0000 7f7270902640 1 -- 172.21.15.55:0/144877109 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7254007d50 con 0x7f726c10a010 2024-04-26T22:01:18.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.051+0000 7f7270902640 1 -- 172.21.15.55:0/144877109 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254004e40 con 0x7f726c10a010 2024-04-26T22:01:18.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.054+0000 7f726bfff640 1 -- 172.21.15.55:0/144877109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726c1b7f50 con 0x7f726c10a010 2024-04-26T22:01:18.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.054+0000 7f726bfff640 1 -- 172.21.15.55:0/144877109 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726c1b8360 con 0x7f726c10a010 2024-04-26T22:01:18.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.061+0000 7f7274616740 1 Processor -- start 2024-04-26T22:01:18.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.063+0000 7f7274616740 1 -- start start 2024-04-26T22:01:18.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.063+0000 7f7274616740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5612976080b0 0x5612976084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.063+0000 7f7274616740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297609700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.063+0000 7f7274616740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609c40 0x5612976104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.063+0000 7f7274616740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56129756cd80 con 0x5612976080b0 2024-04-26T22:01:18.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.063+0000 7f7274616740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5612975268b0 con 0x561297609280 2024-04-26T22:01:18.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.063+0000 7f7274616740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561297568d40 con 0x561297609c40 2024-04-26T22:01:18.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.065+0000 7f7270902640 1 -- 172.21.15.55:0/144877109 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7254005020 con 0x7f726c10a010 2024-04-26T22:01:18.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.066+0000 7f7270902640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72580a3e40 0x7f72580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.067+0000 7f7270902640 1 -- 172.21.15.55:0/144877109 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f72540d04b0 con 0x7f726c10a010 2024-04-26T22:01:18.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.067+0000 7f726affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297609700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.067+0000 7f726affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297609700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46402/0 (socket says 172.21.15.55:46402) 2024-04-26T22:01:18.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.067+0000 7f726affd640 1 -- 172.21.15.55:0/3202903746 learned_addr learned my addr 172.21.15.55:0/3202903746 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:18.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.069+0000 7f726b7fe640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72580a3e40 0x7f72580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.069+0000 7f726b7fe640 1 --2- 172.21.15.55:0/3202903746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5612976080b0 0x5612976084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.072+0000 7f726b7fe640 1 -- 172.21.15.55:0/3202903746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609c40 msgr2=0x5612976104d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:18.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.072+0000 7f726b7fe640 1 --2- 172.21.15.55:0/3202903746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609c40 0x5612976104d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.072+0000 7f726b7fe640 1 -- 172.21.15.55:0/3202903746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 msgr2=0x561297609700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:18.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.072+0000 7f726b7fe640 1 --2- 172.21.15.55:0/3202903746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297609700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.072+0000 7f726b7fe640 1 -- 172.21.15.55:0/3202903746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612974f7a40 con 0x5612976080b0 2024-04-26T22:01:18.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.072+0000 7f726b7fe640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72580a3e40 0x7f72580a6300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7264005ab0 tx=0x7f7264004100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.073+0000 7f726b7fe640 1 --2- 172.21.15.55:0/3202903746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5612976080b0 0x5612976084d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7264006010 tx=0x7f7264047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cdb46ca8cc706f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.079+0000 7f7261ffb640 1 -- 172.21.15.55:0/3202903746 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264035040 con 0x5612976080b0 2024-04-26T22:01:18.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.079+0000 7f7261ffb640 1 -- 172.21.15.55:0/3202903746 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726404e070 con 0x5612976080b0 2024-04-26T22:01:18.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.079+0000 7f7261ffb640 1 -- 172.21.15.55:0/3202903746 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726403d7b0 con 0x5612976080b0 2024-04-26T22:01:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.083+0000 7f7274616740 1 -- 172.21.15.55:0/3202903746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5612976080b0 msgr2=0x5612976084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.083+0000 7f7274616740 1 --2- 172.21.15.55:0/3202903746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5612976080b0 0x5612976084d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7264006010 tx=0x7f7264047000 comp rx=0 tx=0).stop 2024-04-26T22:01:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.084+0000 7f7274616740 1 -- 172.21.15.55:0/3202903746 shutdown_connections 2024-04-26T22:01:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.084+0000 7f7274616740 1 --2- 172.21.15.55:0/3202903746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609c40 0x5612976104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.084+0000 7f7274616740 1 --2- 172.21.15.55:0/3202903746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297609700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.084+0000 7f7274616740 1 --2- 172.21.15.55:0/3202903746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5612976080b0 0x5612976084d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.084+0000 7f7274616740 1 -- 172.21.15.55:0/3202903746 >> 172.21.15.55:0/3202903746 conn(0x5612976035a0 msgr2=0x5612976039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:18.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.096+0000 7f7274616740 1 -- 172.21.15.55:0/3202903746 shutdown_connections 2024-04-26T22:01:18.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.097+0000 7f7274616740 1 -- 172.21.15.55:0/3202903746 wait complete. 2024-04-26T22:01:18.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.098+0000 7f7274616740 1 -- start start 2024-04-26T22:01:18.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.099+0000 7f7274616740 1 Processor -- start 2024-04-26T22:01:18.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.100+0000 7f7274616740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612976080b0 0x5612976125f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.100+0000 7f7274616740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297612b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.100+0000 7f7274616740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x561297619b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.100+0000 7f7274616740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561297610bf0 con 0x561297609c40 2024-04-26T22:01:18.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.101+0000 7f7274616740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5612974f3d10 con 0x561297609280 2024-04-26T22:01:18.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.101+0000 7f7274616740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561297610d70 con 0x5612976080b0 2024-04-26T22:01:18.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.101+0000 7f7274616740 10 client.0 _ll_register_callbacks cb 0x561297520210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:18.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.102+0000 7f726b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x561297619b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.102+0000 7f726b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x561297619b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50722/0 (socket says 172.21.15.55:50722) 2024-04-26T22:01:18.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.102+0000 7f726b7fe640 1 -- 172.21.15.55:0/955196913 learned_addr learned my addr 172.21.15.55:0/955196913 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:18.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.103+0000 7f726affd640 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612976080b0 0x5612976125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.104+0000 7f726a7fc640 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297612b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.106+0000 7f726b7fe640 1 -- 172.21.15.55:0/955196913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612976080b0 msgr2=0x5612976125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:18.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.106+0000 7f726b7fe640 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612976080b0 0x5612976125f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.106+0000 7f726b7fe640 1 -- 172.21.15.55:0/955196913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 msgr2=0x561297612b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:18.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.106+0000 7f726b7fe640 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297612b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.106+0000 7f726b7fe640 1 -- 172.21.15.55:0/955196913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612973487d0 con 0x561297609c40 2024-04-26T22:01:18.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.111+0000 7f726b7fe640 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x561297619b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f72640317f0 tx=0x7f7264031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ee49e512b3de11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.111+0000 7f724affd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:18.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.112+0000 7f724affd640 1 -- 172.21.15.55:0/955196913 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264056070 con 0x561297609c40 2024-04-26T22:01:18.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.112+0000 7f7274616740 1 -- 172.21.15.55:0/955196913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f726404e070 con 0x561297609c40 2024-04-26T22:01:18.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.112+0000 7f724affd640 1 -- 172.21.15.55:0/955196913 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f726403d4f0 con 0x561297609c40 2024-04-26T22:01:18.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.114+0000 7f724affd640 1 -- 172.21.15.55:0/955196913 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7264060050 con 0x561297609c40 2024-04-26T22:01:18.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.114+0000 7f7274616740 10 client.16764 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:18.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.114+0000 7f7274616740 1 -- 172.21.15.55:0/955196913 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56129727e3f0 con 0x561297609c40 2024-04-26T22:01:18.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.116+0000 7f724affd640 1 -- 172.21.15.55:0/955196913 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726404e070 con 0x561297609c40 2024-04-26T22:01:18.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.116+0000 7f724affd640 1 -- 172.21.15.55:0/955196913 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7264063020 con 0x561297609c40 2024-04-26T22:01:18.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.116+0000 7f7274616740 10 client.16764 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:18.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.116+0000 7f7274616740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:18.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.116+0000 7f7274616740 20 client.16764 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:18.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.116+0000 7f7274616740 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56129761e010 0x561297620430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.116+0000 7f7274616740 4 client.16764 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:18.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.116+0000 7f7274616740 1 -- 172.21.15.55:0/955196913 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56129757a990 con 0x56129761e010 2024-04-26T22:01:18.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.119+0000 7f726affd640 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56129761e010 0x561297620430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.123+0000 7f726affd640 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56129761e010 0x561297620430 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c14d32ac615707b server_cookie=2231ed50eedf7124 in_seq=0 out_seq=0 2024-04-26T22:01:18.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.123+0000 7f724affd640 10 client.16764 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.135+0000 7f724affd640 1 -- 172.21.15.55:0/955196913 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56129757a990 con 0x56129761e010 2024-04-26T22:01:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.135+0000 7f724affd640 10 client.16764 handle_command_reply: tid=0 2024-04-26T22:01:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.135+0000 7f7274616740 1 client.16764 shutdown 2024-04-26T22:01:18.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.141+0000 7f7274616740 1 -- 172.21.15.55:0/955196913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 msgr2=0x561297619b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.141+0000 7f7274616740 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x561297619b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f72640317f0 tx=0x7f7264031820 comp rx=0 tx=0).stop 2024-04-26T22:01:18.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.142+0000 7f7274616740 1 -- 172.21.15.55:0/955196913 shutdown_connections 2024-04-26T22:01:18.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.142+0000 7f7274616740 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56129761e010 0x561297620430 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.142+0000 7f7274616740 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x561297619b70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.142+0000 7f7274616740 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561297609280 0x561297612b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.142+0000 7f7274616740 1 --2- 172.21.15.55:0/955196913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5612976080b0 0x5612976125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.143+0000 7f7274616740 1 -- 172.21.15.55:0/955196913 >> 172.21.15.55:0/955196913 conn(0x5612976035a0 msgr2=0x5612976026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:18.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.144+0000 7f724affd640 0 client.16764 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:18.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.145+0000 7f7274616740 1 -- 172.21.15.55:0/955196913 shutdown_connections 2024-04-26T22:01:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.145+0000 7f7274616740 1 -- 172.21.15.55:0/955196913 wait complete. 2024-04-26T22:01:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.146+0000 7f7274616740 20 client.16764 trim_cache size 0 max 16384 2024-04-26T22:01:18.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.205+0000 7f7274616740 1 Processor -- start 2024-04-26T22:01:18.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.205+0000 7f7274616740 1 -- start start 2024-04-26T22:01:18.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.206+0000 7f7274616740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976080b0 0x561297601e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.206+0000 7f7274616740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609280 0x561297619b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.206+0000 7f7274616740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609c40 0x561297602360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.206+0000 7f7274616740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f726c1c3b30 con 0x561297609280 2024-04-26T22:01:18.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.207+0000 7f726affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976080b0 0x561297601e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.207+0000 7f7274616740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f726c1c1400 con 0x5612976080b0 2024-04-26T22:01:18.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.207+0000 7f7274616740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f726c1bf250 con 0x561297609c40 2024-04-26T22:01:18.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.207+0000 7f726affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976080b0 0x561297601e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:46428/0 (socket says 172.21.15.55:46428) 2024-04-26T22:01:18.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.207+0000 7f726affd640 1 -- 172.21.15.55:0/3593672559 learned_addr learned my addr 172.21.15.55:0/3593672559 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:18.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.208+0000 7f726a7fc640 1 --2- 172.21.15.55:0/3593672559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609280 0x561297619b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.208+0000 7f726affd640 1 -- 172.21.15.55:0/3593672559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609c40 msgr2=0x561297602360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:18.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.208+0000 7f726affd640 1 --2- 172.21.15.55:0/3593672559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609c40 0x561297602360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.208+0000 7f726affd640 1 -- 172.21.15.55:0/3593672559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609280 msgr2=0x561297619b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:18.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.208+0000 7f726affd640 1 --2- 172.21.15.55:0/3593672559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609280 0x561297619b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.208+0000 7f726affd640 1 -- 172.21.15.55:0/3593672559 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612975fed10 con 0x5612976080b0 2024-04-26T22:01:18.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.212+0000 7f726affd640 1 --2- 172.21.15.55:0/3593672559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976080b0 0x561297601e20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f725c009990 tx=0x7f725c0138f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd9ddf409917688 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.214+0000 7f72617fa640 1 -- 172.21.15.55:0/3593672559 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c01d030 con 0x5612976080b0 2024-04-26T22:01:18.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.214+0000 7f72617fa640 1 -- 172.21.15.55:0/3593672559 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f725c01d680 con 0x5612976080b0 2024-04-26T22:01:18.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.214+0000 7f72617fa640 1 -- 172.21.15.55:0/3593672559 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c01cc90 con 0x5612976080b0 2024-04-26T22:01:18.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 -- 172.21.15.55:0/3593672559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976080b0 msgr2=0x561297601e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 --2- 172.21.15.55:0/3593672559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976080b0 0x561297601e20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f725c009990 tx=0x7f725c0138f0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 -- 172.21.15.55:0/3593672559 shutdown_connections 2024-04-26T22:01:18.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 --2- 172.21.15.55:0/3593672559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609c40 0x561297602360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 --2- 172.21.15.55:0/3593672559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609280 0x561297619b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 --2- 172.21.15.55:0/3593672559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976080b0 0x561297601e20 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f725c009990 tx=0x7f725c0138f0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 -- 172.21.15.55:0/3593672559 >> 172.21.15.55:0/3593672559 conn(0x5612976035a0 msgr2=0x5612976125f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:18.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 -- 172.21.15.55:0/3593672559 shutdown_connections 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.215+0000 7f7274616740 1 -- 172.21.15.55:0/3593672559 wait complete. 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.216+0000 7f7274616740 1 -- start start 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f7274616740 1 Processor -- start 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f7274616740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609280 0x561297616d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f7274616740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x5612976172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f7274616740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976182b0 0x561297618730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f7274616740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f726c1bf8c0 con 0x561297609c40 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f7274616740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f726c1c0d30 con 0x5612976182b0 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f7274616740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f726c1bff90 con 0x561297609280 2024-04-26T22:01:18.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f7274616740 10 client.0 _ll_register_callbacks cb 0x7f725c009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f726a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x5612976172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f726a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x5612976172a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50732/0 (socket says 172.21.15.55:50732) 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.217+0000 7f726a7fc640 1 -- 172.21.15.55:0/1159273075 learned_addr learned my addr 172.21.15.55:0/1159273075 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.218+0000 7f726a7fc640 1 -- 172.21.15.55:0/1159273075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609280 msgr2=0x561297616d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.218+0000 7f726a7fc640 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609280 0x561297616d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.218+0000 7f726a7fc640 1 -- 172.21.15.55:0/1159273075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976182b0 msgr2=0x561297618730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.218+0000 7f726a7fc640 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976182b0 0x561297618730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.218+0000 7f726a7fc640 1 -- 172.21.15.55:0/1159273075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56129723e0f0 con 0x561297609c40 2024-04-26T22:01:18.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.218+0000 7f726a7fc640 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x5612976172a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f725409c310 tx=0x7f725409cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f893ef978e79b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.218+0000 7f724affd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.218+0000 7f724affd640 1 -- 172.21.15.55:0/1159273075 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72540b9030 con 0x561297609c40 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.219+0000 7f724affd640 1 -- 172.21.15.55:0/1159273075 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f725409a910 con 0x561297609c40 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.219+0000 7f7274616740 1 -- 172.21.15.55:0/1159273075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f725c01d680 con 0x561297609c40 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.219+0000 7f724affd640 1 -- 172.21.15.55:0/1159273075 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f72540a5280 con 0x561297609c40 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.219+0000 7f7274616740 10 client.16770 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.219+0000 7f7274616740 1 -- 172.21.15.55:0/1159273075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56129752d4b0 con 0x561297609c40 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.219+0000 7f724affd640 1 -- 172.21.15.55:0/1159273075 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f725c01d680 con 0x561297609c40 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.220+0000 7f724affd640 1 -- 172.21.15.55:0/1159273075 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f72540984f0 con 0x561297609c40 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.220+0000 7f7274616740 10 client.16770 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.220+0000 7f7274616740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.220+0000 7f7274616740 20 client.16770 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.220+0000 7f7274616740 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561297684c00 0x561297687020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.220+0000 7f7274616740 4 client.16770 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:01:18.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.220+0000 7f7274616740 1 -- 172.21.15.55:0/1159273075 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561297574800 con 0x561297684c00 2024-04-26T22:01:18.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.223+0000 7f726affd640 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561297684c00 0x561297687020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.226+0000 7f724affd640 10 client.16770 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:18.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.226+0000 7f726affd640 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561297684c00 0x561297687020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fee54e650583930 server_cookie=32d114182db8ce5c in_seq=0 out_seq=0 2024-04-26T22:01:18.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f724affd640 1 -- 172.21.15.55:0/1159273075 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x561297574800 con 0x561297684c00 2024-04-26T22:01:18.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f724affd640 10 client.16770 handle_command_reply: tid=0 2024-04-26T22:01:18.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 client.16770 shutdown 2024-04-26T22:01:18.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 -- 172.21.15.55:0/1159273075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 msgr2=0x5612976172a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x5612976172a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f725409c310 tx=0x7f725409cd10 comp rx=0 tx=0).stop 2024-04-26T22:01:18.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 -- 172.21.15.55:0/1159273075 shutdown_connections 2024-04-26T22:01:18.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5612976182b0 0x561297618730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561297684c00 0x561297687020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561297609c40 0x5612976172a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 --2- 172.21.15.55:0/1159273075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561297609280 0x561297616d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f7274616740 1 -- 172.21.15.55:0/1159273075 >> 172.21.15.55:0/1159273075 conn(0x5612976035a0 msgr2=0x561297601e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:18.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.231+0000 7f724affd640 0 client.16770 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:18.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.232+0000 7f7274616740 1 -- 172.21.15.55:0/1159273075 shutdown_connections 2024-04-26T22:01:18.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.232+0000 7f7274616740 1 -- 172.21.15.55:0/1159273075 wait complete. 2024-04-26T22:01:18.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.232+0000 7f7274616740 20 client.16770 trim_cache size 0 max 16384 2024-04-26T22:01:18.234 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:01:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (2 inodes in the stack)", 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "ffb70e35-f8e5-4fa3-a89d-334a383f3451": { 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "ffb70e35-f8e5-4fa3-a89d-334a383f3451", 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 -- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72580a3e40 msgr2=0x7f72580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72580a3e40 0x7f72580a6300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7264005ab0 tx=0x7f7264004100 comp rx=0 tx=0).stop 2024-04-26T22:01:18.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 -- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a010 msgr2=0x7f726c1165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:18.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a010 0x7f726c1165a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f725400c970 tx=0x7f725400ce30 comp rx=0 tx=0).stop 2024-04-26T22:01:18.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 -- 172.21.15.55:0/144877109 shutdown_connections 2024-04-26T22:01:18.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72580a3e40 0x7f72580a6300 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f726c10a9b0 0x7f726c1b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f726c10a010 0x7f726c1165a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 --2- 172.21.15.55:0/144877109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f726c073d70 0x7f726c116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:18.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.233+0000 7f726bfff640 1 -- 172.21.15.55:0/144877109 >> 172.21.15.55:0/144877109 conn(0x7f726c06f840 msgr2=0x7f726c0701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:18.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.234+0000 7f726bfff640 1 -- 172.21.15.55:0/144877109 shutdown_connections 2024-04-26T22:01:18.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:18.235+0000 7f726bfff640 1 -- 172.21.15.55:0/144877109 wait complete. 2024-04-26T22:01:18.263 INFO:tasks.cephfs.filesystem:scrub status for tag:ffb70e35-f8e5-4fa3-a89d-334a383f3451 - {'path': '/', 'tag': 'ffb70e35-f8e5-4fa3-a89d-334a383f3451', 'options': 'recursive,force'} 2024-04-26T22:01:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:18 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:01:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:18 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:18 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:01:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:18 smithi165 ceph-mon[33296]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 910 MiB/s wr, 1.83k op/s 2024-04-26T22:01:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:18 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:01:19.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:18 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:19.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:18 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:01:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:18 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:18 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:01:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:18 smithi055 ceph-mon[29221]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 910 MiB/s wr, 1.83k op/s 2024-04-26T22:01:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:18 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:01:19.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:18 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:18 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:01:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:18 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:18 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:01:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:18 smithi064 ceph-mon[32765]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 910 MiB/s wr, 1.83k op/s 2024-04-26T22:01:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:18 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:01:19.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:18 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:21.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:20 smithi055 ceph-mon[29221]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 889 MiB/s wr, 1.70k op/s 2024-04-26T22:01:21.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:20 smithi064 ceph-mon[32765]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 889 MiB/s wr, 1.70k op/s 2024-04-26T22:01:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:20 smithi165 ceph-mon[33296]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 889 MiB/s wr, 1.70k op/s 2024-04-26T22:01:23.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:22 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:23.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:22 smithi055 ceph-mon[29221]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 884 MiB/s wr, 1.90k op/s 2024-04-26T22:01:23.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:22 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:23.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:22 smithi064 ceph-mon[32765]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 884 MiB/s wr, 1.90k op/s 2024-04-26T22:01:23.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:22 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:23.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:22 smithi165 ceph-mon[33296]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 884 MiB/s wr, 1.90k op/s 2024-04-26T22:01:25.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:24 smithi055 ceph-mon[29221]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 804 MiB/s wr, 1.64k op/s 2024-04-26T22:01:25.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:24 smithi064 ceph-mon[32765]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 804 MiB/s wr, 1.64k op/s 2024-04-26T22:01:25.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:24 smithi165 ceph-mon[33296]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 804 MiB/s wr, 1.64k op/s 2024-04-26T22:01:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:26 smithi055 ceph-mon[29221]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 814 MiB/s wr, 1.49k op/s 2024-04-26T22:01:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:26 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:01:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:26 smithi064 ceph-mon[32765]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 814 MiB/s wr, 1.49k op/s 2024-04-26T22:01:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:26 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:01:27.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:26 smithi165 ceph-mon[33296]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 814 MiB/s wr, 1.49k op/s 2024-04-26T22:01:27.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:26 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:01:28.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:27 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:01:28.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:27 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:01:28.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:27 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:01:29.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:28 smithi055 ceph-mon[29221]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 915 MiB/s wr, 1.51k op/s 2024-04-26T22:01:29.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:28 smithi064 ceph-mon[32765]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 915 MiB/s wr, 1.51k op/s 2024-04-26T22:01:29.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:28 smithi165 ceph-mon[33296]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 915 MiB/s wr, 1.51k op/s 2024-04-26T22:01:31.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:30 smithi055 ceph-mon[29221]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.51k op/s 2024-04-26T22:01:31.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:30 smithi064 ceph-mon[32765]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.51k op/s 2024-04-26T22:01:31.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:30 smithi165 ceph-mon[33296]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 894 MiB/s wr, 1.51k op/s 2024-04-26T22:01:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:32 smithi064 ceph-mon[32765]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.45k op/s 2024-04-26T22:01:33.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:32 smithi165 ceph-mon[33296]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.45k op/s 2024-04-26T22:01:33.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:32 smithi055 ceph-mon[29221]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 1.45k op/s 2024-04-26T22:01:35.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:34 smithi064 ceph-mon[32765]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 808 MiB/s wr, 1.19k op/s 2024-04-26T22:01:35.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:34 smithi165 ceph-mon[33296]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 808 MiB/s wr, 1.19k op/s 2024-04-26T22:01:35.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:34 smithi055 ceph-mon[29221]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 808 MiB/s wr, 1.19k op/s 2024-04-26T22:01:36.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:35 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:01:36.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:35 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:01:36.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:35 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:01:37.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:37 smithi165 ceph-mon[33296]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 835 MiB/s wr, 1.18k op/s 2024-04-26T22:01:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:37 smithi055 ceph-mon[29221]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 835 MiB/s wr, 1.18k op/s 2024-04-26T22:01:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:37 smithi064 ceph-mon[32765]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 835 MiB/s wr, 1.18k op/s 2024-04-26T22:01:38.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:38.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:38.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:01:38.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:38.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:38.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:01:38.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:38.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:38.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:01:39.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:39 smithi165 ceph-mon[33296]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 899 MiB/s wr, 1.25k op/s 2024-04-26T22:01:39.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:39 smithi055 ceph-mon[29221]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 899 MiB/s wr, 1.25k op/s 2024-04-26T22:01:40.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:39 smithi064 ceph-mon[32765]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 899 MiB/s wr, 1.25k op/s 2024-04-26T22:01:41.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:41 smithi165 ceph-mon[33296]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 880 MiB/s wr, 1.25k op/s 2024-04-26T22:01:41.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:41 smithi055 ceph-mon[29221]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 880 MiB/s wr, 1.25k op/s 2024-04-26T22:01:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:41 smithi064 ceph-mon[32765]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 880 MiB/s wr, 1.25k op/s 2024-04-26T22:01:43.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:43 smithi165 ceph-mon[33296]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 881 MiB/s wr, 1.28k op/s 2024-04-26T22:01:43.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:43 smithi055 ceph-mon[29221]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 881 MiB/s wr, 1.28k op/s 2024-04-26T22:01:44.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:43 smithi064 ceph-mon[32765]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 881 MiB/s wr, 1.28k op/s 2024-04-26T22:01:45.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:45 smithi165 ceph-mon[33296]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 814 MiB/s wr, 1.21k op/s 2024-04-26T22:01:45.940 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:45 smithi055 ceph-mon[29221]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 814 MiB/s wr, 1.21k op/s 2024-04-26T22:01:46.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:45 smithi064 ceph-mon[32765]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 814 MiB/s wr, 1.21k op/s 2024-04-26T22:01:47.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:47 smithi165 ceph-mon[33296]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 832 MiB/s wr, 1.38k op/s 2024-04-26T22:01:47.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:47 smithi055 ceph-mon[29221]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 832 MiB/s wr, 1.38k op/s 2024-04-26T22:01:48.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:47 smithi064 ceph-mon[32765]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 832 MiB/s wr, 1.38k op/s 2024-04-26T22:01:48.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:01:48.264 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:01:48.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.721+0000 7f201e978640 1 -- 172.21.15.55:0/2534376927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 msgr2=0x7f2018103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.721+0000 7f201e978640 1 --2- 172.21.15.55:0/2534376927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 0x7f2018103630 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f200c009860 tx=0x7f200c02a360 comp rx=0 tx=0).stop 2024-04-26T22:01:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.724+0000 7f201e978640 1 -- 172.21.15.55:0/2534376927 shutdown_connections 2024-04-26T22:01:48.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.724+0000 7f201e978640 1 --2- 172.21.15.55:0/2534376927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2018106520 0x7f20181069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.724+0000 7f201e978640 1 --2- 172.21.15.55:0/2534376927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2018103b70 0x7f2018105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.724+0000 7f201e978640 1 --2- 172.21.15.55:0/2534376927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 0x7f2018103630 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.724+0000 7f201e978640 1 -- 172.21.15.55:0/2534376927 >> 172.21.15.55:0/2534376927 conn(0x7f20180fce20 msgr2=0x7f20180ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.732+0000 7f201e978640 1 -- 172.21.15.55:0/2534376927 shutdown_connections 2024-04-26T22:01:48.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.736+0000 7f201e978640 1 -- 172.21.15.55:0/2534376927 wait complete. 2024-04-26T22:01:48.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.738+0000 7f201e978640 1 Processor -- start 2024-04-26T22:01:48.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.738+0000 7f201e978640 1 -- start start 2024-04-26T22:01:48.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.740+0000 7f201e978640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 0x7f201819f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.740+0000 7f201e978640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2018103b70 0x7f201819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.740+0000 7f201e978640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2018106520 0x7f20181a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.740+0000 7f201e978640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2018114b70 con 0x7f2018106520 2024-04-26T22:01:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.740+0000 7f2017fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 0x7f201819f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.740+0000 7f2017fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 0x7f201819f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37128/0 (socket says 172.21.15.55:37128) 2024-04-26T22:01:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.740+0000 7f2017fff640 1 -- 172.21.15.55:0/3102363605 learned_addr learned my addr 172.21.15.55:0/3102363605 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.741+0000 7f201e978640 1 -- 172.21.15.55:0/3102363605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20181149f0 con 0x7f2018101200 2024-04-26T22:01:48.742 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.741+0000 7f201e978640 1 -- 172.21.15.55:0/3102363605 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2018114cf0 con 0x7f2018103b70 2024-04-26T22:01:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.741+0000 7f20177fe640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2018103b70 0x7f201819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.742+0000 7f2017fff640 1 -- 172.21.15.55:0/3102363605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2018103b70 msgr2=0x7f201819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.742+0000 7f2017fff640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2018103b70 0x7f201819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.742+0000 7f2017fff640 1 -- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2018106520 msgr2=0x7f20181a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.742+0000 7f2017fff640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2018106520 0x7f20181a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.742+0000 7f2017fff640 1 -- 172.21.15.55:0/3102363605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20181a6ea0 con 0x7f2018101200 2024-04-26T22:01:48.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.748+0000 7f2017fff640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 0x7f201819f280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f200c005d00 tx=0x7f200c032cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.751+0000 7f20157fa640 1 -- 172.21.15.55:0/3102363605 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c037030 con 0x7f2018101200 2024-04-26T22:01:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.751+0000 7f20157fa640 1 -- 172.21.15.55:0/3102363605 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200c038040 con 0x7f2018101200 2024-04-26T22:01:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.751+0000 7f20157fa640 1 -- 172.21.15.55:0/3102363605 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0046c0 con 0x7f2018101200 2024-04-26T22:01:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.754+0000 7f201e978640 1 -- 172.21.15.55:0/3102363605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20181a7130 con 0x7f2018101200 2024-04-26T22:01:48.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.754+0000 7f201e978640 1 -- 172.21.15.55:0/3102363605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20181a7510 con 0x7f2018101200 2024-04-26T22:01:48.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.764+0000 7f20157fa640 1 -- 172.21.15.55:0/3102363605 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f200c043030 con 0x7f2018101200 2024-04-26T22:01:48.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.769+0000 7f202176c740 1 Processor -- start 2024-04-26T22:01:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.773+0000 7f202176c740 1 -- start start 2024-04-26T22:01:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.777+0000 7f20157fa640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20000a3dc0 0x7f20000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.777+0000 7f20157fa640 1 -- 172.21.15.55:0/3102363605 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f200c030860 con 0x7f2018101200 2024-04-26T22:01:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.778+0000 7f201cf15640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20000a3dc0 0x7f20000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f202176c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f202176c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d03520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f202176c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d03a60 0x55c5a1d0a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f202176c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c5a1c66d80 con 0x55c5a1d030a0 2024-04-26T22:01:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f202176c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c5a1c208b0 con 0x55c5a1d03a60 2024-04-26T22:01:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f202176c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5a1c62d40 con 0x55c5a1d01ed0 2024-04-26T22:01:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f2017fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f2017fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48616/0 (socket says 172.21.15.55:48616) 2024-04-26T22:01:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.779+0000 7f2017fff640 1 -- 172.21.15.55:0/2708432318 learned_addr learned my addr 172.21.15.55:0/2708432318 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.780+0000 7f2017fff640 1 -- 172.21.15.55:0/2708432318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d03a60 msgr2=0x55c5a1d0a2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:01:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.780+0000 7f2017fff640 1 --2- 172.21.15.55:0/2708432318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d03a60 0x55c5a1d0a2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.780+0000 7f2017fff640 1 -- 172.21.15.55:0/2708432318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 msgr2=0x55c5a1d03520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:01:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.780+0000 7f2017fff640 1 --2- 172.21.15.55:0/2708432318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d03520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.780+0000 7f2017fff640 1 -- 172.21.15.55:0/2708432318 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a1bf1a40 con 0x55c5a1d01ed0 2024-04-26T22:01:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.780+0000 7f2017fff640 1 --2- 172.21.15.55:0/2708432318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d022f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f200c0057a0 tx=0x7f200c0329b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=212a9d3564c435a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.783+0000 7f1ffdffb640 1 -- 172.21.15.55:0/2708432318 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0c15e0 con 0x55c5a1d01ed0 2024-04-26T22:01:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.783+0000 7f1ffdffb640 1 -- 172.21.15.55:0/2708432318 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200c0c5e10 con 0x55c5a1d01ed0 2024-04-26T22:01:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.783+0000 7f1ffdffb640 1 -- 172.21.15.55:0/2708432318 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0c13d0 con 0x55c5a1d01ed0 2024-04-26T22:01:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 -- 172.21.15.55:0/2708432318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 msgr2=0x55c5a1d022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 --2- 172.21.15.55:0/2708432318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d022f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f200c0057a0 tx=0x7f200c0329b0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f201cf15640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20000a3dc0 0x7f20000a6280 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f20080058c0 tx=0x7f2008005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 -- 172.21.15.55:0/2708432318 shutdown_connections 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 --2- 172.21.15.55:0/2708432318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d03a60 0x55c5a1d0a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 --2- 172.21.15.55:0/2708432318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d03520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 --2- 172.21.15.55:0/2708432318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d022f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 -- 172.21.15.55:0/2708432318 >> 172.21.15.55:0/2708432318 conn(0x55c5a1cfd6b0 msgr2=0x55c5a1cfdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 -- 172.21.15.55:0/2708432318 shutdown_connections 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.785+0000 7f202176c740 1 -- 172.21.15.55:0/2708432318 wait complete. 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.786+0000 7f202176c740 1 -- start start 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.786+0000 7f202176c740 1 Processor -- start 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.787+0000 7f202176c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d1cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.787+0000 7f202176c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d1d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.787+0000 7f202176c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d03a60 0x55c5a1d24570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.787+0000 7f202176c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55c5a1c62d40 con 0x55c5a1d030a0 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.787+0000 7f202176c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c5a1bedd10 con 0x55c5a1d03a60 2024-04-26T22:01:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.787+0000 7f202176c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5a1d0a830 con 0x55c5a1d01ed0 2024-04-26T22:01:48.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.787+0000 7f202176c740 10 client.0 _ll_register_callbacks cb 0x55c5a1c1a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:48.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.788+0000 7f2017fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d1cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.788+0000 7f2017fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d1cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48630/0 (socket says 172.21.15.55:48630) 2024-04-26T22:01:48.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.788+0000 7f2017fff640 1 -- 172.21.15.55:0/4239502415 learned_addr learned my addr 172.21.15.55:0/4239502415 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:48.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.788+0000 7f20177fe640 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d1d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.788+0000 7f20177fe640 1 -- 172.21.15.55:0/4239502415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 msgr2=0x55c5a1d1cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:48.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.788+0000 7f20177fe640 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d1cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.788+0000 7f20177fe640 1 -- 172.21.15.55:0/4239502415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d03a60 msgr2=0x55c5a1d24570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:48.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.789+0000 7f20177fe640 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d03a60 0x55c5a1d24570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.789+0000 7f20177fe640 1 -- 172.21.15.55:0/4239502415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a1a427d0 con 0x55c5a1d030a0 2024-04-26T22:01:48.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.789+0000 7f20177fe640 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d1d530 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f20181a0790 tx=0x7f201000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5623196cdc5b3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.790+0000 7f1ff77fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.790+0000 7f1ff77fe640 1 -- 172.21.15.55:0/4239502415 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010015070 con 0x55c5a1d030a0 2024-04-26T22:01:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.791+0000 7f202176c740 1 -- 172.21.15.55:0/4239502415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f200c0c5e10 con 0x55c5a1d030a0 2024-04-26T22:01:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.791+0000 7f1ff77fe640 1 -- 172.21.15.55:0/4239502415 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2010011840 con 0x55c5a1d030a0 2024-04-26T22:01:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.791+0000 7f1ff77fe640 1 -- 172.21.15.55:0/4239502415 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2010021050 con 0x55c5a1d030a0 2024-04-26T22:01:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.792+0000 7f202176c740 10 client.16773 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.792+0000 7f202176c740 1 -- 172.21.15.55:0/4239502415 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c5a19783f0 con 0x55c5a1d030a0 2024-04-26T22:01:48.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.793+0000 7f1ff77fe640 1 -- 172.21.15.55:0/4239502415 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200c0c5e10 con 0x55c5a1d030a0 2024-04-26T22:01:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.793+0000 7f1ff77fe640 1 -- 172.21.15.55:0/4239502415 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2010010390 con 0x55c5a1d030a0 2024-04-26T22:01:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.793+0000 7f202176c740 10 client.16773 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.793+0000 7f202176c740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.793+0000 7f202176c740 20 client.16773 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.793+0000 7f202176c740 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5a1d28b60 0x55c5a1d2afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.793+0000 7f202176c740 4 client.16773 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.793+0000 7f202176c740 1 -- 172.21.15.55:0/4239502415 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5a1c74990 con 0x55c5a1d28b60 2024-04-26T22:01:48.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.796+0000 7f2017fff640 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5a1d28b60 0x55c5a1d2afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.812+0000 7f2017fff640 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5a1d28b60 0x55c5a1d2afc0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b1e74126c4762f server_cookie=70ff8fca7075ec30 in_seq=0 out_seq=0 2024-04-26T22:01:48.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.812+0000 7f1ff77fe640 10 client.16773 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:48.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.819+0000 7f1ff77fe640 1 -- 172.21.15.55:0/4239502415 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5a1c74990 con 0x55c5a1d28b60 2024-04-26T22:01:48.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.819+0000 7f1ff77fe640 10 client.16773 handle_command_reply: tid=0 2024-04-26T22:01:48.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.819+0000 7f202176c740 1 client.16773 shutdown 2024-04-26T22:01:48.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.819+0000 7f202176c740 1 -- 172.21.15.55:0/4239502415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 msgr2=0x55c5a1d1d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.819+0000 7f202176c740 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d1d530 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f20181a0790 tx=0x7f201000c9b0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.822+0000 7f202176c740 1 -- 172.21.15.55:0/4239502415 shutdown_connections 2024-04-26T22:01:48.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.822+0000 7f202176c740 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d03a60 0x55c5a1d24570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.822+0000 7f202176c740 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5a1d28b60 0x55c5a1d2afc0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.822+0000 7f202176c740 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d1d530 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.822+0000 7f202176c740 1 --2- 172.21.15.55:0/4239502415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d1cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.823+0000 7f202176c740 1 -- 172.21.15.55:0/4239502415 >> 172.21.15.55:0/4239502415 conn(0x55c5a1cfd6b0 msgr2=0x55c5a1cfc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.824+0000 7f1ff77fe640 0 client.16773 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:48.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.826+0000 7f202176c740 1 -- 172.21.15.55:0/4239502415 shutdown_connections 2024-04-26T22:01:48.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.827+0000 7f202176c740 1 -- 172.21.15.55:0/4239502415 wait complete. 2024-04-26T22:01:48.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.827+0000 7f202176c740 20 client.16773 trim_cache size 0 max 16384 2024-04-26T22:01:48.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.886+0000 7f202176c740 1 Processor -- start 2024-04-26T22:01:48.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.889+0000 7f202176c740 1 -- start start 2024-04-26T22:01:48.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.889+0000 7f202176c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d06c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.889+0000 7f202176c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d07150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.889+0000 7f202176c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d24530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.890+0000 7f202176c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20181aa6d0 con 0x55c5a1d030a0 2024-04-26T22:01:48.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.890+0000 7f202176c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20181a7fa0 con 0x55c5a1d01ed0 2024-04-26T22:01:48.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.890+0000 7f202176c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f201810e650 con 0x55c5a1d03a60 2024-04-26T22:01:48.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.890+0000 7f20177fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d07150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.890+0000 7f20177fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d07150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44668/0 (socket says 172.21.15.55:44668) 2024-04-26T22:01:48.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.890+0000 7f20177fe640 1 -- 172.21.15.55:0/355263819 learned_addr learned my addr 172.21.15.55:0/355263819 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:48.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.890+0000 7f201cf15640 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d24530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.890+0000 7f2017fff640 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d06c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.891+0000 7f2017fff640 1 -- 172.21.15.55:0/355263819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 msgr2=0x55c5a1d24530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:48.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.891+0000 7f2017fff640 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d24530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.891+0000 7f2017fff640 1 -- 172.21.15.55:0/355263819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 msgr2=0x55c5a1d07150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:48.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.891+0000 7f2017fff640 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d07150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.891+0000 7f2017fff640 1 -- 172.21.15.55:0/355263819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a19380f0 con 0x55c5a1d01ed0 2024-04-26T22:01:48.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.892+0000 7f2017fff640 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d06c10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f200c0b9c00 tx=0x7f200c0095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a36104e35f8e4764 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.893+0000 7f201e978640 1 -- 172.21.15.55:0/355263819 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0dc120 con 0x55c5a1d01ed0 2024-04-26T22:01:48.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.894+0000 7f201e978640 1 -- 172.21.15.55:0/355263819 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200c0c5e10 con 0x55c5a1d01ed0 2024-04-26T22:01:48.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.894+0000 7f201e978640 1 -- 172.21.15.55:0/355263819 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0bd490 con 0x55c5a1d01ed0 2024-04-26T22:01:48.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.895+0000 7f20177fe640 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d07150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:01:48.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.895+0000 7f201cf15640 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d24530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.896+0000 7f202176c740 1 -- 172.21.15.55:0/355263819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 msgr2=0x55c5a1d06c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.896+0000 7f202176c740 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d06c10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f200c0b9c00 tx=0x7f200c0095b0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.896+0000 7f202176c740 1 -- 172.21.15.55:0/355263819 shutdown_connections 2024-04-26T22:01:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.896+0000 7f202176c740 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d24530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.896+0000 7f202176c740 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d07150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.897+0000 7f202176c740 1 --2- 172.21.15.55:0/355263819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d06c10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.897+0000 7f202176c740 1 -- 172.21.15.55:0/355263819 >> 172.21.15.55:0/355263819 conn(0x55c5a1cfd6b0 msgr2=0x55c5a1d1cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.904+0000 7f202176c740 1 -- 172.21.15.55:0/355263819 shutdown_connections 2024-04-26T22:01:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.904+0000 7f202176c740 1 -- 172.21.15.55:0/355263819 wait complete. 2024-04-26T22:01:48.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.904+0000 7f202176c740 1 -- start start 2024-04-26T22:01:48.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.905+0000 7f202176c740 1 Processor -- start 2024-04-26T22:01:48.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.906+0000 7f202176c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d21430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.906+0000 7f202176c740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d21970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.906+0000 7f202176c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d8d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.906+0000 7f202176c740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f201810ecc0 con 0x55c5a1d030a0 2024-04-26T22:01:48.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.907+0000 7f2017fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d21430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.907+0000 7f2017fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d21430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37154/0 (socket says 172.21.15.55:37154) 2024-04-26T22:01:48.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.907+0000 7f2017fff640 1 -- 172.21.15.55:0/3429238867 learned_addr learned my addr 172.21.15.55:0/3429238867 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:48.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.907+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20181a78f0 con 0x55c5a1d01ed0 2024-04-26T22:01:48.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.907+0000 7f201cf15640 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d8d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.907+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f201810f390 con 0x55c5a1d03a60 2024-04-26T22:01:48.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.907+0000 7f202176c740 10 client.0 _ll_register_callbacks cb 0x7f2010008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.908+0000 7f20177fe640 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d21970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.908+0000 7f201cf15640 1 -- 172.21.15.55:0/3429238867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 msgr2=0x55c5a1d21430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.908+0000 7f201cf15640 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d21430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.908+0000 7f201cf15640 1 -- 172.21.15.55:0/3429238867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 msgr2=0x55c5a1d21970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.908+0000 7f201cf15640 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d21970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.908+0000 7f201cf15640 1 -- 172.21.15.55:0/3429238867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a1c274b0 con 0x55c5a1d03a60 2024-04-26T22:01:48.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.910+0000 7f201cf15640 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d8d130 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f20080099b0 tx=0x7f200801da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=778f02d1bdca0996 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:48.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.917+0000 7f1ff77fe640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:01:48.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.917+0000 7f1ff77fe640 1 -- 172.21.15.55:0/3429238867 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20080109c0 con 0x55c5a1d03a60 2024-04-26T22:01:48.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.917+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f200c0c5e10 con 0x55c5a1d03a60 2024-04-26T22:01:48.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.917+0000 7f1ff77fe640 1 -- 172.21.15.55:0/3429238867 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2008010b60 con 0x55c5a1d03a60 2024-04-26T22:01:48.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.917+0000 7f1ff77fe640 1 -- 172.21.15.55:0/3429238867 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2008010d60 con 0x55c5a1d03a60 2024-04-26T22:01:48.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.917+0000 7f202176c740 10 client.25942 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:48.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.917+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c5a1d8e810 con 0x55c5a1d03a60 2024-04-26T22:01:48.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.921+0000 7f1ff77fe640 1 -- 172.21.15.55:0/3429238867 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200c0c5e10 con 0x55c5a1d03a60 2024-04-26T22:01:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.923+0000 7f1ff77fe640 1 -- 172.21.15.55:0/3429238867 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2008027510 con 0x55c5a1d03a60 2024-04-26T22:01:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.923+0000 7f202176c740 10 client.25942 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.923+0000 7f202176c740 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.923+0000 7f202176c740 20 client.25942 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.923+0000 7f202176c740 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5a1d8f150 0x55c5a1d91570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:48.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.923+0000 7f202176c740 4 client.25942 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:01:48.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.923+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5a1c6e800 con 0x55c5a1d8f150 2024-04-26T22:01:48.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.925+0000 7f2017fff640 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5a1d8f150 0x55c5a1d91570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.927+0000 7f2017fff640 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5a1d8f150 0x55c5a1d91570 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f3a1d6643d919d server_cookie=be2b72903431df87 in_seq=0 out_seq=0 2024-04-26T22:01:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.927+0000 7f1ff77fe640 10 client.25942 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:48.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.933+0000 7f1ff77fe640 1 -- 172.21.15.55:0/3429238867 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c5a1c6e800 con 0x55c5a1d8f150 2024-04-26T22:01:48.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.933+0000 7f1ff77fe640 10 client.25942 handle_command_reply: tid=0 2024-04-26T22:01:48.933 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.933+0000 7f202176c740 1 client.25942 shutdown 2024-04-26T22:01:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.934+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 msgr2=0x55c5a1d8d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.934+0000 7f202176c740 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d8d130 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f20080099b0 tx=0x7f200801da80 comp rx=0 tx=0).stop 2024-04-26T22:01:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 shutdown_connections 2024-04-26T22:01:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f202176c740 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5a1d03a60 0x55c5a1d8d130 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f202176c740 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55c5a1d8f150 0x55c5a1d91570 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f202176c740 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c5a1d030a0 0x55c5a1d21970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f202176c740 1 --2- 172.21.15.55:0/3429238867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5a1d01ed0 0x55c5a1d21430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 >> 172.21.15.55:0/3429238867 conn(0x55c5a1cfd6b0 msgr2=0x55c5a1d1cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f1ff77fe640 0 client.25942 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:48.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 shutdown_connections 2024-04-26T22:01:48.939 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.937+0000 7f202176c740 1 -- 172.21.15.55:0/3429238867 wait complete. 2024-04-26T22:01:48.942 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.939+0000 7f202176c740 20 client.25942 trim_cache size 0 max 16384 2024-04-26T22:01:48.946 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:01:48.946 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:01:48.946 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:01:48.946 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 -- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20000a3dc0 msgr2=0x7f20000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20000a3dc0 0x7f20000a6280 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f20080058c0 tx=0x7f2008005850 comp rx=0 tx=0).stop 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 -- 172.21.15.55:0/3102363605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 msgr2=0x7f201819f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 0x7f201819f280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f200c005d00 tx=0x7f200c032cb0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 -- 172.21.15.55:0/3102363605 shutdown_connections 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20000a3dc0 0x7f20000a6280 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2018106520 0x7f20181a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2018103b70 0x7f201819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 --2- 172.21.15.55:0/3102363605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2018101200 0x7f201819f280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.947+0000 7f1ffeffd640 1 -- 172.21.15.55:0/3102363605 >> 172.21.15.55:0/3102363605 conn(0x7f20180fce20 msgr2=0x7f2018104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:48.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.948+0000 7f1ffeffd640 1 -- 172.21.15.55:0/3102363605 shutdown_connections 2024-04-26T22:01:48.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:48.948+0000 7f1ffeffd640 1 -- 172.21.15.55:0/3102363605 wait complete. 2024-04-26T22:01:48.990 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:01:48.991 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:01:49.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.370+0000 7fb5e39af640 1 -- 172.21.15.55:0/627793322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5dc10ec70 msgr2=0x7fb5dc10f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.370+0000 7fb5e39af640 1 --2- 172.21.15.55:0/627793322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5dc10ec70 0x7fb5dc10f120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009900 tx=0x7fb5d802a440 comp rx=0 tx=0).stop 2024-04-26T22:01:49.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.371+0000 7fb5e39af640 1 -- 172.21.15.55:0/627793322 shutdown_connections 2024-04-26T22:01:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.371+0000 7fb5e39af640 1 --2- 172.21.15.55:0/627793322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5dc10ec70 0x7fb5dc10f120 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.371+0000 7fb5e39af640 1 --2- 172.21.15.55:0/627793322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc10c2a0 0x7fb5dc10e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.371+0000 7fb5e39af640 1 --2- 172.21.15.55:0/627793322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5dc109930 0x7fb5dc10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.371+0000 7fb5e39af640 1 -- 172.21.15.55:0/627793322 >> 172.21.15.55:0/627793322 conn(0x7fb5dc06f7d0 msgr2=0x7fb5dc071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:49.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.374+0000 7fb5e39af640 1 -- 172.21.15.55:0/627793322 shutdown_connections 2024-04-26T22:01:49.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.375+0000 7fb5e39af640 1 -- 172.21.15.55:0/627793322 wait complete. 2024-04-26T22:01:49.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.376+0000 7fb5e39af640 1 Processor -- start 2024-04-26T22:01:49.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.376+0000 7fb5e39af640 1 -- start start 2024-04-26T22:01:49.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.377+0000 7fb5e39af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5dc109930 0x7fb5dc1a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.377+0000 7fb5e39af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc10c2a0 0x7fb5dc1a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.377+0000 7fb5e39af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5dc10ec70 0x7fb5dc07ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.377+0000 7fb5e39af640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc11d290 con 0x7fb5dc10ec70 2024-04-26T22:01:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.377+0000 7fb5e39af640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc11d110 con 0x7fb5dc10c2a0 2024-04-26T22:01:49.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.377+0000 7fb5e39af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5dc11d410 con 0x7fb5dc109930 2024-04-26T22:01:49.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.379+0000 7fb5e174b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5dc109930 0x7fb5dc1a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.380+0000 7fb5e1f4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5dc10ec70 0x7fb5dc07ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.380+0000 7fb5e1f4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5dc10ec70 0x7fb5dc07ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44698/0 (socket says 172.21.15.55:44698) 2024-04-26T22:01:49.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.380+0000 7fb5e1f4c640 1 -- 172.21.15.55:0/1073905094 learned_addr learned my addr 172.21.15.55:0/1073905094 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:49.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.381+0000 7fb5e0f4a640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc10c2a0 0x7fb5dc1a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.383+0000 7fb5e1f4c640 1 -- 172.21.15.55:0/1073905094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5dc109930 msgr2=0x7fb5dc1a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.383+0000 7fb5e1f4c640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5dc109930 0x7fb5dc1a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.383+0000 7fb5e1f4c640 1 -- 172.21.15.55:0/1073905094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc10c2a0 msgr2=0x7fb5dc1a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:49.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.383+0000 7fb5e1f4c640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc10c2a0 0x7fb5dc1a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:49.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.383+0000 7fb5e1f4c640 1 -- 172.21.15.55:0/1073905094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc07b230 con 0x7fb5dc10ec70 2024-04-26T22:01:49.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.385+0000 7fb5e1f4c640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5dc10ec70 0x7fb5dc07ab90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb5d802a9b0 tx=0x7fb5d8004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.390+0000 7fb5d27fc640 1 -- 172.21.15.55:0/1073905094 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8040030 con 0x7fb5dc10ec70 2024-04-26T22:01:49.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.390+0000 7fb5d27fc640 1 -- 172.21.15.55:0/1073905094 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d803cb00 con 0x7fb5dc10ec70 2024-04-26T22:01:49.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.390+0000 7fb5d27fc640 1 -- 172.21.15.55:0/1073905094 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8031b20 con 0x7fb5dc10ec70 2024-04-26T22:01:49.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.391+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5dc07b4c0 con 0x7fb5dc10ec70 2024-04-26T22:01:49.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.391+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5dc07b960 con 0x7fb5dc10ec70 2024-04-26T22:01:49.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.399+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c00052c0 con 0x7fb5dc10ec70 2024-04-26T22:01:49.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.400+0000 7fb5d27fc640 1 -- 172.21.15.55:0/1073905094 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb5d8043030 con 0x7fb5dc10ec70 2024-04-26T22:01:49.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.401+0000 7fb5d27fc640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5c80a3e40 0x7fb5c80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:49.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.403+0000 7fb5d27fc640 1 -- 172.21.15.55:0/1073905094 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb5d80f0f20 con 0x7fb5dc10ec70 2024-04-26T22:01:49.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.407+0000 7fb5e174b640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5c80a3e40 0x7fb5c80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:49.416 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.415+0000 7fb5e174b640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5c80a3e40 0x7fb5c80a6300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc1a8c80 tx=0x7fb5cc005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:49.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:49.424+0000 7fb5d27fc640 1 -- 172.21.15.55:0/1073905094 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb5d80bd700 con 0x7fb5dc10ec70 2024-04-26T22:01:49.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:49 smithi055 ceph-mon[29221]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 MiB/s wr, 1.66k op/s 2024-04-26T22:01:50.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:49 smithi064 ceph-mon[32765]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 MiB/s wr, 1.66k op/s 2024-04-26T22:01:50.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:49 smithi165 ceph-mon[33296]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 MiB/s wr, 1.66k op/s 2024-04-26T22:01:50.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.164+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5c00055b0 con 0x7fb5dc10ec70 2024-04-26T22:01:50.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.172+0000 7fb5d27fc640 1 -- 172.21.15.55:0/1073905094 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fb5d80bd000 con 0x7fb5dc10ec70 2024-04-26T22:01:50.173 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:01:50.173 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:01:50.173 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:01:50.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.190+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5c80a3e40 msgr2=0x7fb5c80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.190+0000 7fb5e39af640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5c80a3e40 0x7fb5c80a6300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc1a8c80 tx=0x7fb5cc005c60 comp rx=0 tx=0).stop 2024-04-26T22:01:50.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.190+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5dc10ec70 msgr2=0x7fb5dc07ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.190+0000 7fb5e39af640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5dc10ec70 0x7fb5dc07ab90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb5d802a9b0 tx=0x7fb5d8004200 comp rx=0 tx=0).stop 2024-04-26T22:01:50.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.193+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 shutdown_connections 2024-04-26T22:01:50.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.193+0000 7fb5e39af640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb5c80a3e40 0x7fb5c80a6300 secure :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc1a8c80 tx=0x7fb5cc005c60 comp rx=0 tx=0).stop 2024-04-26T22:01:50.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.193+0000 7fb5e39af640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5dc10ec70 0x7fb5dc07ab90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.193+0000 7fb5e39af640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5dc10c2a0 0x7fb5dc1a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.193+0000 7fb5e39af640 1 --2- 172.21.15.55:0/1073905094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5dc109930 0x7fb5dc1a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.193+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 >> 172.21.15.55:0/1073905094 conn(0x7fb5dc06f7d0 msgr2=0x7fb5dc10a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.202+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 shutdown_connections 2024-04-26T22:01:50.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.204+0000 7fb5e39af640 1 -- 172.21.15.55:0/1073905094 wait complete. 2024-04-26T22:01:50.260 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:01:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.765+0000 7fc73bb7f640 1 -- 172.21.15.55:0/812377752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 msgr2=0x7fc734110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.765+0000 7fc73bb7f640 1 --2- 172.21.15.55:0/812377752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 0x7fc734110c70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc724009930 tx=0x7fc72402e890 comp rx=0 tx=0).stop 2024-04-26T22:01:50.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.772+0000 7fc73bb7f640 1 -- 172.21.15.55:0/812377752 shutdown_connections 2024-04-26T22:01:50.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.773+0000 7fc73bb7f640 1 --2- 172.21.15.55:0/812377752 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7341111b0 0x7fc7341135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.773+0000 7fc73bb7f640 1 --2- 172.21.15.55:0/812377752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 0x7fc734110c70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.773+0000 7fc73bb7f640 1 --2- 172.21.15.55:0/812377752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc734109930 0x7fc734109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.773+0000 7fc73bb7f640 1 -- 172.21.15.55:0/812377752 >> 172.21.15.55:0/812377752 conn(0x7fc73406f7c0 msgr2=0x7fc734071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.782+0000 7fc73bb7f640 1 -- 172.21.15.55:0/812377752 shutdown_connections 2024-04-26T22:01:50.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.783+0000 7fc73bb7f640 1 -- 172.21.15.55:0/812377752 wait complete. 2024-04-26T22:01:50.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.785+0000 7fc73bb7f640 1 Processor -- start 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.788+0000 7fc73bb7f640 1 -- start start 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.788+0000 7fc73bb7f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc734109930 0x7fc7341b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.788+0000 7fc73bb7f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 0x7fc7341b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.788+0000 7fc73bb7f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7341111b0 0x7fc7341b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.788+0000 7fc73bb7f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc734125c20 con 0x7fc73410a290 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.788+0000 7fc73bb7f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc734125aa0 con 0x7fc734109930 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.788+0000 7fc73bb7f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc734125da0 con 0x7fc7341111b0 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.789+0000 7fc73911a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 0x7fc7341b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.789+0000 7fc73911a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 0x7fc7341b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42106/0 (socket says 172.21.15.55:42106) 2024-04-26T22:01:50.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.789+0000 7fc73911a640 1 -- 172.21.15.55:0/999787411 learned_addr learned my addr 172.21.15.55:0/999787411 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:50.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.790+0000 7fc73911a640 1 -- 172.21.15.55:0/999787411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7341111b0 msgr2=0x7fc7341b7900 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:01:50.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.790+0000 7fc73911a640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7341111b0 0x7fc7341b7900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.790+0000 7fc73911a640 1 -- 172.21.15.55:0/999787411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc734109930 msgr2=0x7fc7341b0350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:01:50.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.790+0000 7fc73911a640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc734109930 0x7fc7341b0350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.790+0000 7fc73911a640 1 -- 172.21.15.55:0/999787411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7341b7f70 con 0x7fc73410a290 2024-04-26T22:01:50.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.791+0000 7fc73911a640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 0x7fc7341b0890 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc724009850 tx=0x7fc7240051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.792+0000 7fc72affd640 1 -- 172.21.15.55:0/999787411 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72403a030 con 0x7fc73410a290 2024-04-26T22:01:50.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.792+0000 7fc73bb7f640 1 -- 172.21.15.55:0/999787411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341b8200 con 0x7fc73410a290 2024-04-26T22:01:50.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.792+0000 7fc73bb7f640 1 -- 172.21.15.55:0/999787411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341b8770 con 0x7fc73410a290 2024-04-26T22:01:50.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.802+0000 7fc73e975740 1 Processor -- start 2024-04-26T22:01:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.803+0000 7fc73e975740 1 -- start start 2024-04-26T22:01:50.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.804+0000 7fc73e975740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca599ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.804+0000 7fc73e975740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca599be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.804+0000 7fc73e975740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599c3b0 0x55cca59a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.804+0000 7fc73e975740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cca58fed80 con 0x55cca599ba10 2024-04-26T22:01:50.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.804+0000 7fc73e975740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cca58b8a20 con 0x55cca599c3b0 2024-04-26T22:01:50.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.804+0000 7fc73e975740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cca58fad40 con 0x55cca599a7f0 2024-04-26T22:01:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.805+0000 7fc72affd640 1 -- 172.21.15.55:0/999787411 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc724043070 con 0x7fc73410a290 2024-04-26T22:01:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.805+0000 7fc72affd640 1 -- 172.21.15.55:0/999787411 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72403e710 con 0x7fc73410a290 2024-04-26T22:01:50.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.809+0000 7fc73911a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599c3b0 0x55cca59a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.812+0000 7fc73911a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599c3b0 0x55cca59a2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37580/0 (socket says 172.21.15.55:37580) 2024-04-26T22:01:50.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.812+0000 7fc72affd640 1 -- 172.21.15.55:0/999787411 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc72403e8b0 con 0x7fc73410a290 2024-04-26T22:01:50.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.814+0000 7fc72affd640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7200a3dc0 0x7fc7200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.814+0000 7fc73911a640 1 -- 172.21.15.55:0/577357539 learned_addr learned my addr 172.21.15.55:0/577357539 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:50.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.816+0000 7fc73a11c640 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca599ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.823+0000 7fc73991b640 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca599be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.823+0000 7fc73a11c640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7200a3dc0 0x7fc7200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.831+0000 7fc73a11c640 1 -- 172.21.15.55:0/577357539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599c3b0 msgr2=0x55cca59a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.831+0000 7fc73a11c640 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599c3b0 0x55cca59a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.831+0000 7fc73a11c640 1 -- 172.21.15.55:0/577357539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 msgr2=0x55cca599be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:50.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.831+0000 7fc73a11c640 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca599be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.831+0000 7fc73a11c640 1 -- 172.21.15.55:0/577357539 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca58c3c30 con 0x55cca599a7f0 2024-04-26T22:01:50.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.837+0000 7fc73a11c640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7200a3dc0 0x7fc7200a6280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc73406c650 tx=0x7fc73000d2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.838+0000 7fc72affd640 1 -- 172.21.15.55:0/999787411 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc7240bfa30 con 0x7fc73410a290 2024-04-26T22:01:50.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.842+0000 7fc73a11c640 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca599ac10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc73000da40 tx=0x7fc730019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfdc7b83e908ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.848+0000 7fc73bb7f640 1 -- 172.21.15.55:0/577357539 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730024070 con 0x55cca599a7f0 2024-04-26T22:01:50.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.848+0000 7fc73bb7f640 1 -- 172.21.15.55:0/577357539 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc730010050 con 0x55cca599a7f0 2024-04-26T22:01:50.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.848+0000 7fc73bb7f640 1 -- 172.21.15.55:0/577357539 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73001b070 con 0x55cca599a7f0 2024-04-26T22:01:50.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.852+0000 7fc73e975740 1 -- 172.21.15.55:0/577357539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 msgr2=0x55cca599ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.852+0000 7fc73e975740 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca599ac10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc73000da40 tx=0x7fc730019000 comp rx=0 tx=0).stop 2024-04-26T22:01:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.855+0000 7fc73e975740 1 -- 172.21.15.55:0/577357539 shutdown_connections 2024-04-26T22:01:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.855+0000 7fc73e975740 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599c3b0 0x55cca59a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.855+0000 7fc73e975740 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca599be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.855+0000 7fc73e975740 1 --2- 172.21.15.55:0/577357539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca599ac10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.855+0000 7fc73e975740 1 -- 172.21.15.55:0/577357539 >> 172.21.15.55:0/577357539 conn(0x55cca58db910 msgr2=0x55cca5996420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.855+0000 7fc73e975740 1 -- 172.21.15.55:0/577357539 shutdown_connections 2024-04-26T22:01:50.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.855+0000 7fc73e975740 1 -- 172.21.15.55:0/577357539 wait complete. 2024-04-26T22:01:50.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.856+0000 7fc73e975740 1 -- start start 2024-04-26T22:01:50.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.856+0000 7fc73e975740 1 Processor -- start 2024-04-26T22:01:50.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.857+0000 7fc73e975740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca59a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.857+0000 7fc73e975740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599ba10 0x55cca59a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.857+0000 7fc73e975740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.857+0000 7fc73e975740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55cca59a3380 con 0x55cca599c3b0 2024-04-26T22:01:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.857+0000 7fc73e975740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cca5885dc0 con 0x55cca599ba10 2024-04-26T22:01:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.857+0000 7fc73e975740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cca59a3500 con 0x55cca599a7f0 2024-04-26T22:01:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.857+0000 7fc73e975740 10 client.0 _ll_register_callbacks cb 0x55cca58a1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.858+0000 7fc73a11c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.858+0000 7fc73a11c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59ac0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42126/0 (socket says 172.21.15.55:42126) 2024-04-26T22:01:50.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.858+0000 7fc73a11c640 1 -- 172.21.15.55:0/2122979169 learned_addr learned my addr 172.21.15.55:0/2122979169 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:50.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.860+0000 7fc73a11c640 1 -- 172.21.15.55:0/2122979169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 msgr2=0x55cca59a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:50.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.860+0000 7fc73a11c640 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca59a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.860+0000 7fc73a11c640 1 -- 172.21.15.55:0/2122979169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599ba10 msgr2=0x55cca59a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:50.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.860+0000 7fc73a11c640 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599ba10 0x55cca59a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.860+0000 7fc73a11c640 1 -- 172.21.15.55:0/2122979169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca56103f0 con 0x55cca599c3b0 2024-04-26T22:01:50.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.861+0000 7fc73a11c640 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59ac0a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc730012a70 tx=0x7fc73000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc72104d575df2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.864+0000 7fc71d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:50.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.864+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2122979169 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73002d060 con 0x55cca599c3b0 2024-04-26T22:01:50.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.865+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2122979169 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc730005d80 con 0x55cca599c3b0 2024-04-26T22:01:50.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.867+0000 7fc73e975740 1 -- 172.21.15.55:0/2122979169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc730010050 con 0x55cca599c3b0 2024-04-26T22:01:50.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.872+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2122979169 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc730024070 con 0x55cca599c3b0 2024-04-26T22:01:50.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.873+0000 7fc73e975740 10 client.16794 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:50.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.873+0000 7fc73e975740 1 -- 172.21.15.55:0/2122979169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cca56da7d0 con 0x55cca599c3b0 2024-04-26T22:01:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.880+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2122979169 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc730010050 con 0x55cca599c3b0 2024-04-26T22:01:50.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.880+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2122979169 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc730015070 con 0x55cca599c3b0 2024-04-26T22:01:50.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.881+0000 7fc73e975740 10 client.16794 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:50.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.881+0000 7fc73e975740 10 client.16794 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:01:50.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.881+0000 7fc73e975740 20 client.16794 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:50.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.881+0000 7fc73e975740 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55cca59b0cd0 0x55cca59b3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.881+0000 7fc73e975740 4 client.16794 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:50.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.881+0000 7fc73e975740 1 -- 172.21.15.55:0/2122979169 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cca590c990 con 0x55cca59b0cd0 2024-04-26T22:01:50.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.886+0000 7fc73991b640 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55cca59b0cd0 0x55cca59b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.897+0000 7fc73991b640 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55cca59b0cd0 0x55cca59b3130 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4f3857b4bb53cf1 server_cookie=71a147e6606f101 in_seq=0 out_seq=0 2024-04-26T22:01:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.898+0000 7fc71d7fa640 10 client.16794 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:01:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.902+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2122979169 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cca590c990 con 0x55cca59b0cd0 2024-04-26T22:01:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.902+0000 7fc71d7fa640 10 client.16794 handle_command_reply: tid=0 2024-04-26T22:01:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.902+0000 7fc73e975740 1 client.16794 shutdown 2024-04-26T22:01:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.904+0000 7fc73e975740 1 -- 172.21.15.55:0/2122979169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 msgr2=0x55cca59ac0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.904+0000 7fc73e975740 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59ac0a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc730012a70 tx=0x7fc73000df60 comp rx=0 tx=0).stop 2024-04-26T22:01:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.904+0000 7fc73e975740 1 -- 172.21.15.55:0/2122979169 shutdown_connections 2024-04-26T22:01:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.904+0000 7fc73e975740 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55cca59b0cd0 0x55cca59b3130 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.904+0000 7fc73e975740 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59ac0a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.904+0000 7fc73e975740 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599ba10 0x55cca59a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.904+0000 7fc73e975740 1 --2- 172.21.15.55:0/2122979169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca59a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.904+0000 7fc73e975740 1 -- 172.21.15.55:0/2122979169 >> 172.21.15.55:0/2122979169 conn(0x55cca58db910 msgr2=0x55cca5995580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.905+0000 7fc71d7fa640 0 client.16794 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:01:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.905+0000 7fc73e975740 1 -- 172.21.15.55:0/2122979169 shutdown_connections 2024-04-26T22:01:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.909+0000 7fc73e975740 1 -- 172.21.15.55:0/2122979169 wait complete. 2024-04-26T22:01:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.912+0000 7fc73e975740 20 client.16794 trim_cache size 0 max 16384 2024-04-26T22:01:50.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:50 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1073905094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:50.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.959+0000 7fc73e975740 1 Processor -- start 2024-04-26T22:01:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.959+0000 7fc73e975740 1 -- start start 2024-04-26T22:01:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.959+0000 7fc73e975740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca59965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.959+0000 7fc73e975740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599ba10 0x55cca59ac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.959+0000 7fc73e975740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.960+0000 7fc73e975740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc718010f40 con 0x55cca599c3b0 2024-04-26T22:01:50.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.960+0000 7fc73e975740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc71800e810 con 0x55cca599ba10 2024-04-26T22:01:50.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.960+0000 7fc73e975740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc71800c660 con 0x55cca599a7f0 2024-04-26T22:01:50.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.960+0000 7fc73a11c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.960+0000 7fc73a11c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42128/0 (socket says 172.21.15.55:42128) 2024-04-26T22:01:50.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.960+0000 7fc73a11c640 1 -- 172.21.15.55:0/3206131680 learned_addr learned my addr 172.21.15.55:0/3206131680 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:50.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.960+0000 7fc73a11c640 1 -- 172.21.15.55:0/3206131680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 msgr2=0x55cca59965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:50.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.960+0000 7fc73991b640 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca59965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc73a11c640 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca59965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc73a11c640 1 -- 172.21.15.55:0/3206131680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599ba10 msgr2=0x55cca59ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:50.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc73a11c640 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599ba10 0x55cca59ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc73a11c640 1 -- 172.21.15.55:0/3206131680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cca59970d0 con 0x55cca599c3b0 2024-04-26T22:01:50.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc73991b640 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca59965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:01:50.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc73a11c640 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59992e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc73000e010 tx=0x7fc7300166b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cbe24099b70e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc71f7fe640 1 -- 172.21.15.55:0/3206131680 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73002d040 con 0x55cca599c3b0 2024-04-26T22:01:50.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc71f7fe640 1 -- 172.21.15.55:0/3206131680 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc730006e50 con 0x55cca599c3b0 2024-04-26T22:01:50.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.961+0000 7fc71f7fe640 1 -- 172.21.15.55:0/3206131680 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730037080 con 0x55cca599c3b0 2024-04-26T22:01:50.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.963+0000 7fc73e975740 1 -- 172.21.15.55:0/3206131680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 msgr2=0x55cca59992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.963+0000 7fc73e975740 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59992e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc73000e010 tx=0x7fc7300166b0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.963+0000 7fc73e975740 1 -- 172.21.15.55:0/3206131680 shutdown_connections 2024-04-26T22:01:50.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.963+0000 7fc73e975740 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599c3b0 0x55cca59992e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.963+0000 7fc73e975740 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599ba10 0x55cca59ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.963+0000 7fc73e975740 1 --2- 172.21.15.55:0/3206131680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599a7f0 0x55cca59965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.963+0000 7fc73e975740 1 -- 172.21.15.55:0/3206131680 >> 172.21.15.55:0/3206131680 conn(0x55cca58db910 msgr2=0x55cca5995ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.964+0000 7fc73e975740 1 -- 172.21.15.55:0/3206131680 shutdown_connections 2024-04-26T22:01:50.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.964+0000 7fc73e975740 1 -- 172.21.15.55:0/3206131680 wait complete. 2024-04-26T22:01:50.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.965+0000 7fc73e975740 1 -- start start 2024-04-26T22:01:50.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.965+0000 7fc73e975740 1 Processor -- start 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.965+0000 7fc73e975740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599a7f0 0x55cca59ac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.965+0000 7fc73e975740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca59a8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.965+0000 7fc73e975740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599c3b0 0x55cca59a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73e975740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc71800ccd0 con 0x55cca599ba10 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73e975740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc71800e140 con 0x55cca599a7f0 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73e975740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc71800d3a0 con 0x55cca599c3b0 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73e975740 10 client.0 _ll_register_callbacks cb 0x7fc730022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73911a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca59a8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73911a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca59a8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42140/0 (socket says 172.21.15.55:42140) 2024-04-26T22:01:50.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73911a640 1 -- 172.21.15.55:0/2096841719 learned_addr learned my addr 172.21.15.55:0/2096841719 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:50.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73911a640 1 -- 172.21.15.55:0/2096841719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599c3b0 msgr2=0x55cca59a9180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:50.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.966+0000 7fc73911a640 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599c3b0 0x55cca59a9180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.967+0000 7fc73911a640 1 -- 172.21.15.55:0/2096841719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599a7f0 msgr2=0x55cca59ac060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:50.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.967+0000 7fc73911a640 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599a7f0 0x55cca59ac060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.967+0000 7fc73911a640 1 -- 172.21.15.55:0/2096841719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cca5a15320 con 0x55cca599ba10 2024-04-26T22:01:50.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.967+0000 7fc73911a640 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca59a8b50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc724003f20 tx=0x7fc7240c1f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591d3f3839cf91d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:50.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.967+0000 7fc71d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:50.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.968+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2096841719 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240dd070 con 0x55cca599ba10 2024-04-26T22:01:50.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.968+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2096841719 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc7240e0b50 con 0x55cca599ba10 2024-04-26T22:01:50.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.968+0000 7fc73e975740 1 -- 172.21.15.55:0/2096841719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc730006e50 con 0x55cca599ba10 2024-04-26T22:01:50.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.968+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2096841719 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc7240da020 con 0x55cca599ba10 2024-04-26T22:01:50.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.968+0000 7fc73e975740 10 client.16800 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:50.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.969+0000 7fc73e975740 1 -- 172.21.15.55:0/2096841719 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cca5a16600 con 0x55cca599ba10 2024-04-26T22:01:50.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.969+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2096841719 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc730006e50 con 0x55cca599ba10 2024-04-26T22:01:50.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.969+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2096841719 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc7240e0020 con 0x55cca599ba10 2024-04-26T22:01:50.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.970+0000 7fc73e975740 10 client.16800 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:50.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.970+0000 7fc73e975740 10 client.16800 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:01:50.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.970+0000 7fc73e975740 20 client.16800 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:50.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.970+0000 7fc73e975740 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55cca5a16fe0 0x55cca5a19400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:50.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.970+0000 7fc73e975740 4 client.16800 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:01:50.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.970+0000 7fc73e975740 1 -- 172.21.15.55:0/2096841719 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cca5906800 con 0x55cca5a16fe0 2024-04-26T22:01:50.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.971+0000 7fc73991b640 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55cca5a16fe0 0x55cca5a19400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:50.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.975+0000 7fc73991b640 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55cca5a16fe0 0x55cca5a19400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a89d3ab7329b803 server_cookie=d70773dff8768bce in_seq=0 out_seq=0 2024-04-26T22:01:50.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.975+0000 7fc71d7fa640 10 client.16800 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:01:50.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.976+0000 7fc71d7fa640 1 -- 172.21.15.55:0/2096841719 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cca5906800 con 0x55cca5a16fe0 2024-04-26T22:01:50.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.976+0000 7fc71d7fa640 10 client.16800 handle_command_reply: tid=0 2024-04-26T22:01:50.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.976+0000 7fc73e975740 1 client.16800 shutdown 2024-04-26T22:01:50.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.976+0000 7fc73e975740 1 -- 172.21.15.55:0/2096841719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 msgr2=0x55cca59a8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.976+0000 7fc73e975740 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca59a8b50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc724003f20 tx=0x7fc7240c1f60 comp rx=0 tx=0).stop 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.976+0000 7fc73e975740 1 -- 172.21.15.55:0/2096841719 shutdown_connections 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.976+0000 7fc73e975740 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55cca5a16fe0 0x55cca5a19400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.977+0000 7fc73e975740 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cca599c3b0 0x55cca59a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.977+0000 7fc71d7fa640 0 client.16800 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.977+0000 7fc73e975740 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55cca599ba10 0x55cca59a8b50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.977+0000 7fc73e975740 1 --2- 172.21.15.55:0/2096841719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cca599a7f0 0x55cca59ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.977+0000 7fc73e975740 1 -- 172.21.15.55:0/2096841719 >> 172.21.15.55:0/2096841719 conn(0x55cca58db910 msgr2=0x55cca5999b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.977+0000 7fc73e975740 1 -- 172.21.15.55:0/2096841719 shutdown_connections 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.977+0000 7fc73e975740 1 -- 172.21.15.55:0/2096841719 wait complete. 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.977+0000 7fc73e975740 20 client.16800 trim_cache size 0 max 16384 2024-04-26T22:01:50.979 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:01:50.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 -- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7200a3dc0 msgr2=0x7fc7200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7200a3dc0 0x7fc7200a6280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc73406c650 tx=0x7fc73000d2d0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 -- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 msgr2=0x7fc7341b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:50.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 0x7fc7341b0890 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc724009850 tx=0x7fc7240051a0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 -- 172.21.15.55:0/999787411 shutdown_connections 2024-04-26T22:01:50.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc7200a3dc0 0x7fc7200a6280 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7341111b0 0x7fc7341b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73410a290 0x7fc7341b0890 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 --2- 172.21.15.55:0/999787411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc734109930 0x7fc7341b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:50.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 -- 172.21.15.55:0/999787411 >> 172.21.15.55:0/999787411 conn(0x7fc73406f7c0 msgr2=0x7fc734070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:50.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.978+0000 7fc728bff640 1 -- 172.21.15.55:0/999787411 shutdown_connections 2024-04-26T22:01:50.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:50.979+0000 7fc728bff640 1 -- 172.21.15.55:0/999787411 wait complete. 2024-04-26T22:01:50.999 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:01:51.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:50 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1073905094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:51.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:50 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1073905094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:01:51.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.304+0000 7fb0ef1be640 1 -- 172.21.15.55:0/71494233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e810a8d0 msgr2=0x7fb0e810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.304+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/71494233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e810a8d0 0x7fb0e810f240 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009900 tx=0x7fb0d802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.315+0000 7fb0ef1be640 1 -- 172.21.15.55:0/71494233 shutdown_connections 2024-04-26T22:01:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.315+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/71494233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e810a8d0 0x7fb0e810f240 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.315+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/71494233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0e8109f10 0x7fb0e810a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.315+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/71494233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e8073c10 0x7fb0e8074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.315+0000 7fb0ef1be640 1 -- 172.21.15.55:0/71494233 >> 172.21.15.55:0/71494233 conn(0x7fb0e806f780 msgr2=0x7fb0e8071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.318+0000 7fb0ef1be640 1 -- 172.21.15.55:0/71494233 shutdown_connections 2024-04-26T22:01:51.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.320+0000 7fb0ef1be640 1 -- 172.21.15.55:0/71494233 wait complete. 2024-04-26T22:01:51.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.321+0000 7fb0ef1be640 1 Processor -- start 2024-04-26T22:01:51.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.321+0000 7fb0ef1be640 1 -- start start 2024-04-26T22:01:51.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.322+0000 7fb0ef1be640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e8073c10 0x7fb0e81a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.322+0000 7fb0ef1be640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0e8109f10 0x7fb0e81a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.322+0000 7fb0ef1be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e810a8d0 0x7fb0e81aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.322+0000 7fb0ef1be640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e811d0e0 con 0x7fb0e8073c10 2024-04-26T22:01:51.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.322+0000 7fb0ef1be640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e811cf60 con 0x7fb0e810a8d0 2024-04-26T22:01:51.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.322+0000 7fb0ef1be640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e811d260 con 0x7fb0e8109f10 2024-04-26T22:01:51.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.323+0000 7fb0ee1bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e8073c10 0x7fb0e81a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.325+0000 7fb0ee1bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e8073c10 0x7fb0e81a7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42152/0 (socket says 172.21.15.55:42152) 2024-04-26T22:01:51.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.325+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/2969062081 learned_addr learned my addr 172.21.15.55:0/2969062081 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:51.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.326+0000 7fb0ed9bb640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0e8109f10 0x7fb0e81a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.327+0000 7fb0ee9bd640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e810a8d0 0x7fb0e81aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.331+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/2969062081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0e8109f10 msgr2=0x7fb0e81a7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.331+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0e8109f10 0x7fb0e81a7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.331+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/2969062081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e810a8d0 msgr2=0x7fb0e81aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.331+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e810a8d0 0x7fb0e81aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.331+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/2969062081 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e81af360 con 0x7fb0e8073c10 2024-04-26T22:01:51.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.331+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e8073c10 0x7fb0e81a7740 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000eab0 tx=0x7fb0e000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.339+0000 7fb0df7fe640 1 -- 172.21.15.55:0/2969062081 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000ce40 con 0x7fb0e8073c10 2024-04-26T22:01:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.339+0000 7fb0df7fe640 1 -- 172.21.15.55:0/2969062081 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e0011920 con 0x7fb0e8073c10 2024-04-26T22:01:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.339+0000 7fb0df7fe640 1 -- 172.21.15.55:0/2969062081 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e001aaa0 con 0x7fb0e8073c10 2024-04-26T22:01:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.340+0000 7fb0ef1be640 1 -- 172.21.15.55:0/2969062081 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e81af650 con 0x7fb0e8073c10 2024-04-26T22:01:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.340+0000 7fb0ef1be640 1 -- 172.21.15.55:0/2969062081 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81afb50 con 0x7fb0e8073c10 2024-04-26T22:01:51.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.350+0000 7fb0df7fe640 1 -- 172.21.15.55:0/2969062081 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb0e001ac40 con 0x7fb0e8073c10 2024-04-26T22:01:51.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.351+0000 7fb0f3215740 1 Processor -- start 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.353+0000 7fb0df7fe640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0d40a3e40 0x7fb0d40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.353+0000 7fb0df7fe640 1 -- 172.21.15.55:0/2969062081 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb0e00d0ef0 con 0x7fb0e8073c10 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.356+0000 7fb0ee9bd640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0d40a3e40 0x7fb0d40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.356+0000 7fb0f3215740 1 -- start start 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.357+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.357+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c9c00 0x5647eb9ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.357+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.357+0000 7fb0f3215740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5647eb92cd80 con 0x5647eb9c9c00 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.357+0000 7fb0f3215740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5647eb8e6a20 con 0x5647eb9ca5c0 2024-04-26T22:01:51.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.357+0000 7fb0f3215740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5647eb928d40 con 0x5647eb9c8a30 2024-04-26T22:01:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.359+0000 7fb0ee9bd640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0d40a3e40 0x7fb0d40a6300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8031500 tx=0x7fb0d80041a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.359+0000 7fb0ee9bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.359+0000 7fb0ee9bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37648/0 (socket says 172.21.15.55:37648) 2024-04-26T22:01:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.359+0000 7fb0ee9bd640 1 -- 172.21.15.55:0/2154149083 learned_addr learned my addr 172.21.15.55:0/2154149083 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.361+0000 7fb0ed9bb640 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c9c00 0x5647eb9ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.361+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.363+0000 7fb0ee9bd640 1 -- 172.21.15.55:0/2154149083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 msgr2=0x5647eb9c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.363+0000 7fb0ee9bd640 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.363+0000 7fb0ee9bd640 1 -- 172.21.15.55:0/2154149083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c9c00 msgr2=0x5647eb9ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.363+0000 7fb0ee9bd640 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c9c00 0x5647eb9ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.363+0000 7fb0ee9bd640 1 -- 172.21.15.55:0/2154149083 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647eb8f1c30 con 0x5647eb9ca5c0 2024-04-26T22:01:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.364+0000 7fb0ee9bd640 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d0e50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8033000 tx=0x7fb0d8041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f157132f554c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.375+0000 7fb0dcbfe640 1 -- 172.21.15.55:0/2154149083 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8035050 con 0x5647eb9ca5c0 2024-04-26T22:01:51.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.375+0000 7fb0dcbfe640 1 -- 172.21.15.55:0/2154149083 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d8004d70 con 0x5647eb9ca5c0 2024-04-26T22:01:51.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.376+0000 7fb0f3215740 1 -- 172.21.15.55:0/2154149083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 msgr2=0x5647eb9d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.376+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d0e50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8033000 tx=0x7fb0d8041000 comp rx=0 tx=0).stop 2024-04-26T22:01:51.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.376+0000 7fb0f3215740 1 -- 172.21.15.55:0/2154149083 shutdown_connections 2024-04-26T22:01:51.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.376+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d0e50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.376+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c9c00 0x5647eb9ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.376+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2154149083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.376+0000 7fb0f3215740 1 -- 172.21.15.55:0/2154149083 >> 172.21.15.55:0/2154149083 conn(0x5647eb909910 msgr2=0x5647eb9c4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.377+0000 7fb0f3215740 1 -- 172.21.15.55:0/2154149083 shutdown_connections 2024-04-26T22:01:51.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.377+0000 7fb0f3215740 1 -- 172.21.15.55:0/2154149083 wait complete. 2024-04-26T22:01:51.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.377+0000 7fb0f3215740 1 -- start start 2024-04-26T22:01:51.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.378+0000 7fb0f3215740 1 Processor -- start 2024-04-26T22:01:51.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.378+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.378+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.378+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9ca5c0 0x5647eb9da1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.378+0000 7fb0f3215740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5647eb9d15c0 con 0x5647eb9ca5c0 2024-04-26T22:01:51.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.378+0000 7fb0f3215740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5647eb8b3dc0 con 0x5647eb9c9c00 2024-04-26T22:01:51.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.378+0000 7fb0f3215740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5647eb9d1740 con 0x5647eb9c8a30 2024-04-26T22:01:51.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.378+0000 7fb0f3215740 10 client.0 _ll_register_callbacks cb 0x5647eb8cf600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:51.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.380+0000 7fb0ee1bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.380+0000 7fb0ee1bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9d2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:43132/0 (socket says 172.21.15.55:43132) 2024-04-26T22:01:51.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.380+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/2565162685 learned_addr learned my addr 172.21.15.55:0/2565162685 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:51.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.380+0000 7fb0ed9bb640 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.381+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/2565162685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 msgr2=0x5647eb9d3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.381+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.381+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/2565162685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9ca5c0 msgr2=0x5647eb9da1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.381+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9ca5c0 0x5647eb9da1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.381+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/2565162685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647eb63e3f0 con 0x5647eb9c8a30 2024-04-26T22:01:51.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.383+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9d2c20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c700 tx=0x7fb0e009cfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99350eb26fddffcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.383+0000 7fb0d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:01:51.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.384+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/2565162685 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00b5070 con 0x5647eb9c8a30 2024-04-26T22:01:51.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.385+0000 7fb0f3215740 1 -- 172.21.15.55:0/2565162685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d8004d70 con 0x5647eb9c8a30 2024-04-26T22:01:51.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.385+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/2565162685 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb0e00b6270 con 0x5647eb9c8a30 2024-04-26T22:01:51.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.389+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/2565162685 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb0e00b6470 con 0x5647eb9c8a30 2024-04-26T22:01:51.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.390+0000 7fb0f3215740 10 client.25957 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:51.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.390+0000 7fb0f3215740 1 -- 172.21.15.55:0/2565162685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5647eb7087d0 con 0x5647eb9c8a30 2024-04-26T22:01:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.400+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/2565162685 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d8004d70 con 0x5647eb9c8a30 2024-04-26T22:01:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.400+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/2565162685 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb0e0022070 con 0x5647eb9c8a30 2024-04-26T22:01:51.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.400+0000 7fb0f3215740 10 client.25957 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.400+0000 7fb0f3215740 10 client.25957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:01:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.400+0000 7fb0f3215740 20 client.25957 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.400+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5647eb9f0050 0x5647eb9f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.400+0000 7fb0f3215740 4 client.25957 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:51.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.400+0000 7fb0f3215740 1 -- 172.21.15.55:0/2565162685 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647eb93a990 con 0x5647eb9f0050 2024-04-26T22:01:51.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.409+0000 7fb0ed9bb640 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5647eb9f0050 0x5647eb9f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.433+0000 7fb0d1ffb640 10 client.25957 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:01:51.433 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.433+0000 7fb0ed9bb640 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5647eb9f0050 0x5647eb9f24d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=214da69c5cb4a28d server_cookie=81164cdf39caa56a in_seq=0 out_seq=0 2024-04-26T22:01:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.441+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/2565162685 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647eb93a990 con 0x5647eb9f0050 2024-04-26T22:01:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.441+0000 7fb0d1ffb640 10 client.25957 handle_command_reply: tid=0 2024-04-26T22:01:51.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.441+0000 7fb0f3215740 1 client.25957 shutdown 2024-04-26T22:01:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0f3215740 1 -- 172.21.15.55:0/2565162685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 msgr2=0x5647eb9d2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9d2c20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c700 tx=0x7fb0e009cfd0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.447 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0f3215740 1 -- 172.21.15.55:0/2565162685 shutdown_connections 2024-04-26T22:01:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5647eb9f0050 0x5647eb9f24d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9ca5c0 0x5647eb9da1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0f3215740 1 --2- 172.21.15.55:0/2565162685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c8a30 0x5647eb9d2c20 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c700 tx=0x7fb0e009cfd0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0f3215740 1 -- 172.21.15.55:0/2565162685 >> 172.21.15.55:0/2565162685 conn(0x5647eb909910 msgr2=0x5647eb9c33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.447+0000 7fb0d1ffb640 0 client.25957 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:01:51.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.452+0000 7fb0f3215740 1 -- 172.21.15.55:0/2565162685 shutdown_connections 2024-04-26T22:01:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.454+0000 7fb0f3215740 1 -- 172.21.15.55:0/2565162685 wait complete. 2024-04-26T22:01:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.457+0000 7fb0f3215740 20 client.25957 trim_cache size 0 max 16384 2024-04-26T22:01:51.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.516+0000 7fb0f3215740 1 Processor -- start 2024-04-26T22:01:51.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.520+0000 7fb0f3215740 1 -- start start 2024-04-26T22:01:51.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.521+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.521+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9ca5c0 0x5647eb9da130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.521+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.522+0000 7fb0f3215740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc010f80 con 0x5647eb9c4640 2024-04-26T22:01:51.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.522+0000 7fb0f3215740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc00e850 con 0x5647eb9c9c00 2024-04-26T22:01:51.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.522+0000 7fb0f3215740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc00c6a0 con 0x5647eb9ca5c0 2024-04-26T22:01:51.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.524+0000 7fb0ed9bb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9ca5c0 0x5647eb9da130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.525+0000 7fb0ee1bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.525+0000 7fb0ee1bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37658/0 (socket says 172.21.15.55:37658) 2024-04-26T22:01:51.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.525+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/1225978839 learned_addr learned my addr 172.21.15.55:0/1225978839 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:51.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.530+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/1225978839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9ca5c0 msgr2=0x5647eb9da130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.530+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/1225978839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9ca5c0 0x5647eb9da130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.532 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.530+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/1225978839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 msgr2=0x5647eb9c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.530+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/1225978839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.530+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/1225978839 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647eb9f2b50 con 0x5647eb9c9c00 2024-04-26T22:01:51.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.538+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/1225978839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d2f40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0099430 tx=0x7fb0e00094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f8c9585ef2d43f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.542+0000 7fb0d3fff640 1 -- 172.21.15.55:0/1225978839 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00b5070 con 0x5647eb9c9c00 2024-04-26T22:01:51.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.542+0000 7fb0d3fff640 1 -- 172.21.15.55:0/1225978839 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e00cea50 con 0x5647eb9c9c00 2024-04-26T22:01:51.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.542+0000 7fb0d3fff640 1 -- 172.21.15.55:0/1225978839 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0097b40 con 0x5647eb9c9c00 2024-04-26T22:01:51.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.544+0000 7fb0f3215740 1 -- 172.21.15.55:0/1225978839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 msgr2=0x5647eb9d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.544+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1225978839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d2f40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0099430 tx=0x7fb0e00094c0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.548+0000 7fb0f3215740 1 -- 172.21.15.55:0/1225978839 shutdown_connections 2024-04-26T22:01:51.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.548+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1225978839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.548+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1225978839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9ca5c0 0x5647eb9da130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.548+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1225978839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9c9c00 0x5647eb9d2f40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.548+0000 7fb0f3215740 1 -- 172.21.15.55:0/1225978839 >> 172.21.15.55:0/1225978839 conn(0x5647eb909910 msgr2=0x5647eb9d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.551+0000 7fb0f3215740 1 -- 172.21.15.55:0/1225978839 shutdown_connections 2024-04-26T22:01:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.555+0000 7fb0f3215740 1 -- 172.21.15.55:0/1225978839 wait complete. 2024-04-26T22:01:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.555+0000 7fb0f3215740 1 -- start start 2024-04-26T22:01:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.555+0000 7fb0f3215740 1 Processor -- start 2024-04-26T22:01:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.556+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9d7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.556+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c9c00 0x5647eb9d8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.556+0000 7fb0f3215740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.556+0000 7fb0f3215740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc00cd10 con 0x5647eb9c4640 2024-04-26T22:01:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.556+0000 7fb0f3215740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc00e180 con 0x5647eb9ca5c0 2024-04-26T22:01:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.556+0000 7fb0f3215740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc00d3e0 con 0x5647eb9c9c00 2024-04-26T22:01:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.556+0000 7fb0f3215740 10 client.0 _ll_register_callbacks cb 0x7fb0e001a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.557+0000 7fb0ee1bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9d7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.557+0000 7fb0ee1bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9d7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42200/0 (socket says 172.21.15.55:42200) 2024-04-26T22:01:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.557+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/1108042294 learned_addr learned my addr 172.21.15.55:0/1108042294 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.559+0000 7fb0ed9bb640 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c9c00 0x5647eb9d8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.561+0000 7fb0ee9bd640 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.564+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/1108042294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c9c00 msgr2=0x5647eb9d8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.564+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c9c00 0x5647eb9d8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.564+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/1108042294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 msgr2=0x5647eb9d9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:51.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.564+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.564+0000 7fb0ee1bc640 1 -- 172.21.15.55:0/1108042294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647eb9d9720 con 0x5647eb9c4640 2024-04-26T22:01:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.565+0000 7fb0ee1bc640 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9d7c50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0099950 tx=0x7fb0e00992f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc5943e6bfe55c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.565+0000 7fb0d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:51.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.570+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/1108042294 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00ad070 con 0x5647eb9c4640 2024-04-26T22:01:51.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.570+0000 7fb0f3215740 1 -- 172.21.15.55:0/1108042294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e00cea50 con 0x5647eb9c4640 2024-04-26T22:01:51.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.570+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/1108042294 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb0e0098d20 con 0x5647eb9c4640 2024-04-26T22:01:51.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.575+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/1108042294 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb0e00b5050 con 0x5647eb9c4640 2024-04-26T22:01:51.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.576+0000 7fb0f3215740 10 client.16812 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:51.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.576+0000 7fb0f3215740 1 -- 172.21.15.55:0/1108042294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5647eba56a00 con 0x5647eb9c4640 2024-04-26T22:01:51.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.587+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/1108042294 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e00cea50 con 0x5647eb9c4640 2024-04-26T22:01:51.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.587+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/1108042294 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb0e0022070 con 0x5647eb9c4640 2024-04-26T22:01:51.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.589+0000 7fb0f3215740 10 client.16812 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:51.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.589+0000 7fb0f3215740 10 client.16812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:01:51.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.589+0000 7fb0f3215740 20 client.16812 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:51.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.589+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5647eba57100 0x5647eba594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:51.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.589+0000 7fb0f3215740 4 client.16812 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:01:51.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.589+0000 7fb0f3215740 1 -- 172.21.15.55:0/1108042294 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647eb934800 con 0x5647eba57100 2024-04-26T22:01:51.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.600+0000 7fb0ed9bb640 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5647eba57100 0x5647eba594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:51.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.608+0000 7fb0d1ffb640 10 client.16812 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:01:51.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.608+0000 7fb0ed9bb640 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5647eba57100 0x5647eba594f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b48fdfb951155e86 server_cookie=6f02cfd8872d9832 in_seq=0 out_seq=0 2024-04-26T22:01:51.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.612+0000 7fb0d1ffb640 1 -- 172.21.15.55:0/1108042294 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647eb934800 con 0x5647eba57100 2024-04-26T22:01:51.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.612+0000 7fb0d1ffb640 10 client.16812 handle_command_reply: tid=0 2024-04-26T22:01:51.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.612+0000 7fb0f3215740 1 client.16812 shutdown 2024-04-26T22:01:51.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.613+0000 7fb0f3215740 1 -- 172.21.15.55:0/1108042294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 msgr2=0x5647eb9d7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.613+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9d7c50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0099950 tx=0x7fb0e00992f0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.621+0000 7fb0f3215740 1 -- 172.21.15.55:0/1108042294 shutdown_connections 2024-04-26T22:01:51.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.621+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5647eba57100 0x5647eba594f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.621+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5647eb9ca5c0 0x5647eb9d9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.621+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647eb9c9c00 0x5647eb9d8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.621+0000 7fb0f3215740 1 --2- 172.21.15.55:0/1108042294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5647eb9c4640 0x5647eb9d7c50 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.621+0000 7fb0d1ffb640 0 client.16812 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:01:51.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.621+0000 7fb0f3215740 1 -- 172.21.15.55:0/1108042294 >> 172.21.15.55:0/1108042294 conn(0x5647eb909910 msgr2=0x5647eb9c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.627+0000 7fb0f3215740 1 -- 172.21.15.55:0/1108042294 shutdown_connections 2024-04-26T22:01:51.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.627+0000 7fb0f3215740 1 -- 172.21.15.55:0/1108042294 wait complete. 2024-04-26T22:01:51.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.631+0000 7fb0f3215740 20 client.16812 trim_cache size 0 max 16384 2024-04-26T22:01:51.636 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:01:51.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.644+0000 7fb0ef1be640 1 -- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0d40a3e40 msgr2=0x7fb0d40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.644+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0d40a3e40 0x7fb0d40a6300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8031500 tx=0x7fb0d80041a0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.644+0000 7fb0ef1be640 1 -- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e8073c10 msgr2=0x7fb0e81a7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:51.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.644+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e8073c10 0x7fb0e81a7740 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000eab0 tx=0x7fb0e000ef70 comp rx=0 tx=0).stop 2024-04-26T22:01:51.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.648+0000 7fb0ef1be640 1 -- 172.21.15.55:0/2969062081 shutdown_connections 2024-04-26T22:01:51.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.648+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb0d40a3e40 0x7fb0d40a6300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.648+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0e810a8d0 0x7fb0e81aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.648+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0e8109f10 0x7fb0e81a7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.648+0000 7fb0ef1be640 1 --2- 172.21.15.55:0/2969062081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb0e8073c10 0x7fb0e81a7740 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:51.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.648+0000 7fb0ef1be640 1 -- 172.21.15.55:0/2969062081 >> 172.21.15.55:0/2969062081 conn(0x7fb0e806f780 msgr2=0x7fb0e8071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:51.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.656+0000 7fb0ef1be640 1 -- 172.21.15.55:0/2969062081 shutdown_connections 2024-04-26T22:01:51.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:51.660+0000 7fb0ef1be640 1 -- 172.21.15.55:0/2969062081 wait complete. 2024-04-26T22:01:51.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:01:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:51 smithi055 ceph-mon[29221]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 889 MiB/s wr, 1.74k op/s 2024-04-26T22:01:52.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:51 smithi064 ceph-mon[32765]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 889 MiB/s wr, 1.74k op/s 2024-04-26T22:01:52.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:51 smithi165 ceph-mon[33296]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 889 MiB/s wr, 1.74k op/s 2024-04-26T22:01:52.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.080+0000 7fd86e3a4640 1 -- 172.21.15.55:0/1628103497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd86810a640 msgr2=0x7fd86810f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.080+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/1628103497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd86810a640 0x7fd86810f2b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd850009900 tx=0x7fd85002a400 comp rx=0 tx=0).stop 2024-04-26T22:01:52.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.083+0000 7fd86e3a4640 1 -- 172.21.15.55:0/1628103497 shutdown_connections 2024-04-26T22:01:52.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.083+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/1628103497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd86810a640 0x7fd86810f2b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.083+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/1628103497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd868074540 0x7fd868109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.083+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/1628103497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868073be0 0x7fd868074000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.083+0000 7fd86e3a4640 1 -- 172.21.15.55:0/1628103497 >> 172.21.15.55:0/1628103497 conn(0x7fd86806f780 msgr2=0x7fd868071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:52.087 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.086+0000 7fd86e3a4640 1 -- 172.21.15.55:0/1628103497 shutdown_connections 2024-04-26T22:01:52.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.088+0000 7fd86e3a4640 1 -- 172.21.15.55:0/1628103497 wait complete. 2024-04-26T22:01:52.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.089+0000 7fd86e3a4640 1 Processor -- start 2024-04-26T22:01:52.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.090+0000 7fd86e3a4640 1 -- start start 2024-04-26T22:01:52.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.090+0000 7fd86e3a4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd868073be0 0x7fd8681a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.090+0000 7fd86e3a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868074540 0x7fd8681a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.090+0000 7fd86e3a4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd86810a640 0x7fd8681aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.090+0000 7fd86e3a4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd86811d0e0 con 0x7fd868073be0 2024-04-26T22:01:52.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.090+0000 7fd86e3a4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd86811cf60 con 0x7fd868074540 2024-04-26T22:01:52.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.090+0000 7fd86e3a4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd86811d260 con 0x7fd86810a640 2024-04-26T22:01:52.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.094+0000 7fd86c941640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd86810a640 0x7fd8681aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.095+0000 7fd867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd868073be0 0x7fd8681a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.095+0000 7fd867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd868073be0 0x7fd8681a7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42230/0 (socket says 172.21.15.55:42230) 2024-04-26T22:01:52.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.095+0000 7fd867fff640 1 -- 172.21.15.55:0/227390115 learned_addr learned my addr 172.21.15.55:0/227390115 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:52.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.095+0000 7fd8677fe640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868074540 0x7fd8681a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.098+0000 7fd86c941640 1 -- 172.21.15.55:0/227390115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868074540 msgr2=0x7fd8681a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.098+0000 7fd86c941640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868074540 0x7fd8681a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.098+0000 7fd86c941640 1 -- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd868073be0 msgr2=0x7fd8681a7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.098+0000 7fd86c941640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd868073be0 0x7fd8681a7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.098+0000 7fd86c941640 1 -- 172.21.15.55:0/227390115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8681af3b0 con 0x7fd86810a640 2024-04-26T22:01:52.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.101+0000 7fd86c941640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd86810a640 0x7fd8681aed40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd850009850 tx=0x7fd850003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:52.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.103+0000 7fd8657fa640 1 -- 172.21.15.55:0/227390115 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85003f070 con 0x7fd86810a640 2024-04-26T22:01:52.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.103+0000 7fd8657fa640 1 -- 172.21.15.55:0/227390115 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd850031c50 con 0x7fd86810a640 2024-04-26T22:01:52.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.103+0000 7fd86e3a4640 1 -- 172.21.15.55:0/227390115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8681af640 con 0x7fd86810a640 2024-04-26T22:01:52.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.103+0000 7fd86e3a4640 1 -- 172.21.15.55:0/227390115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8681afb30 con 0x7fd86810a640 2024-04-26T22:01:52.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.105+0000 7fd8657fa640 1 -- 172.21.15.55:0/227390115 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850035030 con 0x7fd86810a640 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd871199740 1 Processor -- start 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd871199740 1 -- start start 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd871199740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd871199740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cfbd0 0x556ad61d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd871199740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61d0590 0x556ad61d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd871199740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556ad6132d80 con 0x556ad61cea00 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd871199740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556ad60eca20 con 0x556ad61d0590 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd871199740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556ad612ed40 con 0x556ad61cfbd0 2024-04-26T22:01:52.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.114+0000 7fd8657fa640 1 -- 172.21.15.55:0/227390115 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd850042070 con 0x7fd86810a640 2024-04-26T22:01:52.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.116+0000 7fd8657fa640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8540a3e40 0x7fd8540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.116+0000 7fd86c941640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8540a3e40 0x7fd8540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.117+0000 7fd8657fa640 1 -- 172.21.15.55:0/227390115 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd8500f0ca0 con 0x7fd86810a640 2024-04-26T22:01:52.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.119+0000 7fd86c941640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8540a3e40 0x7fd8540a6300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd8500b9190 tx=0x7fd8500b88f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:52.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.120+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61d0590 0x556ad61d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.120+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.120+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42240/0 (socket says 172.21.15.55:42240) 2024-04-26T22:01:52.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.120+0000 7fd8677fe640 1 -- 172.21.15.55:0/32205614 learned_addr learned my addr 172.21.15.55:0/32205614 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:52.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.120+0000 7fd867fff640 1 --2- 172.21.15.55:0/32205614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cfbd0 0x556ad61d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.122+0000 7fd8677fe640 1 -- 172.21.15.55:0/32205614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cfbd0 msgr2=0x556ad61d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:52.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.122+0000 7fd8677fe640 1 --2- 172.21.15.55:0/32205614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cfbd0 0x556ad61d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.122+0000 7fd8677fe640 1 -- 172.21.15.55:0/32205614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61d0590 msgr2=0x556ad61d6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:52.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.122+0000 7fd8677fe640 1 --2- 172.21.15.55:0/32205614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61d0590 0x556ad61d6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.122+0000 7fd8677fe640 1 -- 172.21.15.55:0/32205614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad60f7c30 con 0x556ad61cea00 2024-04-26T22:01:52.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.122+0000 7fd8677fe640 1 --2- 172.21.15.55:0/32205614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61cee20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd858005fd0 tx=0x7fd858004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1a49d3b161974f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:52.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.123+0000 7fd85e7fc640 1 -- 172.21.15.55:0/32205614 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858005220 con 0x556ad61cea00 2024-04-26T22:01:52.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.123+0000 7fd85e7fc640 1 -- 172.21.15.55:0/32205614 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd858017070 con 0x556ad61cea00 2024-04-26T22:01:52.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.123+0000 7fd85e7fc640 1 -- 172.21.15.55:0/32205614 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858010770 con 0x556ad61cea00 2024-04-26T22:01:52.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.126+0000 7fd871199740 1 -- 172.21.15.55:0/32205614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 msgr2=0x556ad61cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.126+0000 7fd871199740 1 --2- 172.21.15.55:0/32205614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61cee20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd858005fd0 tx=0x7fd858004730 comp rx=0 tx=0).stop 2024-04-26T22:01:52.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.130+0000 7fd871199740 1 -- 172.21.15.55:0/32205614 shutdown_connections 2024-04-26T22:01:52.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.130+0000 7fd871199740 1 --2- 172.21.15.55:0/32205614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61d0590 0x556ad61d6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.130+0000 7fd871199740 1 --2- 172.21.15.55:0/32205614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cfbd0 0x556ad61d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.130+0000 7fd871199740 1 --2- 172.21.15.55:0/32205614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61cee20 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.130+0000 7fd871199740 1 -- 172.21.15.55:0/32205614 >> 172.21.15.55:0/32205614 conn(0x556ad610f910 msgr2=0x556ad61ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:52.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.134+0000 7fd871199740 1 -- 172.21.15.55:0/32205614 shutdown_connections 2024-04-26T22:01:52.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.138+0000 7fd871199740 1 -- 172.21.15.55:0/32205614 wait complete. 2024-04-26T22:01:52.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.138+0000 7fd871199740 1 -- start start 2024-04-26T22:01:52.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.138+0000 7fd871199740 1 Processor -- start 2024-04-26T22:01:52.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.139+0000 7fd871199740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cea00 0x556ad61d8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.139+0000 7fd871199740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.139+0000 7fd871199740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61d0590 0x556ad61e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.139+0000 7fd871199740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556ad61d7540 con 0x556ad61d0590 2024-04-26T22:01:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.139+0000 7fd871199740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556ad60b9dc0 con 0x556ad61cfbd0 2024-04-26T22:01:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.139+0000 7fd871199740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556ad61d76c0 con 0x556ad61cea00 2024-04-26T22:01:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.139+0000 7fd871199740 10 client.0 _ll_register_callbacks cb 0x556ad60d5600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:52.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.146+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61d0590 0x556ad61e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.146+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61d0590 0x556ad61e01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42242/0 (socket says 172.21.15.55:42242) 2024-04-26T22:01:52.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.146+0000 7fd8677fe640 1 -- 172.21.15.55:0/3150735279 learned_addr learned my addr 172.21.15.55:0/3150735279 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:52.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.146+0000 7fd8677fe640 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cea00 0x556ad61d8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.147 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.146+0000 7fd867fff640 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.151+0000 7fd867fff640 1 -- 172.21.15.55:0/3150735279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cea00 msgr2=0x556ad61d8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:52.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.151+0000 7fd867fff640 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cea00 0x556ad61d8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.151+0000 7fd867fff640 1 -- 172.21.15.55:0/3150735279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61d0590 msgr2=0x556ad61e01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:52.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.151+0000 7fd867fff640 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61d0590 0x556ad61e01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.151+0000 7fd867fff640 1 -- 172.21.15.55:0/3150735279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad5e443f0 con 0x556ad61cfbd0 2024-04-26T22:01:52.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.152+0000 7fd867fff640 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61d91b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd8600042c0 tx=0x7fd860003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a604e2d67dfae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:52.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.152+0000 7fd8477fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:01:52.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.155+0000 7fd8477fe640 1 -- 172.21.15.55:0/3150735279 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860013c40 con 0x556ad61cfbd0 2024-04-26T22:01:52.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.155+0000 7fd8477fe640 1 -- 172.21.15.55:0/3150735279 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd860013de0 con 0x556ad61cfbd0 2024-04-26T22:01:52.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.156+0000 7fd871199740 1 -- 172.21.15.55:0/3150735279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd858017070 con 0x556ad61cfbd0 2024-04-26T22:01:52.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.157+0000 7fd8477fe640 1 -- 172.21.15.55:0/3150735279 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd86001c440 con 0x556ad61cfbd0 2024-04-26T22:01:52.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.160+0000 7fd871199740 10 client.25952 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:52.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.160+0000 7fd871199740 1 -- 172.21.15.55:0/3150735279 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556ad5f0e7d0 con 0x556ad61cfbd0 2024-04-26T22:01:52.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.161+0000 7fd8477fe640 1 -- 172.21.15.55:0/3150735279 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd858017070 con 0x556ad61cfbd0 2024-04-26T22:01:52.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.163+0000 7fd8477fe640 1 -- 172.21.15.55:0/3150735279 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd86001a020 con 0x556ad61cfbd0 2024-04-26T22:01:52.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.163+0000 7fd871199740 10 client.25952 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:52.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.163+0000 7fd871199740 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:52.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.163+0000 7fd871199740 20 client.25952 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:52.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.164+0000 7fd871199740 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556ad61e4f30 0x556ad61e7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.164+0000 7fd871199740 4 client.25952 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:52.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.164+0000 7fd871199740 1 -- 172.21.15.55:0/3150735279 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad6140990 con 0x556ad61e4f30 2024-04-26T22:01:52.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.164+0000 7fd8677fe640 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556ad61e4f30 0x556ad61e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.167+0000 7fd8677fe640 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556ad61e4f30 0x556ad61e7350 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657bfa8c470dc0cf server_cookie=ba8debb48ca60aa in_seq=0 out_seq=0 2024-04-26T22:01:52.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.168+0000 7fd8477fe640 10 client.25952 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:52.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.171+0000 7fd8477fe640 1 -- 172.21.15.55:0/3150735279 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ad6140990 con 0x556ad61e4f30 2024-04-26T22:01:52.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.171+0000 7fd8477fe640 10 client.25952 handle_command_reply: tid=0 2024-04-26T22:01:52.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.171+0000 7fd871199740 1 client.25952 shutdown 2024-04-26T22:01:52.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.171+0000 7fd871199740 1 -- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 msgr2=0x556ad61d91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.171+0000 7fd871199740 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61d91b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd8600042c0 tx=0x7fd860003f90 comp rx=0 tx=0).stop 2024-04-26T22:01:52.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 1 -- 172.21.15.55:0/3150735279 shutdown_connections 2024-04-26T22:01:52.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556ad61e4f30 0x556ad61e7350 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61d0590 0x556ad61e01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61d91b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 1 --2- 172.21.15.55:0/3150735279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61cea00 0x556ad61d8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd8477fe640 0 client.25952 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:52.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 1 -- 172.21.15.55:0/3150735279 >> 172.21.15.55:0/3150735279 conn(0x556ad610f910 msgr2=0x556ad61c9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:52.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 1 -- 172.21.15.55:0/3150735279 shutdown_connections 2024-04-26T22:01:52.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 1 -- 172.21.15.55:0/3150735279 wait complete. 2024-04-26T22:01:52.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.172+0000 7fd871199740 20 client.25952 trim_cache size 0 max 16384 2024-04-26T22:01:52.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.205+0000 7fd871199740 1 Processor -- start 2024-04-26T22:01:52.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.206+0000 7fd871199740 1 -- start start 2024-04-26T22:01:52.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.206+0000 7fd871199740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61d7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.206+0000 7fd871199740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61e01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.207+0000 7fd871199740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61d8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.207+0000 7fd871199740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd848010f20 con 0x556ad61cea00 2024-04-26T22:01:52.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.207+0000 7fd871199740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800e7f0 con 0x556ad61cfbd0 2024-04-26T22:01:52.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.207+0000 7fd867fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61e01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.207+0000 7fd867fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61e01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37714/0 (socket says 172.21.15.55:37714) 2024-04-26T22:01:52.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.207+0000 7fd867fff640 1 -- 172.21.15.55:0/753370881 learned_addr learned my addr 172.21.15.55:0/753370881 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:52.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.207+0000 7fd8677fe640 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.207+0000 7fd871199740 1 -- 172.21.15.55:0/753370881 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800c640 con 0x556ad61d0590 2024-04-26T22:01:52.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.208+0000 7fd8677fe640 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61d7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.208+0000 7fd8677fe640 1 -- 172.21.15.55:0/753370881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 msgr2=0x556ad61e01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:52.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.208+0000 7fd8677fe640 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61e01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.209+0000 7fd8677fe640 1 -- 172.21.15.55:0/753370881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 msgr2=0x556ad61d7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:52.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.209+0000 7fd8677fe640 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61d7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.209+0000 7fd8677fe640 1 -- 172.21.15.55:0/753370881 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad61ca3c0 con 0x556ad61d0590 2024-04-26T22:01:52.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.210+0000 7fd8677fe640 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61d8030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd85800dfd0 tx=0x7fd858015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7c8a661ad27bc92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:52.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.213+0000 7fd85dffb640 1 -- 172.21.15.55:0/753370881 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858027040 con 0x556ad61d0590 2024-04-26T22:01:52.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.213+0000 7fd85dffb640 1 -- 172.21.15.55:0/753370881 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd858017070 con 0x556ad61d0590 2024-04-26T22:01:52.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.213+0000 7fd85dffb640 1 -- 172.21.15.55:0/753370881 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858031410 con 0x556ad61d0590 2024-04-26T22:01:52.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.214+0000 7fd871199740 1 -- 172.21.15.55:0/753370881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 msgr2=0x556ad61d8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.214+0000 7fd871199740 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61d8030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd85800dfd0 tx=0x7fd858015040 comp rx=0 tx=0).stop 2024-04-26T22:01:52.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.214+0000 7fd871199740 1 -- 172.21.15.55:0/753370881 shutdown_connections 2024-04-26T22:01:52.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.214+0000 7fd871199740 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61d8030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.214+0000 7fd871199740 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cfbd0 0x556ad61e01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.214+0000 7fd871199740 1 --2- 172.21.15.55:0/753370881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cea00 0x556ad61d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.214+0000 7fd871199740 1 -- 172.21.15.55:0/753370881 >> 172.21.15.55:0/753370881 conn(0x556ad610f910 msgr2=0x556ad61cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:52.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.215+0000 7fd871199740 1 -- 172.21.15.55:0/753370881 shutdown_connections 2024-04-26T22:01:52.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.215+0000 7fd871199740 1 -- 172.21.15.55:0/753370881 wait complete. 2024-04-26T22:01:52.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.215+0000 7fd871199740 1 -- start start 2024-04-26T22:01:52.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.216+0000 7fd871199740 1 Processor -- start 2024-04-26T22:01:52.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.216+0000 7fd871199740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cea00 0x556ad61dd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.216+0000 7fd871199740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cfbd0 0x556ad61dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.216+0000 7fd871199740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.216+0000 7fd871199740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800ccb0 con 0x556ad61cfbd0 2024-04-26T22:01:52.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.216+0000 7fd871199740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800e120 con 0x556ad61cea00 2024-04-26T22:01:52.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.216+0000 7fd871199740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd84800d380 con 0x556ad61d0590 2024-04-26T22:01:52.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.216+0000 7fd871199740 10 client.0 _ll_register_callbacks cb 0x7fd860009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:52.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.217+0000 7fd867fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cfbd0 0x556ad61dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.217+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cea00 0x556ad61dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.217+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cea00 0x556ad61dd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37716/0 (socket says 172.21.15.55:37716) 2024-04-26T22:01:52.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.217+0000 7fd8677fe640 1 -- 172.21.15.55:0/1057280142 learned_addr learned my addr 172.21.15.55:0/1057280142 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:52.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.218+0000 7fd8677fe640 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.218+0000 7fd8677fe640 1 -- 172.21.15.55:0/1057280142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 msgr2=0x556ad61de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:52.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.218+0000 7fd8677fe640 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.218+0000 7fd8677fe640 1 -- 172.21.15.55:0/1057280142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cfbd0 msgr2=0x556ad61dd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:52.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.218+0000 7fd8677fe640 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cfbd0 0x556ad61dd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.218+0000 7fd8677fe640 1 -- 172.21.15.55:0/1057280142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad61ded50 con 0x556ad61cea00 2024-04-26T22:01:52.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.219+0000 7fd8677fe640 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cea00 0x556ad61dd210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd858013f30 tx=0x7fd85800d9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa0046fe687f0a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:52.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.219+0000 7fd8477fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:01:52.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.220+0000 7fd8477fe640 1 -- 172.21.15.55:0/1057280142 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858045060 con 0x556ad61cea00 2024-04-26T22:01:52.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.220+0000 7fd871199740 1 -- 172.21.15.55:0/1057280142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd858017070 con 0x556ad61cea00 2024-04-26T22:01:52.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.220+0000 7fd8477fe640 1 -- 172.21.15.55:0/1057280142 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd858010b80 con 0x556ad61cea00 2024-04-26T22:01:52.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.221+0000 7fd8477fe640 1 -- 172.21.15.55:0/1057280142 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd858010d80 con 0x556ad61cea00 2024-04-26T22:01:52.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.221+0000 7fd871199740 10 client.25958 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:52.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.221+0000 7fd871199740 1 -- 172.21.15.55:0/1057280142 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556ad61e0050 con 0x556ad61cea00 2024-04-26T22:01:52.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.222+0000 7fd8477fe640 1 -- 172.21.15.55:0/1057280142 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd858017070 con 0x556ad61cea00 2024-04-26T22:01:52.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.223+0000 7fd8477fe640 1 -- 172.21.15.55:0/1057280142 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd85802d040 con 0x556ad61cea00 2024-04-26T22:01:52.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.224+0000 7fd871199740 10 client.25958 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:52.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.224+0000 7fd871199740 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:52.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.224+0000 7fd871199740 20 client.25958 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:52.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.224+0000 7fd871199740 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556ad624b630 0x556ad624da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:52.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.224+0000 7fd871199740 4 client.25958 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:01:52.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.224+0000 7fd871199740 1 -- 172.21.15.55:0/1057280142 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ad613a800 con 0x556ad624b630 2024-04-26T22:01:52.226 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.225+0000 7fd867fff640 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556ad624b630 0x556ad624da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:52.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.229+0000 7fd867fff640 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556ad624b630 0x556ad624da50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f54276a3e832f4 server_cookie=e8a2a921fb950a97 in_seq=0 out_seq=0 2024-04-26T22:01:52.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.230+0000 7fd8477fe640 10 client.25958 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:52.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.231+0000 7fd8477fe640 1 -- 172.21.15.55:0/1057280142 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ad613a800 con 0x556ad624b630 2024-04-26T22:01:52.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.232+0000 7fd8477fe640 10 client.25958 handle_command_reply: tid=0 2024-04-26T22:01:52.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.232+0000 7fd871199740 1 client.25958 shutdown 2024-04-26T22:01:52.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.242+0000 7fd871199740 1 -- 172.21.15.55:0/1057280142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cea00 msgr2=0x556ad61dd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.242+0000 7fd871199740 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cea00 0x556ad61dd210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd858013f30 tx=0x7fd85800d9d0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.243+0000 7fd871199740 1 -- 172.21.15.55:0/1057280142 shutdown_connections 2024-04-26T22:01:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.243+0000 7fd871199740 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ad61d0590 0x556ad61de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.243+0000 7fd871199740 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556ad624b630 0x556ad624da50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.243+0000 7fd871199740 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556ad61cfbd0 0x556ad61dd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.243+0000 7fd871199740 1 --2- 172.21.15.55:0/1057280142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556ad61cea00 0x556ad61dd210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.243+0000 7fd871199740 1 -- 172.21.15.55:0/1057280142 >> 172.21.15.55:0/1057280142 conn(0x556ad610f910 msgr2=0x556ad61cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.243+0000 7fd8477fe640 0 client.25958 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:52.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.243+0000 7fd871199740 1 -- 172.21.15.55:0/1057280142 shutdown_connections 2024-04-26T22:01:52.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.246+0000 7fd871199740 1 -- 172.21.15.55:0/1057280142 wait complete. 2024-04-26T22:01:52.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.246+0000 7fd871199740 20 client.25958 trim_cache size 0 max 16384 2024-04-26T22:01:52.251 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:01:52.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.252+0000 7fd86e3a4640 1 -- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8540a3e40 msgr2=0x7fd8540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.252+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8540a3e40 0x7fd8540a6300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd8500b9190 tx=0x7fd8500b88f0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.253+0000 7fd86e3a4640 1 -- 172.21.15.55:0/227390115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd86810a640 msgr2=0x7fd8681aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:52.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.253+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd86810a640 0x7fd8681aed40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd850009850 tx=0x7fd850003b90 comp rx=0 tx=0).stop 2024-04-26T22:01:52.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.253+0000 7fd86e3a4640 1 -- 172.21.15.55:0/227390115 shutdown_connections 2024-04-26T22:01:52.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.253+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd8540a3e40 0x7fd8540a6300 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.253+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd86810a640 0x7fd8681aed40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.254+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd868074540 0x7fd8681a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.254+0000 7fd86e3a4640 1 --2- 172.21.15.55:0/227390115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd868073be0 0x7fd8681a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:52.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.254+0000 7fd86e3a4640 1 -- 172.21.15.55:0/227390115 >> 172.21.15.55:0/227390115 conn(0x7fd86806f780 msgr2=0x7fd8680713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.260+0000 7fd86e3a4640 1 -- 172.21.15.55:0/227390115 shutdown_connections 2024-04-26T22:01:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:52.262+0000 7fd86e3a4640 1 -- 172.21.15.55:0/227390115 wait complete. 2024-04-26T22:01:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:52 smithi064 ceph-mon[32765]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.79k op/s 2024-04-26T22:01:53.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:52 smithi165 ceph-mon[33296]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.79k op/s 2024-04-26T22:01:53.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:52 smithi055 ceph-mon[29221]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.79k op/s 2024-04-26T22:01:53.320 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:01:53.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:01:53.321 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:01:53.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.655+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1502995130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83410ec70 msgr2=0x7fe83410f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:53.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.655+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1502995130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83410ec70 0x7fe83410f120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009be0 tx=0x7fe82c02a710 comp rx=0 tx=0).stop 2024-04-26T22:01:53.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.656+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1502995130 shutdown_connections 2024-04-26T22:01:53.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.656+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1502995130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe83410ec70 0x7fe83410f120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.656+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1502995130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe83410c2a0 0x7fe83410e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.656+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1502995130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe834109930 0x7fe83410bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.656+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1502995130 >> 172.21.15.55:0/1502995130 conn(0x7fe83406f7c0 msgr2=0x7fe834071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:53.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.660+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1502995130 shutdown_connections 2024-04-26T22:01:53.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.660+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1502995130 wait complete. 2024-04-26T22:01:53.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.661+0000 7fe83b6c4640 1 Processor -- start 2024-04-26T22:01:53.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.665+0000 7fe83b6c4640 1 -- start start 2024-04-26T22:01:53.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.665+0000 7fe83b6c4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe834109930 0x7fe8341a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.665+0000 7fe83b6c4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe83410c2a0 0x7fe8341a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.665+0000 7fe83b6c4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe83410ec70 0x7fe8341af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.665+0000 7fe83b6c4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe83411d300 con 0x7fe83410c2a0 2024-04-26T22:01:53.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.665+0000 7fe83b6c4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe83411d180 con 0x7fe834109930 2024-04-26T22:01:53.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.665+0000 7fe83b6c4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe83411d480 con 0x7fe83410ec70 2024-04-26T22:01:53.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.666+0000 7fe839c61640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe83410ec70 0x7fe8341af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.666+0000 7fe839c61640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe83410ec70 0x7fe8341af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:43204/0 (socket says 172.21.15.55:43204) 2024-04-26T22:01:53.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.666+0000 7fe839c61640 1 -- 172.21.15.55:0/1895672599 learned_addr learned my addr 172.21.15.55:0/1895672599 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:53.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.667+0000 7fe838c5f640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe83410c2a0 0x7fe8341a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.667+0000 7fe839c61640 1 -- 172.21.15.55:0/1895672599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe834109930 msgr2=0x7fe8341a7aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:01:53.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.667+0000 7fe839c61640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe834109930 0x7fe8341a7aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.667+0000 7fe839c61640 1 -- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe83410c2a0 msgr2=0x7fe8341a7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:53.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.667+0000 7fe839c61640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe83410c2a0 0x7fe8341a7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.667+0000 7fe839c61640 1 -- 172.21.15.55:0/1895672599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe834111190 con 0x7fe83410ec70 2024-04-26T22:01:53.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.668+0000 7fe839c61640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe83410ec70 0x7fe8341af000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009b30 tx=0x7fe82c031650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:53.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.686+0000 7fe82a7fc640 1 -- 172.21.15.55:0/1895672599 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c041020 con 0x7fe83410ec70 2024-04-26T22:01:53.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.686+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1895672599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe834111420 con 0x7fe83410ec70 2024-04-26T22:01:53.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.687+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1895672599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe834111990 con 0x7fe83410ec70 2024-04-26T22:01:53.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.687+0000 7fe82a7fc640 1 -- 172.21.15.55:0/1895672599 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c031940 con 0x7fe83410ec70 2024-04-26T22:01:53.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.687+0000 7fe82a7fc640 1 -- 172.21.15.55:0/1895672599 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c036040 con 0x7fe83410ec70 2024-04-26T22:01:53.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.704+0000 7fe82a7fc640 1 -- 172.21.15.55:0/1895672599 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe82c044020 con 0x7fe83410ec70 2024-04-26T22:01:53.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.706+0000 7fe82a7fc640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe8200a3e40 0x7fe8200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.706+0000 7fe82a7fc640 1 -- 172.21.15.55:0/1895672599 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe82c0f15e0 con 0x7fe83410ec70 2024-04-26T22:01:53.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.707+0000 7fe83e3db740 1 Processor -- start 2024-04-26T22:01:53.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.707+0000 7fe83e3db740 1 -- start start 2024-04-26T22:01:53.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.707+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5625fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.707+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e562602a0 0x558e56264da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.707+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e562678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.708+0000 7fe83e3db740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558e561c4b00 con 0x558e5625f8b0 2024-04-26T22:01:53.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.708+0000 7fe83e3db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e56160920 con 0x558e562602a0 2024-04-26T22:01:53.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.708+0000 7fe83e3db740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558e561c0ac0 con 0x558e56265410 2024-04-26T22:01:53.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.709+0000 7fe838c5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e562602a0 0x558e56264da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.710+0000 7fe838c5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e562602a0 0x558e56264da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37740/0 (socket says 172.21.15.55:37740) 2024-04-26T22:01:53.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.710+0000 7fe838c5f640 1 -- 172.21.15.55:0/943534454 learned_addr learned my addr 172.21.15.55:0/943534454 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:53.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.711+0000 7fe838c5f640 1 -- 172.21.15.55:0/943534454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 msgr2=0x558e562678a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:53.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.711+0000 7fe838c5f640 1 --2- 172.21.15.55:0/943534454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e562678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.711+0000 7fe838c5f640 1 -- 172.21.15.55:0/943534454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 msgr2=0x558e5625fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:53.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.711+0000 7fe838c5f640 1 --2- 172.21.15.55:0/943534454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5625fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.711+0000 7fe838c5f640 1 -- 172.21.15.55:0/943534454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e55f9e7e0 con 0x558e562602a0 2024-04-26T22:01:53.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.718+0000 7fe838c5f640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe8200a3e40 0x7fe8200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.718+0000 7fe838c5f640 1 --2- 172.21.15.55:0/943534454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e562602a0 0x558e56264da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe830009510 tx=0x7fe830012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a4e2f059cac138f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.719+0000 7fe81f7fe640 1 -- 172.21.15.55:0/943534454 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83000f430 con 0x558e562602a0 2024-04-26T22:01:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.719+0000 7fe81f7fe640 1 -- 172.21.15.55:0/943534454 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe83000fa80 con 0x558e562602a0 2024-04-26T22:01:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.720+0000 7fe81f7fe640 1 -- 172.21.15.55:0/943534454 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83001cca0 con 0x558e562602a0 2024-04-26T22:01:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.721+0000 7fe83e3db740 1 -- 172.21.15.55:0/943534454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e562602a0 msgr2=0x558e56264da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:53.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.721+0000 7fe83e3db740 1 --2- 172.21.15.55:0/943534454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e562602a0 0x558e56264da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe830009510 tx=0x7fe830012820 comp rx=0 tx=0).stop 2024-04-26T22:01:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.723+0000 7fe83e3db740 1 -- 172.21.15.55:0/943534454 shutdown_connections 2024-04-26T22:01:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.723+0000 7fe83e3db740 1 --2- 172.21.15.55:0/943534454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e562678a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.723+0000 7fe83e3db740 1 --2- 172.21.15.55:0/943534454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e562602a0 0x558e56264da0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.723+0000 7fe83e3db740 1 --2- 172.21.15.55:0/943534454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5625fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.723+0000 7fe83e3db740 1 -- 172.21.15.55:0/943534454 >> 172.21.15.55:0/943534454 conn(0x558e5625af20 msgr2=0x558e5625b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.723+0000 7fe83e3db740 1 -- 172.21.15.55:0/943534454 shutdown_connections 2024-04-26T22:01:53.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.727+0000 7fe83e3db740 1 -- 172.21.15.55:0/943534454 wait complete. 2024-04-26T22:01:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.728+0000 7fe83e3db740 1 -- start start 2024-04-26T22:01:53.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.728+0000 7fe83e3db740 1 Processor -- start 2024-04-26T22:01:53.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.729+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e5625f8b0 0x558e56269db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.729+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 0x558e5626a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.729+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e562712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.729+0000 7fe83e3db740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558e562683c0 con 0x558e562602a0 2024-04-26T22:01:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.729+0000 7fe83e3db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e5615e5d0 con 0x558e5625f8b0 2024-04-26T22:01:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.729+0000 7fe83e3db740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558e56268540 con 0x558e56265410 2024-04-26T22:01:53.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.729+0000 7fe83e3db740 10 client.0 _ll_register_callbacks cb 0x558e5614c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.734+0000 7fe839c61640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e562712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.734+0000 7fe839460640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e5625f8b0 0x558e56269db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.735+0000 7fe839460640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e5625f8b0 0x558e56269db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37754/0 (socket says 172.21.15.55:37754) 2024-04-26T22:01:53.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.735+0000 7fe839460640 1 -- 172.21.15.55:0/2778662536 learned_addr learned my addr 172.21.15.55:0/2778662536 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.746+0000 7fe839460640 1 -- 172.21.15.55:0/2778662536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 msgr2=0x558e562712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.746+0000 7fe839460640 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e562712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.746+0000 7fe839460640 1 -- 172.21.15.55:0/2778662536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 msgr2=0x558e5626a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:53.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.746+0000 7fe839460640 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 0x558e5626a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.746+0000 7fe839460640 1 -- 172.21.15.55:0/2778662536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e55ffef70 con 0x558e5625f8b0 2024-04-26T22:01:53.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.746+0000 7fe839c61640 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e562712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:53.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.748+0000 7fe838c5f640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe8200a3e40 0x7fe8200a6300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe83001c510 tx=0x7fe830012820 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:53.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.756+0000 7fe839460640 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e5625f8b0 0x558e56269db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe824009dd0 tx=0x7fe82400b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb2470383e31984 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:53.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.757+0000 7fe81cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:01:53.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.764+0000 7fe81cff9640 1 -- 172.21.15.55:0/2778662536 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824015020 con 0x558e5625f8b0 2024-04-26T22:01:53.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.764+0000 7fe81cff9640 1 -- 172.21.15.55:0/2778662536 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe824011420 con 0x558e5625f8b0 2024-04-26T22:01:53.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.764+0000 7fe83e3db740 1 -- 172.21.15.55:0/2778662536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe83000fa80 con 0x558e5625f8b0 2024-04-26T22:01:53.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.775+0000 7fe81cff9640 1 -- 172.21.15.55:0/2778662536 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe824011640 con 0x558e5625f8b0 2024-04-26T22:01:53.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.783+0000 7fe83e3db740 10 client.25967 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:53.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.783+0000 7fe83e3db740 1 -- 172.21.15.55:0/2778662536 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558e55e96170 con 0x558e5625f8b0 2024-04-26T22:01:53.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.785+0000 7fe81cff9640 1 -- 172.21.15.55:0/2778662536 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe83000fa80 con 0x558e5625f8b0 2024-04-26T22:01:53.787 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.785+0000 7fe81cff9640 1 -- 172.21.15.55:0/2778662536 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe8240199d0 con 0x558e5625f8b0 2024-04-26T22:01:53.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.790+0000 7fe83e3db740 10 client.25967 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:53.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.790+0000 7fe83e3db740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:53.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.790+0000 7fe83e3db740 20 client.25967 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:53.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.790+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558e56275790 0x558e56277c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.790+0000 7fe83e3db740 4 client.25967 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:53.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.790+0000 7fe83e3db740 1 -- 172.21.15.55:0/2778662536 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e561d2710 con 0x558e56275790 2024-04-26T22:01:53.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.794+0000 7fe838c5f640 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558e56275790 0x558e56277c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.811+0000 7fe838c5f640 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558e56275790 0x558e56277c30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85aa5706cdfd26f server_cookie=668c3e09e84abe53 in_seq=0 out_seq=0 2024-04-26T22:01:53.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.816+0000 7fe81cff9640 10 client.25967 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.820+0000 7fe81cff9640 1 -- 172.21.15.55:0/2778662536 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e561d2710 con 0x558e56275790 2024-04-26T22:01:53.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.820+0000 7fe81cff9640 10 client.25967 handle_command_reply: tid=0 2024-04-26T22:01:53.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.822+0000 7fe83e3db740 1 client.25967 shutdown 2024-04-26T22:01:53.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe83e3db740 1 -- 172.21.15.55:0/2778662536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e5625f8b0 msgr2=0x558e56269db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:53.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e5625f8b0 0x558e56269db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe824009dd0 tx=0x7fe82400b4b0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe83e3db740 1 -- 172.21.15.55:0/2778662536 shutdown_connections 2024-04-26T22:01:53.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e562712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558e56275790 0x558e56277c30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 0x558e5626a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2778662536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e5625f8b0 0x558e56269db0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe83e3db740 1 -- 172.21.15.55:0/2778662536 >> 172.21.15.55:0/2778662536 conn(0x558e5625af20 msgr2=0x558e5625a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:53.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.828+0000 7fe81cff9640 0 client.25967 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:53.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.829+0000 7fe83e3db740 1 -- 172.21.15.55:0/2778662536 shutdown_connections 2024-04-26T22:01:53.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.832+0000 7fe83e3db740 1 -- 172.21.15.55:0/2778662536 wait complete. 2024-04-26T22:01:53.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.836+0000 7fe83e3db740 20 client.25967 trim_cache size 0 max 16384 2024-04-26T22:01:53.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.903+0000 7fe83e3db740 1 Processor -- start 2024-04-26T22:01:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.903+0000 7fe83e3db740 1 -- start start 2024-04-26T22:01:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.904+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5627c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.904+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e562602a0 0x558e562711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.904+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56265410 0x558e5627ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.906+0000 7fe83e3db740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe818010f80 con 0x558e5625f8b0 2024-04-26T22:01:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.906+0000 7fe83e3db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe81800e850 con 0x558e56265410 2024-04-26T22:01:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.906+0000 7fe83e3db740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe81800c6a0 con 0x558e562602a0 2024-04-26T22:01:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.906+0000 7fe839460640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5627c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.906+0000 7fe839460640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5627c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42296/0 (socket says 172.21.15.55:42296) 2024-04-26T22:01:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.906+0000 7fe839460640 1 -- 172.21.15.55:0/2458809921 learned_addr learned my addr 172.21.15.55:0/2458809921 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.907+0000 7fe838c5f640 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e562602a0 0x558e562711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.907+0000 7fe839c61640 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56265410 0x558e5627ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.908+0000 7fe839460640 1 -- 172.21.15.55:0/2458809921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e562602a0 msgr2=0x558e562711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.908+0000 7fe839460640 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e562602a0 0x558e562711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.908+0000 7fe839460640 1 -- 172.21.15.55:0/2458809921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56265410 msgr2=0x558e5627ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.908+0000 7fe839460640 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56265410 0x558e5627ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.908+0000 7fe839460640 1 -- 172.21.15.55:0/2458809921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5625ad50 con 0x558e5625f8b0 2024-04-26T22:01:53.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.908+0000 7fe838c5f640 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e562602a0 0x558e562711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:01:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.909+0000 7fe839460640 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5627c530 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe824001eb0 tx=0x7fe824006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae76fe9a52b1684f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.919+0000 7fe81effd640 1 -- 172.21.15.55:0/2458809921 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824026020 con 0x558e5625f8b0 2024-04-26T22:01:53.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.919+0000 7fe81effd640 1 -- 172.21.15.55:0/2458809921 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe824011c90 con 0x558e5625f8b0 2024-04-26T22:01:53.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.921+0000 7fe83e3db740 1 -- 172.21.15.55:0/2458809921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 msgr2=0x558e5627c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:53.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.921+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5627c530 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe824001eb0 tx=0x7fe824006cf0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.922+0000 7fe83e3db740 1 -- 172.21.15.55:0/2458809921 shutdown_connections 2024-04-26T22:01:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.922+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56265410 0x558e5627ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.922+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e562602a0 0x558e562711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.922+0000 7fe83e3db740 1 --2- 172.21.15.55:0/2458809921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e5625f8b0 0x558e5627c530 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.922+0000 7fe83e3db740 1 -- 172.21.15.55:0/2458809921 >> 172.21.15.55:0/2458809921 conn(0x558e5625af20 msgr2=0x558e5625a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:53.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.925+0000 7fe83e3db740 1 -- 172.21.15.55:0/2458809921 shutdown_connections 2024-04-26T22:01:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.925+0000 7fe83e3db740 1 -- 172.21.15.55:0/2458809921 wait complete. 2024-04-26T22:01:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.926+0000 7fe83e3db740 1 -- start start 2024-04-26T22:01:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.926+0000 7fe83e3db740 1 Processor -- start 2024-04-26T22:01:53.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.927+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 0x558e5626ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.927+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e5626f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.927+0000 7fe83e3db740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56270130 0x558e562705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.927+0000 7fe83e3db740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe81800cd10 con 0x558e562602a0 2024-04-26T22:01:53.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.928+0000 7fe83e3db740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe81800e180 con 0x558e56270130 2024-04-26T22:01:53.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.928+0000 7fe83e3db740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe81800d3e0 con 0x558e56265410 2024-04-26T22:01:53.929 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.928+0000 7fe83e3db740 10 client.0 _ll_register_callbacks cb 0x7fe8240099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:53.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.928+0000 7fe839460640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 0x558e5626ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.928+0000 7fe839460640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 0x558e5626ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42298/0 (socket says 172.21.15.55:42298) 2024-04-26T22:01:53.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.928+0000 7fe839460640 1 -- 172.21.15.55:0/658704603 learned_addr learned my addr 172.21.15.55:0/658704603 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:53.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.928+0000 7fe838c5f640 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e5626f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.929+0000 7fe838c5f640 1 -- 172.21.15.55:0/658704603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56270130 msgr2=0x558e562705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.929+0000 7fe838c5f640 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56270130 0x558e562705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.929+0000 7fe838c5f640 1 -- 172.21.15.55:0/658704603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 msgr2=0x558e5626ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.929+0000 7fe838c5f640 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 0x558e5626ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.929+0000 7fe838c5f640 1 -- 172.21.15.55:0/658704603 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e56270af0 con 0x558e56265410 2024-04-26T22:01:53.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.935+0000 7fe838c5f640 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e5626f120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe830000c00 tx=0x7fe8300046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62cbdb36e056ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:53.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.935+0000 7fe81cff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:01:53.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.937+0000 7fe81cff9640 1 -- 172.21.15.55:0/658704603 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8300055a0 con 0x558e56265410 2024-04-26T22:01:53.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.937+0000 7fe81cff9640 1 -- 172.21.15.55:0/658704603 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe8300323b0 con 0x558e56265410 2024-04-26T22:01:53.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.939+0000 7fe83e3db740 1 -- 172.21.15.55:0/658704603 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe824011c90 con 0x558e56265410 2024-04-26T22:01:53.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.944+0000 7fe81cff9640 1 -- 172.21.15.55:0/658704603 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe8300325b0 con 0x558e56265410 2024-04-26T22:01:53.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.945+0000 7fe83e3db740 10 client.25996 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:53.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.945+0000 7fe83e3db740 1 -- 172.21.15.55:0/658704603 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558e562dc460 con 0x558e56265410 2024-04-26T22:01:53.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.948+0000 7fe81cff9640 1 -- 172.21.15.55:0/658704603 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe824011c90 con 0x558e56265410 2024-04-26T22:01:53.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.948+0000 7fe81cff9640 1 -- 172.21.15.55:0/658704603 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe830004c70 con 0x558e56265410 2024-04-26T22:01:53.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.950+0000 7fe83e3db740 10 client.25996 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:53.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.950+0000 7fe83e3db740 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:53.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.950+0000 7fe83e3db740 20 client.25996 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:53.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.950+0000 7fe83e3db740 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558e562dccb0 0x558e562df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:53.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.950+0000 7fe83e3db740 4 client.25996 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:01:53.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.950+0000 7fe83e3db740 1 -- 172.21.15.55:0/658704603 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558e561cc580 con 0x558e562dccb0 2024-04-26T22:01:53.954 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.952+0000 7fe839460640 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558e562dccb0 0x558e562df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:53.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.957+0000 7fe81cff9640 10 client.25996 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:53.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:53.957+0000 7fe839460640 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558e562dccb0 0x558e562df0f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d116b09b0cf1c4d server_cookie=3efcd41b95a06801 in_seq=0 out_seq=0 2024-04-26T22:01:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.479+0000 7fe81cff9640 1 -- 172.21.15.55:0/658704603 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558e561cc580 con 0x558e562dccb0 2024-04-26T22:01:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.479+0000 7fe81cff9640 10 client.25996 handle_command_reply: tid=0 2024-04-26T22:01:54.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 client.25996 shutdown 2024-04-26T22:01:54.482 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 -- 172.21.15.55:0/658704603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 msgr2=0x558e5626f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e5626f120 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe830000c00 tx=0x7fe8300046c0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 -- 172.21.15.55:0/658704603 shutdown_connections 2024-04-26T22:01:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e56270130 0x558e562705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e56265410 0x558e5626f120 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558e562dccb0 0x558e562df0f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 --2- 172.21.15.55:0/658704603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558e562602a0 0x558e5626ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe83e3db740 1 -- 172.21.15.55:0/658704603 >> 172.21.15.55:0/658704603 conn(0x558e5625af20 msgr2=0x558e5625de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.480+0000 7fe81cff9640 0 client.25996 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:54.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.481+0000 7fe83e3db740 1 -- 172.21.15.55:0/658704603 shutdown_connections 2024-04-26T22:01:54.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.481+0000 7fe83e3db740 1 -- 172.21.15.55:0/658704603 wait complete. 2024-04-26T22:01:54.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.481+0000 7fe83e3db740 20 client.25996 trim_cache size 0 max 16384 2024-04-26T22:01:54.485 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:01:54.485 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:01:54.485 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "a81f3312-803e-43fa-a05e-cecf6dad8af3", 2024-04-26T22:01:54.485 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:01:54.485 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:01:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.485+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe8200a3e40 msgr2=0x7fe8200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.485+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe8200a3e40 0x7fe8200a6300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe83001c510 tx=0x7fe830012820 comp rx=0 tx=0).stop 2024-04-26T22:01:54.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.485+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1895672599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe83410ec70 msgr2=0x7fe8341af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.485+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe83410ec70 0x7fe8341af000 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009b30 tx=0x7fe82c031650 comp rx=0 tx=0).stop 2024-04-26T22:01:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.487+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1895672599 shutdown_connections 2024-04-26T22:01:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.487+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe8200a3e40 0x7fe8200a6300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.487+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe83410ec70 0x7fe8341af000 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.487+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe83410c2a0 0x7fe8341a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.488+0000 7fe83b6c4640 1 --2- 172.21.15.55:0/1895672599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe834109930 0x7fe8341a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.491 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.488+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1895672599 >> 172.21.15.55:0/1895672599 conn(0x7fe83406f7c0 msgr2=0x7fe83410c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.489+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1895672599 shutdown_connections 2024-04-26T22:01:54.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.490+0000 7fe83b6c4640 1 -- 172.21.15.55:0/1895672599 wait complete. 2024-04-26T22:01:54.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:01:54.525 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.695+0000 7ffa5ab40640 1 -- 172.21.15.55:0/3911848679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54108670 msgr2=0x7ffa5410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.695+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/3911848679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54108670 0x7ffa5410aac0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009ca0 tx=0x7ffa4c02a8d0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.696+0000 7ffa5ab40640 1 -- 172.21.15.55:0/3911848679 shutdown_connections 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.696+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/3911848679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54108670 0x7ffa5410aac0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.696+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/3911848679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa54105d40 0x7ffa54108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.696+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/3911848679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa54103410 0x7ffa54105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.696+0000 7ffa5ab40640 1 -- 172.21.15.55:0/3911848679 >> 172.21.15.55:0/3911848679 conn(0x7ffa540fce20 msgr2=0x7ffa540ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.696+0000 7ffa5ab40640 1 -- 172.21.15.55:0/3911848679 shutdown_connections 2024-04-26T22:01:54.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.696+0000 7ffa5ab40640 1 -- 172.21.15.55:0/3911848679 wait complete. 2024-04-26T22:01:54.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.698+0000 7ffa5ab40640 1 Processor -- start 2024-04-26T22:01:54.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.698+0000 7ffa5ab40640 1 -- start start 2024-04-26T22:01:54.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.698+0000 7ffa5ab40640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54103410 0x7ffa54198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.699+0000 7ffa5ab40640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa54105d40 0x7ffa54199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.699+0000 7ffa5ab40640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa54108670 0x7ffa541a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.699+0000 7ffa5ab40640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5410e5b0 con 0x7ffa54103410 2024-04-26T22:01:54.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.699+0000 7ffa5ab40640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5410e430 con 0x7ffa54105d40 2024-04-26T22:01:54.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.699+0000 7ffa5ab40640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5410e730 con 0x7ffa54108670 2024-04-26T22:01:54.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.699+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa54105d40 0x7ffa54199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.699+0000 7ffa590dd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa54108670 0x7ffa541a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.700+0000 7ffa590dd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa54108670 0x7ffa541a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:43248/0 (socket says 172.21.15.55:43248) 2024-04-26T22:01:54.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.700+0000 7ffa590dd640 1 -- 172.21.15.55:0/873480909 learned_addr learned my addr 172.21.15.55:0/873480909 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:54.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.700+0000 7ffa588dc640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54103410 0x7ffa54198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.702+0000 7ffa588dc640 1 -- 172.21.15.55:0/873480909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa54108670 msgr2=0x7ffa541a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.702+0000 7ffa588dc640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa54108670 0x7ffa541a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.702+0000 7ffa588dc640 1 -- 172.21.15.55:0/873480909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa54105d40 msgr2=0x7ffa54199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.702+0000 7ffa588dc640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa54105d40 0x7ffa54199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.702+0000 7ffa588dc640 1 -- 172.21.15.55:0/873480909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa541a0730 con 0x7ffa54103410 2024-04-26T22:01:54.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.703+0000 7ffa588dc640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54103410 0x7ffa54198b10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400b3f0 tx=0x7ffa4400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:54.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.703+0000 7ffa51ffb640 1 -- 172.21.15.55:0/873480909 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44015020 con 0x7ffa54103410 2024-04-26T22:01:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.703+0000 7ffa5ab40640 1 -- 172.21.15.55:0/873480909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa541a0a20 con 0x7ffa54103410 2024-04-26T22:01:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.703+0000 7ffa51ffb640 1 -- 172.21.15.55:0/873480909 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa44011ce0 con 0x7ffa54103410 2024-04-26T22:01:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.704+0000 7ffa5ab40640 1 -- 172.21.15.55:0/873480909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa541a0fc0 con 0x7ffa54103410 2024-04-26T22:01:54.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.704+0000 7ffa51ffb640 1 -- 172.21.15.55:0/873480909 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44010d80 con 0x7ffa54103410 2024-04-26T22:01:54.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.707+0000 7ffa5d856740 1 Processor -- start 2024-04-26T22:01:54.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.707+0000 7ffa5d856740 1 -- start start 2024-04-26T22:01:54.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd0ee0 0x55bd42fd1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 0x55bd42fd2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 0x55bd42fd9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa5d856740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bd42f35d80 con 0x55bd42fd0ee0 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa5d856740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bd42eef8b0 con 0x55bd42fd20b0 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa5d856740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd42f31d40 con 0x55bd42fd2a70 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa590dd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd0ee0 0x55bd42fd1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa590dd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd0ee0 0x55bd42fd1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42326/0 (socket says 172.21.15.55:42326) 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa590dd640 1 -- 172.21.15.55:0/2401692348 learned_addr learned my addr 172.21.15.55:0/2401692348 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa51ffb640 1 -- 172.21.15.55:0/873480909 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffa44018020 con 0x7ffa54103410 2024-04-26T22:01:54.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.708+0000 7ffa53fff640 1 --2- 172.21.15.55:0/2401692348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 0x55bd42fd9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa590dd640 1 -- 172.21.15.55:0/2401692348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 msgr2=0x55bd42fd9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:54.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa590dd640 1 --2- 172.21.15.55:0/2401692348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 0x55bd42fd9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa590dd640 1 -- 172.21.15.55:0/2401692348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 msgr2=0x55bd42fd2530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa590dd640 1 --2- 172.21.15.55:0/2401692348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 0x55bd42fd2530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa590dd640 1 -- 172.21.15.55:0/2401692348 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd42ec0a40 con 0x55bd42fd0ee0 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa590dd640 1 --2- 172.21.15.55:0/2401692348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd0ee0 0x55bd42fd1300 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009940 tx=0x7ffa4c02f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd59e7d43dacd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa3affd640 1 -- 172.21.15.55:0/2401692348 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c031870 con 0x55bd42fd0ee0 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa3affd640 1 -- 172.21.15.55:0/2401692348 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4c040070 con 0x55bd42fd0ee0 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa3affd640 1 -- 172.21.15.55:0/2401692348 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c02f620 con 0x55bd42fd0ee0 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.709+0000 7ffa51ffb640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffa3c0a3c80 0x7ffa3c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa51ffb640 1 -- 172.21.15.55:0/873480909 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ffa440cfd90 con 0x7ffa54103410 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa588dc640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffa3c0a3c80 0x7ffa3c0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa5d856740 1 -- 172.21.15.55:0/2401692348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd0ee0 msgr2=0x55bd42fd1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa5d856740 1 --2- 172.21.15.55:0/2401692348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd0ee0 0x55bd42fd1300 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009940 tx=0x7ffa4c02f510 comp rx=0 tx=0).stop 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa5d856740 1 -- 172.21.15.55:0/2401692348 shutdown_connections 2024-04-26T22:01:54.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa5d856740 1 --2- 172.21.15.55:0/2401692348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 0x55bd42fd9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa5d856740 1 --2- 172.21.15.55:0/2401692348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 0x55bd42fd2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa5d856740 1 --2- 172.21.15.55:0/2401692348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd0ee0 0x55bd42fd1300 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.710+0000 7ffa5d856740 1 -- 172.21.15.55:0/2401692348 >> 172.21.15.55:0/2401692348 conn(0x55bd42fcc3d0 msgr2=0x55bd42fcc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.712+0000 7ffa588dc640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffa3c0a3c80 0x7ffa3c0a6140 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffa440a4720 tx=0x7ffa440107e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:54.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.712+0000 7ffa5d856740 1 -- 172.21.15.55:0/2401692348 shutdown_connections 2024-04-26T22:01:54.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.712+0000 7ffa5d856740 1 -- 172.21.15.55:0/2401692348 wait complete. 2024-04-26T22:01:54.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.713+0000 7ffa5d856740 1 -- start start 2024-04-26T22:01:54.715 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.713+0000 7ffa5d856740 1 Processor -- start 2024-04-26T22:01:54.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.714+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd20b0 0x55bd42fdb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd42fe2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa5d856740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bd42fd9a20 con 0x55bd42fd2a70 2024-04-26T22:01:54.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa590dd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd42fe2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa590dd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd42fe2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42340/0 (socket says 172.21.15.55:42340) 2024-04-26T22:01:54.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa590dd640 1 -- 172.21.15.55:0/3497253309 learned_addr learned my addr 172.21.15.55:0/3497253309 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:54.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bd42ebcd10 con 0x55bd42fd0ee0 2024-04-26T22:01:54.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa53fff640 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd20b0 0x55bd42fdb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.717 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.715+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bd42fd9ba0 con 0x55bd42fd20b0 2024-04-26T22:01:54.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.717+0000 7ffa590dd640 1 -- 172.21.15.55:0/3497253309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd20b0 msgr2=0x55bd42fdb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:54.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.717+0000 7ffa590dd640 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd20b0 0x55bd42fdb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.717+0000 7ffa590dd640 1 -- 172.21.15.55:0/3497253309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd0ee0 msgr2=0x55bd42fdb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:54.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.717+0000 7ffa590dd640 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.717+0000 7ffa590dd640 1 -- 172.21.15.55:0/3497253309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd42d117d0 con 0x55bd42fd2a70 2024-04-26T22:01:54.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.717+0000 7ffa5d856740 10 client.0 _ll_register_callbacks cb 0x55bd42ee9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:54.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.717+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c040070 con 0x55bd42fd2a70 2024-04-26T22:01:54.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.718+0000 7ffa590dd640 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd42fe2700 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0044a0 tx=0x7ffa4c02f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ccb197ce0f9711 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:54.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.718+0000 7ffa33fff640 10 client.16851 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:01:54.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.719+0000 7ffa33fff640 1 -- 172.21.15.55:0/3497253309 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c003b80 con 0x55bd42fd2a70 2024-04-26T22:01:54.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.719+0000 7ffa33fff640 1 -- 172.21.15.55:0/3497253309 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ffa4c003d20 con 0x55bd42fd2a70 2024-04-26T22:01:54.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.720+0000 7ffa33fff640 1 -- 172.21.15.55:0/3497253309 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c02f7c0 con 0x55bd42fd2a70 2024-04-26T22:01:54.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.720+0000 7ffa5d856740 10 client.16851 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:54.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.720+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55bd42c473f0 con 0x55bd42fd2a70 2024-04-26T22:01:54.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.721+0000 7ffa33fff640 1 -- 172.21.15.55:0/3497253309 <== mon.0 v2:172.21.15.55:3300/0 4 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ffa4c02f960 con 0x55bd42fd2a70 2024-04-26T22:01:54.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.721+0000 7ffa5d856740 10 client.16851 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:54.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.721+0000 7ffa5d856740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:54.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.721+0000 7ffa5d856740 20 client.16851 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:54.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.723+0000 7ffa5d856740 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bd42feaf50 0x55bd42fed3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.723+0000 7ffa5d856740 4 client.16851 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:01:54.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.723+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd42f43990 con 0x55bd42feaf50 2024-04-26T22:01:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.723+0000 7ffa53fff640 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bd42feaf50 0x55bd42fed3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.727+0000 7ffa33fff640 10 client.16851 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:54.728 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.727+0000 7ffa53fff640 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bd42feaf50 0x55bd42fed3b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5859c30bf8c2e6b1 server_cookie=6720397d4cbd4728 in_seq=0 out_seq=0 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa33fff640 1 -- 172.21.15.55:0/3497253309 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd42f43990 con 0x55bd42feaf50 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa33fff640 10 client.16851 handle_command_reply: tid=0 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 client.16851 shutdown 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 msgr2=0x55bd42fe2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd42fe2700 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0044a0 tx=0x7ffa4c02f620 comp rx=0 tx=0).stop 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 shutdown_connections 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bd42feaf50 0x55bd42fed3b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd42fe2700 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd20b0 0x55bd42fdb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 --2- 172.21.15.55:0/3497253309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 >> 172.21.15.55:0/3497253309 conn(0x55bd42fcc3d0 msgr2=0x55bd42fcb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.733+0000 7ffa33fff640 0 client.16851 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:54.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.734+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 shutdown_connections 2024-04-26T22:01:54.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.734+0000 7ffa5d856740 1 -- 172.21.15.55:0/3497253309 wait complete. 2024-04-26T22:01:54.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.734+0000 7ffa5d856740 20 client.16851 trim_cache size 0 max 16384 2024-04-26T22:01:54.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.772+0000 7ffa5d856740 1 Processor -- start 2024-04-26T22:01:54.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.772+0000 7ffa5d856740 1 -- start start 2024-04-26T22:01:54.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.773+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd0ee0 0x55bd42fcacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.773+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd20b0 0x55bd42fe2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.773+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 0x55bd42fcb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.773+0000 7ffa5d856740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541aa520 con 0x55bd42fd20b0 2024-04-26T22:01:54.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.773+0000 7ffa5d856740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541a7df0 con 0x55bd42fd0ee0 2024-04-26T22:01:54.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.773+0000 7ffa5d856740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541a5c40 con 0x55bd42fd2a70 2024-04-26T22:01:54.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.775+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd20b0 0x55bd42fe2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.775+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd20b0 0x55bd42fe2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42350/0 (socket says 172.21.15.55:42350) 2024-04-26T22:01:54.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.775+0000 7ffa53fff640 1 -- 172.21.15.55:0/2314504479 learned_addr learned my addr 172.21.15.55:0/2314504479 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:54.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.778+0000 7ffa53fff640 1 -- 172.21.15.55:0/2314504479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 msgr2=0x55bd42fcb200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:01:54.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.778+0000 7ffa53fff640 1 --2- 172.21.15.55:0/2314504479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 0x55bd42fcb200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.778+0000 7ffa53fff640 1 -- 172.21.15.55:0/2314504479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd0ee0 msgr2=0x55bd42fcacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:54.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.778+0000 7ffa53fff640 1 --2- 172.21.15.55:0/2314504479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd0ee0 0x55bd42fcacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.780 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.778+0000 7ffa53fff640 1 -- 172.21.15.55:0/2314504479 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd42c070f0 con 0x55bd42fd20b0 2024-04-26T22:01:54.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.779+0000 7ffa53fff640 1 --2- 172.21.15.55:0/2314504479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd20b0 0x55bd42fe2690 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ffa5419a020 tx=0x7ffa4800b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3988c811a01f969 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:54.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.781+0000 7ffa3a7fc640 1 -- 172.21.15.55:0/2314504479 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4801c250 con 0x55bd42fd20b0 2024-04-26T22:01:54.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.782+0000 7ffa3a7fc640 1 -- 172.21.15.55:0/2314504479 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa48020030 con 0x55bd42fd20b0 2024-04-26T22:01:54.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.784+0000 7ffa5d856740 1 -- 172.21.15.55:0/2314504479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd20b0 msgr2=0x55bd42fe2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.784+0000 7ffa5d856740 1 --2- 172.21.15.55:0/2314504479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd20b0 0x55bd42fe2690 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ffa5419a020 tx=0x7ffa4800b5f0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.794+0000 7ffa5d856740 1 -- 172.21.15.55:0/2314504479 shutdown_connections 2024-04-26T22:01:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.794+0000 7ffa5d856740 1 --2- 172.21.15.55:0/2314504479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd2a70 0x55bd42fcb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.794+0000 7ffa5d856740 1 --2- 172.21.15.55:0/2314504479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd20b0 0x55bd42fe2690 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.794+0000 7ffa5d856740 1 --2- 172.21.15.55:0/2314504479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd0ee0 0x55bd42fcacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.794+0000 7ffa5d856740 1 -- 172.21.15.55:0/2314504479 >> 172.21.15.55:0/2314504479 conn(0x55bd42fcc3d0 msgr2=0x55bd42fdb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.798+0000 7ffa5d856740 1 -- 172.21.15.55:0/2314504479 shutdown_connections 2024-04-26T22:01:54.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.798+0000 7ffa5d856740 1 -- 172.21.15.55:0/2314504479 wait complete. 2024-04-26T22:01:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.799+0000 7ffa5d856740 1 -- start start 2024-04-26T22:01:54.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.799+0000 7ffa5d856740 1 Processor -- start 2024-04-26T22:01:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.800+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.801+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 0x55bd42fdfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.801+0000 7ffa5d856740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd4304f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.801+0000 7ffa5d856740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541a62b0 con 0x55bd42fd2a70 2024-04-26T22:01:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.801+0000 7ffa5d856740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541a7720 con 0x55bd42fd20b0 2024-04-26T22:01:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.801+0000 7ffa5d856740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541a6980 con 0x55bd42fd0ee0 2024-04-26T22:01:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.801+0000 7ffa590dd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.801+0000 7ffa590dd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:43278/0 (socket says 172.21.15.55:43278) 2024-04-26T22:01:54.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.801+0000 7ffa590dd640 1 -- 172.21.15.55:0/847144553 learned_addr learned my addr 172.21.15.55:0/847144553 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:01:54.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.802+0000 7ffa53fff640 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 0x55bd42fdfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.803+0000 7ffa5d856740 10 client.0 _ll_register_callbacks cb 0x7ffa4800b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:01:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.805+0000 7ffa590dd640 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd4304f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.806+0000 7ffa590dd640 1 -- 172.21.15.55:0/847144553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 msgr2=0x55bd42fdfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:54.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.806+0000 7ffa590dd640 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 0x55bd42fdfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.806+0000 7ffa590dd640 1 -- 172.21.15.55:0/847144553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 msgr2=0x55bd4304f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:01:54.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.806+0000 7ffa590dd640 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd4304f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.806+0000 7ffa590dd640 1 -- 172.21.15.55:0/847144553 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd42ef64b0 con 0x55bd42fd0ee0 2024-04-26T22:01:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.811+0000 7ffa590dd640 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdf600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0049d0 tx=0x7ffa4c047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68160bb95e944ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:01:54.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.811+0000 7ffa33fff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:01:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.812+0000 7ffa33fff640 1 -- 172.21.15.55:0/847144553 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c061040 con 0x55bd42fd0ee0 2024-04-26T22:01:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.812+0000 7ffa33fff640 1 -- 172.21.15.55:0/847144553 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ffa4c004d50 con 0x55bd42fd0ee0 2024-04-26T22:01:54.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.812+0000 7ffa5d856740 1 -- 172.21.15.55:0/847144553 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa48020030 con 0x55bd42fd0ee0 2024-04-26T22:01:54.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.816+0000 7ffa33fff640 1 -- 172.21.15.55:0/847144553 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c06d050 con 0x55bd42fd0ee0 2024-04-26T22:01:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.816+0000 7ffa5d856740 10 client.26008 fetch_fsmap learned FSMap version 23 2024-04-26T22:01:54.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.816+0000 7ffa5d856740 1 -- 172.21.15.55:0/847144553 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55bd43050bb0 con 0x55bd42fd0ee0 2024-04-26T22:01:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.817+0000 7ffa33fff640 1 -- 172.21.15.55:0/847144553 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4c040070 con 0x55bd42fd0ee0 2024-04-26T22:01:54.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.818+0000 7ffa33fff640 1 -- 172.21.15.55:0/847144553 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ffa4c030d40 con 0x55bd42fd0ee0 2024-04-26T22:01:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.818+0000 7ffa5d856740 10 client.26008 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:01:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.818+0000 7ffa5d856740 10 client.26008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:01:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.818+0000 7ffa5d856740 20 client.26008 populate_metadata read hostname 'smithi055' 2024-04-26T22:01:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.818+0000 7ffa5d856740 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bd43051550 0x55bd43053970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:01:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.818+0000 7ffa5d856740 4 client.26008 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:01:54.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.818+0000 7ffa5d856740 1 -- 172.21.15.55:0/847144553 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ffa48020030 con 0x55bd43051550 2024-04-26T22:01:54.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.823+0000 7ffa53fff640 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bd43051550 0x55bd43053970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:01:54.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.845+0000 7ffa53fff640 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bd43051550 0x55bd43053970 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cfbce4d1b62032 server_cookie=fdd003f2502a1215 in_seq=0 out_seq=0 2024-04-26T22:01:54.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.847+0000 7ffa33fff640 10 client.26008 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.858+0000 7ffa33fff640 1 -- 172.21.15.55:0/847144553 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x7ffa48020030 con 0x55bd43051550 2024-04-26T22:01:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.858+0000 7ffa33fff640 10 client.26008 handle_command_reply: tid=0 2024-04-26T22:01:54.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.858+0000 7ffa5d856740 1 client.26008 shutdown 2024-04-26T22:01:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.861+0000 7ffa5d856740 1 -- 172.21.15.55:0/847144553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd0ee0 msgr2=0x55bd42fdf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.862+0000 7ffa5d856740 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdf600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0049d0 tx=0x7ffa4c047040 comp rx=0 tx=0).stop 2024-04-26T22:01:54.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.862+0000 7ffa5d856740 1 -- 172.21.15.55:0/847144553 shutdown_connections 2024-04-26T22:01:54.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.862+0000 7ffa5d856740 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bd43051550 0x55bd43053970 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.862+0000 7ffa5d856740 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bd42fd2a70 0x55bd4304f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.862+0000 7ffa5d856740 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bd42fd20b0 0x55bd42fdfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.862+0000 7ffa5d856740 1 --2- 172.21.15.55:0/847144553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bd42fd0ee0 0x55bd42fdf600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.863+0000 7ffa5d856740 1 -- 172.21.15.55:0/847144553 >> 172.21.15.55:0/847144553 conn(0x55bd42fcc3d0 msgr2=0x55bd42fcf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.863+0000 7ffa33fff640 0 client.26008 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:01:54.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.867+0000 7ffa5d856740 1 -- 172.21.15.55:0/847144553 shutdown_connections 2024-04-26T22:01:54.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.867+0000 7ffa5d856740 1 -- 172.21.15.55:0/847144553 wait complete. 2024-04-26T22:01:54.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.871+0000 7ffa5d856740 20 client.26008 trim_cache size 0 max 16384 2024-04-26T22:01:54.879 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:01:54.879 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (0 inodes in the stack)", 2024-04-26T22:01:54.880 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:01:54.880 INFO:teuthology.orchestra.run.smithi055.stdout: "a81f3312-803e-43fa-a05e-cecf6dad8af3": { 2024-04-26T22:01:54.880 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:01:54.880 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "a81f3312-803e-43fa-a05e-cecf6dad8af3", 2024-04-26T22:01:54.880 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:01:54.880 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:01:54.880 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:01:54.880 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:01:54.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.879+0000 7ffa5ab40640 1 -- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffa3c0a3c80 msgr2=0x7ffa3c0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.879+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffa3c0a3c80 0x7ffa3c0a6140 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffa440a4720 tx=0x7ffa440107e0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.879+0000 7ffa5ab40640 1 -- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54103410 msgr2=0x7ffa54198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:01:54.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.879+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54103410 0x7ffa54198b10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400b3f0 tx=0x7ffa4400b8b0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.887+0000 7ffa5ab40640 1 -- 172.21.15.55:0/873480909 shutdown_connections 2024-04-26T22:01:54.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.887+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ffa3c0a3c80 0x7ffa3c0a6140 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.887+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffa54108670 0x7ffa541a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.888+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffa54105d40 0x7ffa54199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.888+0000 7ffa5ab40640 1 --2- 172.21.15.55:0/873480909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa54103410 0x7ffa54198b10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:01:54.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.888+0000 7ffa5ab40640 1 -- 172.21.15.55:0/873480909 >> 172.21.15.55:0/873480909 conn(0x7ffa540fce20 msgr2=0x7ffa54103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:01:54.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.893+0000 7ffa5ab40640 1 -- 172.21.15.55:0/873480909 shutdown_connections 2024-04-26T22:01:54.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:01:54.893+0000 7ffa5ab40640 1 -- 172.21.15.55:0/873480909 wait complete. 2024-04-26T22:01:54.958 INFO:tasks.cephfs.filesystem:scrub status for tag:a81f3312-803e-43fa-a05e-cecf6dad8af3 - {'path': '/', 'tag': 'a81f3312-803e-43fa-a05e-cecf6dad8af3', 'options': 'recursive,force'} 2024-04-26T22:01:55.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:54 smithi055 ceph-mon[29221]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 MiB/s wr, 2.02k op/s 2024-04-26T22:01:55.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:54 smithi064 ceph-mon[32765]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 MiB/s wr, 2.02k op/s 2024-04-26T22:01:55.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:54 smithi165 ceph-mon[33296]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 MiB/s wr, 2.02k op/s 2024-04-26T22:01:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:55 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:01:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:55 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:55 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:01:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:55 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:01:56.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:55 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:55 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:01:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:55 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:56.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:55 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:01:56.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:55 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:01:56.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:55 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:56.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:55 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:01:56.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:55 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:55 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:01:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:55 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:01:56.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:55 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:57.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:56 smithi055 ceph-mon[29221]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 828 MiB/s wr, 2.16k op/s 2024-04-26T22:01:57.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:56 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:56 smithi064 ceph-mon[32765]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 828 MiB/s wr, 2.16k op/s 2024-04-26T22:01:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:56 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:57.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:56 smithi165 ceph-mon[33296]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 828 MiB/s wr, 2.16k op/s 2024-04-26T22:01:57.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:56 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:01:59.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:01:58 smithi055 ceph-mon[29221]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 906 MiB/s wr, 2.29k op/s 2024-04-26T22:01:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:01:58 smithi064 ceph-mon[32765]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 906 MiB/s wr, 2.29k op/s 2024-04-26T22:01:59.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:01:58 smithi165 ceph-mon[33296]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 906 MiB/s wr, 2.29k op/s 2024-04-26T22:02:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:00 smithi055 ceph-mon[29221]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 888 MiB/s wr, 2.10k op/s 2024-04-26T22:02:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:00 smithi064 ceph-mon[32765]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 888 MiB/s wr, 2.10k op/s 2024-04-26T22:02:01.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:00 smithi165 ceph-mon[33296]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 888 MiB/s wr, 2.10k op/s 2024-04-26T22:02:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:01 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:02:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:01 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:02:02.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:01 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:02:02.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:01 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:02:02.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:01 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:02:02.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:01 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:02:03.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:02 smithi064 ceph-mon[32765]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 841 MiB/s wr, 2.09k op/s 2024-04-26T22:02:03.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:02 smithi165 ceph-mon[33296]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 841 MiB/s wr, 2.09k op/s 2024-04-26T22:02:03.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:02 smithi055 ceph-mon[29221]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 841 MiB/s wr, 2.09k op/s 2024-04-26T22:02:05.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:04 smithi064 ceph-mon[32765]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 2.23k op/s 2024-04-26T22:02:05.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:04 smithi165 ceph-mon[33296]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 2.23k op/s 2024-04-26T22:02:05.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:04 smithi055 ceph-mon[29221]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 2.23k op/s 2024-04-26T22:02:07.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:06 smithi064 ceph-mon[32765]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 818 MiB/s wr, 2.08k op/s 2024-04-26T22:02:07.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:06 smithi165 ceph-mon[33296]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 818 MiB/s wr, 2.08k op/s 2024-04-26T22:02:07.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:06 smithi055 ceph-mon[29221]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 818 MiB/s wr, 2.08k op/s 2024-04-26T22:02:09.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:08 smithi064 ceph-mon[32765]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 MiB/s wr, 2.22k op/s 2024-04-26T22:02:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:08 smithi165 ceph-mon[33296]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 MiB/s wr, 2.22k op/s 2024-04-26T22:02:09.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:08 smithi055 ceph-mon[29221]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 MiB/s wr, 2.22k op/s 2024-04-26T22:02:11.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:11 smithi165 ceph-mon[33296]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 2.08k op/s 2024-04-26T22:02:11.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:11 smithi055 ceph-mon[29221]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 2.08k op/s 2024-04-26T22:02:12.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:11 smithi064 ceph-mon[32765]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 2.08k op/s 2024-04-26T22:02:13.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:13 smithi165 ceph-mon[33296]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 839 MiB/s wr, 1.98k op/s 2024-04-26T22:02:13.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:13 smithi055 ceph-mon[29221]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 839 MiB/s wr, 1.98k op/s 2024-04-26T22:02:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:13 smithi064 ceph-mon[32765]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 839 MiB/s wr, 1.98k op/s 2024-04-26T22:02:15.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:15 smithi165 ceph-mon[33296]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 1.91k op/s 2024-04-26T22:02:15.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:15 smithi055 ceph-mon[29221]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 1.91k op/s 2024-04-26T22:02:16.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:15 smithi064 ceph-mon[32765]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 1.91k op/s 2024-04-26T22:02:17.944 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:17 smithi055 ceph-mon[29221]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 811 MiB/s wr, 1.67k op/s 2024-04-26T22:02:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:17 smithi064 ceph-mon[32765]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 811 MiB/s wr, 1.67k op/s 2024-04-26T22:02:18.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:17 smithi165 ceph-mon[33296]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 811 MiB/s wr, 1.67k op/s 2024-04-26T22:02:19.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:19 smithi055 ceph-mon[29221]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 893 MiB/s wr, 1.60k op/s 2024-04-26T22:02:20.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:19 smithi064 ceph-mon[32765]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 893 MiB/s wr, 1.60k op/s 2024-04-26T22:02:20.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:19 smithi165 ceph-mon[33296]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 893 MiB/s wr, 1.60k op/s 2024-04-26T22:02:21.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:21 smithi055 ceph-mon[29221]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 849 MiB/s wr, 1.29k op/s 2024-04-26T22:02:22.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:21 smithi064 ceph-mon[32765]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 849 MiB/s wr, 1.29k op/s 2024-04-26T22:02:22.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:21 smithi165 ceph-mon[33296]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 849 MiB/s wr, 1.29k op/s 2024-04-26T22:02:23.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:23 smithi055 ceph-mon[29221]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 817 MiB/s wr, 1.23k op/s 2024-04-26T22:02:24.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:23 smithi064 ceph-mon[32765]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 817 MiB/s wr, 1.23k op/s 2024-04-26T22:02:24.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:23 smithi165 ceph-mon[33296]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 817 MiB/s wr, 1.23k op/s 2024-04-26T22:02:24.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:02:24.959 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:02:25.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.307+0000 7f8b5a37f640 1 -- 172.21.15.55:0/333771804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 msgr2=0x7f8b541113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.311 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.307+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/333771804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 0x7f8b541113d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009860 tx=0x7f8b4402a360 comp rx=0 tx=0).stop 2024-04-26T22:02:25.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.310+0000 7f8b5a37f640 1 -- 172.21.15.55:0/333771804 shutdown_connections 2024-04-26T22:02:25.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.310+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/333771804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 0x7f8b541113d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.310+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/333771804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5410a180 0x7f8b5410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.310+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/333771804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b54108f90 0x7f8b541093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.310+0000 7f8b5a37f640 1 -- 172.21.15.55:0/333771804 >> 172.21.15.55:0/333771804 conn(0x7f8b54077f40 msgr2=0x7f8b5407a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.315+0000 7f8b5a37f640 1 -- 172.21.15.55:0/333771804 shutdown_connections 2024-04-26T22:02:25.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.329+0000 7f8b5a37f640 1 -- 172.21.15.55:0/333771804 wait complete. 2024-04-26T22:02:25.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.331+0000 7f8b5a37f640 1 Processor -- start 2024-04-26T22:02:25.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.331+0000 7f8b5a37f640 1 -- start start 2024-04-26T22:02:25.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.332+0000 7f8b5a37f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b54108f90 0x7f8b54071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.332+0000 7f8b5a37f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5410a180 0x7f8b540718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.334 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.332+0000 7f8b5a37f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 0x7f8b541af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.332+0000 7f8b5a37f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54114ad0 con 0x7f8b54108f90 2024-04-26T22:02:25.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.332+0000 7f8b5a37f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54114950 con 0x7f8b5410a180 2024-04-26T22:02:25.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.332+0000 7f8b5a37f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b54114c50 con 0x7f8b5410ab40 2024-04-26T22:02:25.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.336+0000 7f8b53fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b54108f90 0x7f8b54071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.336+0000 7f8b53fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b54108f90 0x7f8b54071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55922/0 (socket says 172.21.15.55:55922) 2024-04-26T22:02:25.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.336+0000 7f8b53fff640 1 -- 172.21.15.55:0/3700137681 learned_addr learned my addr 172.21.15.55:0/3700137681 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:25.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.336+0000 7f8b5891c640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 0x7f8b541af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.338+0000 7f8b537fe640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5410a180 0x7f8b540718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.345+0000 7f8b5891c640 1 -- 172.21.15.55:0/3700137681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5410a180 msgr2=0x7f8b540718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.345+0000 7f8b5891c640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5410a180 0x7f8b540718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.345+0000 7f8b5891c640 1 -- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b54108f90 msgr2=0x7f8b54071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.345+0000 7f8b5891c640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b54108f90 0x7f8b54071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.345+0000 7f8b5891c640 1 -- 172.21.15.55:0/3700137681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5407b180 con 0x7f8b5410ab40 2024-04-26T22:02:25.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.350+0000 7f8b5891c640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 0x7f8b541af1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b4402a950 tx=0x7f8b44032cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.354+0000 7f8b517fa640 1 -- 172.21.15.55:0/3700137681 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44037050 con 0x7f8b5410ab40 2024-04-26T22:02:25.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.355+0000 7f8b5a37f640 1 -- 172.21.15.55:0/3700137681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5407b410 con 0x7f8b5410ab40 2024-04-26T22:02:25.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.355+0000 7f8b5a37f640 1 -- 172.21.15.55:0/3700137681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5407b900 con 0x7f8b5410ab40 2024-04-26T22:02:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.359+0000 7f8b517fa640 1 -- 172.21.15.55:0/3700137681 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b44038040 con 0x7f8b5410ab40 2024-04-26T22:02:25.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.359+0000 7f8b517fa640 1 -- 172.21.15.55:0/3700137681 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b440046c0 con 0x7f8b5410ab40 2024-04-26T22:02:25.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.362+0000 7f8b5d173740 1 Processor -- start 2024-04-26T22:02:25.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.362+0000 7f8b5d173740 1 -- start start 2024-04-26T22:02:25.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ebfee0 0x562350ec0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350ec1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ec8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b5d173740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562350e24d80 con 0x562350ec10b0 2024-04-26T22:02:25.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b5d173740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562350dde8b0 con 0x562350ebfee0 2024-04-26T22:02:25.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b5d173740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562350e20d40 con 0x562350ec1a70 2024-04-26T22:02:25.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b53fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350ec1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b53fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350ec1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55924/0 (socket says 172.21.15.55:55924) 2024-04-26T22:02:25.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.363+0000 7f8b53fff640 1 -- 172.21.15.55:0/328051273 learned_addr learned my addr 172.21.15.55:0/328051273 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:25.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.364+0000 7f8b53fff640 1 -- 172.21.15.55:0/328051273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 msgr2=0x562350ec8300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:25.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.364+0000 7f8b53fff640 1 --2- 172.21.15.55:0/328051273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ec8300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.364+0000 7f8b53fff640 1 -- 172.21.15.55:0/328051273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ebfee0 msgr2=0x562350ec0300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:25.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.364+0000 7f8b53fff640 1 --2- 172.21.15.55:0/328051273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ebfee0 0x562350ec0300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.364+0000 7f8b53fff640 1 -- 172.21.15.55:0/328051273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562350dafa40 con 0x562350ec10b0 2024-04-26T22:02:25.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.365+0000 7f8b53fff640 1 --2- 172.21.15.55:0/328051273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350ec1530 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009d00 tx=0x7f8b480117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fc0812baffe10d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.365+0000 7f8b3a7fc640 1 -- 172.21.15.55:0/328051273 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48012450 con 0x562350ec10b0 2024-04-26T22:02:25.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.366+0000 7f8b3a7fc640 1 -- 172.21.15.55:0/328051273 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b48012aa0 con 0x562350ec10b0 2024-04-26T22:02:25.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.366+0000 7f8b3a7fc640 1 -- 172.21.15.55:0/328051273 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800fb90 con 0x562350ec10b0 2024-04-26T22:02:25.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.367+0000 7f8b5d173740 1 -- 172.21.15.55:0/328051273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 msgr2=0x562350ec1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.367+0000 7f8b5d173740 1 --2- 172.21.15.55:0/328051273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350ec1530 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009d00 tx=0x7f8b480117b0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.369+0000 7f8b5d173740 1 -- 172.21.15.55:0/328051273 shutdown_connections 2024-04-26T22:02:25.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.369+0000 7f8b5d173740 1 --2- 172.21.15.55:0/328051273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ec8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.369+0000 7f8b5d173740 1 --2- 172.21.15.55:0/328051273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350ec1530 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.369+0000 7f8b5d173740 1 --2- 172.21.15.55:0/328051273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ebfee0 0x562350ec0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.369+0000 7f8b5d173740 1 -- 172.21.15.55:0/328051273 >> 172.21.15.55:0/328051273 conn(0x562350ebb3d0 msgr2=0x562350ebb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.369+0000 7f8b517fa640 1 -- 172.21.15.55:0/3700137681 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8b44004040 con 0x7f8b5410ab40 2024-04-26T22:02:25.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.370+0000 7f8b5d173740 1 -- 172.21.15.55:0/328051273 shutdown_connections 2024-04-26T22:02:25.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.370+0000 7f8b5d173740 1 -- 172.21.15.55:0/328051273 wait complete. 2024-04-26T22:02:25.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.371+0000 7f8b517fa640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b3c0a3d50 0x7f8b3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.371+0000 7f8b5d173740 1 -- start start 2024-04-26T22:02:25.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.371+0000 7f8b517fa640 1 -- 172.21.15.55:0/3700137681 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8b440f0a40 con 0x7f8b5410ab40 2024-04-26T22:02:25.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.371+0000 7f8b5d173740 1 Processor -- start 2024-04-26T22:02:25.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.372+0000 7f8b53fff640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b3c0a3d50 0x7f8b3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.372+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ebfee0 0x562350eca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.372+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.372+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ec1a70 0x562350ed19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.372+0000 7f8b5d173740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562350ec8a80 con 0x562350ec10b0 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.372+0000 7f8b5d173740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562350dabd10 con 0x562350ec1a70 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.372+0000 7f8b5d173740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562350ec8c00 con 0x562350ebfee0 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.372+0000 7f8b5d173740 10 client.0 _ll_register_callbacks cb 0x562350dd8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.373+0000 7f8b537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.373+0000 7f8b537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55928/0 (socket says 172.21.15.55:55928) 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.373+0000 7f8b537fe640 1 -- 172.21.15.55:0/3312318626 learned_addr learned my addr 172.21.15.55:0/3312318626 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.374+0000 7f8b537fe640 1 -- 172.21.15.55:0/3312318626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ebfee0 msgr2=0x562350eca430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.374+0000 7f8b537fe640 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ebfee0 0x562350eca430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.374+0000 7f8b537fe640 1 -- 172.21.15.55:0/3312318626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ec1a70 msgr2=0x562350ed19b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:25.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.374+0000 7f8b537fe640 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ec1a70 0x562350ed19b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.374+0000 7f8b537fe640 1 -- 172.21.15.55:0/3312318626 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562350c007d0 con 0x562350ec10b0 2024-04-26T22:02:25.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.374+0000 7f8b53fff640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b3c0a3d50 0x7f8b3c0a6210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800f670 tx=0x7f8b48005890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.374+0000 7f8b537fe640 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eca970 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0119b0 tx=0x7f8b4c011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb9bedada5b9b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.374+0000 7f8b337fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:02:25.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.375+0000 7f8b337fe640 1 -- 172.21.15.55:0/3312318626 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c01e060 con 0x562350ec10b0 2024-04-26T22:02:25.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.375+0000 7f8b337fe640 1 -- 172.21.15.55:0/3312318626 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8b4c00f880 con 0x562350ec10b0 2024-04-26T22:02:25.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.375+0000 7f8b5d173740 1 -- 172.21.15.55:0/3312318626 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b48012aa0 con 0x562350ec10b0 2024-04-26T22:02:25.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.376+0000 7f8b337fe640 1 -- 172.21.15.55:0/3312318626 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8b4c003e40 con 0x562350ec10b0 2024-04-26T22:02:25.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.376+0000 7f8b5d173740 10 client.16869 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:25.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.376+0000 7f8b5d173740 1 -- 172.21.15.55:0/3312318626 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562350b363f0 con 0x562350ec10b0 2024-04-26T22:02:25.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.377+0000 7f8b337fe640 1 -- 172.21.15.55:0/3312318626 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b48012aa0 con 0x562350ec10b0 2024-04-26T22:02:25.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.377+0000 7f8b337fe640 1 -- 172.21.15.55:0/3312318626 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8b4c0223a0 con 0x562350ec10b0 2024-04-26T22:02:25.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.377+0000 7f8b5d173740 10 client.16869 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:25.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.377+0000 7f8b5d173740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:25.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.377+0000 7f8b5d173740 20 client.16869 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:25.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.377+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562350ed5fd0 0x562350ed8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.377+0000 7f8b5d173740 4 client.16869 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:02:25.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.377+0000 7f8b5d173740 1 -- 172.21.15.55:0/3312318626 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562350e32990 con 0x562350ed5fd0 2024-04-26T22:02:25.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.383+0000 7f8b53fff640 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562350ed5fd0 0x562350ed8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.393+0000 7f8b53fff640 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562350ed5fd0 0x562350ed8410 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b27a90b8cf7f44be server_cookie=3e0e365535ea382b in_seq=0 out_seq=0 2024-04-26T22:02:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.393+0000 7f8b337fe640 10 client.16869 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.399+0000 7f8b337fe640 1 -- 172.21.15.55:0/3312318626 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562350e32990 con 0x562350ed5fd0 2024-04-26T22:02:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.399+0000 7f8b337fe640 10 client.16869 handle_command_reply: tid=0 2024-04-26T22:02:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.399+0000 7f8b5d173740 1 client.16869 shutdown 2024-04-26T22:02:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 -- 172.21.15.55:0/3312318626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 msgr2=0x562350eca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eca970 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0119b0 tx=0x7f8b4c011e70 comp rx=0 tx=0).stop 2024-04-26T22:02:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 -- 172.21.15.55:0/3312318626 shutdown_connections 2024-04-26T22:02:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ec1a70 0x562350ed19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562350ed5fd0 0x562350ed8410 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eca970 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3312318626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ebfee0 0x562350eca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 -- 172.21.15.55:0/3312318626 >> 172.21.15.55:0/3312318626 conn(0x562350ebb3d0 msgr2=0x562350eba550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b337fe640 0 client.16869 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:25.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 -- 172.21.15.55:0/3312318626 shutdown_connections 2024-04-26T22:02:25.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 1 -- 172.21.15.55:0/3312318626 wait complete. 2024-04-26T22:02:25.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.400+0000 7f8b5d173740 20 client.16869 trim_cache size 0 max 16384 2024-04-26T22:02:25.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.430+0000 7f8b5d173740 1 Processor -- start 2024-04-26T22:02:25.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.430+0000 7f8b5d173740 1 -- start start 2024-04-26T22:02:25.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.430+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ebfee0 0x562350ec4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.430+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ec10b0 0x562350ec5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ec9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b5d173740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8b40010e60 con 0x562350ebfee0 2024-04-26T22:02:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b5d173740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4000e730 con 0x562350ec10b0 2024-04-26T22:02:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b5d173740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4000c580 con 0x562350ec1a70 2024-04-26T22:02:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b53fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ebfee0 0x562350ec4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b53fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ebfee0 0x562350ec4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55942/0 (socket says 172.21.15.55:55942) 2024-04-26T22:02:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b53fff640 1 -- 172.21.15.55:0/3889916644 learned_addr learned my addr 172.21.15.55:0/3889916644 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b5891c640 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ec9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b53fff640 1 -- 172.21.15.55:0/3889916644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 msgr2=0x562350ec9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b53fff640 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ec9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b53fff640 1 -- 172.21.15.55:0/3889916644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ec10b0 msgr2=0x562350ec5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b53fff640 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ec10b0 0x562350ec5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b53fff640 1 -- 172.21.15.55:0/3889916644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562350af60f0 con 0x562350ebfee0 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.431+0000 7f8b5891c640 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ec9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.432+0000 7f8b53fff640 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ebfee0 0x562350ec4bd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x562350ebc2a0 tx=0x7f8b48022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684641fa18157f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.432+0000 7f8b39ffb640 1 -- 172.21.15.55:0/3889916644 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4802c040 con 0x562350ebfee0 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.432+0000 7f8b39ffb640 1 -- 172.21.15.55:0/3889916644 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b4803b070 con 0x562350ebfee0 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.432+0000 7f8b39ffb640 1 -- 172.21.15.55:0/3889916644 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48035160 con 0x562350ebfee0 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 -- 172.21.15.55:0/3889916644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ebfee0 msgr2=0x562350ec4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ebfee0 0x562350ec4bd0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x562350ebc2a0 tx=0x7f8b48022000 comp rx=0 tx=0).stop 2024-04-26T22:02:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 -- 172.21.15.55:0/3889916644 shutdown_connections 2024-04-26T22:02:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ec9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ec10b0 0x562350ec5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 --2- 172.21.15.55:0/3889916644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ebfee0 0x562350ec4bd0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 -- 172.21.15.55:0/3889916644 >> 172.21.15.55:0/3889916644 conn(0x562350ebb3d0 msgr2=0x562350eb6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 -- 172.21.15.55:0/3889916644 shutdown_connections 2024-04-26T22:02:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.433+0000 7f8b5d173740 1 -- 172.21.15.55:0/3889916644 wait complete. 2024-04-26T22:02:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.434+0000 7f8b5d173740 1 -- start start 2024-04-26T22:02:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.434+0000 7f8b5d173740 1 Processor -- start 2024-04-26T22:02:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ebfee0 0x562350ece810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b5d173740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ecfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b5d173740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4000cbf0 con 0x562350ec10b0 2024-04-26T22:02:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b5d173740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4000e060 con 0x562350ebfee0 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b5d173740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4000d2c0 con 0x562350ec1a70 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b5d173740 10 client.0 _ll_register_callbacks cb 0x7f8b4c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55946/0 (socket says 172.21.15.55:55946) 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b537fe640 1 -- 172.21.15.55:0/2220881419 learned_addr learned my addr 172.21.15.55:0/2220881419 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b537fe640 1 -- 172.21.15.55:0/2220881419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 msgr2=0x562350ecfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b537fe640 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ecfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b537fe640 1 -- 172.21.15.55:0/2220881419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ebfee0 msgr2=0x562350ece810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b537fe640 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ebfee0 0x562350ece810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.435+0000 7f8b537fe640 1 -- 172.21.15.55:0/2220881419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562350de54b0 con 0x562350ec10b0 2024-04-26T22:02:25.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.436+0000 7f8b537fe640 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eced50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c011980 tx=0x7f8b4c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1332094e27b4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.436+0000 7f8b337fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:02:25.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.436+0000 7f8b337fe640 1 -- 172.21.15.55:0/2220881419 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c031060 con 0x562350ec10b0 2024-04-26T22:02:25.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.436+0000 7f8b337fe640 1 -- 172.21.15.55:0/2220881419 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8b4c00f880 con 0x562350ec10b0 2024-04-26T22:02:25.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.436+0000 7f8b5d173740 1 -- 172.21.15.55:0/2220881419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4803b070 con 0x562350ec10b0 2024-04-26T22:02:25.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.437+0000 7f8b337fe640 1 -- 172.21.15.55:0/2220881419 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8b4c01e020 con 0x562350ec10b0 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.437+0000 7f8b5d173740 10 client.16875 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.437+0000 7f8b5d173740 1 -- 172.21.15.55:0/2220881419 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562350f3c590 con 0x562350ec10b0 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.437+0000 7f8b337fe640 1 -- 172.21.15.55:0/2220881419 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b4803b070 con 0x562350ec10b0 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.437+0000 7f8b337fe640 1 -- 172.21.15.55:0/2220881419 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8b4c005660 con 0x562350ec10b0 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.438+0000 7f8b5d173740 10 client.16875 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.438+0000 7f8b5d173740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.438+0000 7f8b5d173740 20 client.16875 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.438+0000 7f8b5d173740 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562350f3cf00 0x562350f3f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.438+0000 7f8b5d173740 4 client.16875 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:02:25.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.438+0000 7f8b5d173740 1 -- 172.21.15.55:0/2220881419 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562350e2c800 con 0x562350f3cf00 2024-04-26T22:02:25.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.441+0000 7f8b53fff640 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562350f3cf00 0x562350f3f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.443+0000 7f8b53fff640 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562350f3cf00 0x562350f3f320 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514b044962e233bb server_cookie=1d476230fde1d4d9 in_seq=0 out_seq=0 2024-04-26T22:02:25.444 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.443+0000 7f8b337fe640 10 client.16875 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:25.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.449+0000 7f8b337fe640 1 -- 172.21.15.55:0/2220881419 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562350e2c800 con 0x562350f3cf00 2024-04-26T22:02:25.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.449+0000 7f8b337fe640 10 client.16875 handle_command_reply: tid=0 2024-04-26T22:02:25.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.449+0000 7f8b5d173740 1 client.16875 shutdown 2024-04-26T22:02:25.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.450+0000 7f8b5d173740 1 -- 172.21.15.55:0/2220881419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 msgr2=0x562350eced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.450+0000 7f8b5d173740 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eced50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c011980 tx=0x7f8b4c009570 comp rx=0 tx=0).stop 2024-04-26T22:02:25.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.450+0000 7f8b5d173740 1 -- 172.21.15.55:0/2220881419 shutdown_connections 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.450+0000 7f8b5d173740 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562350ec1a70 0x562350ecfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.450+0000 7f8b5d173740 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562350f3cf00 0x562350f3f320 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.450+0000 7f8b5d173740 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562350ec10b0 0x562350eced50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.450+0000 7f8b337fe640 0 client.16875 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.451+0000 7f8b5d173740 1 --2- 172.21.15.55:0/2220881419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562350ebfee0 0x562350ece810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.451+0000 7f8b5d173740 1 -- 172.21.15.55:0/2220881419 >> 172.21.15.55:0/2220881419 conn(0x562350ebb3d0 msgr2=0x562350eca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.451+0000 7f8b5d173740 1 -- 172.21.15.55:0/2220881419 shutdown_connections 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.451+0000 7f8b5d173740 1 -- 172.21.15.55:0/2220881419 wait complete. 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.451+0000 7f8b5d173740 20 client.16875 trim_cache size 0 max 16384 2024-04-26T22:02:25.452 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:02:25.453 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:02:25.453 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:02:25.453 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:02:25.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 -- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b3c0a3d50 msgr2=0x7f8b3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b3c0a3d50 0x7f8b3c0a6210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800f670 tx=0x7f8b48005890 comp rx=0 tx=0).stop 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 -- 172.21.15.55:0/3700137681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 msgr2=0x7f8b541af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 0x7f8b541af1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b4402a950 tx=0x7f8b44032cd0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 -- 172.21.15.55:0/3700137681 shutdown_connections 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8b3c0a3d50 0x7f8b3c0a6210 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b5410ab40 0x7f8b541af1a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8b5410a180 0x7f8b540718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 --2- 172.21.15.55:0/3700137681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b54108f90 0x7f8b54071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 -- 172.21.15.55:0/3700137681 >> 172.21.15.55:0/3700137681 conn(0x7f8b54077f40 msgr2=0x7f8b5410d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 -- 172.21.15.55:0/3700137681 shutdown_connections 2024-04-26T22:02:25.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.452+0000 7f8b5a37f640 1 -- 172.21.15.55:0/3700137681 wait complete. 2024-04-26T22:02:25.484 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:02:25.484 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:02:25.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.801+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2487219356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec010bf10 msgr2=0x7f2ec010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:25.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.801+0000 7f2ec6ec2640 1 --2- 172.21.15.55:0/2487219356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec010bf10 0x7f2ec010e3a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009ca0 tx=0x7f2eb802a8f0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.802+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2487219356 shutdown_connections 2024-04-26T22:02:25.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.802+0000 7f2ec6ec2640 1 --2- 172.21.15.55:0/2487219356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec010ea10 0x7f2ec0119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.802+0000 7f2ec6ec2640 1 --2- 172.21.15.55:0/2487219356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec010bf10 0x7f2ec010e3a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.802+0000 7f2ec6ec2640 1 --2- 172.21.15.55:0/2487219356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ec0074760 0x7f2ec010b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.802+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2487219356 >> 172.21.15.55:0/2487219356 conn(0x7f2ec006f690 msgr2=0x7f2ec0071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:25.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.808+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2487219356 shutdown_connections 2024-04-26T22:02:25.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.808+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2487219356 wait complete. 2024-04-26T22:02:25.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.809+0000 7f2ec6ec2640 1 Processor -- start 2024-04-26T22:02:25.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.809+0000 7f2ec6ec2640 1 -- start start 2024-04-26T22:02:25.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec6ec2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec0074760 0x7f2ec0116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec6ec2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ec010bf10 0x7f2ec01165a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec6ec2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec010ea10 0x7f2ec01b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec6ec2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec011d200 con 0x7f2ec0074760 2024-04-26T22:02:25.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec6ec2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec011d080 con 0x7f2ec010ea10 2024-04-26T22:02:25.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec6ec2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec011d380 con 0x7f2ec010bf10 2024-04-26T22:02:25.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec5ec0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec0074760 0x7f2ec0116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec5ec0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec0074760 0x7f2ec0116060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55966/0 (socket says 172.21.15.55:55966) 2024-04-26T22:02:25.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.810+0000 7f2ec5ec0640 1 -- 172.21.15.55:0/2800425687 learned_addr learned my addr 172.21.15.55:0/2800425687 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:25.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.811+0000 7f2ec5ec0640 1 -- 172.21.15.55:0/2800425687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ec010bf10 msgr2=0x7f2ec01165a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:02:25.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.811+0000 7f2ec5ec0640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ec010bf10 0x7f2ec01165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.812+0000 7f2ec5ec0640 1 -- 172.21.15.55:0/2800425687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec010ea10 msgr2=0x7f2ec01b7910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:02:25.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.812+0000 7f2ec5ec0640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec010ea10 0x7f2ec01b7910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:25.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.812+0000 7f2ec5ec0640 1 -- 172.21.15.55:0/2800425687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec01b7e50 con 0x7f2ec0074760 2024-04-26T22:02:25.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.812+0000 7f2ec5ec0640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec0074760 0x7f2ec0116060 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000b450 tx=0x7f2eb000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.813+0000 7f2eb6ffd640 1 -- 172.21.15.55:0/2800425687 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0015020 con 0x7f2ec0074760 2024-04-26T22:02:25.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.813+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2800425687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec01b8020 con 0x7f2ec0074760 2024-04-26T22:02:25.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.813+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2800425687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec01b8560 con 0x7f2ec0074760 2024-04-26T22:02:25.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.816+0000 7f2eb6ffd640 1 -- 172.21.15.55:0/2800425687 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2eb0011d40 con 0x7f2ec0074760 2024-04-26T22:02:25.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.816+0000 7f2eb6ffd640 1 -- 172.21.15.55:0/2800425687 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb000f340 con 0x7f2ec0074760 2024-04-26T22:02:25.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.821+0000 7f2eb6ffd640 1 -- 172.21.15.55:0/2800425687 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2eb0018020 con 0x7f2ec0074760 2024-04-26T22:02:25.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.823+0000 7f2eb6ffd640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2eac0a3e40 0x7f2eac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:25.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.823+0000 7f2eb6ffd640 1 -- 172.21.15.55:0/2800425687 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2eb00d0f70 con 0x7f2ec0074760 2024-04-26T22:02:25.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.824+0000 7f2ec56bf640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2eac0a3e40 0x7f2eac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:25.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.825+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2800425687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea40052c0 con 0x7f2ec0074760 2024-04-26T22:02:25.835 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.828+0000 7f2ec56bf640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2eac0a3e40 0x7f2eac0a6300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8006010 tx=0x7f2eb8000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:25.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:25.838+0000 7f2eb6ffd640 1 -- 172.21.15.55:0/2800425687 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2eb001f020 con 0x7f2ec0074760 2024-04-26T22:02:25.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:25 smithi055 ceph-mon[29221]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 853 MiB/s wr, 1.26k op/s 2024-04-26T22:02:26.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:25 smithi064 ceph-mon[32765]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 853 MiB/s wr, 1.26k op/s 2024-04-26T22:02:26.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:25 smithi165 ceph-mon[33296]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 853 MiB/s wr, 1.26k op/s 2024-04-26T22:02:26.630 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:02:26.630 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:02:26.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.625+0000 7f2ec6ec2640 1 -- 172.21.15.55:0/2800425687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ea4005880 con 0x7f2ec0074760 2024-04-26T22:02:26.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.628+0000 7f2eb6ffd640 1 -- 172.21.15.55:0/2800425687 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f2eb000d040 con 0x7f2ec0074760 2024-04-26T22:02:26.630 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:02:26.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.634+0000 7f2eb4ff9640 1 -- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2eac0a3e40 msgr2=0x7f2eac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:26.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.635+0000 7f2eb4ff9640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2eac0a3e40 0x7f2eac0a6300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8006010 tx=0x7f2eb8000f40 comp rx=0 tx=0).stop 2024-04-26T22:02:26.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.635+0000 7f2eb4ff9640 1 -- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec0074760 msgr2=0x7f2ec0116060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:26.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.635+0000 7f2eb4ff9640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec0074760 0x7f2ec0116060 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000b450 tx=0x7f2eb000b910 comp rx=0 tx=0).stop 2024-04-26T22:02:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.636+0000 7f2eb4ff9640 1 -- 172.21.15.55:0/2800425687 shutdown_connections 2024-04-26T22:02:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.636+0000 7f2eb4ff9640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2eac0a3e40 0x7f2eac0a6300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.636+0000 7f2eb4ff9640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ec010ea10 0x7f2ec01b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.636+0000 7f2eb4ff9640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ec010bf10 0x7f2ec01165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.636+0000 7f2eb4ff9640 1 --2- 172.21.15.55:0/2800425687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ec0074760 0x7f2ec0116060 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.636+0000 7f2eb4ff9640 1 -- 172.21.15.55:0/2800425687 >> 172.21.15.55:0/2800425687 conn(0x7f2ec006f690 msgr2=0x7f2ec010ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.637+0000 7f2eb4ff9640 1 -- 172.21.15.55:0/2800425687 shutdown_connections 2024-04-26T22:02:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:26.637+0000 7f2eb4ff9640 1 -- 172.21.15.55:0/2800425687 wait complete. 2024-04-26T22:02:26.707 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:02:26.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:26 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2800425687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:27.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:26 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2800425687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:27.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.051+0000 7f93cd499640 1 -- 172.21.15.55:0/715833029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810c2a0 msgr2=0x7f93c810e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.051+0000 7f93cd499640 1 --2- 172.21.15.55:0/715833029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810c2a0 0x7f93c810e730 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009be0 tx=0x7f93c002a6e0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.052+0000 7f93cd499640 1 -- 172.21.15.55:0/715833029 shutdown_connections 2024-04-26T22:02:27.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.052+0000 7f93cd499640 1 --2- 172.21.15.55:0/715833029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c810ec70 0x7f93c810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.052+0000 7f93cd499640 1 --2- 172.21.15.55:0/715833029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810c2a0 0x7f93c810e730 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.052+0000 7f93cd499640 1 --2- 172.21.15.55:0/715833029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93c8109930 0x7f93c810bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.052+0000 7f93cd499640 1 -- 172.21.15.55:0/715833029 >> 172.21.15.55:0/715833029 conn(0x7f93c806f7c0 msgr2=0x7f93c8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.057+0000 7f93cd499640 1 -- 172.21.15.55:0/715833029 shutdown_connections 2024-04-26T22:02:27.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.057+0000 7f93cd499640 1 -- 172.21.15.55:0/715833029 wait complete. 2024-04-26T22:02:27.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.058+0000 7f93cd499640 1 Processor -- start 2024-04-26T22:02:27.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.059+0000 7f93cd499640 1 -- start start 2024-04-26T22:02:27.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93cd499640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c8109930 0x7f93c8116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93cd499640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93c810c2a0 0x7f93c81166a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93cd499640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810ec70 0x7f93c81b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93cd499640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f93c811d310 con 0x7f93c810ec70 2024-04-26T22:02:27.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93cd499640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f93c811d190 con 0x7f93c8109930 2024-04-26T22:02:27.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93cd499640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93c811d490 con 0x7f93c810c2a0 2024-04-26T22:02:27.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810ec70 0x7f93c81b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810ec70 0x7f93c81b7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55988/0 (socket says 172.21.15.55:55988) 2024-04-26T22:02:27.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.060+0000 7f93c77fe640 1 -- 172.21.15.55:0/2060163408 learned_addr learned my addr 172.21.15.55:0/2060163408 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.061+0000 7f93c77fe640 1 -- 172.21.15.55:0/2060163408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93c810c2a0 msgr2=0x7f93c81166a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:02:27.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.061+0000 7f93c77fe640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93c810c2a0 0x7f93c81166a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.061+0000 7f93c77fe640 1 -- 172.21.15.55:0/2060163408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c8109930 msgr2=0x7f93c8116160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:02:27.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.061+0000 7f93c77fe640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c8109930 0x7f93c8116160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.061+0000 7f93c77fe640 1 -- 172.21.15.55:0/2060163408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c81b7f40 con 0x7f93c810ec70 2024-04-26T22:02:27.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.067+0000 7f93c77fe640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810ec70 0x7f93c81b7a00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f93b800d720 tx=0x7f93b800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.069+0000 7f93affff640 1 -- 172.21.15.55:0/2060163408 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8017070 con 0x7f93c810ec70 2024-04-26T22:02:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.069+0000 7f93affff640 1 -- 172.21.15.55:0/2060163408 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b800de30 con 0x7f93c810ec70 2024-04-26T22:02:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.069+0000 7f93affff640 1 -- 172.21.15.55:0/2060163408 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8004c60 con 0x7f93c810ec70 2024-04-26T22:02:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.071+0000 7f93cd499640 1 -- 172.21.15.55:0/2060163408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c81b81d0 con 0x7f93c810ec70 2024-04-26T22:02:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.071+0000 7f93cd499640 1 -- 172.21.15.55:0/2060163408 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c81b86a0 con 0x7f93c810ec70 2024-04-26T22:02:27.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:26 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2800425687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.081+0000 7f93affff640 1 -- 172.21.15.55:0/2060163408 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f93b8010460 con 0x7f93c810ec70 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.081+0000 7f93d01ba740 1 Processor -- start 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.088+0000 7f93affff640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93b00a3e40 0x7f93b00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.088+0000 7f93affff640 1 -- 172.21.15.55:0/2060163408 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f93b80cfef0 con 0x7f93c810ec70 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.089+0000 7f93d01ba740 1 -- start start 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.089+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 0x5597fe92dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.089+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92e9f0 0x5597fe92ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.089+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe935c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.089+0000 7f93d01ba740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5597fe891d80 con 0x5597fe92e9f0 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.089+0000 7f93d01ba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597fe84ba20 con 0x5597fe92f3b0 2024-04-26T22:02:27.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.089+0000 7f93d01ba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597fe88dd40 con 0x5597fe92d820 2024-04-26T22:02:27.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.091+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe935c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.092+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe935c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39712/0 (socket says 172.21.15.55:39712) 2024-04-26T22:02:27.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.092+0000 7f93c77fe640 1 -- 172.21.15.55:0/795655966 learned_addr learned my addr 172.21.15.55:0/795655966 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.093+0000 7f93c67fc640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93b00a3e40 0x7f93b00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.093+0000 7f93c67fc640 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92e9f0 0x5597fe92ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.093+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 0x5597fe92dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.093+0000 7f93c77fe640 1 -- 172.21.15.55:0/795655966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 msgr2=0x5597fe92dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.093+0000 7f93c77fe640 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 0x5597fe92dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.093+0000 7f93c77fe640 1 -- 172.21.15.55:0/795655966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92e9f0 msgr2=0x5597fe92ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.093+0000 7f93c77fe640 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92e9f0 0x5597fe92ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.093+0000 7f93c77fe640 1 -- 172.21.15.55:0/795655966 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fe856c30 con 0x5597fe92f3b0 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.094+0000 7f93c77fe640 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe935c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93b809bf60 tx=0x7f93b80a0440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1d5af5f20b8ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.096+0000 7f93ad7fa640 1 -- 172.21.15.55:0/795655966 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b8020040 con 0x5597fe92f3b0 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.096+0000 7f93ad7fa640 1 -- 172.21.15.55:0/795655966 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b80cda50 con 0x5597fe92f3b0 2024-04-26T22:02:27.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.096+0000 7f93ad7fa640 1 -- 172.21.15.55:0/795655966 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b809b680 con 0x5597fe92f3b0 2024-04-26T22:02:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.101+0000 7f93d01ba740 1 -- 172.21.15.55:0/795655966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 msgr2=0x5597fe935c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.101+0000 7f93d01ba740 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe935c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f93b809bf60 tx=0x7f93b80a0440 comp rx=0 tx=0).stop 2024-04-26T22:02:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.102+0000 7f93d01ba740 1 -- 172.21.15.55:0/795655966 shutdown_connections 2024-04-26T22:02:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.102+0000 7f93d01ba740 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe935c40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.102+0000 7f93d01ba740 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92e9f0 0x5597fe92ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.102+0000 7f93d01ba740 1 --2- 172.21.15.55:0/795655966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 0x5597fe92dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.102+0000 7f93d01ba740 1 -- 172.21.15.55:0/795655966 >> 172.21.15.55:0/795655966 conn(0x5597fe86e910 msgr2=0x5597fe929120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.102+0000 7f93c67fc640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93b00a3e40 0x7f93b00a6300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f93c0004af0 tx=0x7f93c0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.109+0000 7f93d01ba740 1 -- 172.21.15.55:0/795655966 shutdown_connections 2024-04-26T22:02:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.109+0000 7f93d01ba740 1 -- 172.21.15.55:0/795655966 wait complete. 2024-04-26T22:02:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.110+0000 7f93d01ba740 1 -- start start 2024-04-26T22:02:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.110+0000 7f93d01ba740 1 Processor -- start 2024-04-26T22:02:27.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.111+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe937b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.111+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 0x5597fe938060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.111+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92f3b0 0x5597fe93f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.111+0000 7f93d01ba740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5597fe9363c0 con 0x5597fe92d820 2024-04-26T22:02:27.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.111+0000 7f93d01ba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597fe818dc0 con 0x5597fe92e9f0 2024-04-26T22:02:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.111+0000 7f93d01ba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597fe936540 con 0x5597fe92f3b0 2024-04-26T22:02:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.111+0000 7f93d01ba740 10 client.0 _ll_register_callbacks cb 0x5597fe834600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.112+0000 7f93c6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe937b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.112+0000 7f93c6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe937b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56014/0 (socket says 172.21.15.55:56014) 2024-04-26T22:02:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.112+0000 7f93c6ffd640 1 -- 172.21.15.55:0/3700031035 learned_addr learned my addr 172.21.15.55:0/3700031035 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.113+0000 7f93c6ffd640 1 -- 172.21.15.55:0/3700031035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92f3b0 msgr2=0x5597fe93f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.113+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92f3b0 0x5597fe93f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.113+0000 7f93c6ffd640 1 -- 172.21.15.55:0/3700031035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 msgr2=0x5597fe938060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.113+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 0x5597fe938060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.113+0000 7f93c6ffd640 1 -- 172.21.15.55:0/3700031035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fe5a33f0 con 0x5597fe92d820 2024-04-26T22:02:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.114+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe937b20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00d3f0 tx=0x7f93bc00d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77c071c01802b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.117+0000 7f93aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:02:27.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.118+0000 7f93aaff5640 1 -- 172.21.15.55:0/3700031035 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc004220 con 0x5597fe92d820 2024-04-26T22:02:27.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.118+0000 7f93d01ba740 1 -- 172.21.15.55:0/3700031035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93b80cda50 con 0x5597fe92d820 2024-04-26T22:02:27.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.118+0000 7f93aaff5640 1 -- 172.21.15.55:0/3700031035 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f93bc0043c0 con 0x5597fe92d820 2024-04-26T22:02:27.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.123+0000 7f93aaff5640 1 -- 172.21.15.55:0/3700031035 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f93bc00f6f0 con 0x5597fe92d820 2024-04-26T22:02:27.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.124+0000 7f93d01ba740 10 client.16893 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:27.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.124+0000 7f93d01ba740 1 -- 172.21.15.55:0/3700031035 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5597fe66d7d0 con 0x5597fe92d820 2024-04-26T22:02:27.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.126+0000 7f93aaff5640 1 -- 172.21.15.55:0/3700031035 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b80cda50 con 0x5597fe92d820 2024-04-26T22:02:27.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.126+0000 7f93aaff5640 1 -- 172.21.15.55:0/3700031035 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f93bc003cb0 con 0x5597fe92d820 2024-04-26T22:02:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.126+0000 7f93d01ba740 10 client.16893 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.126+0000 7f93d01ba740 10 client.16893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:02:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.126+0000 7f93d01ba740 20 client.16893 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.126+0000 7f93d01ba740 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5597fe943a50 0x5597fe945ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.126+0000 7f93d01ba740 4 client.16893 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:02:27.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.126+0000 7f93d01ba740 1 -- 172.21.15.55:0/3700031035 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597fe89f990 con 0x5597fe943a50 2024-04-26T22:02:27.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.127+0000 7f93c67fc640 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5597fe943a50 0x5597fe945ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.128+0000 7f93c67fc640 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5597fe943a50 0x5597fe945ed0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54f75c26a3a9b461 server_cookie=e25c3f0922ef8ded in_seq=0 out_seq=0 2024-04-26T22:02:27.130 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.129+0000 7f93aaff5640 10 client.16893 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:02:27.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.131+0000 7f93aaff5640 1 -- 172.21.15.55:0/3700031035 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597fe89f990 con 0x5597fe943a50 2024-04-26T22:02:27.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.131+0000 7f93aaff5640 10 client.16893 handle_command_reply: tid=0 2024-04-26T22:02:27.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.131+0000 7f93d01ba740 1 client.16893 shutdown 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.132+0000 7f93d01ba740 1 -- 172.21.15.55:0/3700031035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 msgr2=0x5597fe937b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.132+0000 7f93d01ba740 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe937b20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00d3f0 tx=0x7f93bc00d8b0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93d01ba740 1 -- 172.21.15.55:0/3700031035 shutdown_connections 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93d01ba740 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5597fe943a50 0x5597fe945ed0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93d01ba740 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92f3b0 0x5597fe93f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93d01ba740 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 0x5597fe938060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93d01ba740 1 --2- 172.21.15.55:0/3700031035 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe937b20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93d01ba740 1 -- 172.21.15.55:0/3700031035 >> 172.21.15.55:0/3700031035 conn(0x5597fe86e910 msgr2=0x5597fe928280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93aaff5640 0 client.16893 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93d01ba740 1 -- 172.21.15.55:0/3700031035 shutdown_connections 2024-04-26T22:02:27.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.133+0000 7f93d01ba740 1 -- 172.21.15.55:0/3700031035 wait complete. 2024-04-26T22:02:27.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.134+0000 7f93d01ba740 20 client.16893 trim_cache size 0 max 16384 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.169+0000 7f93d01ba740 1 Processor -- start 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.169+0000 7f93d01ba740 1 -- start start 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe935940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92e9f0 0x5597fe93f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe92bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93d01ba740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f93b4010f90 con 0x5597fe92d820 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93d01ba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f93b400e860 con 0x5597fe92f3b0 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93c6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe935940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93d01ba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93b400c6b0 con 0x5597fe92e9f0 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93c6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe935940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56030/0 (socket says 172.21.15.55:56030) 2024-04-26T22:02:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.170+0000 7f93c6ffd640 1 -- 172.21.15.55:0/2701400194 learned_addr learned my addr 172.21.15.55:0/2701400194 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.171+0000 7f93c6ffd640 1 -- 172.21.15.55:0/2701400194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92e9f0 msgr2=0x5597fe93f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.171+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/2701400194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92e9f0 0x5597fe93f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.171+0000 7f93c6ffd640 1 -- 172.21.15.55:0/2701400194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 msgr2=0x5597fe92bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.171+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/2701400194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe92bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.171+0000 7f93c6ffd640 1 -- 172.21.15.55:0/2701400194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fe932a00 con 0x5597fe92d820 2024-04-26T22:02:27.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.172+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/2701400194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe935940 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f93bc021570 tx=0x7f93bc00b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e3c5faea27fdf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.173+0000 7f93acff9640 1 -- 172.21.15.55:0/2701400194 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc021b30 con 0x5597fe92d820 2024-04-26T22:02:27.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.173+0000 7f93acff9640 1 -- 172.21.15.55:0/2701400194 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc0054e0 con 0x5597fe92d820 2024-04-26T22:02:27.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.173+0000 7f93acff9640 1 -- 172.21.15.55:0/2701400194 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc004660 con 0x5597fe92d820 2024-04-26T22:02:27.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.174+0000 7f93d01ba740 1 -- 172.21.15.55:0/2701400194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 msgr2=0x5597fe935940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.174+0000 7f93d01ba740 1 --2- 172.21.15.55:0/2701400194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe935940 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f93bc021570 tx=0x7f93bc00b340 comp rx=0 tx=0).stop 2024-04-26T22:02:27.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.175+0000 7f93d01ba740 1 -- 172.21.15.55:0/2701400194 shutdown_connections 2024-04-26T22:02:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.175+0000 7f93d01ba740 1 --2- 172.21.15.55:0/2701400194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92f3b0 0x5597fe92bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.175+0000 7f93d01ba740 1 --2- 172.21.15.55:0/2701400194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92e9f0 0x5597fe93f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.175+0000 7f93d01ba740 1 --2- 172.21.15.55:0/2701400194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92d820 0x5597fe935940 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.175+0000 7f93d01ba740 1 -- 172.21.15.55:0/2701400194 >> 172.21.15.55:0/2701400194 conn(0x5597fe86e910 msgr2=0x5597fe9322f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.176+0000 7f93d01ba740 1 -- 172.21.15.55:0/2701400194 shutdown_connections 2024-04-26T22:02:27.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.176+0000 7f93d01ba740 1 -- 172.21.15.55:0/2701400194 wait complete. 2024-04-26T22:02:27.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.177+0000 7f93d01ba740 1 -- start start 2024-04-26T22:02:27.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.177+0000 7f93d01ba740 1 Processor -- start 2024-04-26T22:02:27.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 0x5597fe93bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 0x5597fe93c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93d01ba740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92f3b0 0x5597fe9a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93d01ba740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f93b400cd20 con 0x5597fe92f3b0 2024-04-26T22:02:27.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93d01ba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f93b400e190 con 0x5597fe92e9f0 2024-04-26T22:02:27.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93d01ba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f93b400d3f0 con 0x5597fe92d820 2024-04-26T22:02:27.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93d01ba740 10 client.0 _ll_register_callbacks cb 0x7f93bc005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:27.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92f3b0 0x5597fe9a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92f3b0 0x5597fe9a8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56044/0 (socket says 172.21.15.55:56044) 2024-04-26T22:02:27.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93c77fe640 1 -- 172.21.15.55:0/4171860609 learned_addr learned my addr 172.21.15.55:0/4171860609 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.178+0000 7f93c67fc640 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 0x5597fe93c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.179+0000 7f93c77fe640 1 -- 172.21.15.55:0/4171860609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 msgr2=0x5597fe93bd10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:27.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.179+0000 7f93c77fe640 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 0x5597fe93bd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.179+0000 7f93c77fe640 1 -- 172.21.15.55:0/4171860609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 msgr2=0x5597fe93c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.179+0000 7f93c77fe640 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 0x5597fe93c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.179+0000 7f93c77fe640 1 -- 172.21.15.55:0/4171860609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fe93f320 con 0x5597fe92f3b0 2024-04-26T22:02:27.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.179+0000 7f93c77fe640 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92f3b0 0x5597fe9a8040 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f93b809c190 tx=0x7f93b8098870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88c10cfe51a0634 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.179+0000 7f93aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:02:27.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.179+0000 7f93aaff5640 1 -- 172.21.15.55:0/4171860609 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80b9020 con 0x5597fe92f3b0 2024-04-26T22:02:27.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.180+0000 7f93d01ba740 1 -- 172.21.15.55:0/4171860609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93bc0054e0 con 0x5597fe92f3b0 2024-04-26T22:02:27.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.180+0000 7f93aaff5640 1 -- 172.21.15.55:0/4171860609 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f93b80a49b0 con 0x5597fe92f3b0 2024-04-26T22:02:27.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.180+0000 7f93aaff5640 1 -- 172.21.15.55:0/4171860609 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f93b8020040 con 0x5597fe92f3b0 2024-04-26T22:02:27.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.180+0000 7f93d01ba740 10 client.16899 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:27.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.180+0000 7f93d01ba740 1 -- 172.21.15.55:0/4171860609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5597fe9a9630 con 0x5597fe92f3b0 2024-04-26T22:02:27.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.181+0000 7f93aaff5640 1 -- 172.21.15.55:0/4171860609 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc0054e0 con 0x5597fe92f3b0 2024-04-26T22:02:27.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.182+0000 7f93aaff5640 1 -- 172.21.15.55:0/4171860609 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f93b809b5c0 con 0x5597fe92f3b0 2024-04-26T22:02:27.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.182+0000 7f93d01ba740 10 client.16899 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:27.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.182+0000 7f93d01ba740 10 client.16899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:02:27.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.182+0000 7f93d01ba740 20 client.16899 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:27.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.182+0000 7f93d01ba740 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5597fe9a9b80 0x5597fe9abf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.182+0000 7f93d01ba740 4 client.16899 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:02:27.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.182+0000 7f93d01ba740 1 -- 172.21.15.55:0/4171860609 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597fe899800 con 0x5597fe9a9b80 2024-04-26T22:02:27.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.183+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5597fe9a9b80 0x5597fe9abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.184+0000 7f93c6ffd640 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5597fe9a9b80 0x5597fe9abf70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac8ab5f4db3732c server_cookie=c5bde66155cb965c in_seq=0 out_seq=0 2024-04-26T22:02:27.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.184+0000 7f93aaff5640 10 client.16899 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:02:27.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93aaff5640 1 -- 172.21.15.55:0/4171860609 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597fe899800 con 0x5597fe9a9b80 2024-04-26T22:02:27.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93aaff5640 10 client.16899 handle_command_reply: tid=0 2024-04-26T22:02:27.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 client.16899 shutdown 2024-04-26T22:02:27.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 -- 172.21.15.55:0/4171860609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92f3b0 msgr2=0x5597fe9a8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92f3b0 0x5597fe9a8040 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f93b809c190 tx=0x7f93b8098870 comp rx=0 tx=0).stop 2024-04-26T22:02:27.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 -- 172.21.15.55:0/4171860609 shutdown_connections 2024-04-26T22:02:27.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5597fe9a9b80 0x5597fe9abf70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597fe92f3b0 0x5597fe9a8040 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597fe92e9f0 0x5597fe93c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93aaff5640 0 client.16899 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:02:27.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 --2- 172.21.15.55:0/4171860609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597fe92d820 0x5597fe93bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 -- 172.21.15.55:0/4171860609 >> 172.21.15.55:0/4171860609 conn(0x5597fe86e910 msgr2=0x5597fe9357b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 -- 172.21.15.55:0/4171860609 shutdown_connections 2024-04-26T22:02:27.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.185+0000 7f93d01ba740 1 -- 172.21.15.55:0/4171860609 wait complete. 2024-04-26T22:02:27.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.186+0000 7f93d01ba740 20 client.16899 trim_cache size 0 max 16384 2024-04-26T22:02:27.189 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:02:27.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.186+0000 7f93cd499640 1 -- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93b00a3e40 msgr2=0x7f93b00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.186+0000 7f93cd499640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93b00a3e40 0x7f93b00a6300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f93c0004af0 tx=0x7f93c0009500 comp rx=0 tx=0).stop 2024-04-26T22:02:27.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.186+0000 7f93cd499640 1 -- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810ec70 msgr2=0x7f93c81b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.186+0000 7f93cd499640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810ec70 0x7f93c81b7a00 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f93b800d720 tx=0x7f93b800dbe0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.186+0000 7f93cd499640 1 -- 172.21.15.55:0/2060163408 shutdown_connections 2024-04-26T22:02:27.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.187+0000 7f93cd499640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f93b00a3e40 0x7f93b00a6300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.187+0000 7f93cd499640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93c810ec70 0x7f93c81b7a00 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.187+0000 7f93cd499640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93c810c2a0 0x7f93c81166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.187+0000 7f93cd499640 1 --2- 172.21.15.55:0/2060163408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93c8109930 0x7f93c8116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.187+0000 7f93cd499640 1 -- 172.21.15.55:0/2060163408 >> 172.21.15.55:0/2060163408 conn(0x7f93c806f7c0 msgr2=0x7f93c8109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.187+0000 7f93cd499640 1 -- 172.21.15.55:0/2060163408 shutdown_connections 2024-04-26T22:02:27.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.187+0000 7f93cd499640 1 -- 172.21.15.55:0/2060163408 wait complete. 2024-04-26T22:02:27.207 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:02:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.576+0000 7fd49a085640 1 -- 172.21.15.55:0/2358666636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd494073be0 msgr2=0x7fd494074000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.576+0000 7fd4927fc640 1 -- 172.21.15.55:0/2358666636 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484035020 con 0x7fd494073be0 2024-04-26T22:02:27.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.576+0000 7fd49a085640 1 --2- 172.21.15.55:0/2358666636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd494073be0 0x7fd494074000 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd484009900 tx=0x7fd48402a420 comp rx=0 tx=0).stop 2024-04-26T22:02:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.582+0000 7fd49a085640 1 -- 172.21.15.55:0/2358666636 shutdown_connections 2024-04-26T22:02:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.582+0000 7fd49a085640 1 --2- 172.21.15.55:0/2358666636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd49410a640 0x7fd49410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.582+0000 7fd49a085640 1 --2- 172.21.15.55:0/2358666636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd494074540 0x7fd494109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.582+0000 7fd49a085640 1 --2- 172.21.15.55:0/2358666636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd494073be0 0x7fd494074000 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.582+0000 7fd49a085640 1 -- 172.21.15.55:0/2358666636 >> 172.21.15.55:0/2358666636 conn(0x7fd49406f780 msgr2=0x7fd494071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.588+0000 7fd49a085640 1 -- 172.21.15.55:0/2358666636 shutdown_connections 2024-04-26T22:02:27.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.588+0000 7fd49a085640 1 -- 172.21.15.55:0/2358666636 wait complete. 2024-04-26T22:02:27.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.589+0000 7fd49a085640 1 Processor -- start 2024-04-26T22:02:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.595+0000 7fd49a085640 1 -- start start 2024-04-26T22:02:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.596+0000 7fd49a085640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd494073be0 0x7fd4941a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.596+0000 7fd49a085640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd494074540 0x7fd4941a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.596+0000 7fd49a085640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd49410a640 0x7fd4941aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.596+0000 7fd49a085640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd49411d210 con 0x7fd49410a640 2024-04-26T22:02:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.596+0000 7fd49a085640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd49411d090 con 0x7fd494074540 2024-04-26T22:02:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.596+0000 7fd49a085640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd49411d390 con 0x7fd494073be0 2024-04-26T22:02:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.597+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd494073be0 0x7fd4941a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.597+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd494073be0 0x7fd4941a7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49796/0 (socket says 172.21.15.55:49796) 2024-04-26T22:02:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.597+0000 7fd4937fe640 1 -- 172.21.15.55:0/4009310420 learned_addr learned my addr 172.21.15.55:0/4009310420 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.598+0000 7fd4937fe640 1 -- 172.21.15.55:0/4009310420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd494074540 msgr2=0x7fd4941a7eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:02:27.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.598+0000 7fd4937fe640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd494074540 0x7fd4941a7eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.598+0000 7fd4937fe640 1 -- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd49410a640 msgr2=0x7fd4941aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.598+0000 7fd4937fe640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd49410a640 0x7fd4941aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.598+0000 7fd4937fe640 1 -- 172.21.15.55:0/4009310420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd494111190 con 0x7fd494073be0 2024-04-26T22:02:27.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.599+0000 7fd4937fe640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd494073be0 0x7fd4941a7970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd484000c00 tx=0x7fd484003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.607+0000 7fd490ff9640 1 -- 172.21.15.55:0/4009310420 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48403f070 con 0x7fd494073be0 2024-04-26T22:02:27.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.607+0000 7fd490ff9640 1 -- 172.21.15.55:0/4009310420 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484031dc0 con 0x7fd494073be0 2024-04-26T22:02:27.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.607+0000 7fd490ff9640 1 -- 172.21.15.55:0/4009310420 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484035030 con 0x7fd494073be0 2024-04-26T22:02:27.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.607+0000 7fd49a085640 1 -- 172.21.15.55:0/4009310420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd494111420 con 0x7fd494073be0 2024-04-26T22:02:27.608 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.608+0000 7fd49a085640 1 -- 172.21.15.55:0/4009310420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd494111940 con 0x7fd494073be0 2024-04-26T22:02:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.616+0000 7fd49cd9d740 1 Processor -- start 2024-04-26T22:02:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.616+0000 7fd49cd9d740 1 -- start start 2024-04-26T22:02:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.616+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x564153998e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.617+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 0x56415399a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.617+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56415399a590 0x5641539a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.617+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5641538fcd80 con 0x564153998a00 2024-04-26T22:02:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.617+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5641538b6a20 con 0x56415399a590 2024-04-26T22:02:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.617+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641538f8d40 con 0x564153999bd0 2024-04-26T22:02:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.618+0000 7fd493fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x564153998e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.618+0000 7fd493fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x564153998e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56076/0 (socket says 172.21.15.55:56076) 2024-04-26T22:02:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.618+0000 7fd493fff640 1 -- 172.21.15.55:0/1206892366 learned_addr learned my addr 172.21.15.55:0/1206892366 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.619+0000 7fd492ffd640 1 --2- 172.21.15.55:0/1206892366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56415399a590 0x5641539a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.620+0000 7fd492ffd640 1 -- 172.21.15.55:0/1206892366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 msgr2=0x56415399a050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.620+0000 7fd492ffd640 1 --2- 172.21.15.55:0/1206892366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 0x56415399a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.620+0000 7fd492ffd640 1 -- 172.21.15.55:0/1206892366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 msgr2=0x564153998e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.620+0000 7fd492ffd640 1 --2- 172.21.15.55:0/1206892366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x564153998e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.620+0000 7fd492ffd640 1 -- 172.21.15.55:0/1206892366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641538c1c30 con 0x56415399a590 2024-04-26T22:02:27.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.621+0000 7fd492ffd640 1 --2- 172.21.15.55:0/1206892366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56415399a590 0x5641539a0e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd494109e20 tx=0x7fd48c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf0563da79aa7dca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.626+0000 7fd479ffb640 1 -- 172.21.15.55:0/1206892366 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c018070 con 0x56415399a590 2024-04-26T22:02:27.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.626+0000 7fd479ffb640 1 -- 172.21.15.55:0/1206892366 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c0119d0 con 0x56415399a590 2024-04-26T22:02:27.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.626+0000 7fd479ffb640 1 -- 172.21.15.55:0/1206892366 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c013980 con 0x56415399a590 2024-04-26T22:02:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.627+0000 7fd490ff9640 1 -- 172.21.15.55:0/4009310420 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd484042070 con 0x7fd494073be0 2024-04-26T22:02:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.628+0000 7fd49cd9d740 1 -- 172.21.15.55:0/1206892366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56415399a590 msgr2=0x5641539a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.628+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/1206892366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56415399a590 0x5641539a0e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd494109e20 tx=0x7fd48c00b7e0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.628+0000 7fd49cd9d740 1 -- 172.21.15.55:0/1206892366 shutdown_connections 2024-04-26T22:02:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.628+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/1206892366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56415399a590 0x5641539a0e20 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd494109e20 tx=0x7fd48c00b7e0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.628+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/1206892366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 0x56415399a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.628+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/1206892366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x564153998e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.629+0000 7fd49cd9d740 1 -- 172.21.15.55:0/1206892366 >> 172.21.15.55:0/1206892366 conn(0x5641538d9910 msgr2=0x5641539942b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.630+0000 7fd490ff9640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd47c0a3dc0 0x7fd47c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.630+0000 7fd490ff9640 1 -- 172.21.15.55:0/4009310420 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd4840f0f80 con 0x7fd494073be0 2024-04-26T22:02:27.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.633+0000 7fd492ffd640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd47c0a3dc0 0x7fd47c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.634+0000 7fd49cd9d740 1 -- 172.21.15.55:0/1206892366 shutdown_connections 2024-04-26T22:02:27.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.634+0000 7fd49cd9d740 1 -- 172.21.15.55:0/1206892366 wait complete. 2024-04-26T22:02:27.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.634+0000 7fd492ffd640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd47c0a3dc0 0x7fd47c0a6280 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009af0 tx=0x7fd48c012000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.634+0000 7fd49cd9d740 1 -- start start 2024-04-26T22:02:27.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.635+0000 7fd49cd9d740 1 Processor -- start 2024-04-26T22:02:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.635+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153998a00 0x5641539a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.635+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 0x5641539a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.635+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 0x5641539aa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.635+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5641539a15a0 con 0x5641539a84a0 2024-04-26T22:02:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.635+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564153883dc0 con 0x564153998a00 2024-04-26T22:02:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.635+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641539a1720 con 0x564153999bd0 2024-04-26T22:02:27.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.635+0000 7fd49cd9d740 10 client.0 _ll_register_callbacks cb 0x56415389f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.637+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153998a00 0x5641539a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.637+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153998a00 0x5641539a2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39790/0 (socket says 172.21.15.55:39790) 2024-04-26T22:02:27.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.637+0000 7fd4937fe640 1 -- 172.21.15.55:0/3291070303 learned_addr learned my addr 172.21.15.55:0/3291070303 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.640+0000 7fd492ffd640 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 0x5641539a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.640+0000 7fd4937fe640 1 -- 172.21.15.55:0/3291070303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 msgr2=0x5641539a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.640+0000 7fd4937fe640 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 0x5641539a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.640+0000 7fd4937fe640 1 -- 172.21.15.55:0/3291070303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 msgr2=0x5641539aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.640+0000 7fd4937fe640 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 0x5641539aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.640+0000 7fd4937fe640 1 -- 172.21.15.55:0/3291070303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56415360e3f0 con 0x564153998a00 2024-04-26T22:02:27.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.643+0000 7fd4937fe640 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153998a00 0x5641539a2fa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4840bcd60 tx=0x7fd4840bcc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a97df4c0753a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.653+0000 7fd472ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:02:27.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.654+0000 7fd472ffd640 1 -- 172.21.15.55:0/3291070303 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840d6060 con 0x564153998a00 2024-04-26T22:02:27.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.654+0000 7fd49cd9d740 1 -- 172.21.15.55:0/3291070303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd48c0119d0 con 0x564153998a00 2024-04-26T22:02:27.654 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.654+0000 7fd472ffd640 1 -- 172.21.15.55:0/3291070303 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd4840eeae0 con 0x564153998a00 2024-04-26T22:02:27.656 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.655+0000 7fd472ffd640 1 -- 172.21.15.55:0/3291070303 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd484049070 con 0x564153998a00 2024-04-26T22:02:27.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.657+0000 7fd49cd9d740 10 client.25997 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:27.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.657+0000 7fd49cd9d740 1 -- 172.21.15.55:0/3291070303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5641536d87d0 con 0x564153998a00 2024-04-26T22:02:27.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.661+0000 7fd472ffd640 1 -- 172.21.15.55:0/3291070303 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c0119d0 con 0x564153998a00 2024-04-26T22:02:27.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.662+0000 7fd472ffd640 1 -- 172.21.15.55:0/3291070303 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd4840bd340 con 0x564153998a00 2024-04-26T22:02:27.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.663+0000 7fd49cd9d740 10 client.25997 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:27.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.663+0000 7fd49cd9d740 10 client.25997 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:02:27.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.663+0000 7fd49cd9d740 20 client.25997 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:27.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.663+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5641539af330 0x5641539b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.663+0000 7fd49cd9d740 4 client.25997 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:02:27.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.663+0000 7fd49cd9d740 1 -- 172.21.15.55:0/3291070303 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56415390a990 con 0x5641539af330 2024-04-26T22:02:27.664 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.663+0000 7fd492ffd640 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5641539af330 0x5641539b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.665+0000 7fd492ffd640 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5641539af330 0x5641539b17f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f0ecc292cc0b26b server_cookie=8677bc70a40c5f7e in_seq=0 out_seq=0 2024-04-26T22:02:27.665 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.665+0000 7fd472ffd640 10 client.25997 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:02:27.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.666+0000 7fd472ffd640 1 -- 172.21.15.55:0/3291070303 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56415390a990 con 0x5641539af330 2024-04-26T22:02:27.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.666+0000 7fd472ffd640 10 client.25997 handle_command_reply: tid=0 2024-04-26T22:02:27.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.666+0000 7fd49cd9d740 1 client.25997 shutdown 2024-04-26T22:02:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd49cd9d740 1 -- 172.21.15.55:0/3291070303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153998a00 msgr2=0x5641539a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153998a00 0x5641539a2fa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4840bcd60 tx=0x7fd4840bcc60 comp rx=0 tx=0).stop 2024-04-26T22:02:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd49cd9d740 1 -- 172.21.15.55:0/3291070303 shutdown_connections 2024-04-26T22:02:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5641539af330 0x5641539b17f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 0x5641539aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153999bd0 0x5641539a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/3291070303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153998a00 0x5641539a2fa0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd49cd9d740 1 -- 172.21.15.55:0/3291070303 >> 172.21.15.55:0/3291070303 conn(0x5641538d9910 msgr2=0x564153993450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.672+0000 7fd472ffd640 0 client.25997 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:02:27.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.673+0000 7fd49cd9d740 1 -- 172.21.15.55:0/3291070303 shutdown_connections 2024-04-26T22:02:27.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.673+0000 7fd49cd9d740 1 -- 172.21.15.55:0/3291070303 wait complete. 2024-04-26T22:02:27.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.678+0000 7fd49cd9d740 20 client.25997 trim_cache size 0 max 16384 2024-04-26T22:02:27.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.744+0000 7fd49cd9d740 1 Processor -- start 2024-04-26T22:02:27.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.745+0000 7fd49cd9d740 1 -- start start 2024-04-26T22:02:27.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.745+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x5641539a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.746+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x5641539a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.747+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x5641539a2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56102/0 (socket says 172.21.15.55:56102) 2024-04-26T22:02:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.747+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 0x5641539a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.747+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641539a84a0 0x5641539b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.747+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd480010d50 con 0x564153998a00 2024-04-26T22:02:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.747+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000e620 con 0x564153999bd0 2024-04-26T22:02:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.747+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000c520 con 0x5641539a84a0 2024-04-26T22:02:27.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.748+0000 7fd4937fe640 1 -- 172.21.15.55:0/562506013 learned_addr learned my addr 172.21.15.55:0/562506013 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.751+0000 7fd4937fe640 1 -- 172.21.15.55:0/562506013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641539a84a0 msgr2=0x5641539b60b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:27.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.751+0000 7fd4937fe640 1 --2- 172.21.15.55:0/562506013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641539a84a0 0x5641539b60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.751+0000 7fd4937fe640 1 -- 172.21.15.55:0/562506013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 msgr2=0x5641539a30f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:27.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.751+0000 7fd4937fe640 1 --2- 172.21.15.55:0/562506013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 0x5641539a30f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.751+0000 7fd4937fe640 1 -- 172.21.15.55:0/562506013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641539a0a10 con 0x564153998a00 2024-04-26T22:02:27.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.755+0000 7fd4937fe640 1 --2- 172.21.15.55:0/562506013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x5641539a2bb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd4840b7d40 tx=0x7fd4840bccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7250299d90795d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.757+0000 7fd4797fa640 1 -- 172.21.15.55:0/562506013 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840d6040 con 0x564153998a00 2024-04-26T22:02:27.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.757+0000 7fd4797fa640 1 -- 172.21.15.55:0/562506013 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4840c5aa0 con 0x564153998a00 2024-04-26T22:02:27.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.757+0000 7fd4797fa640 1 -- 172.21.15.55:0/562506013 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840bd2c0 con 0x564153998a00 2024-04-26T22:02:27.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.758+0000 7fd49cd9d740 1 -- 172.21.15.55:0/562506013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 msgr2=0x5641539a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.759+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/562506013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x5641539a2bb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd4840b7d40 tx=0x7fd4840bccd0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.759+0000 7fd49cd9d740 1 -- 172.21.15.55:0/562506013 shutdown_connections 2024-04-26T22:02:27.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.759+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/562506013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641539a84a0 0x5641539b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.759+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/562506013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 0x5641539a30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.759+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/562506013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564153998a00 0x5641539a2bb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.759+0000 7fd49cd9d740 1 -- 172.21.15.55:0/562506013 >> 172.21.15.55:0/562506013 conn(0x5641538d9910 msgr2=0x56415399d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.760+0000 7fd49cd9d740 1 -- 172.21.15.55:0/562506013 shutdown_connections 2024-04-26T22:02:27.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.760+0000 7fd49cd9d740 1 -- 172.21.15.55:0/562506013 wait complete. 2024-04-26T22:02:27.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.761+0000 7fd49cd9d740 1 -- start start 2024-04-26T22:02:27.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.761+0000 7fd49cd9d740 1 Processor -- start 2024-04-26T22:02:27.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.762+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153998a00 0x5641539c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.762+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 0x5641539c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.763+0000 7fd49cd9d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 0x5641539c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.763+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000cae0 con 0x5641539a84a0 2024-04-26T22:02:27.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.763+0000 7fd49cd9d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000df50 con 0x564153999bd0 2024-04-26T22:02:27.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.763+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153998a00 0x5641539c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.763+0000 7fd4937fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153998a00 0x5641539c0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49830/0 (socket says 172.21.15.55:49830) 2024-04-26T22:02:27.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.763+0000 7fd4937fe640 1 -- 172.21.15.55:0/2934835685 learned_addr learned my addr 172.21.15.55:0/2934835685 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:27.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.763+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000d1b0 con 0x564153998a00 2024-04-26T22:02:27.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.763+0000 7fd492ffd640 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 0x5641539c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.764+0000 7fd49cd9d740 10 client.0 _ll_register_callbacks cb 0x7fd484003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:27.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.765+0000 7fd493fff640 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 0x5641539c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.766+0000 7fd492ffd640 1 -- 172.21.15.55:0/2934835685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153998a00 msgr2=0x5641539c0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.766+0000 7fd492ffd640 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153998a00 0x5641539c0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.766+0000 7fd492ffd640 1 -- 172.21.15.55:0/2934835685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 msgr2=0x5641539c16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:27.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.766+0000 7fd492ffd640 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 0x5641539c16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.766+0000 7fd492ffd640 1 -- 172.21.15.55:0/2934835685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641539c1ce0 con 0x564153999bd0 2024-04-26T22:02:27.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.768+0000 7fd492ffd640 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 0x5641539c0750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd48c008000 tx=0x7fd48c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936030f952e03756 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:27.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.768+0000 7fd472ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:02:27.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.769+0000 7fd472ffd640 1 -- 172.21.15.55:0/2934835685 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c034060 con 0x564153999bd0 2024-04-26T22:02:27.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.769+0000 7fd472ffd640 1 -- 172.21.15.55:0/2934835685 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd48c013410 con 0x564153999bd0 2024-04-26T22:02:27.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.769+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4840c5aa0 con 0x564153999bd0 2024-04-26T22:02:27.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.770+0000 7fd472ffd640 1 -- 172.21.15.55:0/2934835685 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd48c026040 con 0x564153999bd0 2024-04-26T22:02:27.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.789+0000 7fd49cd9d740 10 client.26000 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:27.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.789+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5641539c2fa0 con 0x564153999bd0 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.792+0000 7fd472ffd640 1 -- 172.21.15.55:0/2934835685 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd48c013780 con 0x564153999bd0 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.794+0000 7fd472ffd640 1 -- 172.21.15.55:0/2934835685 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd48c02e520 con 0x564153999bd0 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.794+0000 7fd49cd9d740 10 client.26000 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.794+0000 7fd49cd9d740 10 client.26000 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.794+0000 7fd49cd9d740 20 client.26000 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.794+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564153a159f0 0x564153a17de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.794+0000 7fd49cd9d740 4 client.26000 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.794+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd4840c5aa0 con 0x564153a159f0 2024-04-26T22:02:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.795+0000 7fd4937fe640 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564153a159f0 0x564153a17de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:27.820 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:02:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.808+0000 7fd4937fe640 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564153a159f0 0x564153a17de0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64da6d3ced0b7a54 server_cookie=4aa3706db5c7db4f in_seq=0 out_seq=0 2024-04-26T22:02:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.808+0000 7fd472ffd640 10 client.26000 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:02:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.814+0000 7fd472ffd640 1 -- 172.21.15.55:0/2934835685 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd4840c5aa0 con 0x564153a159f0 2024-04-26T22:02:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.814+0000 7fd472ffd640 10 client.26000 handle_command_reply: tid=0 2024-04-26T22:02:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.814+0000 7fd49cd9d740 1 client.26000 shutdown 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.815+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 msgr2=0x5641539c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.815+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 0x5641539c0750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd48c008000 tx=0x7fd48c00b7e0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.815+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 shutdown_connections 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.815+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x564153a159f0 0x564153a17de0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.816+0000 7fd472ffd640 0 client.26000 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.816+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641539a84a0 0x5641539c16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.816+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564153999bd0 0x5641539c0750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.816+0000 7fd49cd9d740 1 --2- 172.21.15.55:0/2934835685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564153998a00 0x5641539c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.816+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 >> 172.21.15.55:0/2934835685 conn(0x5641538d9910 msgr2=0x5641539a2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.816+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 shutdown_connections 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.816+0000 7fd49cd9d740 1 -- 172.21.15.55:0/2934835685 wait complete. 2024-04-26T22:02:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.816+0000 7fd49cd9d740 20 client.26000 trim_cache size 0 max 16384 2024-04-26T22:02:27.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.822+0000 7fd49a085640 1 -- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd47c0a3dc0 msgr2=0x7fd47c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.822+0000 7fd49a085640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd47c0a3dc0 0x7fd47c0a6280 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009af0 tx=0x7fd48c012000 comp rx=0 tx=0).stop 2024-04-26T22:02:27.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.822+0000 7fd49a085640 1 -- 172.21.15.55:0/4009310420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd494073be0 msgr2=0x7fd4941a7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:27.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.822+0000 7fd49a085640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd494073be0 0x7fd4941a7970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd484000c00 tx=0x7fd484003c80 comp rx=0 tx=0).stop 2024-04-26T22:02:27.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.826+0000 7fd49a085640 1 -- 172.21.15.55:0/4009310420 shutdown_connections 2024-04-26T22:02:27.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.826+0000 7fd49a085640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd47c0a3dc0 0x7fd47c0a6280 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.826+0000 7fd49a085640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd49410a640 0x7fd4941aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.826+0000 7fd49a085640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd494074540 0x7fd4941a7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.826+0000 7fd49a085640 1 --2- 172.21.15.55:0/4009310420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd494073be0 0x7fd4941a7970 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:27.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.826+0000 7fd49a085640 1 -- 172.21.15.55:0/4009310420 >> 172.21.15.55:0/4009310420 conn(0x7fd49406f780 msgr2=0x7fd4940701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:27.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.829+0000 7fd49a085640 1 -- 172.21.15.55:0/4009310420 shutdown_connections 2024-04-26T22:02:27.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:27.829+0000 7fd49a085640 1 -- 172.21.15.55:0/4009310420 wait complete. 2024-04-26T22:02:27.893 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:02:28.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:27 smithi064 ceph-mon[32765]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 818 MiB/s wr, 1.14k op/s 2024-04-26T22:02:28.019 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:27 smithi055 ceph-mon[29221]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 818 MiB/s wr, 1.14k op/s 2024-04-26T22:02:28.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:27 smithi165 ceph-mon[33296]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 818 MiB/s wr, 1.14k op/s 2024-04-26T22:02:28.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.342+0000 7f0e820fc640 1 -- 172.21.15.55:0/2704438632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c1111b0 msgr2=0x7f0e7c1135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.342+0000 7f0e820fc640 1 --2- 172.21.15.55:0/2704438632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c1111b0 0x7f0e7c1135a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009cd0 tx=0x7f0e7402a8f0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.344+0000 7f0e820fc640 1 -- 172.21.15.55:0/2704438632 shutdown_connections 2024-04-26T22:02:28.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.344+0000 7f0e820fc640 1 --2- 172.21.15.55:0/2704438632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c1111b0 0x7f0e7c1135a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.344+0000 7f0e820fc640 1 --2- 172.21.15.55:0/2704438632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e7c10a290 0x7f0e7c110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.344+0000 7f0e820fc640 1 --2- 172.21.15.55:0/2704438632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c109930 0x7f0e7c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.344+0000 7f0e820fc640 1 -- 172.21.15.55:0/2704438632 >> 172.21.15.55:0/2704438632 conn(0x7f0e7c06f7c0 msgr2=0x7f0e7c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:28.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.350+0000 7f0e820fc640 1 -- 172.21.15.55:0/2704438632 shutdown_connections 2024-04-26T22:02:28.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.353+0000 7f0e820fc640 1 -- 172.21.15.55:0/2704438632 wait complete. 2024-04-26T22:02:28.355 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.354+0000 7f0e820fc640 1 Processor -- start 2024-04-26T22:02:28.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.360+0000 7f0e820fc640 1 -- start start 2024-04-26T22:02:28.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.360+0000 7f0e820fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c109930 0x7f0e7c10c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.360+0000 7f0e820fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e7c10a290 0x7f0e7c10ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.361+0000 7f0e820fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c1111b0 0x7f0e7c10b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.361+0000 7f0e820fc640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c116cd0 con 0x7f0e7c1111b0 2024-04-26T22:02:28.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.361+0000 7f0e820fc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c116b50 con 0x7f0e7c109930 2024-04-26T22:02:28.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.361+0000 7f0e820fc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c116e50 con 0x7f0e7c10a290 2024-04-26T22:02:28.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.362+0000 7f0e7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c109930 0x7f0e7c10c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.363+0000 7f0e7affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e7c10a290 0x7f0e7c10ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.363+0000 7f0e7b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c109930 0x7f0e7c10c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39838/0 (socket says 172.21.15.55:39838) 2024-04-26T22:02:28.364 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.363+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/1848868608 learned_addr learned my addr 172.21.15.55:0/1848868608 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:28.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.364+0000 7f0e7bfff640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c1111b0 0x7f0e7c10b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.364+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/1848868608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e7c10a290 msgr2=0x7f0e7c10ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.364+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e7c10a290 0x7f0e7c10ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.364+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c1111b0 msgr2=0x7f0e7c10b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.364+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c1111b0 0x7f0e7c10b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.364+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/1848868608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c10b680 con 0x7f0e7c109930 2024-04-26T22:02:28.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.365+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c109930 0x7f0e7c10c5c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e9f0 tx=0x7f0e6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:28.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.366+0000 7f0e78ff9640 1 -- 172.21.15.55:0/1848868608 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00cd80 con 0x7f0e7c109930 2024-04-26T22:02:28.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.366+0000 7f0e78ff9640 1 -- 172.21.15.55:0/1848868608 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c0108e0 con 0x7f0e7c109930 2024-04-26T22:02:28.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.366+0000 7f0e820fc640 1 -- 172.21.15.55:0/1848868608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c10b970 con 0x7f0e7c109930 2024-04-26T22:02:28.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.366+0000 7f0e820fc640 1 -- 172.21.15.55:0/1848868608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1afdf0 con 0x7f0e7c109930 2024-04-26T22:02:28.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.367+0000 7f0e78ff9640 1 -- 172.21.15.55:0/1848868608 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c018980 con 0x7f0e7c109930 2024-04-26T22:02:28.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.376+0000 7f0e78ff9640 1 -- 172.21.15.55:0/1848868608 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e6c018b20 con 0x7f0e7c109930 2024-04-26T22:02:28.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.377+0000 7f0e78ff9640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0e640a3d80 0x7f0e640a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.378+0000 7f0e78ff9640 1 -- 172.21.15.55:0/1848868608 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0e6c0d1980 con 0x7f0e7c109930 2024-04-26T22:02:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.380+0000 7f0e84e11740 1 Processor -- start 2024-04-26T22:02:28.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.380+0000 7f0e84e11740 1 -- start start 2024-04-26T22:02:28.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec2830 0x56329eec2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eec3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eecac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e84e11740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56329ee26d80 con 0x56329eec2830 2024-04-26T22:02:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e7affd640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0e640a3d80 0x7f0e640a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e84e11740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56329ede0a20 con 0x56329eec43c0 2024-04-26T22:02:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e84e11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56329ee22d40 con 0x56329eec3a00 2024-04-26T22:02:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e7b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec2830 0x56329eec2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e7b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec2830 0x56329eec2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56128/0 (socket says 172.21.15.55:56128) 2024-04-26T22:02:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.381+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/1322899351 learned_addr learned my addr 172.21.15.55:0/1322899351 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.382+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/1322899351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 msgr2=0x56329eec3e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.382+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/1322899351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eec3e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.382+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/1322899351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 msgr2=0x56329eecac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.382+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/1322899351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eecac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.382+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/1322899351 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329edebc30 con 0x56329eec2830 2024-04-26T22:02:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.382+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/1322899351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec2830 0x56329eec2c50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c6a0 tx=0x7f0e6c09da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76de8555ba8f4f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.385+0000 7f0e61ffb640 1 -- 172.21.15.55:0/1322899351 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c09bd60 con 0x56329eec2830 2024-04-26T22:02:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.385+0000 7f0e61ffb640 1 -- 172.21.15.55:0/1322899351 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c010a90 con 0x56329eec2830 2024-04-26T22:02:28.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.386+0000 7f0e84e11740 1 -- 172.21.15.55:0/1322899351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec2830 msgr2=0x56329eec2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.386+0000 7f0e84e11740 1 --2- 172.21.15.55:0/1322899351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec2830 0x56329eec2c50 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c6a0 tx=0x7f0e6c09da60 comp rx=0 tx=0).stop 2024-04-26T22:02:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.387+0000 7f0e84e11740 1 -- 172.21.15.55:0/1322899351 shutdown_connections 2024-04-26T22:02:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.387+0000 7f0e84e11740 1 --2- 172.21.15.55:0/1322899351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eecac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.387+0000 7f0e84e11740 1 --2- 172.21.15.55:0/1322899351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eec3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.387+0000 7f0e84e11740 1 --2- 172.21.15.55:0/1322899351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec2830 0x56329eec2c50 secure :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c6a0 tx=0x7f0e6c09da60 comp rx=0 tx=0).stop 2024-04-26T22:02:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.387+0000 7f0e84e11740 1 -- 172.21.15.55:0/1322899351 >> 172.21.15.55:0/1322899351 conn(0x56329ee03910 msgr2=0x56329eebe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.390+0000 7f0e7affd640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0e640a3d80 0x7f0e640a6240 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c10c330 tx=0x7f0e70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:28.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.392+0000 7f0e84e11740 1 -- 172.21.15.55:0/1322899351 shutdown_connections 2024-04-26T22:02:28.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.395+0000 7f0e84e11740 1 -- 172.21.15.55:0/1322899351 wait complete. 2024-04-26T22:02:28.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.395+0000 7f0e84e11740 1 -- start start 2024-04-26T22:02:28.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.395+0000 7f0e84e11740 1 Processor -- start 2024-04-26T22:02:28.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.396+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eecce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.396+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eecd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.396+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eed47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.396+0000 7f0e84e11740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56329eecb3c0 con 0x56329eed2400 2024-04-26T22:02:28.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.396+0000 7f0e84e11740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56329edaddc0 con 0x56329eec43c0 2024-04-26T22:02:28.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.396+0000 7f0e84e11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56329eecb540 con 0x56329eec3a00 2024-04-26T22:02:28.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.397+0000 7f0e84e11740 10 client.0 _ll_register_callbacks cb 0x56329edc9600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:28.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.403+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eed47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.403+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eed47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56142/0 (socket says 172.21.15.55:56142) 2024-04-26T22:02:28.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.403+0000 7f0e7bfff640 1 -- 172.21.15.55:0/3887639941 learned_addr learned my addr 172.21.15.55:0/3887639941 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:28.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.403+0000 7f0e7affd640 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eecd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.406+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eecce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.411+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/3887639941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 msgr2=0x56329eecd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:28.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.411+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eecd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.411+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/3887639941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 msgr2=0x56329eed47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:28.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.411+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eed47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.411+0000 7f0e7b7fe640 1 -- 172.21.15.55:0/3887639941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329eb383f0 con 0x56329eec3a00 2024-04-26T22:02:28.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.415+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eecce90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x56329eec35b0 tx=0x7f0e6c0cf240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7551f424d9d7e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:28.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.415+0000 7f0e5affd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:02:28.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.415+0000 7f0e5affd640 1 -- 172.21.15.55:0/3887639941 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c010a90 con 0x56329eec3a00 2024-04-26T22:02:28.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.415+0000 7f0e5affd640 1 -- 172.21.15.55:0/3887639941 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0e6c010430 con 0x56329eec3a00 2024-04-26T22:02:28.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.419+0000 7f0e84e11740 1 -- 172.21.15.55:0/3887639941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56329ee34990 con 0x56329eec3a00 2024-04-26T22:02:28.423 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.422+0000 7f0e5affd640 1 -- 172.21.15.55:0/3887639941 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0e6c010630 con 0x56329eec3a00 2024-04-26T22:02:28.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.431+0000 7f0e84e11740 10 client.26026 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:28.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.431+0000 7f0e84e11740 1 -- 172.21.15.55:0/3887639941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56329ec027d0 con 0x56329eec3a00 2024-04-26T22:02:28.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.435+0000 7f0e5affd640 1 -- 172.21.15.55:0/3887639941 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c09c920 con 0x56329eec3a00 2024-04-26T22:02:28.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.435+0000 7f0e5affd640 1 -- 172.21.15.55:0/3887639941 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0e6c0a30b0 con 0x56329eec3a00 2024-04-26T22:02:28.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.436+0000 7f0e84e11740 10 client.26026 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:28.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.436+0000 7f0e84e11740 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:28.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.436+0000 7f0e84e11740 20 client.26026 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:28.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.436+0000 7f0e84e11740 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56329eed8d70 0x56329eedb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.436+0000 7f0e84e11740 4 client.26026 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:02:28.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.437+0000 7f0e84e11740 1 -- 172.21.15.55:0/3887639941 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56329ee34990 con 0x56329eed8d70 2024-04-26T22:02:28.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.441+0000 7f0e7affd640 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56329eed8d70 0x56329eedb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.460+0000 7f0e5affd640 10 client.26026 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:28.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.460+0000 7f0e7affd640 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56329eed8d70 0x56329eedb160 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fafe31faa137698 server_cookie=defd8b2bb38533f1 in_seq=0 out_seq=0 2024-04-26T22:02:28.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.472+0000 7f0e5affd640 1 -- 172.21.15.55:0/3887639941 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56329ee34990 con 0x56329eed8d70 2024-04-26T22:02:28.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.472+0000 7f0e5affd640 10 client.26026 handle_command_reply: tid=0 2024-04-26T22:02:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.472+0000 7f0e84e11740 1 client.26026 shutdown 2024-04-26T22:02:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.474+0000 7f0e84e11740 1 -- 172.21.15.55:0/3887639941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 msgr2=0x56329eecce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.474+0000 7f0e84e11740 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eecce90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x56329eec35b0 tx=0x7f0e6c0cf240 comp rx=0 tx=0).stop 2024-04-26T22:02:28.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.475+0000 7f0e84e11740 1 -- 172.21.15.55:0/3887639941 shutdown_connections 2024-04-26T22:02:28.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.475+0000 7f0e84e11740 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56329eed8d70 0x56329eedb160 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.475+0000 7f0e84e11740 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eed47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.475+0000 7f0e84e11740 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eecd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.475+0000 7f0e84e11740 1 --2- 172.21.15.55:0/3887639941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eecce90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.477 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.475+0000 7f0e84e11740 1 -- 172.21.15.55:0/3887639941 >> 172.21.15.55:0/3887639941 conn(0x56329ee03910 msgr2=0x56329eebd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.476+0000 7f0e5affd640 0 client.26026 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.476+0000 7f0e84e11740 1 -- 172.21.15.55:0/3887639941 shutdown_connections 2024-04-26T22:02:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.477+0000 7f0e84e11740 1 -- 172.21.15.55:0/3887639941 wait complete. 2024-04-26T22:02:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.477+0000 7f0e84e11740 20 client.26026 trim_cache size 0 max 16384 2024-04-26T22:02:28.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.539+0000 7f0e84e11740 1 Processor -- start 2024-04-26T22:02:28.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.541+0000 7f0e84e11740 1 -- start start 2024-04-26T22:02:28.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.541+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec3a00 0x56329eebe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.541+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec43c0 0x56329eeccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.541+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eed2400 0x56329eeccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.541+0000 7f0e84e11740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1b2ef0 con 0x56329eec43c0 2024-04-26T22:02:28.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.541+0000 7f0e84e11740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1b07c0 con 0x56329eec3a00 2024-04-26T22:02:28.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.541+0000 7f0e84e11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1ac1c0 con 0x56329eed2400 2024-04-26T22:02:28.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.543+0000 7f0e7affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec43c0 0x56329eeccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.543+0000 7f0e7affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec43c0 0x56329eeccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56146/0 (socket says 172.21.15.55:56146) 2024-04-26T22:02:28.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.543+0000 7f0e7affd640 1 -- 172.21.15.55:0/376161146 learned_addr learned my addr 172.21.15.55:0/376161146 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:28.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.543+0000 7f0e7bfff640 1 --2- 172.21.15.55:0/376161146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eed2400 0x56329eeccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.544+0000 7f0e7affd640 1 -- 172.21.15.55:0/376161146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eed2400 msgr2=0x56329eeccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:28.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.544+0000 7f0e7affd640 1 --2- 172.21.15.55:0/376161146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eed2400 0x56329eeccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.544+0000 7f0e7affd640 1 -- 172.21.15.55:0/376161146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec3a00 msgr2=0x56329eebe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:28.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.544+0000 7f0e7affd640 1 --2- 172.21.15.55:0/376161146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec3a00 0x56329eebe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.544+0000 7f0e7affd640 1 -- 172.21.15.55:0/376161146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329eeb9ae0 con 0x56329eec43c0 2024-04-26T22:02:28.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.546+0000 7f0e7affd640 1 --2- 172.21.15.55:0/376161146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec43c0 0x56329eeccaa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009fd0 tx=0x7f0e70024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a1b598aead32f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:28.547 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.547+0000 7f0e617fa640 1 -- 172.21.15.55:0/376161146 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70041070 con 0x56329eec43c0 2024-04-26T22:02:28.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.547+0000 7f0e617fa640 1 -- 172.21.15.55:0/376161146 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56329ee34990 con 0x56329eec43c0 2024-04-26T22:02:28.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.548+0000 7f0e617fa640 1 -- 172.21.15.55:0/376161146 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70031a90 con 0x56329eec43c0 2024-04-26T22:02:28.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.549+0000 7f0e84e11740 1 -- 172.21.15.55:0/376161146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec43c0 msgr2=0x56329eeccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.549+0000 7f0e84e11740 1 --2- 172.21.15.55:0/376161146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec43c0 0x56329eeccaa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009fd0 tx=0x7f0e70024000 comp rx=0 tx=0).stop 2024-04-26T22:02:28.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.549+0000 7f0e84e11740 1 -- 172.21.15.55:0/376161146 shutdown_connections 2024-04-26T22:02:28.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.549+0000 7f0e84e11740 1 --2- 172.21.15.55:0/376161146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eed2400 0x56329eeccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.549+0000 7f0e84e11740 1 --2- 172.21.15.55:0/376161146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eec43c0 0x56329eeccaa0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.549+0000 7f0e84e11740 1 --2- 172.21.15.55:0/376161146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec3a00 0x56329eebe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.549+0000 7f0e84e11740 1 -- 172.21.15.55:0/376161146 >> 172.21.15.55:0/376161146 conn(0x56329ee03910 msgr2=0x56329eebdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:28.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.549+0000 7f0e84e11740 1 -- 172.21.15.55:0/376161146 shutdown_connections 2024-04-26T22:02:28.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.550+0000 7f0e84e11740 1 -- 172.21.15.55:0/376161146 wait complete. 2024-04-26T22:02:28.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.550+0000 7f0e84e11740 1 -- start start 2024-04-26T22:02:28.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.551+0000 7f0e84e11740 1 Processor -- start 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.551+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eee8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.551+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eee85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.551+0000 7f0e84e11740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eeeb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.552+0000 7f0e84e11740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1ac830 con 0x56329eed2400 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.552+0000 7f0e84e11740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1b00f0 con 0x56329eec43c0 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.552+0000 7f0e84e11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1acf00 con 0x56329eec3a00 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.552+0000 7f0e84e11740 10 client.0 _ll_register_callbacks cb 0x7f0e6c0a1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.553+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eeeb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.553+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eeeb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56150/0 (socket says 172.21.15.55:56150) 2024-04-26T22:02:28.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.553+0000 7f0e7bfff640 1 -- 172.21.15.55:0/734672255 learned_addr learned my addr 172.21.15.55:0/734672255 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:28.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.554+0000 7f0e7bfff640 1 -- 172.21.15.55:0/734672255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 msgr2=0x56329eee8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:28.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.554+0000 7f0e7bfff640 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eee8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.554+0000 7f0e7bfff640 1 -- 172.21.15.55:0/734672255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 msgr2=0x56329eee85c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:28.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.554+0000 7f0e7bfff640 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eee85c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.554+0000 7f0e7bfff640 1 -- 172.21.15.55:0/734672255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329eeebbd0 con 0x56329eed2400 2024-04-26T22:02:28.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.558+0000 7f0e7bfff640 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eeeb5e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009cd0 tx=0x7f0e74030900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d7af5b99173b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:28.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.560+0000 7f0e5affd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:02:28.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.561+0000 7f0e5affd640 1 -- 172.21.15.55:0/734672255 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7404a060 con 0x56329eed2400 2024-04-26T22:02:28.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.561+0000 7f0e84e11740 1 -- 172.21.15.55:0/734672255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56329ee34990 con 0x56329eed2400 2024-04-26T22:02:28.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.562+0000 7f0e5affd640 1 -- 172.21.15.55:0/734672255 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0e74030c00 con 0x56329eed2400 2024-04-26T22:02:28.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.562+0000 7f0e5affd640 1 -- 172.21.15.55:0/734672255 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0e74003970 con 0x56329eed2400 2024-04-26T22:02:28.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.565+0000 7f0e84e11740 10 client.16929 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:28.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.565+0000 7f0e84e11740 1 -- 172.21.15.55:0/734672255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56329ef3f200 con 0x56329eed2400 2024-04-26T22:02:28.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.570+0000 7f0e5affd640 1 -- 172.21.15.55:0/734672255 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56329ee34990 con 0x56329eed2400 2024-04-26T22:02:28.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.571+0000 7f0e5affd640 1 -- 172.21.15.55:0/734672255 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0e74031440 con 0x56329eed2400 2024-04-26T22:02:28.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.571+0000 7f0e84e11740 10 client.16929 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:28.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.571+0000 7f0e84e11740 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:28.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.571+0000 7f0e84e11740 20 client.16929 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:28.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.571+0000 7f0e84e11740 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56329ef3fc10 0x56329ef42030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:28.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.571+0000 7f0e84e11740 4 client.16929 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:02:28.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.571+0000 7f0e84e11740 1 -- 172.21.15.55:0/734672255 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56329ee2e800 con 0x56329ef3fc10 2024-04-26T22:02:28.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.573+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56329ef3fc10 0x56329ef42030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:28.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.576+0000 7f0e7b7fe640 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56329ef3fc10 0x56329ef42030 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ffe5ab967f38897 server_cookie=e6f50c83a840a78d in_seq=0 out_seq=0 2024-04-26T22:02:28.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.576+0000 7f0e5affd640 10 client.16929 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:28.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.581+0000 7f0e5affd640 1 -- 172.21.15.55:0/734672255 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56329ee2e800 con 0x56329ef3fc10 2024-04-26T22:02:28.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.581+0000 7f0e5affd640 10 client.16929 handle_command_reply: tid=0 2024-04-26T22:02:28.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 client.16929 shutdown 2024-04-26T22:02:28.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 -- 172.21.15.55:0/734672255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 msgr2=0x56329eeeb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eeeb5e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009cd0 tx=0x7f0e74030900 comp rx=0 tx=0).stop 2024-04-26T22:02:28.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 -- 172.21.15.55:0/734672255 shutdown_connections 2024-04-26T22:02:28.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56329ef3fc10 0x56329ef42030 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56329eed2400 0x56329eeeb5e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56329eec43c0 0x56329eee85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 --2- 172.21.15.55:0/734672255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56329eec3a00 0x56329eee8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e84e11740 1 -- 172.21.15.55:0/734672255 >> 172.21.15.55:0/734672255 conn(0x56329ee03910 msgr2=0x56329eec6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:28.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.582+0000 7f0e5affd640 0 client.16929 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:28.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.583+0000 7f0e84e11740 1 -- 172.21.15.55:0/734672255 shutdown_connections 2024-04-26T22:02:28.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.583+0000 7f0e84e11740 1 -- 172.21.15.55:0/734672255 wait complete. 2024-04-26T22:02:28.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.583+0000 7f0e84e11740 20 client.16929 trim_cache size 0 max 16384 2024-04-26T22:02:28.586 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:02:28.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 -- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0e640a3d80 msgr2=0x7f0e640a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0e640a3d80 0x7f0e640a6240 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c10c330 tx=0x7f0e70005f90 comp rx=0 tx=0).stop 2024-04-26T22:02:28.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 -- 172.21.15.55:0/1848868608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c109930 msgr2=0x7f0e7c10c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:28.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c109930 0x7f0e7c10c5c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e9f0 tx=0x7f0e6c00eeb0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 -- 172.21.15.55:0/1848868608 shutdown_connections 2024-04-26T22:02:28.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0e640a3d80 0x7f0e640a6240 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c1111b0 0x7f0e7c10b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e7c10a290 0x7f0e7c10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 --2- 172.21.15.55:0/1848868608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7c109930 0x7f0e7c10c5c0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e9f0 tx=0x7f0e6c00eeb0 comp rx=0 tx=0).stop 2024-04-26T22:02:28.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.585+0000 7f0e820fc640 1 -- 172.21.15.55:0/1848868608 >> 172.21.15.55:0/1848868608 conn(0x7f0e7c06f7c0 msgr2=0x7f0e7c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:28.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.586+0000 7f0e820fc640 1 -- 172.21.15.55:0/1848868608 shutdown_connections 2024-04-26T22:02:28.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:28.586+0000 7f0e820fc640 1 -- 172.21.15.55:0/1848868608 wait complete. 2024-04-26T22:02:29.631 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:02:29.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:02:29.631 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:02:29.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:29.997+0000 7f703a7a7640 1 -- 172.21.15.55:0/2627729299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f703410a590 msgr2=0x7f703410a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:29.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:29.997+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2627729299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f703410a590 0x7f703410a9f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7024009900 tx=0x7f702402a440 comp rx=0 tx=0).stop 2024-04-26T22:02:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:29.999+0000 7f703a7a7640 1 -- 172.21.15.55:0/2627729299 shutdown_connections 2024-04-26T22:02:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:29.999+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2627729299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f703410b060 0x7f7034111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:29.999+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2627729299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f703410a590 0x7f703410a9f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:29.999+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2627729299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7034109ba0 0x7f7034109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:29.999+0000 7f703a7a7640 1 -- 172.21.15.55:0/2627729299 >> 172.21.15.55:0/2627729299 conn(0x7f703406f780 msgr2=0x7f7034071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:30.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.002+0000 7f703a7a7640 1 -- 172.21.15.55:0/2627729299 shutdown_connections 2024-04-26T22:02:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.004+0000 7f703a7a7640 1 -- 172.21.15.55:0/2627729299 wait complete. 2024-04-26T22:02:30.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.006+0000 7f703a7a7640 1 Processor -- start 2024-04-26T22:02:30.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:29 smithi064 ceph-mon[32765]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 897 MiB/s wr, 1.18k op/s 2024-04-26T22:02:30.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.008+0000 7f703a7a7640 1 -- start start 2024-04-26T22:02:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.008+0000 7f703a7a7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034109ba0 0x7f70341a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.009+0000 7f703a7a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f703410a590 0x7f70341a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.009+0000 7f703a7a7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f703410b060 0x7f70341a8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.009+0000 7f703a7a7640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7034116c00 con 0x7f7034109ba0 2024-04-26T22:02:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.009+0000 7f703a7a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7034116a80 con 0x7f703410a590 2024-04-26T22:02:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.009+0000 7f703a7a7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7034116d80 con 0x7f703410b060 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.010+0000 7f7039fa6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f703410b060 0x7f70341a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.010+0000 7f7039fa6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f703410b060 0x7f70341a8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49916/0 (socket says 172.21.15.55:49916) 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.010+0000 7f7039fa6640 1 -- 172.21.15.55:0/2889241344 learned_addr learned my addr 172.21.15.55:0/2889241344 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.010+0000 7f7038fa4640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f703410a590 0x7f70341a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.012+0000 7f7039fa6640 1 -- 172.21.15.55:0/2889241344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f703410a590 msgr2=0x7f70341a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.012+0000 7f7039fa6640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f703410a590 0x7f70341a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.012+0000 7f7039fa6640 1 -- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034109ba0 msgr2=0x7f70341a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.012+0000 7f7039fa6640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034109ba0 0x7f70341a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.012+0000 7f7039fa6640 1 -- 172.21.15.55:0/2889241344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70341a8f70 con 0x7f703410b060 2024-04-26T22:02:30.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.014+0000 7f7039fa6640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f703410b060 0x7f70341a8900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b990 tx=0x7f702c00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:30.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.015+0000 7f702a7fc640 1 -- 172.21.15.55:0/2889241344 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c019070 con 0x7f703410b060 2024-04-26T22:02:30.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.015+0000 7f702a7fc640 1 -- 172.21.15.55:0/2889241344 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702c00cba0 con 0x7f703410b060 2024-04-26T22:02:30.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.015+0000 7f702a7fc640 1 -- 172.21.15.55:0/2889241344 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c0075d0 con 0x7f703410b060 2024-04-26T22:02:30.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.018+0000 7f703a7a7640 1 -- 172.21.15.55:0/2889241344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70341a9260 con 0x7f703410b060 2024-04-26T22:02:30.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.018+0000 7f703a7a7640 1 -- 172.21.15.55:0/2889241344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70341a97b0 con 0x7f703410b060 2024-04-26T22:02:30.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.028+0000 7f703e7ff740 1 Processor -- start 2024-04-26T22:02:30.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.029+0000 7f703e7ff740 1 -- start start 2024-04-26T22:02:30.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.033+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467821bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.034+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x564678220fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.034+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 0x564678223ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.034+0000 7f703e7ff740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564678180b00 con 0x56467821c4b0 2024-04-26T22:02:30.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.034+0000 7f703e7ff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56467811c920 con 0x564678221620 2024-04-26T22:02:30.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.034+0000 7f703e7ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56467817cac0 con 0x56467821bac0 2024-04-26T22:02:30.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.034+0000 7f702a7fc640 1 -- 172.21.15.55:0/2889241344 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f702c01c070 con 0x7f703410b060 2024-04-26T22:02:30.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.035+0000 7f7038fa4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467821bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.035+0000 7f7038fa4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467821bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49928/0 (socket says 172.21.15.55:49928) 2024-04-26T22:02:30.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.035+0000 7f7038fa4640 1 -- 172.21.15.55:0/1510846392 learned_addr learned my addr 172.21.15.55:0/1510846392 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:30.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.036+0000 7f7038fa4640 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 0x564678223ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.036+0000 7f702a7fc640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70200a3d40 0x7f70200a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.037+0000 7f702a7fc640 1 -- 172.21.15.55:0/2889241344 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f702c0cfa50 con 0x7f703410b060 2024-04-26T22:02:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.038+0000 7f70397a5640 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x564678220fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.039+0000 7f7039fa6640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70200a3d40 0x7f70200a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.039+0000 7f70397a5640 1 -- 172.21.15.55:0/1510846392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 msgr2=0x56467821bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:30.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.039+0000 7f70397a5640 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467821bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.039+0000 7f70397a5640 1 -- 172.21.15.55:0/1510846392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 msgr2=0x564678223ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:30.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.039+0000 7f70397a5640 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 0x564678223ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.039+0000 7f70397a5640 1 -- 172.21.15.55:0/1510846392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564677f5a7e0 con 0x56467821c4b0 2024-04-26T22:02:30.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.040+0000 7f70397a5640 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x564678220fb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7030009720 tx=0x7f7030009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1106a71154b8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:30.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.040+0000 7f7039fa6640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70200a3d40 0x7f70200a6200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f702c0986c0 tx=0x7f702c0cfca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:30.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.042+0000 7f701f7fe640 1 -- 172.21.15.55:0/1510846392 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030015020 con 0x56467821c4b0 2024-04-26T22:02:30.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.042+0000 7f701f7fe640 1 -- 172.21.15.55:0/1510846392 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7030009e30 con 0x56467821c4b0 2024-04-26T22:02:30.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.043+0000 7f703e7ff740 1 -- 172.21.15.55:0/1510846392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 msgr2=0x564678220fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.043+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x564678220fb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7030009720 tx=0x7f7030009be0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.047+0000 7f703e7ff740 1 -- 172.21.15.55:0/1510846392 shutdown_connections 2024-04-26T22:02:30.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.047+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 0x564678223ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.047+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x564678220fb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.047+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1510846392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467821bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.047+0000 7f703e7ff740 1 -- 172.21.15.55:0/1510846392 >> 172.21.15.55:0/1510846392 conn(0x564678217130 msgr2=0x564678217530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.049+0000 7f703e7ff740 1 -- 172.21.15.55:0/1510846392 shutdown_connections 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.049+0000 7f703e7ff740 1 -- 172.21.15.55:0/1510846392 wait complete. 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.050+0000 7f703e7ff740 1 -- start start 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.050+0000 7f703e7ff740 1 Processor -- start 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.051+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821bac0 0x5646782207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.051+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821c4b0 0x564678220d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.051+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564678221620 0x56467821d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.051+0000 7f703e7ff740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564678224620 con 0x564678221620 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.051+0000 7f703e7ff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56467811a5d0 con 0x56467821bac0 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.051+0000 7f703e7ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5646782247a0 con 0x56467821c4b0 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.051+0000 7f703e7ff740 10 client.0 _ll_register_callbacks cb 0x564678108080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.052+0000 7f70397a5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821c4b0 0x564678220d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.052+0000 7f70397a5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821c4b0 0x564678220d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49940/0 (socket says 172.21.15.55:49940) 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.052+0000 7f70397a5640 1 -- 172.21.15.55:0/1175849327 learned_addr learned my addr 172.21.15.55:0/1175849327 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.052+0000 7f7038fa4640 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821bac0 0x5646782207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.053+0000 7f70397a5640 1 -- 172.21.15.55:0/1175849327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821bac0 msgr2=0x5646782207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.053+0000 7f70397a5640 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821bac0 0x5646782207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.053+0000 7f70397a5640 1 -- 172.21.15.55:0/1175849327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564678221620 msgr2=0x56467821d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.053+0000 7f70397a5640 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564678221620 0x56467821d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.053+0000 7f70397a5640 1 -- 172.21.15.55:0/1175849327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564677fbaf70 con 0x56467821c4b0 2024-04-26T22:02:30.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.057+0000 7f70397a5640 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821c4b0 0x564678220d20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7030016000 tx=0x7f7030006b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cfa77c84bb2fb4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:30.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.057+0000 7f701cff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:02:30.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.059+0000 7f701cff9640 1 -- 172.21.15.55:0/1175849327 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030020070 con 0x56467821c4b0 2024-04-26T22:02:30.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.060+0000 7f703e7ff740 1 -- 172.21.15.55:0/1175849327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7030009e30 con 0x56467821c4b0 2024-04-26T22:02:30.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.060+0000 7f701cff9640 1 -- 172.21.15.55:0/1175849327 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f703000bad0 con 0x56467821c4b0 2024-04-26T22:02:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.064+0000 7f701cff9640 1 -- 172.21.15.55:0/1175849327 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f703000bcd0 con 0x56467821c4b0 2024-04-26T22:02:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.064+0000 7f703e7ff740 10 client.26038 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.064+0000 7f703e7ff740 1 -- 172.21.15.55:0/1175849327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564677e52170 con 0x56467821c4b0 2024-04-26T22:02:30.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.065+0000 7f701cff9640 1 -- 172.21.15.55:0/1175849327 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7030009e30 con 0x56467821c4b0 2024-04-26T22:02:30.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:29 smithi165 ceph-mon[33296]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 897 MiB/s wr, 1.18k op/s 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.065+0000 7f701cff9640 1 -- 172.21.15.55:0/1175849327 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f703002f4f0 con 0x56467821c4b0 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.067+0000 7f703e7ff740 10 client.26038 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.067+0000 7f703e7ff740 10 client.26038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.067+0000 7f703e7ff740 20 client.26038 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.067+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56467823a450 0x56467823c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.067+0000 7f703e7ff740 4 client.26038 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.067+0000 7f703e7ff740 1 -- 172.21.15.55:0/1175849327 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56467818e710 con 0x56467823a450 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.068+0000 7f7038fa4640 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56467823a450 0x56467823c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.072+0000 7f7038fa4640 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56467823a450 0x56467823c8b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe73b654c3de748 server_cookie=ae95dfbca8d2e242 in_seq=0 out_seq=0 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.073+0000 7f701cff9640 10 client.26038 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.078+0000 7f701cff9640 1 -- 172.21.15.55:0/1175849327 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56467818e710 con 0x56467823a450 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.078+0000 7f701cff9640 10 client.26038 handle_command_reply: tid=0 2024-04-26T22:02:30.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.078+0000 7f703e7ff740 1 client.26038 shutdown 2024-04-26T22:02:30.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.078+0000 7f703e7ff740 1 -- 172.21.15.55:0/1175849327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821c4b0 msgr2=0x564678220d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.078+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821c4b0 0x564678220d20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7030016000 tx=0x7f7030006b80 comp rx=0 tx=0).stop 2024-04-26T22:02:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f703e7ff740 1 -- 172.21.15.55:0/1175849327 shutdown_connections 2024-04-26T22:02:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56467823a450 0x56467823c8b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564678221620 0x56467821d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821c4b0 0x564678220d20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f703e7ff740 1 --2- 172.21.15.55:0/1175849327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821bac0 0x5646782207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f703e7ff740 1 -- 172.21.15.55:0/1175849327 >> 172.21.15.55:0/1175849327 conn(0x564678217130 msgr2=0x5646782162f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:30.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f701cff9640 0 client.26038 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:30.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f703e7ff740 1 -- 172.21.15.55:0/1175849327 shutdown_connections 2024-04-26T22:02:30.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.082+0000 7f703e7ff740 1 -- 172.21.15.55:0/1175849327 wait complete. 2024-04-26T22:02:30.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.084+0000 7f703e7ff740 20 client.26038 trim_cache size 0 max 16384 2024-04-26T22:02:30.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.175+0000 7f703e7ff740 1 Processor -- start 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.176+0000 7f703e7ff740 1 -- start start 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.176+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821bac0 0x5646782207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821c4b0 0x564678220ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564678221620 0x564678219d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f703e7ff740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f70341b2e70 con 0x56467821bac0 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f703e7ff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f70341b0740 con 0x56467821c4b0 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f703e7ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70341ae590 con 0x564678221620 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f70397a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821c4b0 0x564678220ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f70397a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821c4b0 0x564678220ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:39904/0 (socket says 172.21.15.55:39904) 2024-04-26T22:02:30.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f70397a5640 1 -- 172.21.15.55:0/4667497 learned_addr learned my addr 172.21.15.55:0/4667497 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:30.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.177+0000 7f7038fa4640 1 --2- 172.21.15.55:0/4667497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564678221620 0x564678219d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.178+0000 7f70397a5640 1 -- 172.21.15.55:0/4667497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564678221620 msgr2=0x564678219d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:30.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.178+0000 7f70397a5640 1 --2- 172.21.15.55:0/4667497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564678221620 0x564678219d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.178+0000 7f70397a5640 1 -- 172.21.15.55:0/4667497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821bac0 msgr2=0x5646782207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:30.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.178+0000 7f70397a5640 1 --2- 172.21.15.55:0/4667497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821bac0 0x5646782207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.178+0000 7f70397a5640 1 -- 172.21.15.55:0/4667497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56467821d5f0 con 0x56467821c4b0 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.179+0000 7f70397a5640 1 --2- 172.21.15.55:0/4667497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821c4b0 0x564678220ce0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7030009c20 tx=0x7f7030006b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c23a951a208a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.179+0000 7f701effd640 1 -- 172.21.15.55:0/4667497 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030034070 con 0x56467821c4b0 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.179+0000 7f701effd640 1 -- 172.21.15.55:0/4667497 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7030006bf0 con 0x56467821c4b0 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.179+0000 7f701effd640 1 -- 172.21.15.55:0/4667497 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030020040 con 0x56467821c4b0 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.180+0000 7f703e7ff740 1 -- 172.21.15.55:0/4667497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821c4b0 msgr2=0x564678220ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.180+0000 7f703e7ff740 1 --2- 172.21.15.55:0/4667497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821c4b0 0x564678220ce0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7030009c20 tx=0x7f7030006b00 comp rx=0 tx=0).stop 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.181+0000 7f703e7ff740 1 -- 172.21.15.55:0/4667497 shutdown_connections 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.181+0000 7f703e7ff740 1 --2- 172.21.15.55:0/4667497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564678221620 0x564678219d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.181+0000 7f703e7ff740 1 --2- 172.21.15.55:0/4667497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56467821c4b0 0x564678220ce0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.181+0000 7f703e7ff740 1 --2- 172.21.15.55:0/4667497 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821bac0 0x5646782207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.181+0000 7f703e7ff740 1 -- 172.21.15.55:0/4667497 >> 172.21.15.55:0/4667497 conn(0x564678217130 msgr2=0x5646782252f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:30.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:29 smithi055 ceph-mon[29221]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 897 MiB/s wr, 1.18k op/s 2024-04-26T22:02:30.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.198+0000 7f703e7ff740 1 -- 172.21.15.55:0/4667497 shutdown_connections 2024-04-26T22:02:30.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.199+0000 7f703e7ff740 1 -- 172.21.15.55:0/4667497 wait complete. 2024-04-26T22:02:30.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.199+0000 7f703e7ff740 1 -- start start 2024-04-26T22:02:30.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.200+0000 7f703e7ff740 1 Processor -- start 2024-04-26T22:02:30.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.201+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467829cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.201+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x56467829d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.202+0000 7f703e7ff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 0x56467824c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.202+0000 7f703e7ff740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f70341aec00 con 0x56467821c4b0 2024-04-26T22:02:30.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.202+0000 7f703e7ff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f70341b0070 con 0x564678221620 2024-04-26T22:02:30.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.202+0000 7f703e7ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70341af2d0 con 0x56467821bac0 2024-04-26T22:02:30.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.202+0000 7f703e7ff740 10 client.0 _ll_register_callbacks cb 0x7f7030009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:30.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.202+0000 7f70397a5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x56467829d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.202+0000 7f70397a5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x56467829d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56208/0 (socket says 172.21.15.55:56208) 2024-04-26T22:02:30.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.202+0000 7f70397a5640 1 -- 172.21.15.55:0/3906109619 learned_addr learned my addr 172.21.15.55:0/3906109619 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:30.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.204+0000 7f7038fa4640 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467829cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.204+0000 7f7038fa4640 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 0x56467824c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.205+0000 7f70397a5640 1 -- 172.21.15.55:0/3906109619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 msgr2=0x56467829cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:30.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.205+0000 7f70397a5640 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467829cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.205+0000 7f70397a5640 1 -- 172.21.15.55:0/3906109619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 msgr2=0x56467824c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:30.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.205+0000 7f70397a5640 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 0x56467824c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.205+0000 7f70397a5640 1 -- 172.21.15.55:0/3906109619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56467824cd00 con 0x56467821c4b0 2024-04-26T22:02:30.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.206+0000 7f70397a5640 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x56467829d450 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f703002fa30 tx=0x7f703002fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b835509087c8a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:30.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.207+0000 7f701cff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:02:30.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.207+0000 7f701cff9640 1 -- 172.21.15.55:0/3906109619 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030034030 con 0x56467821c4b0 2024-04-26T22:02:30.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.207+0000 7f701cff9640 1 -- 172.21.15.55:0/3906109619 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f703002f450 con 0x56467821c4b0 2024-04-26T22:02:30.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.210+0000 7f703e7ff740 1 -- 172.21.15.55:0/3906109619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7030006bf0 con 0x56467821c4b0 2024-04-26T22:02:30.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.219+0000 7f701cff9640 1 -- 172.21.15.55:0/3906109619 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7030020040 con 0x56467821c4b0 2024-04-26T22:02:30.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.220+0000 7f703e7ff740 10 client.16938 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:30.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.220+0000 7f703e7ff740 1 -- 172.21.15.55:0/3906109619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56467824e020 con 0x56467821c4b0 2024-04-26T22:02:30.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.232+0000 7f701cff9640 1 -- 172.21.15.55:0/3906109619 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7030006bf0 con 0x56467821c4b0 2024-04-26T22:02:30.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.234+0000 7f701cff9640 1 -- 172.21.15.55:0/3906109619 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f703001a420 con 0x56467821c4b0 2024-04-26T22:02:30.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.236+0000 7f703e7ff740 10 client.16938 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:30.239 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.236+0000 7f703e7ff740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:30.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.236+0000 7f703e7ff740 20 client.16938 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:30.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.236+0000 7f703e7ff740 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56467822c490 0x56467822e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:30.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.236+0000 7f703e7ff740 4 client.16938 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:02:30.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.236+0000 7f703e7ff740 1 -- 172.21.15.55:0/3906109619 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564678188580 con 0x56467822c490 2024-04-26T22:02:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.241+0000 7f7038fa4640 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56467822c490 0x56467822e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.262+0000 7f7038fa4640 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56467822c490 0x56467822e8b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b24a9c4db7aa89 server_cookie=1dea29ca7913145f in_seq=0 out_seq=0 2024-04-26T22:02:30.265 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.263+0000 7f701cff9640 10 client.16938 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:30.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.617+0000 7f701cff9640 1 -- 172.21.15.55:0/3906109619 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564678188580 con 0x56467822c490 2024-04-26T22:02:30.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.617+0000 7f701cff9640 10 client.16938 handle_command_reply: tid=0 2024-04-26T22:02:30.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.617+0000 7f703e7ff740 1 client.16938 shutdown 2024-04-26T22:02:30.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f703e7ff740 1 -- 172.21.15.55:0/3906109619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 msgr2=0x56467829d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f703e7ff740 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x56467829d450 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f703002fa30 tx=0x7f703002fe40 comp rx=0 tx=0).stop 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f703e7ff740 1 -- 172.21.15.55:0/3906109619 shutdown_connections 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f703e7ff740 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564678221620 0x56467824c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f703e7ff740 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56467822c490 0x56467822e8b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f703e7ff740 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56467821c4b0 0x56467829d450 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f703e7ff740 1 --2- 172.21.15.55:0/3906109619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56467821bac0 0x56467829cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f703e7ff740 1 -- 172.21.15.55:0/3906109619 >> 172.21.15.55:0/3906109619 conn(0x564678217130 msgr2=0x5646782284e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.618+0000 7f701cff9640 0 client.16938 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.619+0000 7f703e7ff740 1 -- 172.21.15.55:0/3906109619 shutdown_connections 2024-04-26T22:02:30.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.619+0000 7f703e7ff740 1 -- 172.21.15.55:0/3906109619 wait complete. 2024-04-26T22:02:30.623 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:02:30.623 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:02:30.623 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "42d776df-528f-4f1e-8db9-6b9c4e1d8e25", 2024-04-26T22:02:30.623 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:02:30.623 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:02:30.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.620+0000 7f703e7ff740 20 client.16938 trim_cache size 0 max 16384 2024-04-26T22:02:30.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.622+0000 7f703a7a7640 1 -- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70200a3d40 msgr2=0x7f70200a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.622+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70200a3d40 0x7f70200a6200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f702c0986c0 tx=0x7f702c0cfca0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.622+0000 7f703a7a7640 1 -- 172.21.15.55:0/2889241344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f703410b060 msgr2=0x7f70341a8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:30.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.622+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f703410b060 0x7f70341a8900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b990 tx=0x7f702c00be50 comp rx=0 tx=0).stop 2024-04-26T22:02:30.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.622+0000 7f703a7a7640 1 -- 172.21.15.55:0/2889241344 shutdown_connections 2024-04-26T22:02:30.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.622+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f70200a3d40 0x7f70200a6200 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.623+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f703410b060 0x7f70341a8900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.623+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f703410a590 0x7f70341a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.623+0000 7f703a7a7640 1 --2- 172.21.15.55:0/2889241344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7034109ba0 0x7f70341a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:30.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.623+0000 7f703a7a7640 1 -- 172.21.15.55:0/2889241344 >> 172.21.15.55:0/2889241344 conn(0x7f703406f780 msgr2=0x7f703410dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:30.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.627+0000 7f703a7a7640 1 -- 172.21.15.55:0/2889241344 shutdown_connections 2024-04-26T22:02:30.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:30.627+0000 7f703a7a7640 1 -- 172.21.15.55:0/2889241344 wait complete. 2024-04-26T22:02:30.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:02:30.682 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:02:30.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:30 smithi055 ceph-mon[29221]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 846 MiB/s wr, 1.11k op/s 2024-04-26T22:02:31.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.035+0000 7f026e751640 1 -- 172.21.15.55:0/3881610364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0268074810 msgr2=0x7f0268074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:31.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.036+0000 7f026e751640 1 --2- 172.21.15.55:0/3881610364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0268074810 0x7f0268074c30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f025c0097d0 tx=0x7f025c02b620 comp rx=0 tx=0).stop 2024-04-26T22:02:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.042+0000 7f026e751640 1 -- 172.21.15.55:0/3881610364 shutdown_connections 2024-04-26T22:02:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.042+0000 7f026e751640 1 --2- 172.21.15.55:0/3881610364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f026807a0a0 0x7f026807c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.042+0000 7f026e751640 1 --2- 172.21.15.55:0/3881610364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02680776f0 0x7f0268079b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.042+0000 7f026e751640 1 --2- 172.21.15.55:0/3881610364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0268074810 0x7f0268074c30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.042+0000 7f026e751640 1 -- 172.21.15.55:0/3881610364 >> 172.21.15.55:0/3881610364 conn(0x7f026806f7c0 msgr2=0x7f0268071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:31.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.045+0000 7f026e751640 1 -- 172.21.15.55:0/3881610364 shutdown_connections 2024-04-26T22:02:31.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.046+0000 7f026e751640 1 -- 172.21.15.55:0/3881610364 wait complete. 2024-04-26T22:02:31.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.047+0000 7f026e751640 1 Processor -- start 2024-04-26T22:02:31.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.047+0000 7f026e751640 1 -- start start 2024-04-26T22:02:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.048+0000 7f026e751640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0268074810 0x7f026812bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.048+0000 7f026e751640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02680776f0 0x7f026812c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.048+0000 7f026e751640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f026807a0a0 0x7f0268132b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.048+0000 7f026e751640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0268084160 con 0x7f0268074810 2024-04-26T22:02:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.048+0000 7f026e751640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0268083fe0 con 0x7f026807a0a0 2024-04-26T22:02:31.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.048+0000 7f026e751640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02680842e0 con 0x7f02680776f0 2024-04-26T22:02:31.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.049+0000 7f026ccee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f026807a0a0 0x7f0268132b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.049+0000 7f026ccee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f026807a0a0 0x7f0268132b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:57352/0 (socket says 172.21.15.55:57352) 2024-04-26T22:02:31.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.049+0000 7f026ccee640 1 -- 172.21.15.55:0/889362449 learned_addr learned my addr 172.21.15.55:0/889362449 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:31.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.053+0000 7f0267fff640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0268074810 0x7f026812bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.054+0000 7f026ccee640 1 -- 172.21.15.55:0/889362449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02680776f0 msgr2=0x7f026812c0a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:02:31.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.054+0000 7f026ccee640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02680776f0 0x7f026812c0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.054+0000 7f026ccee640 1 -- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0268074810 msgr2=0x7f026812bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:31.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.054+0000 7f026ccee640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0268074810 0x7f026812bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.054+0000 7f026ccee640 1 -- 172.21.15.55:0/889362449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0268133280 con 0x7f026807a0a0 2024-04-26T22:02:31.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.058+0000 7f026ccee640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f026807a0a0 0x7f0268132b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f026000cab0 tx=0x7f026000cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:31.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.074+0000 7f02657fa640 1 -- 172.21.15.55:0/889362449 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026000ae20 con 0x7f026807a0a0 2024-04-26T22:02:31.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.074+0000 7f026e751640 1 -- 172.21.15.55:0/889362449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0268133570 con 0x7f026807a0a0 2024-04-26T22:02:31.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.074+0000 7f026e751640 1 -- 172.21.15.55:0/889362449 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0268133ac0 con 0x7f026807a0a0 2024-04-26T22:02:31.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.074+0000 7f02657fa640 1 -- 172.21.15.55:0/889362449 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026000f900 con 0x7f026807a0a0 2024-04-26T22:02:31.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.074+0000 7f02657fa640 1 -- 172.21.15.55:0/889362449 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260018aa0 con 0x7f026807a0a0 2024-04-26T22:02:31.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:30 smithi165 ceph-mon[33296]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 846 MiB/s wr, 1.11k op/s 2024-04-26T22:02:31.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.083+0000 7f027146d740 1 Processor -- start 2024-04-26T22:02:31.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.085+0000 7f02657fa640 1 -- 172.21.15.55:0/889362449 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0260018c40 con 0x7f026807a0a0 2024-04-26T22:02:31.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.090+0000 7f02657fa640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f02500a3e40 0x7f02500a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.090+0000 7f02657fa640 1 -- 172.21.15.55:0/889362449 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f02600d0860 con 0x7f026807a0a0 2024-04-26T22:02:31.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.091+0000 7f027146d740 1 -- start start 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.091+0000 7f027146d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525967300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.091+0000 7f027146d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655259680b0 0x565525968530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.091+0000 7f027146d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565525968a70 0x56552596f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.091+0000 7f027146d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5655258cbd80 con 0x5655259680b0 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.091+0000 7f027146d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5655258858b0 con 0x565525966ee0 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.091+0000 7f027146d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5655258c7d40 con 0x565525968a70 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.092+0000 7f0267fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525967300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.092+0000 7f0267fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525967300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:57358/0 (socket says 172.21.15.55:57358) 2024-04-26T22:02:31.092 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.092+0000 7f0267fff640 1 -- 172.21.15.55:0/3906104050 learned_addr learned my addr 172.21.15.55:0/3906104050 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:31.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.092+0000 7f02677fe640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f02500a3e40 0x7f02500a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.092+0000 7f02677fe640 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655259680b0 0x565525968530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.093+0000 7f026ccee640 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565525968a70 0x56552596f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.093+0000 7f02677fe640 1 -- 172.21.15.55:0/3906104050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565525968a70 msgr2=0x56552596f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:31.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.094+0000 7f02677fe640 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565525968a70 0x56552596f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.094+0000 7f02677fe640 1 -- 172.21.15.55:0/3906104050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 msgr2=0x565525967300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:31.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.094+0000 7f02677fe640 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525967300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.094+0000 7f02677fe640 1 -- 172.21.15.55:0/3906104050 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565525856a40 con 0x5655259680b0 2024-04-26T22:02:31.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.094+0000 7f02677fe640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f02500a3e40 0x7f02500a6300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f02580144b0 tx=0x7f025800b740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:31.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.094+0000 7f02677fe640 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655259680b0 0x565525968530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0258014fd0 tx=0x7f0258017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364cfbaf970c4a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:31.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.094+0000 7f024e7fc640 1 -- 172.21.15.55:0/3906104050 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025800d050 con 0x5655259680b0 2024-04-26T22:02:31.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.095+0000 7f024e7fc640 1 -- 172.21.15.55:0/3906104050 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f025800e040 con 0x5655259680b0 2024-04-26T22:02:31.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.095+0000 7f024e7fc640 1 -- 172.21.15.55:0/3906104050 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02580206a0 con 0x5655259680b0 2024-04-26T22:02:31.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.098+0000 7f027146d740 1 -- 172.21.15.55:0/3906104050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655259680b0 msgr2=0x565525968530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:31.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.098+0000 7f027146d740 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655259680b0 0x565525968530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0258014fd0 tx=0x7f0258017000 comp rx=0 tx=0).stop 2024-04-26T22:02:31.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.102+0000 7f027146d740 1 -- 172.21.15.55:0/3906104050 shutdown_connections 2024-04-26T22:02:31.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.102+0000 7f027146d740 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x565525968a70 0x56552596f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.102+0000 7f027146d740 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5655259680b0 0x565525968530 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.102+0000 7f027146d740 1 --2- 172.21.15.55:0/3906104050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525967300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.102+0000 7f027146d740 1 -- 172.21.15.55:0/3906104050 >> 172.21.15.55:0/3906104050 conn(0x5655259623d0 msgr2=0x5655259627d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.111+0000 7f027146d740 1 -- 172.21.15.55:0/3906104050 shutdown_connections 2024-04-26T22:02:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.111+0000 7f027146d740 1 -- 172.21.15.55:0/3906104050 wait complete. 2024-04-26T22:02:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.111+0000 7f027146d740 1 -- start start 2024-04-26T22:02:31.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.112+0000 7f027146d740 1 Processor -- start 2024-04-26T22:02:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.113+0000 7f027146d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525971370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.113+0000 7f027146d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.113 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.113+0000 7f027146d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x5655259788f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.114+0000 7f027146d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56552596f9c0 con 0x565525968a70 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.114+0000 7f027146d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565525852d10 con 0x565525966ee0 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.114+0000 7f026ccee640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x5655259788f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.114+0000 7f026ccee640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x5655259788f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37016/0 (socket says 172.21.15.55:37016) 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.114+0000 7f026ccee640 1 -- 172.21.15.55:0/1918764502 learned_addr learned my addr 172.21.15.55:0/1918764502 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.114+0000 7f02677fe640 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.114+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56552596fb40 con 0x5655259680b0 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.114+0000 7f0267fff640 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525971370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.115+0000 7f027146d740 10 client.0 _ll_register_callbacks cb 0x56552587f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.115+0000 7f026ccee640 1 -- 172.21.15.55:0/1918764502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 msgr2=0x5655259718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.115+0000 7f026ccee640 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.115+0000 7f026ccee640 1 -- 172.21.15.55:0/1918764502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 msgr2=0x565525971370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.115+0000 7f026ccee640 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525971370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.116 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.115+0000 7f026ccee640 1 -- 172.21.15.55:0/1918764502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5655256a77d0 con 0x565525968a70 2024-04-26T22:02:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.116+0000 7f026ccee640 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x5655259788f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f026009c8d0 tx=0x7f026009c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511727b2ca036c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:31.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.116+0000 7f02477fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:02:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.117+0000 7f02477fe640 1 -- 172.21.15.55:0/1918764502 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600b9070 con 0x565525968a70 2024-04-26T22:02:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.117+0000 7f02477fe640 1 -- 172.21.15.55:0/1918764502 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f026000f650 con 0x565525968a70 2024-04-26T22:02:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.118+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f025800e040 con 0x565525968a70 2024-04-26T22:02:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.118+0000 7f02477fe640 1 -- 172.21.15.55:0/1918764502 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f02600a55b0 con 0x565525968a70 2024-04-26T22:02:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.118+0000 7f027146d740 10 client.16947 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.118+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5655255dd3f0 con 0x565525968a70 2024-04-26T22:02:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.118+0000 7f02477fe640 1 -- 172.21.15.55:0/1918764502 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f025800e040 con 0x565525968a70 2024-04-26T22:02:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.119+0000 7f02477fe640 1 -- 172.21.15.55:0/1918764502 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0260020030 con 0x565525968a70 2024-04-26T22:02:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.119+0000 7f027146d740 10 client.16947 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.119+0000 7f027146d740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.120+0000 7f027146d740 20 client.16947 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.120+0000 7f027146d740 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56552597d200 0x56552597f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.120+0000 7f027146d740 4 client.16947 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:02:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.120+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655258d9990 con 0x56552597d200 2024-04-26T22:02:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.120+0000 7f0267fff640 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56552597d200 0x56552597f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.123+0000 7f0267fff640 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56552597d200 0x56552597f6c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d09ef443a50ffce server_cookie=86495d945c1a1e11 in_seq=0 out_seq=0 2024-04-26T22:02:31.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.123+0000 7f02477fe640 10 client.16947 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.126+0000 7f02477fe640 1 -- 172.21.15.55:0/1918764502 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5655258d9990 con 0x56552597d200 2024-04-26T22:02:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.126+0000 7f02477fe640 10 client.16947 handle_command_reply: tid=0 2024-04-26T22:02:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.126+0000 7f027146d740 1 client.16947 shutdown 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.126+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 msgr2=0x5655259788f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.126+0000 7f027146d740 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x5655259788f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f026009c8d0 tx=0x7f026009c390 comp rx=0 tx=0).stop 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 shutdown_connections 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56552597d200 0x56552597f6c0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x5655259788f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 1 --2- 172.21.15.55:0/1918764502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525971370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 >> 172.21.15.55:0/1918764502 conn(0x5655259623d0 msgr2=0x565525961550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f02477fe640 0 client.16947 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 shutdown_connections 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 1 -- 172.21.15.55:0/1918764502 wait complete. 2024-04-26T22:02:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.127+0000 7f027146d740 20 client.16947 trim_cache size 0 max 16384 2024-04-26T22:02:31.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.161+0000 7f027146d740 1 Processor -- start 2024-04-26T22:02:31.161 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.161+0000 7f027146d740 1 -- start start 2024-04-26T22:02:31.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.162+0000 7f027146d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525965340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.162+0000 7f027146d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.162+0000 7f027146d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x565525965880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.167+0000 7f02677fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.167+0000 7f026ccee640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x565525965880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.167+0000 7f02677fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259788b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:34572/0 (socket says 172.21.15.55:34572) 2024-04-26T22:02:31.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.167+0000 7f0267fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525965340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.168 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.167+0000 7f0267fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525965340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:57366/0 (socket says 172.21.15.55:57366) 2024-04-26T22:02:31.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.170+0000 7f026ccee640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x565525965880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37020/0 (socket says 172.21.15.55:37020) 2024-04-26T22:02:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.182+0000 7f02677fe640 1 -- 172.21.15.55:0/464866963 learned_addr learned my addr 172.21.15.55:0/464866963 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:31.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.183+0000 7f027146d740 1 -- 172.21.15.55:0/464866963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0254010e10 con 0x565525968a70 2024-04-26T22:02:31.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.183+0000 7f027146d740 1 -- 172.21.15.55:0/464866963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f025400e6e0 con 0x565525966ee0 2024-04-26T22:02:31.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.183+0000 7f027146d740 1 -- 172.21.15.55:0/464866963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f025400c530 con 0x5655259680b0 2024-04-26T22:02:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.184+0000 7f026ccee640 1 -- 172.21.15.55:0/464866963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 msgr2=0x5655259788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.184+0000 7f026ccee640 1 --2- 172.21.15.55:0/464866963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259788b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.184+0000 7f026ccee640 1 -- 172.21.15.55:0/464866963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 msgr2=0x565525965340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:02:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.184+0000 7f026ccee640 1 --2- 172.21.15.55:0/464866963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525965340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.184+0000 7f026ccee640 1 -- 172.21.15.55:0/464866963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56552559d0f0 con 0x565525968a70 2024-04-26T22:02:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.185+0000 7f026ccee640 1 --2- 172.21.15.55:0/464866963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x565525965880 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f02600bd130 tx=0x7f026009c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de68fc4a038c4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:31.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.186+0000 7f024dffb640 1 -- 172.21.15.55:0/464866963 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600b9070 con 0x565525968a70 2024-04-26T22:02:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.187+0000 7f024dffb640 1 -- 172.21.15.55:0/464866963 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026009c940 con 0x565525968a70 2024-04-26T22:02:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.187+0000 7f024dffb640 1 -- 172.21.15.55:0/464866963 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026009b800 con 0x565525968a70 2024-04-26T22:02:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.188+0000 7f027146d740 1 -- 172.21.15.55:0/464866963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 msgr2=0x565525965880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.188+0000 7f027146d740 1 --2- 172.21.15.55:0/464866963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x565525965880 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f02600bd130 tx=0x7f026009c510 comp rx=0 tx=0).stop 2024-04-26T22:02:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.188+0000 7f027146d740 1 -- 172.21.15.55:0/464866963 shutdown_connections 2024-04-26T22:02:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.189+0000 7f027146d740 1 --2- 172.21.15.55:0/464866963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525968a70 0x565525965880 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.189+0000 7f027146d740 1 --2- 172.21.15.55:0/464866963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x5655259788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.189+0000 7f027146d740 1 --2- 172.21.15.55:0/464866963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525966ee0 0x565525965340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.189+0000 7f027146d740 1 -- 172.21.15.55:0/464866963 >> 172.21.15.55:0/464866963 conn(0x5655259623d0 msgr2=0x565525971370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:31.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.189+0000 7f027146d740 1 -- 172.21.15.55:0/464866963 shutdown_connections 2024-04-26T22:02:31.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.190+0000 7f027146d740 1 -- 172.21.15.55:0/464866963 wait complete. 2024-04-26T22:02:31.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.191+0000 7f027146d740 1 -- start start 2024-04-26T22:02:31.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.191+0000 7f027146d740 1 Processor -- start 2024-04-26T22:02:31.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.192+0000 7f027146d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525966ee0 0x565525975fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.192+0000 7f027146d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x565525976510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.192+0000 7f027146d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525968a70 0x565525977520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.192+0000 7f027146d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f025400cba0 con 0x565525966ee0 2024-04-26T22:02:31.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.192+0000 7f027146d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f025400e010 con 0x565525968a70 2024-04-26T22:02:31.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.192+0000 7f027146d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f025400d270 con 0x5655259680b0 2024-04-26T22:02:31.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.192+0000 7f027146d740 10 client.0 _ll_register_callbacks cb 0x7f026000a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:02:31.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.194+0000 7f0267fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525966ee0 0x565525975fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.194+0000 7f0267fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525966ee0 0x565525975fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37036/0 (socket says 172.21.15.55:37036) 2024-04-26T22:02:31.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.194+0000 7f0267fff640 1 -- 172.21.15.55:0/3864157097 learned_addr learned my addr 172.21.15.55:0/3864157097 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:02:31.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.195+0000 7f0267fff640 1 -- 172.21.15.55:0/3864157097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 msgr2=0x565525976510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:31.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.195+0000 7f0267fff640 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x565525976510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.195+0000 7f0267fff640 1 -- 172.21.15.55:0/3864157097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525968a70 msgr2=0x565525977520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:02:31.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.195+0000 7f0267fff640 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525968a70 0x565525977520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.195+0000 7f0267fff640 1 -- 172.21.15.55:0/3864157097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56552588c4b0 con 0x565525966ee0 2024-04-26T22:02:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.195+0000 7f0267fff640 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525966ee0 0x565525975fd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f025c009720 tx=0x7f025c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831589b9e9b60e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:02:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.195+0000 7f02477fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:02:31.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.197+0000 7f02477fe640 1 -- 172.21.15.55:0/3864157097 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c037040 con 0x565525966ee0 2024-04-26T22:02:31.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.198+0000 7f02477fe640 1 -- 172.21.15.55:0/3864157097 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f025c032bb0 con 0x565525966ee0 2024-04-26T22:02:31.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.198+0000 7f027146d740 1 -- 172.21.15.55:0/3864157097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026009c940 con 0x565525966ee0 2024-04-26T22:02:31.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.200+0000 7f02477fe640 1 -- 172.21.15.55:0/3864157097 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f025c032db0 con 0x565525966ee0 2024-04-26T22:02:31.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.202+0000 7f027146d740 10 client.16953 fetch_fsmap learned FSMap version 23 2024-04-26T22:02:31.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.202+0000 7f027146d740 1 -- 172.21.15.55:0/3864157097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5655259e3840 con 0x565525966ee0 2024-04-26T22:02:31.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.207+0000 7f02477fe640 1 -- 172.21.15.55:0/3864157097 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026009c940 con 0x565525966ee0 2024-04-26T22:02:31.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.207+0000 7f02477fe640 1 -- 172.21.15.55:0/3864157097 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f025c047560 con 0x565525966ee0 2024-04-26T22:02:31.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.207+0000 7f027146d740 10 client.16953 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:02:31.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.207+0000 7f027146d740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:02:31.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.207+0000 7f027146d740 20 client.16953 populate_metadata read hostname 'smithi055' 2024-04-26T22:02:31.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.207+0000 7f027146d740 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5655259e4180 0x5655259e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:02:31.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.207+0000 7f027146d740 4 client.16953 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:02:31.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.207+0000 7f027146d740 1 -- 172.21.15.55:0/3864157097 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5655258d3800 con 0x5655259e4180 2024-04-26T22:02:31.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.221+0000 7f02677fe640 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5655259e4180 0x5655259e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:02:31.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.239+0000 7f02677fe640 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5655259e4180 0x5655259e65a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c7a9f745b49129 server_cookie=7bd657d47ae8e832 in_seq=0 out_seq=0 2024-04-26T22:02:31.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.239+0000 7f02477fe640 10 client.16953 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:31.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.251+0000 7f02477fe640 1 -- 172.21.15.55:0/3864157097 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x5655258d3800 con 0x5655259e4180 2024-04-26T22:02:31.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.251+0000 7f02477fe640 10 client.16953 handle_command_reply: tid=0 2024-04-26T22:02:31.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:30 smithi064 ceph-mon[32765]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 846 MiB/s wr, 1.11k op/s 2024-04-26T22:02:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.258+0000 7f027146d740 1 client.16953 shutdown 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 -- 172.21.15.55:0/3864157097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525966ee0 msgr2=0x565525975fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525966ee0 0x565525975fd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f025c009720 tx=0x7f025c009fd0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 -- 172.21.15.55:0/3864157097 shutdown_connections 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565525968a70 0x565525977520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5655259680b0 0x565525976510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5655259e4180 0x5655259e65a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 --2- 172.21.15.55:0/3864157097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x565525966ee0 0x565525975fd0 secure :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f025c009720 tx=0x7f025c009fd0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 -- 172.21.15.55:0/3864157097 >> 172.21.15.55:0/3864157097 conn(0x5655259623d0 msgr2=0x565525965340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f02477fe640 0 client.16953 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 -- 172.21.15.55:0/3864157097 shutdown_connections 2024-04-26T22:02:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 1 -- 172.21.15.55:0/3864157097 wait complete. 2024-04-26T22:02:31.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.265+0000 7f027146d740 20 client.16953 trim_cache size 0 max 16384 2024-04-26T22:02:31.273 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (2 inodes in the stack)", 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout: "42d776df-528f-4f1e-8db9-6b9c4e1d8e25": { 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "42d776df-528f-4f1e-8db9-6b9c4e1d8e25", 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:02:31.274 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:02:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.273+0000 7f026e751640 1 -- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f02500a3e40 msgr2=0x7f02500a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.273+0000 7f026e751640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f02500a3e40 0x7f02500a6300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f02580144b0 tx=0x7f025800b740 comp rx=0 tx=0).stop 2024-04-26T22:02:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.273+0000 7f026e751640 1 -- 172.21.15.55:0/889362449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f026807a0a0 msgr2=0x7f0268132b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:02:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.274+0000 7f026e751640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f026807a0a0 0x7f0268132b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f026000cab0 tx=0x7f026000cf70 comp rx=0 tx=0).stop 2024-04-26T22:02:31.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.277+0000 7f026e751640 1 -- 172.21.15.55:0/889362449 shutdown_connections 2024-04-26T22:02:31.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.277+0000 7f026e751640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f02500a3e40 0x7f02500a6300 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.277+0000 7f026e751640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f026807a0a0 0x7f0268132b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.279 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.277+0000 7f026e751640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02680776f0 0x7f026812c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.277+0000 7f026e751640 1 --2- 172.21.15.55:0/889362449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0268074810 0x7f026812bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:02:31.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.277+0000 7f026e751640 1 -- 172.21.15.55:0/889362449 >> 172.21.15.55:0/889362449 conn(0x7f026806f7c0 msgr2=0x7f02680781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:02:31.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.280+0000 7f026e751640 1 -- 172.21.15.55:0/889362449 shutdown_connections 2024-04-26T22:02:31.282 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:02:31.280+0000 7f026e751640 1 -- 172.21.15.55:0/889362449 wait complete. 2024-04-26T22:02:31.324 INFO:tasks.cephfs.filesystem:scrub status for tag:42d776df-528f-4f1e-8db9-6b9c4e1d8e25 - {'path': '/', 'tag': '42d776df-528f-4f1e-8db9-6b9c4e1d8e25', 'options': 'recursive,force'} 2024-04-26T22:02:32.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:31 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:02:32.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:31 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:32.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:31 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:02:32.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:31 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:02:32.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:31 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:32.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:31 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:02:32.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:31 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:02:32.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:31 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:32.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:31 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:02:33.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:32 smithi165 ceph-mon[33296]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 861 MiB/s wr, 1.24k op/s 2024-04-26T22:02:33.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:32 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:02:33.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:32 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:33.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:32 smithi055 ceph-mon[29221]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 861 MiB/s wr, 1.24k op/s 2024-04-26T22:02:33.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:32 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:02:33.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:32 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:32 smithi064 ceph-mon[32765]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 861 MiB/s wr, 1.24k op/s 2024-04-26T22:02:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:32 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:02:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:32 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:35.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:34 smithi055 ceph-mon[29221]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 859 MiB/s wr, 1.45k op/s 2024-04-26T22:02:35.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:34 smithi064 ceph-mon[32765]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 859 MiB/s wr, 1.45k op/s 2024-04-26T22:02:35.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:34 smithi165 ceph-mon[33296]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 859 MiB/s wr, 1.45k op/s 2024-04-26T22:02:36.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:35 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:36.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:35 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:36.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:35 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:02:37.157 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:36 smithi064 ceph-mon[32765]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 814 MiB/s wr, 1.55k op/s 2024-04-26T22:02:37.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:36 smithi055 ceph-mon[29221]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 814 MiB/s wr, 1.55k op/s 2024-04-26T22:02:37.227 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:36 smithi165 ceph-mon[33296]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 814 MiB/s wr, 1.55k op/s 2024-04-26T22:02:39.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:38 smithi055 ceph-mon[29221]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-04-26T22:02:39.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:38 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:38 smithi064 ceph-mon[32765]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-04-26T22:02:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:38 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:39.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:38 smithi165 ceph-mon[33296]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-04-26T22:02:39.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:38 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:40 smithi064 ceph-mon[32765]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 851 MiB/s wr, 1.69k op/s 2024-04-26T22:02:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:41.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:40 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:02:41.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:40 smithi165 ceph-mon[33296]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 851 MiB/s wr, 1.69k op/s 2024-04-26T22:02:41.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:41.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:41.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:40 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:02:41.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:40 smithi055 ceph-mon[29221]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 851 MiB/s wr, 1.69k op/s 2024-04-26T22:02:41.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:41.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:41.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:40 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:02:42.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:41 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:02:42.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:41 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:02:42.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:41 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:02:42.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:41 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:02:42.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:41 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:02:42.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:41 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:02:43.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:42 smithi064 ceph-mon[32765]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.76k op/s 2024-04-26T22:02:43.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:42 smithi165 ceph-mon[33296]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.76k op/s 2024-04-26T22:02:43.339 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:42 smithi055 ceph-mon[29221]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.76k op/s 2024-04-26T22:02:45.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:44 smithi064 ceph-mon[32765]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 869 MiB/s wr, 1.75k op/s 2024-04-26T22:02:45.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:44 smithi165 ceph-mon[33296]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 869 MiB/s wr, 1.75k op/s 2024-04-26T22:02:45.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:44 smithi055 ceph-mon[29221]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 869 MiB/s wr, 1.75k op/s 2024-04-26T22:02:47.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:46 smithi064 ceph-mon[32765]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 823 MiB/s wr, 1.51k op/s 2024-04-26T22:02:47.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:46 smithi165 ceph-mon[33296]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 823 MiB/s wr, 1.51k op/s 2024-04-26T22:02:47.446 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:46 smithi055 ceph-mon[29221]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 823 MiB/s wr, 1.51k op/s 2024-04-26T22:02:49.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:48 smithi064 ceph-mon[32765]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 904 MiB/s wr, 1.48k op/s 2024-04-26T22:02:49.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:48 smithi165 ceph-mon[33296]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 904 MiB/s wr, 1.48k op/s 2024-04-26T22:02:49.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:48 smithi055 ceph-mon[29221]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 904 MiB/s wr, 1.48k op/s 2024-04-26T22:02:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:51 smithi064 ceph-mon[32765]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 850 MiB/s wr, 1.34k op/s 2024-04-26T22:02:51.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:51 smithi165 ceph-mon[33296]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 850 MiB/s wr, 1.34k op/s 2024-04-26T22:02:51.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:51 smithi055 ceph-mon[29221]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 850 MiB/s wr, 1.34k op/s 2024-04-26T22:02:53.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:53 smithi165 ceph-mon[33296]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.40k op/s 2024-04-26T22:02:53.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:53 smithi055 ceph-mon[29221]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.40k op/s 2024-04-26T22:02:53.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:53 smithi064 ceph-mon[32765]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.40k op/s 2024-04-26T22:02:55.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:55 smithi165 ceph-mon[33296]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.32k op/s 2024-04-26T22:02:55.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:55 smithi055 ceph-mon[29221]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.32k op/s 2024-04-26T22:02:56.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:55 smithi064 ceph-mon[32765]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 860 MiB/s wr, 1.32k op/s 2024-04-26T22:02:57.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:57 smithi165 ceph-mon[33296]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 1.18k op/s 2024-04-26T22:02:57.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:57 smithi055 ceph-mon[29221]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 1.18k op/s 2024-04-26T22:02:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:57 smithi064 ceph-mon[32765]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 1.18k op/s 2024-04-26T22:02:59.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:02:59 smithi165 ceph-mon[33296]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 1.28k op/s 2024-04-26T22:02:59.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:02:59 smithi055 ceph-mon[29221]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 1.28k op/s 2024-04-26T22:03:00.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:02:59 smithi064 ceph-mon[32765]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 1.28k op/s 2024-04-26T22:03:01.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:03:01.324 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:03:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.536+0000 7fe045462640 1 -- 172.21.15.55:0/1957830011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe04010f020 msgr2=0x7fe0401114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.536+0000 7fe045462640 1 --2- 172.21.15.55:0/1957830011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe04010f020 0x7fe0401114e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe0300074e0 tx=0x7fe03002b0a0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.540+0000 7fe045462640 1 -- 172.21.15.55:0/1957830011 shutdown_connections 2024-04-26T22:03:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.540+0000 7fe045462640 1 --2- 172.21.15.55:0/1957830011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe04010f020 0x7fe0401114e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.540+0000 7fe045462640 1 --2- 172.21.15.55:0/1957830011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe04010c560 0x7fe04010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.540+0000 7fe045462640 1 --2- 172.21.15.55:0/1957830011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe040109ba0 0x7fe04010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.540+0000 7fe045462640 1 -- 172.21.15.55:0/1957830011 >> 172.21.15.55:0/1957830011 conn(0x7fe04006f7c0 msgr2=0x7fe040071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.553+0000 7fe045462640 1 -- 172.21.15.55:0/1957830011 shutdown_connections 2024-04-26T22:03:01.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.553+0000 7fe045462640 1 -- 172.21.15.55:0/1957830011 wait complete. 2024-04-26T22:03:01.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.555+0000 7fe045462640 1 Processor -- start 2024-04-26T22:03:01.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.561+0000 7fe045462640 1 -- start start 2024-04-26T22:03:01.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.562+0000 7fe045462640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040109ba0 0x7fe04019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.562+0000 7fe045462640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe04010c560 0x7fe04019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.562+0000 7fe045462640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe04010f020 0x7fe0401b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.562+0000 7fe045462640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe040114b90 con 0x7fe04010c560 2024-04-26T22:03:01.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.562+0000 7fe045462640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe040114a10 con 0x7fe04010f020 2024-04-26T22:03:01.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.562+0000 7fe045462640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe040114d10 con 0x7fe040109ba0 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.563+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe04010f020 0x7fe0401b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.563+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe04010f020 0x7fe0401b7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:38394/0 (socket says 172.21.15.55:38394) 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.563+0000 7fe03f7fe640 1 -- 172.21.15.55:0/4194254413 learned_addr learned my addr 172.21.15.55:0/4194254413 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.564+0000 7fe03f7fe640 1 -- 172.21.15.55:0/4194254413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040109ba0 msgr2=0x7fe04019f0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.564+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040109ba0 0x7fe04019f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.564+0000 7fe03f7fe640 1 -- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe04010c560 msgr2=0x7fe04019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.564+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe04010c560 0x7fe04019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.564+0000 7fe03f7fe640 1 -- 172.21.15.55:0/4194254413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0401a2720 con 0x7fe04010f020 2024-04-26T22:03:01.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.565+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe04010f020 0x7fe0401b7a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe03000d090 tx=0x7fe030005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.586+0000 7fe027fff640 1 -- 172.21.15.55:0/4194254413 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03000a2e0 con 0x7fe04010f020 2024-04-26T22:03:01.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.586+0000 7fe027fff640 1 -- 172.21.15.55:0/4194254413 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe03000a930 con 0x7fe04010f020 2024-04-26T22:03:01.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.586+0000 7fe027fff640 1 -- 172.21.15.55:0/4194254413 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300051a0 con 0x7fe04010f020 2024-04-26T22:03:01.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.587+0000 7fe045462640 1 -- 172.21.15.55:0/4194254413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0401b7f90 con 0x7fe04010f020 2024-04-26T22:03:01.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.587+0000 7fe045462640 1 -- 172.21.15.55:0/4194254413 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0401b8420 con 0x7fe04010f020 2024-04-26T22:03:01.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.599+0000 7fe048179740 1 Processor -- start 2024-04-26T22:03:01.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.600+0000 7fe048179740 1 -- start start 2024-04-26T22:03:01.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.600+0000 7fe048179740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.600+0000 7fe048179740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.600+0000 7fe048179740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1b9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.600+0000 7fe048179740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ee7c115d80 con 0x55ee7c1b20b0 2024-04-26T22:03:01.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.600+0000 7fe048179740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee7c0cf8b0 con 0x55ee7c1b0ee0 2024-04-26T22:03:01.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.600+0000 7fe048179740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ee7c111d40 con 0x55ee7c1b2a70 2024-04-26T22:03:01.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.602+0000 7fe03effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.602+0000 7fe03effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33420/0 (socket says 172.21.15.55:33420) 2024-04-26T22:03:01.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.602+0000 7fe03effd640 1 -- 172.21.15.55:0/1345461707 learned_addr learned my addr 172.21.15.55:0/1345461707 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:01.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.602+0000 7fe03e7fc640 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.603+0000 7fe03e7fc640 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.603+0000 7fe03e7fc640 1 -- 172.21.15.55:0/1345461707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b2a70 msgr2=0x55ee7c1b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:01.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.603+0000 7fe03e7fc640 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1b9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.603+0000 7fe03e7fc640 1 -- 172.21.15.55:0/1345461707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b20b0 msgr2=0x55ee7c1b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:01.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.603+0000 7fe03e7fc640 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.603+0000 7fe03e7fc640 1 -- 172.21.15.55:0/1345461707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee7c0a0a40 con 0x55ee7c1b0ee0 2024-04-26T22:03:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.604+0000 7fe027fff640 1 -- 172.21.15.55:0/4194254413 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe030043020 con 0x7fe04010f020 2024-04-26T22:03:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.605+0000 7fe03e7fc640 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b1300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0401a0570 tx=0x7fe0380097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7e4068697442451 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.606+0000 7fe027fff640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0280a3e40 0x7fe0280a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.606+0000 7fe027fff640 1 -- 172.21.15.55:0/4194254413 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe0300efc10 con 0x7fe04010f020 2024-04-26T22:03:01.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.607+0000 7fe0257fa640 1 -- 172.21.15.55:0/1345461707 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0380140f0 con 0x55ee7c1b0ee0 2024-04-26T22:03:01.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.607+0000 7fe0257fa640 1 -- 172.21.15.55:0/1345461707 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe038014290 con 0x55ee7c1b0ee0 2024-04-26T22:03:01.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.607+0000 7fe0257fa640 1 -- 172.21.15.55:0/1345461707 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038005280 con 0x55ee7c1b0ee0 2024-04-26T22:03:01.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.609+0000 7fe048179740 1 -- 172.21.15.55:0/1345461707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 msgr2=0x55ee7c1b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.609+0000 7fe048179740 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b1300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0401a0570 tx=0x7fe0380097c0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.610+0000 7fe03effd640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0280a3e40 0x7fe0280a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.613+0000 7fe03effd640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0280a3e40 0x7fe0280a6300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe0340098b0 tx=0x7fe034008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.613+0000 7fe048179740 1 -- 172.21.15.55:0/1345461707 shutdown_connections 2024-04-26T22:03:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.613+0000 7fe048179740 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.613+0000 7fe048179740 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.613+0000 7fe048179740 1 --2- 172.21.15.55:0/1345461707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b1300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.613+0000 7fe048179740 1 -- 172.21.15.55:0/1345461707 >> 172.21.15.55:0/1345461707 conn(0x55ee7c1ac3d0 msgr2=0x55ee7c1ac7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.616+0000 7fe048179740 1 -- 172.21.15.55:0/1345461707 shutdown_connections 2024-04-26T22:03:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.617+0000 7fe048179740 1 -- 172.21.15.55:0/1345461707 wait complete. 2024-04-26T22:03:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.617+0000 7fe048179740 1 -- start start 2024-04-26T22:03:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.617+0000 7fe048179740 1 Processor -- start 2024-04-26T22:03:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.618+0000 7fe048179740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1bb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.618+0000 7fe048179740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1bb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.618+0000 7fe048179740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.618+0000 7fe048179740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ee7c1b9a80 con 0x55ee7c1b2a70 2024-04-26T22:03:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.618+0000 7fe048179740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee7c09cd10 con 0x55ee7c1b0ee0 2024-04-26T22:03:01.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.618+0000 7fe048179740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ee7c1b9c00 con 0x55ee7c1b20b0 2024-04-26T22:03:01.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.618+0000 7fe048179740 10 client.0 _ll_register_callbacks cb 0x55ee7c0c9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:01.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.619+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.619+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33436/0 (socket says 172.21.15.55:33436) 2024-04-26T22:03:01.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.619+0000 7fe03f7fe640 1 -- 172.21.15.55:0/231077300 learned_addr learned my addr 172.21.15.55:0/231077300 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:01.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.621+0000 7fe03e7fc640 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.621+0000 7fe03f7fe640 1 -- 172.21.15.55:0/231077300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 msgr2=0x55ee7c1bb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:01.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.621+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1bb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.621+0000 7fe03f7fe640 1 -- 172.21.15.55:0/231077300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 msgr2=0x55ee7c1bb450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:01.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.621+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1bb450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.621+0000 7fe03f7fe640 1 -- 172.21.15.55:0/231077300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee7bef17d0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.622+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c29d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe0300c4540 tx=0x7fe030045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdc4accaff54b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.629+0000 7fe022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:01.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.629+0000 7fe022ff5640 1 -- 172.21.15.55:0/231077300 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300bbe00 con 0x55ee7c1b2a70 2024-04-26T22:03:01.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.629+0000 7fe048179740 1 -- 172.21.15.55:0/231077300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe038014290 con 0x55ee7c1b2a70 2024-04-26T22:03:01.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.629+0000 7fe022ff5640 1 -- 172.21.15.55:0/231077300 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe030005340 con 0x55ee7c1b2a70 2024-04-26T22:03:01.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.631+0000 7fe022ff5640 1 -- 172.21.15.55:0/231077300 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe0300c01b0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.631+0000 7fe048179740 10 client.16962 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:01.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.631+0000 7fe048179740 1 -- 172.21.15.55:0/231077300 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ee7be273f0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.639+0000 7fe022ff5640 1 -- 172.21.15.55:0/231077300 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe038014290 con 0x55ee7c1b2a70 2024-04-26T22:03:01.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.639+0000 7fe022ff5640 1 -- 172.21.15.55:0/231077300 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe030004a50 con 0x55ee7c1b2a70 2024-04-26T22:03:01.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.641+0000 7fe048179740 10 client.16962 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:01.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.641+0000 7fe048179740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:01.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.641+0000 7fe048179740 20 client.16962 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:01.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.641+0000 7fe048179740 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee7c1c70f0 0x55ee7c1c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.641+0000 7fe048179740 4 client.16962 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:01.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.641+0000 7fe048179740 1 -- 172.21.15.55:0/231077300 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee7c123990 con 0x55ee7c1c70f0 2024-04-26T22:03:01.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.645+0000 7fe03effd640 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee7c1c70f0 0x55ee7c1c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.661+0000 7fe03effd640 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee7c1c70f0 0x55ee7c1c9590 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a0a3f35c5504a91 server_cookie=43f85eca3e4e10b5 in_seq=0 out_seq=0 2024-04-26T22:03:01.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.661+0000 7fe022ff5640 10 client.16962 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:01.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.666+0000 7fe022ff5640 1 -- 172.21.15.55:0/231077300 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee7c123990 con 0x55ee7c1c70f0 2024-04-26T22:03:01.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.666+0000 7fe022ff5640 10 client.16962 handle_command_reply: tid=0 2024-04-26T22:03:01.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.666+0000 7fe048179740 1 client.16962 shutdown 2024-04-26T22:03:01.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.666+0000 7fe048179740 1 -- 172.21.15.55:0/231077300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 msgr2=0x55ee7c1c29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.666+0000 7fe048179740 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c29d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe0300c4540 tx=0x7fe030045000 comp rx=0 tx=0).stop 2024-04-26T22:03:01.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe048179740 1 -- 172.21.15.55:0/231077300 shutdown_connections 2024-04-26T22:03:01.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe048179740 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee7c1c70f0 0x55ee7c1c9590 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe048179740 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c29d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe048179740 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1bb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe048179740 1 --2- 172.21.15.55:0/231077300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1bb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe048179740 1 -- 172.21.15.55:0/231077300 >> 172.21.15.55:0/231077300 conn(0x55ee7c1ac3d0 msgr2=0x55ee7c1ab550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:01.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe022ff5640 0 client.16962 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:01.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe048179740 1 -- 172.21.15.55:0/231077300 shutdown_connections 2024-04-26T22:03:01.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.667+0000 7fe048179740 1 -- 172.21.15.55:0/231077300 wait complete. 2024-04-26T22:03:01.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.668+0000 7fe048179740 20 client.16962 trim_cache size 0 max 16384 2024-04-26T22:03:01.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:01 smithi055 ceph-mon[29221]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 817 MiB/s wr, 1.18k op/s 2024-04-26T22:03:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.770+0000 7fe048179740 1 Processor -- start 2024-04-26T22:03:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.770+0000 7fe048179740 1 -- start start 2024-04-26T22:03:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.770+0000 7fe048179740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.770+0000 7fe048179740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.772 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.771+0000 7fe048179740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.771+0000 7fe048179740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401c3d20 con 0x55ee7c1b2a70 2024-04-26T22:03:01.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.771+0000 7fe048179740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401c15f0 con 0x55ee7c1b0ee0 2024-04-26T22:03:01.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.771+0000 7fe048179740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401bf440 con 0x55ee7c1b20b0 2024-04-26T22:03:01.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.774+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.774+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33440/0 (socket says 172.21.15.55:33440) 2024-04-26T22:03:01.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.774+0000 7fe03f7fe640 1 -- 172.21.15.55:0/2496246013 learned_addr learned my addr 172.21.15.55:0/2496246013 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:01.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.774+0000 7fe03effd640 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.776+0000 7fe03e7fc640 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.778+0000 7fe03f7fe640 1 -- 172.21.15.55:0/2496246013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 msgr2=0x55ee7c1b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.778+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.778+0000 7fe03f7fe640 1 -- 172.21.15.55:0/2496246013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 msgr2=0x55ee7c1b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.778+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.778+0000 7fe03f7fe640 1 -- 172.21.15.55:0/2496246013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee7bde70f0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.781+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c2990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe0300bbb00 tx=0x7fe0300ba310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101659e696774248 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.788+0000 7fe024ff9640 1 -- 172.21.15.55:0/2496246013 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300bbbd0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.789+0000 7fe024ff9640 1 -- 172.21.15.55:0/2496246013 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0300bbe00 con 0x55ee7c1b2a70 2024-04-26T22:03:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.790+0000 7fe048179740 1 -- 172.21.15.55:0/2496246013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 msgr2=0x55ee7c1c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.790+0000 7fe048179740 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c2990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe0300bbb00 tx=0x7fe0300ba310 comp rx=0 tx=0).stop 2024-04-26T22:03:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.792+0000 7fe048179740 1 -- 172.21.15.55:0/2496246013 shutdown_connections 2024-04-26T22:03:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.792+0000 7fe048179740 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c1c2990 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.792+0000 7fe048179740 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.792+0000 7fe048179740 1 --2- 172.21.15.55:0/2496246013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.792+0000 7fe048179740 1 -- 172.21.15.55:0/2496246013 >> 172.21.15.55:0/2496246013 conn(0x55ee7c1ac3d0 msgr2=0x55ee7c1bb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:01.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.804+0000 7fe048179740 1 -- 172.21.15.55:0/2496246013 shutdown_connections 2024-04-26T22:03:01.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.804+0000 7fe048179740 1 -- 172.21.15.55:0/2496246013 wait complete. 2024-04-26T22:03:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.805+0000 7fe048179740 1 -- start start 2024-04-26T22:03:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.806+0000 7fe048179740 1 Processor -- start 2024-04-26T22:03:01.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.807+0000 7fe048179740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.808+0000 7fe048179740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.808+0000 7fe048179740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c22b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.809+0000 7fe048179740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401bfab0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.809+0000 7fe048179740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401c0f20 con 0x55ee7c1b20b0 2024-04-26T22:03:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.809+0000 7fe048179740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0401c0180 con 0x55ee7c1b0ee0 2024-04-26T22:03:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.809+0000 7fe048179740 10 client.0 _ll_register_callbacks cb 0x7fe030004740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.810+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c22b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.810+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c22b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33450/0 (socket says 172.21.15.55:33450) 2024-04-26T22:03:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.810+0000 7fe03f7fe640 1 -- 172.21.15.55:0/853195622 learned_addr learned my addr 172.21.15.55:0/853195622 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:01.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.811+0000 7fe03f7fe640 1 -- 172.21.15.55:0/853195622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b0ee0 msgr2=0x55ee7c1bf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:01.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.811+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.811+0000 7fe03f7fe640 1 -- 172.21.15.55:0/853195622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b20b0 msgr2=0x55ee7c1bfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:01.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.811+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.811+0000 7fe03f7fe640 1 -- 172.21.15.55:0/853195622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee7c0d64b0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.812+0000 7fe03f7fe640 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c22b700 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe0300074b0 tx=0x7fe0300b8490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe51fd2d7868b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:01.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.812+0000 7fe022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:01.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.812+0000 7fe022ff5640 1 -- 172.21.15.55:0/853195622 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300bbbd0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.812+0000 7fe022ff5640 1 -- 172.21.15.55:0/853195622 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe0300edb30 con 0x55ee7c1b2a70 2024-04-26T22:03:01.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.813+0000 7fe048179740 1 -- 172.21.15.55:0/853195622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee7c11d800 con 0x55ee7c1b2a70 2024-04-26T22:03:01.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.817+0000 7fe022ff5640 1 -- 172.21.15.55:0/853195622 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe0300bbe00 con 0x55ee7c1b2a70 2024-04-26T22:03:01.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.824+0000 7fe048179740 10 client.16968 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.824+0000 7fe048179740 1 -- 172.21.15.55:0/853195622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ee7c0d64b0 con 0x55ee7c1b2a70 2024-04-26T22:03:01.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.824+0000 7fe022ff5640 1 -- 172.21.15.55:0/853195622 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ee7c11d800 con 0x55ee7c1b2a70 2024-04-26T22:03:01.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.826+0000 7fe022ff5640 1 -- 172.21.15.55:0/853195622 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe0300ed690 con 0x55ee7c1b2a70 2024-04-26T22:03:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.827+0000 7fe048179740 10 client.16968 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.827+0000 7fe048179740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.827+0000 7fe048179740 20 client.16968 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.827+0000 7fe048179740 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee7c22d1f0 0x55ee7c22f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.827+0000 7fe048179740 4 client.16968 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:03:01.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.827+0000 7fe048179740 1 -- 172.21.15.55:0/853195622 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee7c0984a0 con 0x55ee7c22d1f0 2024-04-26T22:03:01.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.833+0000 7fe03effd640 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee7c22d1f0 0x55ee7c22f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:01.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.845+0000 7fe03effd640 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee7c22d1f0 0x55ee7c22f5e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36158d06a96613cd server_cookie=34693ae47f8eac9c in_seq=0 out_seq=0 2024-04-26T22:03:01.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.846+0000 7fe022ff5640 10 client.16968 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:01.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.850+0000 7fe022ff5640 1 -- 172.21.15.55:0/853195622 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ee7c0984a0 con 0x55ee7c22d1f0 2024-04-26T22:03:01.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.850+0000 7fe022ff5640 10 client.16968 handle_command_reply: tid=0 2024-04-26T22:03:01.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.850+0000 7fe048179740 1 client.16968 shutdown 2024-04-26T22:03:01.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 -- 172.21.15.55:0/853195622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 msgr2=0x55ee7c22b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c22b700 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe0300074b0 tx=0x7fe0300b8490 comp rx=0 tx=0).stop 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 -- 172.21.15.55:0/853195622 shutdown_connections 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ee7c22d1f0 0x55ee7c22f5e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ee7c1b2a70 0x55ee7c22b700 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee7c1b20b0 0x55ee7c1bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 --2- 172.21.15.55:0/853195622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee7c1b0ee0 0x55ee7c1bf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe022ff5640 0 client.16968 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 -- 172.21.15.55:0/853195622 >> 172.21.15.55:0/853195622 conn(0x55ee7c1ac3d0 msgr2=0x55ee7c1bb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.853+0000 7fe048179740 1 -- 172.21.15.55:0/853195622 shutdown_connections 2024-04-26T22:03:01.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.854+0000 7fe048179740 1 -- 172.21.15.55:0/853195622 wait complete. 2024-04-26T22:03:01.855 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:03:01.855 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:03:01.856 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:03:01.856 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:03:01.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.854+0000 7fe048179740 20 client.16968 trim_cache size 0 max 16384 2024-04-26T22:03:01.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 -- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0280a3e40 msgr2=0x7fe0280a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0280a3e40 0x7fe0280a6300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe0340098b0 tx=0x7fe034008040 comp rx=0 tx=0).stop 2024-04-26T22:03:01.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 -- 172.21.15.55:0/4194254413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe04010f020 msgr2=0x7fe0401b7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:01.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe04010f020 0x7fe0401b7a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe03000d090 tx=0x7fe030005760 comp rx=0 tx=0).stop 2024-04-26T22:03:01.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 -- 172.21.15.55:0/4194254413 shutdown_connections 2024-04-26T22:03:01.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0280a3e40 0x7fe0280a6300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe04010f020 0x7fe0401b7a50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe04010c560 0x7fe04019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 --2- 172.21.15.55:0/4194254413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe040109ba0 0x7fe04019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:01.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.856+0000 7fe045462640 1 -- 172.21.15.55:0/4194254413 >> 172.21.15.55:0/4194254413 conn(0x7fe04006f7c0 msgr2=0x7fe04010e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:01.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.857+0000 7fe045462640 1 -- 172.21.15.55:0/4194254413 shutdown_connections 2024-04-26T22:03:01.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:01.857+0000 7fe045462640 1 -- 172.21.15.55:0/4194254413 wait complete. 2024-04-26T22:03:01.893 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:03:01.894 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:02.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:01 smithi064 ceph-mon[32765]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 817 MiB/s wr, 1.18k op/s 2024-04-26T22:03:02.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:01 smithi165 ceph-mon[33296]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 817 MiB/s wr, 1.18k op/s 2024-04-26T22:03:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.168+0000 7ff64359e640 1 -- 172.21.15.55:0/4174750875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64410bf10 msgr2=0x7ff64410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.168+0000 7ff64359e640 1 --2- 172.21.15.55:0/4174750875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64410bf10 0x7ff64410e3a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff638009900 tx=0x7ff63802a440 comp rx=0 tx=0).stop 2024-04-26T22:03:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.169+0000 7ff64359e640 1 -- 172.21.15.55:0/4174750875 shutdown_connections 2024-04-26T22:03:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.169+0000 7ff64359e640 1 --2- 172.21.15.55:0/4174750875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff64410ea10 0x7ff644119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.169+0000 7ff64359e640 1 --2- 172.21.15.55:0/4174750875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff64410bf10 0x7ff64410e3a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.169+0000 7ff64359e640 1 --2- 172.21.15.55:0/4174750875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff644074760 0x7ff64410b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.169+0000 7ff64359e640 1 -- 172.21.15.55:0/4174750875 >> 172.21.15.55:0/4174750875 conn(0x7ff64406f690 msgr2=0x7ff644071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:02.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.177+0000 7ff64359e640 1 -- 172.21.15.55:0/4174750875 shutdown_connections 2024-04-26T22:03:02.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.182+0000 7ff64359e640 1 -- 172.21.15.55:0/4174750875 wait complete. 2024-04-26T22:03:02.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.183+0000 7ff64359e640 1 Processor -- start 2024-04-26T22:03:02.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.186+0000 7ff64359e640 1 -- start start 2024-04-26T22:03:02.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.187+0000 7ff64359e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff644074760 0x7ff6441a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:02.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.187+0000 7ff64359e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff64410bf10 0x7ff6441a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:02.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.187+0000 7ff64359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64410ea10 0x7ff6441aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:02.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.187+0000 7ff64359e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff64411d0e0 con 0x7ff64410bf10 2024-04-26T22:03:02.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.187+0000 7ff64359e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff64411cf60 con 0x7ff64410ea10 2024-04-26T22:03:02.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.187+0000 7ff64359e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff64411d260 con 0x7ff644074760 2024-04-26T22:03:02.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.189+0000 7ff642d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64410ea10 0x7ff6441aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:02.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.189+0000 7ff642d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64410ea10 0x7ff6441aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:38446/0 (socket says 172.21.15.55:38446) 2024-04-26T22:03:02.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.189+0000 7ff642d9d640 1 -- 172.21.15.55:0/2074363122 learned_addr learned my addr 172.21.15.55:0/2074363122 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:02.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.191+0000 7ff641d9b640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff64410bf10 0x7ff6441a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:02.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.193+0000 7ff641d9b640 1 -- 172.21.15.55:0/2074363122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff644074760 msgr2=0x7ff6441a7850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:02.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.193+0000 7ff641d9b640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff644074760 0x7ff6441a7850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.193+0000 7ff641d9b640 1 -- 172.21.15.55:0/2074363122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64410ea10 msgr2=0x7ff6441aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:02.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.193+0000 7ff641d9b640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64410ea10 0x7ff6441aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.193+0000 7ff641d9b640 1 -- 172.21.15.55:0/2074363122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6441af420 con 0x7ff64410bf10 2024-04-26T22:03:02.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.193+0000 7ff641d9b640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff64410bf10 0x7ff6441a7d90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff638009850 tx=0x7ff638004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:02.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.201+0000 7ff6337fe640 1 -- 172.21.15.55:0/2074363122 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638040030 con 0x7ff64410bf10 2024-04-26T22:03:02.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.201+0000 7ff6337fe640 1 -- 172.21.15.55:0/2074363122 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6380050f0 con 0x7ff64410bf10 2024-04-26T22:03:02.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.201+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6441af6b0 con 0x7ff64410bf10 2024-04-26T22:03:02.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.201+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6441afb70 con 0x7ff64410bf10 2024-04-26T22:03:02.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.201+0000 7ff6337fe640 1 -- 172.21.15.55:0/2074363122 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63803c910 con 0x7ff64410bf10 2024-04-26T22:03:02.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.207+0000 7ff6337fe640 1 -- 172.21.15.55:0/2074363122 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff638043030 con 0x7ff64410bf10 2024-04-26T22:03:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.208+0000 7ff6337fe640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff62c0a3e40 0x7ff62c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:02.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.209+0000 7ff6337fe640 1 -- 172.21.15.55:0/2074363122 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff6380f0c80 con 0x7ff64410bf10 2024-04-26T22:03:02.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.210+0000 7ff64259c640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff62c0a3e40 0x7ff62c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:02.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.210+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6240052c0 con 0x7ff64410bf10 2024-04-26T22:03:02.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.212+0000 7ff64259c640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff62c0a3e40 0x7ff62c0a6300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff63400a890 tx=0x7ff634005b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:02.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.231+0000 7ff6337fe640 1 -- 172.21.15.55:0/2074363122 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff6380bd460 con 0x7ff64410bf10 2024-04-26T22:03:02.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.914+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff624005880 con 0x7ff64410bf10 2024-04-26T22:03:02.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.919+0000 7ff6337fe640 1 -- 172.21.15.55:0/2074363122 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7ff6380bcd60 con 0x7ff64410bf10 2024-04-26T22:03:02.921 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:03:02.921 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:03:02.921 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:03:02.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.928+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff62c0a3e40 msgr2=0x7ff62c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:02.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.928+0000 7ff64359e640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff62c0a3e40 0x7ff62c0a6300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff63400a890 tx=0x7ff634005b80 comp rx=0 tx=0).stop 2024-04-26T22:03:02.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.928+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff64410bf10 msgr2=0x7ff6441a7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:02.928 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.928+0000 7ff64359e640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff64410bf10 0x7ff6441a7d90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff638009850 tx=0x7ff638004300 comp rx=0 tx=0).stop 2024-04-26T22:03:02.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.931+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 shutdown_connections 2024-04-26T22:03:02.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.931+0000 7ff64359e640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff62c0a3e40 0x7ff62c0a6300 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.931 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.931+0000 7ff64359e640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff64410ea10 0x7ff6441aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.931+0000 7ff64359e640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff64410bf10 0x7ff6441a7d90 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.931+0000 7ff64359e640 1 --2- 172.21.15.55:0/2074363122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff644074760 0x7ff6441a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:02.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.931+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 >> 172.21.15.55:0/2074363122 conn(0x7ff64406f690 msgr2=0x7ff644109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:02.935 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.935+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 shutdown_connections 2024-04-26T22:03:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:02.938+0000 7ff64359e640 1 -- 172.21.15.55:0/2074363122 wait complete. 2024-04-26T22:03:02.979 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:03:03.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.405+0000 7f737dabb640 1 -- 172.21.15.55:0/2210002529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7378109930 msgr2=0x7f737810bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.405+0000 7f737dabb640 1 --2- 172.21.15.55:0/2210002529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7378109930 0x7f737810bd60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f737001acb0 tx=0x7f737003b920 comp rx=0 tx=0).stop 2024-04-26T22:03:03.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.406+0000 7f737dabb640 1 -- 172.21.15.55:0/2210002529 shutdown_connections 2024-04-26T22:03:03.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.406+0000 7f737dabb640 1 --2- 172.21.15.55:0/2210002529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f737810ec70 0x7f737810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.406+0000 7f737dabb640 1 --2- 172.21.15.55:0/2210002529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737810c2a0 0x7f737810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.406+0000 7f737dabb640 1 --2- 172.21.15.55:0/2210002529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7378109930 0x7f737810bd60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.406+0000 7f737dabb640 1 -- 172.21.15.55:0/2210002529 >> 172.21.15.55:0/2210002529 conn(0x7f737806f7c0 msgr2=0x7f7378071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:03.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.410+0000 7f737dabb640 1 -- 172.21.15.55:0/2210002529 shutdown_connections 2024-04-26T22:03:03.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.413+0000 7f737dabb640 1 -- 172.21.15.55:0/2210002529 wait complete. 2024-04-26T22:03:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.415+0000 7f737dabb640 1 Processor -- start 2024-04-26T22:03:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.415+0000 7f737dabb640 1 -- start start 2024-04-26T22:03:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.415+0000 7f737dabb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f737810c2a0 0x7f737819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.415+0000 7f737dabb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737810ec70 0x7f737819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.415+0000 7f737dabb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73781a4ab0 0x7f73781a6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.415+0000 7f737dabb640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7378112af0 con 0x7f737810c2a0 2024-04-26T22:03:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.415+0000 7f737dabb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7378112970 con 0x7f737810ec70 2024-04-26T22:03:03.420 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.415+0000 7f737dabb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7378112c70 con 0x7f73781a4ab0 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.419+0000 7f7377fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73781a4ab0 0x7f73781a6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.419+0000 7f7377fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73781a4ab0 0x7f73781a6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36316/0 (socket says 172.21.15.55:36316) 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.419+0000 7f7377fff640 1 -- 172.21.15.55:0/855576284 learned_addr learned my addr 172.21.15.55:0/855576284 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.421+0000 7f7376ffd640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737810ec70 0x7f737819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.421+0000 7f73777fe640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f737810c2a0 0x7f737819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.424+0000 7f7377fff640 1 -- 172.21.15.55:0/855576284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737810ec70 msgr2=0x7f737819faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.424+0000 7f7377fff640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737810ec70 0x7f737819faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.424+0000 7f7377fff640 1 -- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f737810c2a0 msgr2=0x7f737819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.424+0000 7f7377fff640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f737810c2a0 0x7f737819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.424+0000 7f7377fff640 1 -- 172.21.15.55:0/855576284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73781a75a0 con 0x7f73781a4ab0 2024-04-26T22:03:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.428+0000 7f7377fff640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73781a4ab0 0x7f73781a6ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f736800e970 tx=0x7f736800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.430+0000 7f7374ff9640 1 -- 172.21.15.55:0/855576284 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736800ccc0 con 0x7f73781a4ab0 2024-04-26T22:03:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.430+0000 7f7374ff9640 1 -- 172.21.15.55:0/855576284 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73680054d0 con 0x7f73781a4ab0 2024-04-26T22:03:03.436 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.430+0000 7f7374ff9640 1 -- 172.21.15.55:0/855576284 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7368010430 con 0x7f73781a4ab0 2024-04-26T22:03:03.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.431+0000 7f737dabb640 1 -- 172.21.15.55:0/855576284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73781a7890 con 0x7f73781a4ab0 2024-04-26T22:03:03.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.431+0000 7f737dabb640 1 -- 172.21.15.55:0/855576284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73781a7d90 con 0x7f73781a4ab0 2024-04-26T22:03:03.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.440+0000 7f73807cd740 1 Processor -- start 2024-04-26T22:03:03.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.440+0000 7f7374ff9640 1 -- 172.21.15.55:0/855576284 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f73680105d0 con 0x7f73781a4ab0 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.444+0000 7f73807cd740 1 -- start start 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.444+0000 7f73807cd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a52c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.444+0000 7f73807cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 0x55f216a53e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.444+0000 7f73807cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a5ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.444+0000 7f73807cd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f2169b6d80 con 0x55f216a52820 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.444+0000 7f73807cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f216970a20 con 0x55f216a539f0 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.444+0000 7f73807cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f2169b2d40 con 0x55f216a543b0 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.443+0000 7f7374ff9640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73600a3dc0 0x7f73600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.445+0000 7f7374ff9640 1 -- 172.21.15.55:0/855576284 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f73680d0860 con 0x7f73781a4ab0 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.447+0000 7f7376ffd640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73600a3dc0 0x7f73600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.447+0000 7f7376ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 0x55f216a53e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.451+0000 7f7377fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a5ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.452+0000 7f73777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a52c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.452+0000 7f73777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a52c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33492/0 (socket says 172.21.15.55:33492) 2024-04-26T22:03:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.452+0000 7f73777fe640 1 -- 172.21.15.55:0/2382706110 learned_addr learned my addr 172.21.15.55:0/2382706110 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.453+0000 7f73777fe640 1 -- 172.21.15.55:0/2382706110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 msgr2=0x55f216a5ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.453+0000 7f73777fe640 1 --2- 172.21.15.55:0/2382706110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a5ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.453+0000 7f73777fe640 1 -- 172.21.15.55:0/2382706110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 msgr2=0x55f216a53e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.453+0000 7f73777fe640 1 --2- 172.21.15.55:0/2382706110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 0x55f216a53e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.453+0000 7f73777fe640 1 -- 172.21.15.55:0/2382706110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21697bc30 con 0x55f216a52820 2024-04-26T22:03:03.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.458+0000 7f73777fe640 1 --2- 172.21.15.55:0/2382706110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a52c40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f737001acb0 tx=0x7f7370005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96116a314a626c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.459+0000 7f7376ffd640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73600a3dc0 0x7f73600a6280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f736c005d00 tx=0x7f736c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.461+0000 7f735dffb640 1 -- 172.21.15.55:0/2382706110 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370045040 con 0x55f216a52820 2024-04-26T22:03:03.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.461+0000 7f735dffb640 1 -- 172.21.15.55:0/2382706110 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f737001a7f0 con 0x55f216a52820 2024-04-26T22:03:03.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.462+0000 7f73807cd740 1 -- 172.21.15.55:0/2382706110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 msgr2=0x55f216a52c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.462+0000 7f73807cd740 1 --2- 172.21.15.55:0/2382706110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a52c40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f737001acb0 tx=0x7f7370005f60 comp rx=0 tx=0).stop 2024-04-26T22:03:03.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.465+0000 7f73807cd740 1 -- 172.21.15.55:0/2382706110 shutdown_connections 2024-04-26T22:03:03.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.465+0000 7f73807cd740 1 --2- 172.21.15.55:0/2382706110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a5ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.465+0000 7f73807cd740 1 --2- 172.21.15.55:0/2382706110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 0x55f216a53e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.465+0000 7f73807cd740 1 --2- 172.21.15.55:0/2382706110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a52c40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.465+0000 7f73807cd740 1 -- 172.21.15.55:0/2382706110 >> 172.21.15.55:0/2382706110 conn(0x55f216993910 msgr2=0x55f216a4e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:03.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.478+0000 7f73807cd740 1 -- 172.21.15.55:0/2382706110 shutdown_connections 2024-04-26T22:03:03.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.481+0000 7f73807cd740 1 -- 172.21.15.55:0/2382706110 wait complete. 2024-04-26T22:03:03.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.481+0000 7f73807cd740 1 -- start start 2024-04-26T22:03:03.486 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.482+0000 7f73807cd740 1 Processor -- start 2024-04-26T22:03:03.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.487+0000 7f73807cd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a5ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.487+0000 7f73807cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 0x55f216a5cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.487+0000 7f73807cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.487+0000 7f73807cd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f216a5b3b0 con 0x55f216a52820 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.487+0000 7f73807cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f21693ddc0 con 0x55f216a539f0 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.487+0000 7f73807cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f216a5b530 con 0x55f216a543b0 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.487+0000 7f73807cd740 10 client.0 _ll_register_callbacks cb 0x55f216959600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.488+0000 7f73777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a5ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.488+0000 7f73777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a5ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33506/0 (socket says 172.21.15.55:33506) 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.488+0000 7f73777fe640 1 -- 172.21.15.55:0/3025788039 learned_addr learned my addr 172.21.15.55:0/3025788039 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.489+0000 7f73777fe640 1 -- 172.21.15.55:0/3025788039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 msgr2=0x55f216a63fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.489+0000 7f73777fe640 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.489+0000 7f73777fe640 1 -- 172.21.15.55:0/3025788039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 msgr2=0x55f216a5cf80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.489+0000 7f73777fe640 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 0x55f216a5cf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.489+0000 7f73777fe640 1 -- 172.21.15.55:0/3025788039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2166c83f0 con 0x55f216a52820 2024-04-26T22:03:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.491+0000 7f73777fe640 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a5ca40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f737003be90 tx=0x7f7370040a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc25f3ef4ecedfea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.491+0000 7f7356ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:03.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.496+0000 7f7356ffd640 1 -- 172.21.15.55:0/3025788039 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370045040 con 0x55f216a52820 2024-04-26T22:03:03.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.496+0000 7f7356ffd640 1 -- 172.21.15.55:0/3025788039 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f737005e450 con 0x55f216a52820 2024-04-26T22:03:03.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.499+0000 7f73807cd740 1 -- 172.21.15.55:0/3025788039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f737001a7f0 con 0x55f216a52820 2024-04-26T22:03:03.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.504+0000 7f7356ffd640 1 -- 172.21.15.55:0/3025788039 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f737005e650 con 0x55f216a52820 2024-04-26T22:03:03.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.508+0000 7f73807cd740 10 client.16986 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:03.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.508+0000 7f73807cd740 1 -- 172.21.15.55:0/3025788039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f2167927d0 con 0x55f216a52820 2024-04-26T22:03:03.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.515+0000 7f7356ffd640 1 -- 172.21.15.55:0/3025788039 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f737001a7f0 con 0x55f216a52820 2024-04-26T22:03:03.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.516+0000 7f7356ffd640 1 -- 172.21.15.55:0/3025788039 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7370053070 con 0x55f216a52820 2024-04-26T22:03:03.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.516+0000 7f73807cd740 10 client.16986 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:03.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.516+0000 7f73807cd740 10 client.16986 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:03:03.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.516+0000 7f73807cd740 20 client.16986 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:03.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.516+0000 7f73807cd740 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55f216a68880 0x55f216a6acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.516+0000 7f73807cd740 4 client.16986 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:03.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.516+0000 7f73807cd740 1 -- 172.21.15.55:0/3025788039 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2169c4990 con 0x55f216a68880 2024-04-26T22:03:03.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.519+0000 7f7376ffd640 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55f216a68880 0x55f216a6acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.522+0000 7f7376ffd640 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55f216a68880 0x55f216a6acc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32cd7f375780e3f4 server_cookie=3d1901c7f5249dc8 in_seq=0 out_seq=0 2024-04-26T22:03:03.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.523+0000 7f7356ffd640 10 client.16986 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:03:03.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.523+0000 7f7356ffd640 1 -- 172.21.15.55:0/3025788039 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2169c4990 con 0x55f216a68880 2024-04-26T22:03:03.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.524+0000 7f7356ffd640 10 client.16986 handle_command_reply: tid=0 2024-04-26T22:03:03.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.524+0000 7f73807cd740 1 client.16986 shutdown 2024-04-26T22:03:03.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.526+0000 7f73807cd740 1 -- 172.21.15.55:0/3025788039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 msgr2=0x55f216a5ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.526+0000 7f73807cd740 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a5ca40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f737003be90 tx=0x7f7370040a60 comp rx=0 tx=0).stop 2024-04-26T22:03:03.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.527+0000 7f73807cd740 1 -- 172.21.15.55:0/3025788039 shutdown_connections 2024-04-26T22:03:03.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.527+0000 7f73807cd740 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55f216a68880 0x55f216a6acc0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.527+0000 7f73807cd740 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.527+0000 7f73807cd740 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a539f0 0x55f216a5cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.527+0000 7f73807cd740 1 --2- 172.21.15.55:0/3025788039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a52820 0x55f216a5ca40 secure :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f737003be90 tx=0x7f7370040a60 comp rx=0 tx=0).stop 2024-04-26T22:03:03.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.527+0000 7f73807cd740 1 -- 172.21.15.55:0/3025788039 >> 172.21.15.55:0/3025788039 conn(0x55f216993910 msgr2=0x55f216a4d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:03.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.527+0000 7f7356ffd640 0 client.16986 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:03:03.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.530+0000 7f73807cd740 1 -- 172.21.15.55:0/3025788039 shutdown_connections 2024-04-26T22:03:03.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.530+0000 7f73807cd740 1 -- 172.21.15.55:0/3025788039 wait complete. 2024-04-26T22:03:03.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.533+0000 7f73807cd740 20 client.16986 trim_cache size 0 max 16384 2024-04-26T22:03:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.598+0000 7f73807cd740 1 Processor -- start 2024-04-26T22:03:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.598+0000 7f73807cd740 1 -- start start 2024-04-26T22:03:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.598+0000 7f73807cd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a539f0 0x55f216a5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.599+0000 7f73807cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.599+0000 7f73807cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a577e0 0x55f216a5c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.599+0000 7f73807cd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f7364010fc0 con 0x55f216a539f0 2024-04-26T22:03:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.600+0000 7f73777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a539f0 0x55f216a5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.600+0000 7f73777fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a539f0 0x55f216a5c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33518/0 (socket says 172.21.15.55:33518) 2024-04-26T22:03:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.600+0000 7f7376ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.600+0000 7f7376ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36354/0 (socket says 172.21.15.55:36354) 2024-04-26T22:03:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.600+0000 7f7376ffd640 1 -- 172.21.15.55:0/3845744266 learned_addr learned my addr 172.21.15.55:0/3845744266 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.600+0000 7f7377fff640 1 --2- 172.21.15.55:0/3845744266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a577e0 0x55f216a5c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.600+0000 7f73807cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f736400e890 con 0x55f216a577e0 2024-04-26T22:03:03.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.601+0000 7f73807cd740 1 -- 172.21.15.55:0/3845744266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f736400c6e0 con 0x55f216a543b0 2024-04-26T22:03:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.603+0000 7f7376ffd640 1 -- 172.21.15.55:0/3845744266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a577e0 msgr2=0x55f216a5c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.603+0000 7f7376ffd640 1 --2- 172.21.15.55:0/3845744266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a577e0 0x55f216a5c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.603+0000 7f7376ffd640 1 -- 172.21.15.55:0/3845744266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a539f0 msgr2=0x55f216a5c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.603+0000 7f7376ffd640 1 --2- 172.21.15.55:0/3845744266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a539f0 0x55f216a5c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.603+0000 7f7376ffd640 1 -- 172.21.15.55:0/3845744266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f216a498b0 con 0x55f216a543b0 2024-04-26T22:03:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.604+0000 7f7376ffd640 1 --2- 172.21.15.55:0/3845744266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f736c00b620 tx=0x7f736c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3542d68c9955506 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.606+0000 7f735d7fa640 1 -- 172.21.15.55:0/3845744266 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c003a40 con 0x55f216a543b0 2024-04-26T22:03:03.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.606+0000 7f735d7fa640 1 -- 172.21.15.55:0/3845744266 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736c02d070 con 0x55f216a543b0 2024-04-26T22:03:03.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.610+0000 7f735d7fa640 1 -- 172.21.15.55:0/3845744266 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c01f3f0 con 0x55f216a543b0 2024-04-26T22:03:03.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.611+0000 7f73807cd740 1 -- 172.21.15.55:0/3845744266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 msgr2=0x55f216a63f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.611+0000 7f73807cd740 1 --2- 172.21.15.55:0/3845744266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f736c00b620 tx=0x7f736c004730 comp rx=0 tx=0).stop 2024-04-26T22:03:03.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.612+0000 7f73807cd740 1 -- 172.21.15.55:0/3845744266 shutdown_connections 2024-04-26T22:03:03.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.612+0000 7f73807cd740 1 --2- 172.21.15.55:0/3845744266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a577e0 0x55f216a5c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.612+0000 7f73807cd740 1 --2- 172.21.15.55:0/3845744266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a543b0 0x55f216a63f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.612+0000 7f73807cd740 1 --2- 172.21.15.55:0/3845744266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a539f0 0x55f216a5c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.613 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.612+0000 7f73807cd740 1 -- 172.21.15.55:0/3845744266 >> 172.21.15.55:0/3845744266 conn(0x55f216993910 msgr2=0x55f216a4d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:03.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.615+0000 7f73807cd740 1 -- 172.21.15.55:0/3845744266 shutdown_connections 2024-04-26T22:03:03.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.618+0000 7f73807cd740 1 -- 172.21.15.55:0/3845744266 wait complete. 2024-04-26T22:03:03.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.618+0000 7f73807cd740 1 -- start start 2024-04-26T22:03:03.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.619+0000 7f73807cd740 1 Processor -- start 2024-04-26T22:03:03.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.620+0000 7f73807cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a539f0 0x55f216a61070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.620+0000 7f73807cd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a543b0 0x55f216a615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.621+0000 7f73807cd740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a577e0 0x55f216a625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.621+0000 7f73807cd740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f736400cd50 con 0x55f216a577e0 2024-04-26T22:03:03.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.621+0000 7f7377fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a577e0 0x55f216a625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.621+0000 7f73807cd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f736400e1c0 con 0x55f216a543b0 2024-04-26T22:03:03.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.621+0000 7f73807cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f736400d420 con 0x55f216a539f0 2024-04-26T22:03:03.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.622+0000 7f7377fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a577e0 0x55f216a625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33532/0 (socket says 172.21.15.55:33532) 2024-04-26T22:03:03.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.622+0000 7f7377fff640 1 -- 172.21.15.55:0/4148442631 learned_addr learned my addr 172.21.15.55:0/4148442631 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:03.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.622+0000 7f73807cd740 10 client.0 _ll_register_callbacks cb 0x7f737001ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:03.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.623+0000 7f7376ffd640 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a543b0 0x55f216a615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.624+0000 7f7377fff640 1 -- 172.21.15.55:0/4148442631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a539f0 msgr2=0x55f216a61070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:03.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.624+0000 7f7377fff640 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a539f0 0x55f216a61070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.624+0000 7f7377fff640 1 -- 172.21.15.55:0/4148442631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a543b0 msgr2=0x55f216a615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:03.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.624+0000 7f7377fff640 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a543b0 0x55f216a615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.624+0000 7f7377fff640 1 -- 172.21.15.55:0/4148442631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f216a62bb0 con 0x55f216a577e0 2024-04-26T22:03:03.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.625+0000 7f7377fff640 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a577e0 0x55f216a625c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f736809d670 tx=0x7f736809d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358fa1657b1c5214 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:03.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.627+0000 7f7356ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:03.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.628+0000 7f7356ffd640 1 -- 172.21.15.55:0/4148442631 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73680b5070 con 0x55f216a577e0 2024-04-26T22:03:03.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.628+0000 7f73807cd740 1 -- 172.21.15.55:0/4148442631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736c02d070 con 0x55f216a577e0 2024-04-26T22:03:03.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.628+0000 7f7356ffd640 1 -- 172.21.15.55:0/4148442631 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f736809ddd0 con 0x55f216a577e0 2024-04-26T22:03:03.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.630+0000 7f7356ffd640 1 -- 172.21.15.55:0/4148442631 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f73680ad050 con 0x55f216a577e0 2024-04-26T22:03:03.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.631+0000 7f73807cd740 10 client.16992 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:03.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.631+0000 7f73807cd740 1 -- 172.21.15.55:0/4148442631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f216a63f80 con 0x55f216a577e0 2024-04-26T22:03:03.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.634+0000 7f7356ffd640 1 -- 172.21.15.55:0/4148442631 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f736c02d070 con 0x55f216a577e0 2024-04-26T22:03:03.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.634+0000 7f7356ffd640 1 -- 172.21.15.55:0/4148442631 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f73680bbb60 con 0x55f216a577e0 2024-04-26T22:03:03.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.634+0000 7f73807cd740 10 client.16992 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:03.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.634+0000 7f73807cd740 10 client.16992 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:03:03.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.634+0000 7f73807cd740 20 client.16992 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:03.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.634+0000 7f73807cd740 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55f216acf2e0 0x55f216ad1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:03.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.634+0000 7f73807cd740 4 client.16992 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:03:03.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.634+0000 7f73807cd740 1 -- 172.21.15.55:0/4148442631 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2169be800 con 0x55f216acf2e0 2024-04-26T22:03:03.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.638+0000 7f73777fe640 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55f216acf2e0 0x55f216ad1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:03.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.657+0000 7f73777fe640 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55f216acf2e0 0x55f216ad1700 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b081b000628fe3e3 server_cookie=2c3efcf885a8fa04 in_seq=0 out_seq=0 2024-04-26T22:03:03.663 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.657+0000 7f7356ffd640 10 client.16992 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:03:03.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.668+0000 7f7356ffd640 1 -- 172.21.15.55:0/4148442631 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2169be800 con 0x55f216acf2e0 2024-04-26T22:03:03.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.668+0000 7f7356ffd640 10 client.16992 handle_command_reply: tid=0 2024-04-26T22:03:03.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.668+0000 7f73807cd740 1 client.16992 shutdown 2024-04-26T22:03:03.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f73807cd740 1 -- 172.21.15.55:0/4148442631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a577e0 msgr2=0x55f216a625c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f73807cd740 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a577e0 0x55f216a625c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f736809d670 tx=0x7f736809d390 comp rx=0 tx=0).stop 2024-04-26T22:03:03.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f73807cd740 1 -- 172.21.15.55:0/4148442631 shutdown_connections 2024-04-26T22:03:03.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f73807cd740 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55f216acf2e0 0x55f216ad1700 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f73807cd740 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f216a577e0 0x55f216a625c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f73807cd740 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f216a543b0 0x55f216a615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f73807cd740 1 --2- 172.21.15.55:0/4148442631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f216a539f0 0x55f216a61070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f7356ffd640 0 client.16992 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:03:03.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.673+0000 7f73807cd740 1 -- 172.21.15.55:0/4148442631 >> 172.21.15.55:0/4148442631 conn(0x55f216993910 msgr2=0x55f216a50f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:03.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.675+0000 7f73807cd740 1 -- 172.21.15.55:0/4148442631 shutdown_connections 2024-04-26T22:03:03.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.675+0000 7f73807cd740 1 -- 172.21.15.55:0/4148442631 wait complete. 2024-04-26T22:03:03.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.681+0000 7f73807cd740 20 client.16992 trim_cache size 0 max 16384 2024-04-26T22:03:03.685 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:03:03.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.693+0000 7f737dabb640 1 -- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73600a3dc0 msgr2=0x7f73600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.693+0000 7f737dabb640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73600a3dc0 0x7f73600a6280 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f736c005d00 tx=0x7f736c00b5b0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.693+0000 7f737dabb640 1 -- 172.21.15.55:0/855576284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73781a4ab0 msgr2=0x7f73781a6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:03.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.693+0000 7f737dabb640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73781a4ab0 0x7f73781a6ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f736800e970 tx=0x7f736800ee30 comp rx=0 tx=0).stop 2024-04-26T22:03:03.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.696+0000 7f737dabb640 1 -- 172.21.15.55:0/855576284 shutdown_connections 2024-04-26T22:03:03.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.696+0000 7f737dabb640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f73600a3dc0 0x7f73600a6280 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.696+0000 7f737dabb640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73781a4ab0 0x7f73781a6ea0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.696+0000 7f737dabb640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f737810ec70 0x7f737819faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.696+0000 7f737dabb640 1 --2- 172.21.15.55:0/855576284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f737810c2a0 0x7f737819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:03.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.696+0000 7f737dabb640 1 -- 172.21.15.55:0/855576284 >> 172.21.15.55:0/855576284 conn(0x7f737806f7c0 msgr2=0x7f737810a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:03.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.698+0000 7f737dabb640 1 -- 172.21.15.55:0/855576284 shutdown_connections 2024-04-26T22:03:03.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:03.699+0000 7f737dabb640 1 -- 172.21.15.55:0/855576284 wait complete. 2024-04-26T22:03:03.746 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:03:03.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:03 smithi055 ceph-mon[29221]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 852 MiB/s wr, 1.22k op/s 2024-04-26T22:03:03.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:03 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2074363122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:03:04.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:03 smithi064 ceph-mon[32765]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 852 MiB/s wr, 1.22k op/s 2024-04-26T22:03:04.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:03 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2074363122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:03:04.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:03 smithi165 ceph-mon[33296]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 852 MiB/s wr, 1.22k op/s 2024-04-26T22:03:04.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:03 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2074363122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:03:04.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.197+0000 7fd4642b0640 1 -- 172.21.15.55:0/3899217479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c10c560 msgr2=0x7fd45c10e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.197+0000 7fd4642b0640 1 --2- 172.21.15.55:0/3899217479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c10c560 0x7fd45c10e990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd4580074e0 tx=0x7fd45802b080 comp rx=0 tx=0).stop 2024-04-26T22:03:04.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.197+0000 7fd4642b0640 1 -- 172.21.15.55:0/3899217479 shutdown_connections 2024-04-26T22:03:04.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.197+0000 7fd4642b0640 1 --2- 172.21.15.55:0/3899217479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd45c10f020 0x7fd45c1114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.197+0000 7fd4642b0640 1 --2- 172.21.15.55:0/3899217479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c10c560 0x7fd45c10e990 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.197+0000 7fd4642b0640 1 --2- 172.21.15.55:0/3899217479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd45c109ba0 0x7fd45c10bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.197+0000 7fd4642b0640 1 -- 172.21.15.55:0/3899217479 >> 172.21.15.55:0/3899217479 conn(0x7fd45c06f7c0 msgr2=0x7fd45c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.198+0000 7fd4642b0640 1 -- 172.21.15.55:0/3899217479 shutdown_connections 2024-04-26T22:03:04.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.198+0000 7fd4642b0640 1 -- 172.21.15.55:0/3899217479 wait complete. 2024-04-26T22:03:04.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.199+0000 7fd4642b0640 1 Processor -- start 2024-04-26T22:03:04.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.199+0000 7fd4642b0640 1 -- start start 2024-04-26T22:03:04.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd4642b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c109ba0 0x7fd45c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd4642b0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd45c10c560 0x7fd45c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd4642b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd45c10f020 0x7fd45c1a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd4642b0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c114be0 con 0x7fd45c109ba0 2024-04-26T22:03:04.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd4642b0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c114a60 con 0x7fd45c10f020 2024-04-26T22:03:04.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd4642b0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c114d60 con 0x7fd45c10c560 2024-04-26T22:03:04.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd46204c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c109ba0 0x7fd45c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd46204c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c109ba0 0x7fd45c19f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33548/0 (socket says 172.21.15.55:33548) 2024-04-26T22:03:04.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd46204c640 1 -- 172.21.15.55:0/2709016791 learned_addr learned my addr 172.21.15.55:0/2709016791 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.200+0000 7fd46184b640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd45c10c560 0x7fd45c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd46204c640 1 -- 172.21.15.55:0/2709016791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd45c10c560 msgr2=0x7fd45c19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd46204c640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd45c10c560 0x7fd45c19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd46204c640 1 -- 172.21.15.55:0/2709016791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd45c10f020 msgr2=0x7fd45c1a6960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:04.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd46204c640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd45c10f020 0x7fd45c1a6960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd46204c640 1 -- 172.21.15.55:0/2709016791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c1a6fd0 con 0x7fd45c109ba0 2024-04-26T22:03:04.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd46204c640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c109ba0 0x7fd45c19f3b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00b450 tx=0x7fd44c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd4537fe640 1 -- 172.21.15.55:0/2709016791 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c015020 con 0x7fd45c109ba0 2024-04-26T22:03:04.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd4642b0640 1 -- 172.21.15.55:0/2709016791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c1a72c0 con 0x7fd45c109ba0 2024-04-26T22:03:04.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.201+0000 7fd4642b0640 1 -- 172.21.15.55:0/2709016791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c1a7880 con 0x7fd45c109ba0 2024-04-26T22:03:04.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.202+0000 7fd4537fe640 1 -- 172.21.15.55:0/2709016791 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44c011d40 con 0x7fd45c109ba0 2024-04-26T22:03:04.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.203+0000 7fd4537fe640 1 -- 172.21.15.55:0/2709016791 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c01adf0 con 0x7fd45c109ba0 2024-04-26T22:03:04.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.205+0000 7fd4537fe640 1 -- 172.21.15.55:0/2709016791 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd44c018020 con 0x7fd45c109ba0 2024-04-26T22:03:04.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.206+0000 7fd466fc6740 1 Processor -- start 2024-04-26T22:03:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.206+0000 7fd466fc6740 1 -- start start 2024-04-26T22:03:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea36820 0x55da2ea36c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea37e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd46204c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea37e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea383b0 0x55da2ea3ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd4537fe640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd4480a80e0 0x7fd4480aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd46204c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea37e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33558/0 (socket says 172.21.15.55:33558) 2024-04-26T22:03:04.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd46204c640 1 -- 172.21.15.55:0/3481593780 learned_addr learned my addr 172.21.15.55:0/3481593780 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd466fc6740 1 -- 172.21.15.55:0/3481593780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55da2e99ad80 con 0x55da2ea379f0 2024-04-26T22:03:04.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd466fc6740 1 -- 172.21.15.55:0/3481593780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55da2e954a20 con 0x55da2ea383b0 2024-04-26T22:03:04.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd466fc6740 1 -- 172.21.15.55:0/3481593780 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55da2e996d40 con 0x55da2ea36820 2024-04-26T22:03:04.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.207+0000 7fd46284d640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd4480a80e0 0x7fd4480aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.208+0000 7fd4537fe640 1 -- 172.21.15.55:0/2709016791 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd44c0cfef0 con 0x7fd45c109ba0 2024-04-26T22:03:04.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.208+0000 7fd46204c640 1 -- 172.21.15.55:0/3481593780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea36820 msgr2=0x55da2ea36c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.208+0000 7fd46204c640 1 --2- 172.21.15.55:0/3481593780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea36820 0x55da2ea36c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.208+0000 7fd46204c640 1 -- 172.21.15.55:0/3481593780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea383b0 msgr2=0x55da2ea3ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.208+0000 7fd46204c640 1 --2- 172.21.15.55:0/3481593780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea383b0 0x55da2ea3ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.208+0000 7fd46204c640 1 -- 172.21.15.55:0/3481593780 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2e95fc30 con 0x55da2ea379f0 2024-04-26T22:03:04.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.209+0000 7fd46204c640 1 --2- 172.21.15.55:0/3481593780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea37e70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd44c09be40 tx=0x7fd44c01a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4334a818df3e6c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.209+0000 7fd46284d640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd4480a80e0 0x7fd4480aa5a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd45c06c650 tx=0x7fd4540015d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.209+0000 7fd450bfe640 1 -- 172.21.15.55:0/3481593780 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0a04f0 con 0x55da2ea379f0 2024-04-26T22:03:04.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.209+0000 7fd450bfe640 1 -- 172.21.15.55:0/3481593780 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44c0a4db0 con 0x55da2ea379f0 2024-04-26T22:03:04.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.209+0000 7fd450bfe640 1 -- 172.21.15.55:0/3481593780 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0987b0 con 0x55da2ea379f0 2024-04-26T22:03:04.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.210+0000 7fd466fc6740 1 -- 172.21.15.55:0/3481593780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 msgr2=0x55da2ea37e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.210+0000 7fd466fc6740 1 --2- 172.21.15.55:0/3481593780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea37e70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd44c09be40 tx=0x7fd44c01a850 comp rx=0 tx=0).stop 2024-04-26T22:03:04.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.211+0000 7fd466fc6740 1 -- 172.21.15.55:0/3481593780 shutdown_connections 2024-04-26T22:03:04.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.211+0000 7fd466fc6740 1 --2- 172.21.15.55:0/3481593780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea383b0 0x55da2ea3ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.211+0000 7fd466fc6740 1 --2- 172.21.15.55:0/3481593780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea37e70 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.211+0000 7fd466fc6740 1 --2- 172.21.15.55:0/3481593780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea36820 0x55da2ea36c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.211+0000 7fd466fc6740 1 -- 172.21.15.55:0/3481593780 >> 172.21.15.55:0/3481593780 conn(0x55da2e977910 msgr2=0x55da2ea32120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.211+0000 7fd466fc6740 1 -- 172.21.15.55:0/3481593780 shutdown_connections 2024-04-26T22:03:04.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.211+0000 7fd466fc6740 1 -- 172.21.15.55:0/3481593780 wait complete. 2024-04-26T22:03:04.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.212+0000 7fd466fc6740 1 -- start start 2024-04-26T22:03:04.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.212+0000 7fd466fc6740 1 Processor -- start 2024-04-26T22:03:04.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.212+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea40900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.213+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea40e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.213+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea383b0 0x55da2ea47e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.213+0000 7fd46184b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea40e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.213+0000 7fd46184b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea40e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33574/0 (socket says 172.21.15.55:33574) 2024-04-26T22:03:04.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.213+0000 7fd46184b640 1 -- 172.21.15.55:0/115337469 learned_addr learned my addr 172.21.15.55:0/115337469 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.213+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55da2ea3f3b0 con 0x55da2ea379f0 2024-04-26T22:03:04.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.213+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55da2e921dc0 con 0x55da2ea36820 2024-04-26T22:03:04.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.213+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55da2ea3f530 con 0x55da2ea383b0 2024-04-26T22:03:04.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.214+0000 7fd466fc6740 10 client.0 _ll_register_callbacks cb 0x55da2e93d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:04.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.214+0000 7fd46204c640 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea40900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.214+0000 7fd46184b640 1 -- 172.21.15.55:0/115337469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea383b0 msgr2=0x55da2ea47e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.214+0000 7fd46284d640 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea383b0 0x55da2ea47e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.215+0000 7fd46184b640 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea383b0 0x55da2ea47e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.215+0000 7fd46184b640 1 -- 172.21.15.55:0/115337469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 msgr2=0x55da2ea40900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.215+0000 7fd46184b640 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea40900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.216 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.215+0000 7fd46184b640 1 -- 172.21.15.55:0/115337469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2e6ac3f0 con 0x55da2ea379f0 2024-04-26T22:03:04.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.215+0000 7fd46184b640 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea40e40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd45c074ab0 tx=0x7fd458007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cee7f1747fad728 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.216+0000 7fd445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:04.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.217+0000 7fd445ffb640 1 -- 172.21.15.55:0/115337469 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458038040 con 0x55da2ea379f0 2024-04-26T22:03:04.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.217+0000 7fd445ffb640 1 -- 172.21.15.55:0/115337469 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd458005cb0 con 0x55da2ea379f0 2024-04-26T22:03:04.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.217+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd44c0a4db0 con 0x55da2ea379f0 2024-04-26T22:03:04.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.219+0000 7fd445ffb640 1 -- 172.21.15.55:0/115337469 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd458004050 con 0x55da2ea379f0 2024-04-26T22:03:04.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.219+0000 7fd466fc6740 10 client.17004 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:04.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.219+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55da2e7767d0 con 0x55da2ea379f0 2024-04-26T22:03:04.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.219+0000 7fd445ffb640 1 -- 172.21.15.55:0/115337469 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44c0a4db0 con 0x55da2ea379f0 2024-04-26T22:03:04.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.220+0000 7fd445ffb640 1 -- 172.21.15.55:0/115337469 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd45803b070 con 0x55da2ea379f0 2024-04-26T22:03:04.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.220+0000 7fd466fc6740 10 client.17004 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:04.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.220+0000 7fd466fc6740 10 client.17004 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:03:04.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.220+0000 7fd466fc6740 20 client.17004 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:04.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.220+0000 7fd466fc6740 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55da2ea4c7c0 0x55da2ea4ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.220+0000 7fd466fc6740 4 client.17004 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:04.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.220+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da2e9a8990 con 0x55da2ea4c7c0 2024-04-26T22:03:04.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.221+0000 7fd46204c640 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55da2ea4c7c0 0x55da2ea4ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.222+0000 7fd46204c640 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55da2ea4c7c0 0x55da2ea4ec00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e64dba8cd8f1907b server_cookie=3c2f674aa78ffdb7 in_seq=0 out_seq=0 2024-04-26T22:03:04.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.222+0000 7fd445ffb640 10 client.17004 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:03:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.223+0000 7fd445ffb640 1 -- 172.21.15.55:0/115337469 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da2e9a8990 con 0x55da2ea4c7c0 2024-04-26T22:03:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.223+0000 7fd445ffb640 10 client.17004 handle_command_reply: tid=0 2024-04-26T22:03:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.223+0000 7fd466fc6740 1 client.17004 shutdown 2024-04-26T22:03:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 msgr2=0x55da2ea40e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea40e40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd45c074ab0 tx=0x7fd458007f90 comp rx=0 tx=0).stop 2024-04-26T22:03:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 shutdown_connections 2024-04-26T22:03:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55da2ea4c7c0 0x55da2ea4ec00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea383b0 0x55da2ea47e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea379f0 0x55da2ea40e40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd445ffb640 0 client.17004 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:03:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 --2- 172.21.15.55:0/115337469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea40900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 >> 172.21.15.55:0/115337469 conn(0x55da2e977910 msgr2=0x55da2ea31280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 shutdown_connections 2024-04-26T22:03:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 1 -- 172.21.15.55:0/115337469 wait complete. 2024-04-26T22:03:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.224+0000 7fd466fc6740 20 client.17004 trim_cache size 0 max 16384 2024-04-26T22:03:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.252+0000 7fd466fc6740 1 Processor -- start 2024-04-26T22:03:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.252+0000 7fd466fc6740 1 -- start start 2024-04-26T22:03:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.252+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.252+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea379f0 0x55da2ea47e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.252+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea3b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd466fc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd440010e70 con 0x55da2ea383b0 2024-04-26T22:03:04.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd466fc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd44000e740 con 0x55da2ea36820 2024-04-26T22:03:04.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd466fc6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd44000c590 con 0x55da2ea379f0 2024-04-26T22:03:04.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46284d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea3b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46284d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea3b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33588/0 (socket says 172.21.15.55:33588) 2024-04-26T22:03:04.254 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46284d640 1 -- 172.21.15.55:0/4109626608 learned_addr learned my addr 172.21.15.55:0/4109626608 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46204c640 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46284d640 1 -- 172.21.15.55:0/4109626608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea379f0 msgr2=0x55da2ea47e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46284d640 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea379f0 0x55da2ea47e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46284d640 1 -- 172.21.15.55:0/4109626608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 msgr2=0x55da2ea322b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46284d640 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea322b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46284d640 1 -- 172.21.15.55:0/4109626608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2ea4c560 con 0x55da2ea383b0 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.253+0000 7fd46204c640 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.254+0000 7fd46284d640 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea3b110 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd45400ce60 tx=0x7fd4540012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2afdcbe45095c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.254+0000 7fd447fff640 1 -- 172.21.15.55:0/4109626608 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454024070 con 0x55da2ea383b0 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.255+0000 7fd447fff640 1 -- 172.21.15.55:0/4109626608 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd454001bb0 con 0x55da2ea383b0 2024-04-26T22:03:04.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.255+0000 7fd447fff640 1 -- 172.21.15.55:0/4109626608 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45401fa60 con 0x55da2ea383b0 2024-04-26T22:03:04.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 -- 172.21.15.55:0/4109626608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 msgr2=0x55da2ea3b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea3b110 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd45400ce60 tx=0x7fd4540012e0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 -- 172.21.15.55:0/4109626608 shutdown_connections 2024-04-26T22:03:04.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea3b110 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea379f0 0x55da2ea47e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 --2- 172.21.15.55:0/4109626608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea36820 0x55da2ea322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 -- 172.21.15.55:0/4109626608 >> 172.21.15.55:0/4109626608 conn(0x55da2e977910 msgr2=0x55da2ea31ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 -- 172.21.15.55:0/4109626608 shutdown_connections 2024-04-26T22:03:04.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 -- 172.21.15.55:0/4109626608 wait complete. 2024-04-26T22:03:04.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.257+0000 7fd466fc6740 1 -- start start 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.258+0000 7fd466fc6740 1 Processor -- start 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.258+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea36820 0x55da2ea45ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.258+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea379f0 0x55da2ea46000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.258+0000 7fd466fc6740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea47010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.258+0000 7fd466fc6740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd44000cc00 con 0x55da2ea383b0 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.258+0000 7fd466fc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd44000e070 con 0x55da2ea379f0 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46284d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea47010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46284d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea47010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33590/0 (socket says 172.21.15.55:33590) 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46284d640 1 -- 172.21.15.55:0/1991711568 learned_addr learned my addr 172.21.15.55:0/1991711568 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd44000d2d0 con 0x55da2ea36820 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46184b640 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea379f0 0x55da2ea46000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd466fc6740 10 client.0 _ll_register_callbacks cb 0x7fd4580073e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:04.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46284d640 1 -- 172.21.15.55:0/1991711568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea36820 msgr2=0x55da2ea45ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46284d640 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea36820 0x55da2ea45ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46284d640 1 -- 172.21.15.55:0/1991711568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea379f0 msgr2=0x55da2ea46000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46284d640 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea379f0 0x55da2ea46000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.261 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.259+0000 7fd46284d640 1 -- 172.21.15.55:0/1991711568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2ea47600 con 0x55da2ea383b0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd46284d640 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea47010 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd454022040 tx=0x7fd45401f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31da9b37c69f3ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd445ffb640 1 -- 172.21.15.55:0/1991711568 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45402d070 con 0x55da2ea383b0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd445ffb640 1 -- 172.21.15.55:0/1991711568 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd45401fde0 con 0x55da2ea383b0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd454001bb0 con 0x55da2ea383b0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd445ffb640 1 -- 172.21.15.55:0/1991711568 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd454039050 con 0x55da2ea383b0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd466fc6740 10 client.17010 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55da2eabb9a0 con 0x55da2ea383b0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.260+0000 7fd445ffb640 1 -- 172.21.15.55:0/1991711568 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd454001bb0 con 0x55da2ea383b0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.261+0000 7fd445ffb640 1 -- 172.21.15.55:0/1991711568 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fd4540296f0 con 0x55da2ea383b0 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.261+0000 7fd466fc6740 10 client.17010 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:04.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.261+0000 7fd466fc6740 10 client.17010 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:03:04.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.261+0000 7fd466fc6740 20 client.17010 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:04.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.261+0000 7fd466fc6740 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55da2eabc270 0x55da2eabe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.261+0000 7fd466fc6740 4 client.17010 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:03:04.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.261+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da2e9a2800 con 0x55da2eabc270 2024-04-26T22:03:04.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.261+0000 7fd46204c640 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55da2eabc270 0x55da2eabe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.263+0000 7fd445ffb640 10 client.17010 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:03:04.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.263+0000 7fd46204c640 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55da2eabc270 0x55da2eabe690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bda003cab234c961 server_cookie=226389efe05c144d in_seq=0 out_seq=0 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.263+0000 7fd445ffb640 1 -- 172.21.15.55:0/1991711568 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da2e9a2800 con 0x55da2eabc270 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.263+0000 7fd445ffb640 10 client.17010 handle_command_reply: tid=0 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.263+0000 7fd466fc6740 1 client.17010 shutdown 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.263+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 msgr2=0x55da2ea47010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.263+0000 7fd466fc6740 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea47010 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd454022040 tx=0x7fd45401f7e0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.263+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 shutdown_connections 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd466fc6740 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55da2eabc270 0x55da2eabe690 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd466fc6740 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55da2ea383b0 0x55da2ea47010 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd445ffb640 0 client.17010 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd466fc6740 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55da2ea379f0 0x55da2ea46000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd466fc6740 1 --2- 172.21.15.55:0/1991711568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da2ea36820 0x55da2ea45ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 >> 172.21.15.55:0/1991711568 conn(0x55da2e977910 msgr2=0x55da2ea309f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 shutdown_connections 2024-04-26T22:03:04.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd466fc6740 1 -- 172.21.15.55:0/1991711568 wait complete. 2024-04-26T22:03:04.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.264+0000 7fd466fc6740 20 client.17010 trim_cache size 0 max 16384 2024-04-26T22:03:04.267 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 -- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd4480a80e0 msgr2=0x7fd4480aa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd4480a80e0 0x7fd4480aa5a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd45c06c650 tx=0x7fd4540015d0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 -- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c109ba0 msgr2=0x7fd45c19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c109ba0 0x7fd45c19f3b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00b450 tx=0x7fd44c00b910 comp rx=0 tx=0).stop 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 -- 172.21.15.55:0/2709016791 shutdown_connections 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd4480a80e0 0x7fd4480aa5a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd45c10f020 0x7fd45c1a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd45c10c560 0x7fd45c19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 --2- 172.21.15.55:0/2709016791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd45c109ba0 0x7fd45c19f3b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.265+0000 7fd4642b0640 1 -- 172.21.15.55:0/2709016791 >> 172.21.15.55:0/2709016791 conn(0x7fd45c06f7c0 msgr2=0x7fd45c109f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.266+0000 7fd4642b0640 1 -- 172.21.15.55:0/2709016791 shutdown_connections 2024-04-26T22:03:04.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.266+0000 7fd4642b0640 1 -- 172.21.15.55:0/2709016791 wait complete. 2024-04-26T22:03:04.299 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.744+0000 7fe08bc15640 1 -- 172.21.15.55:0/3309810372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0841011e0 msgr2=0x7fe084103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.744+0000 7fe08bc15640 1 --2- 172.21.15.55:0/3309810372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0841011e0 0x7fe084103610 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe074009900 tx=0x7fe07402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.745+0000 7fe08bc15640 1 -- 172.21.15.55:0/3309810372 shutdown_connections 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.745+0000 7fe08bc15640 1 --2- 172.21.15.55:0/3309810372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe084106500 0x7fe084106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.745+0000 7fe08bc15640 1 --2- 172.21.15.55:0/3309810372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084103b50 0x7fe084105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.745+0000 7fe08bc15640 1 --2- 172.21.15.55:0/3309810372 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0841011e0 0x7fe084103610 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.745+0000 7fe08bc15640 1 -- 172.21.15.55:0/3309810372 >> 172.21.15.55:0/3309810372 conn(0x7fe0840fcde0 msgr2=0x7fe0840ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.745+0000 7fe08bc15640 1 -- 172.21.15.55:0/3309810372 shutdown_connections 2024-04-26T22:03:04.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.745+0000 7fe08bc15640 1 -- 172.21.15.55:0/3309810372 wait complete. 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe08bc15640 1 Processor -- start 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe08bc15640 1 -- start start 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe08bc15640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0841011e0 0x7fe08419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe08bc15640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe084103b50 0x7fe08419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe08bc15640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084106500 0x7fe0841a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe08bc15640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe084114b10 con 0x7fe084103b50 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe08bc15640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe084114990 con 0x7fe084106500 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe08bc15640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe084114c90 con 0x7fe0841011e0 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe0891b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe084103b50 0x7fe08419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe0891b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe084103b50 0x7fe08419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33606/0 (socket says 172.21.15.55:33606) 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.746+0000 7fe0891b0640 1 -- 172.21.15.55:0/2160134163 learned_addr learned my addr 172.21.15.55:0/2160134163 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe08a1b2640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084106500 0x7fe0841a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe0899b1640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0841011e0 0x7fe08419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe0891b0640 1 -- 172.21.15.55:0/2160134163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0841011e0 msgr2=0x7fe08419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe0891b0640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0841011e0 0x7fe08419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe0891b0640 1 -- 172.21.15.55:0/2160134163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084106500 msgr2=0x7fe0841a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe0891b0640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084106500 0x7fe0841a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe0891b0640 1 -- 172.21.15.55:0/2160134163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0841a6de0 con 0x7fe084103b50 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe0899b1640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0841011e0 0x7fe08419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.747+0000 7fe0891b0640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe084103b50 0x7fe08419f700 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00db80 tx=0x7fe07c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.748+0000 7fe07affd640 1 -- 172.21.15.55:0/2160134163 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c017070 con 0x7fe084103b50 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.748+0000 7fe08bc15640 1 -- 172.21.15.55:0/2160134163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0841a70d0 con 0x7fe084103b50 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.748+0000 7fe07affd640 1 -- 172.21.15.55:0/2160134163 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe07c003b70 con 0x7fe084103b50 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.748+0000 7fe08bc15640 1 -- 172.21.15.55:0/2160134163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0841a7670 con 0x7fe084103b50 2024-04-26T22:03:04.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.748+0000 7fe07affd640 1 -- 172.21.15.55:0/2160134163 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0106c0 con 0x7fe084103b50 2024-04-26T22:03:04.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.752+0000 7fe07affd640 1 -- 172.21.15.55:0/2160134163 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe07c004b20 con 0x7fe084103b50 2024-04-26T22:03:04.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.752+0000 7fe08ea09740 1 Processor -- start 2024-04-26T22:03:04.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.753+0000 7fe08ea09740 1 -- start start 2024-04-26T22:03:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.753+0000 7fe07affd640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0700a3c40 0x7fe0700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.753+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2964c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.753+0000 7fe07affd640 1 -- 172.21.15.55:0/2160134163 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe07c0d0740 con 0x7fe084103b50 2024-04-26T22:03:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.753+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2965e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.753+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e296cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe08a1b2640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0700a3c40 0x7fe0700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe08ea09740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e1e28c8d80 con 0x55e1e29659f0 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe08ea09740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e1e2882a20 con 0x55e1e2964820 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe08ea09740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e1e28c4d40 con 0x55e1e29663b0 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0891b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2965e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0891b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2965e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33620/0 (socket says 172.21.15.55:33620) 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0891b0640 1 -- 172.21.15.55:0/832962227 learned_addr learned my addr 172.21.15.55:0/832962227 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0899b1640 1 --2- 172.21.15.55:0/832962227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2964c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0891b0640 1 -- 172.21.15.55:0/832962227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 msgr2=0x55e1e296cc40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0891b0640 1 --2- 172.21.15.55:0/832962227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e296cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0891b0640 1 -- 172.21.15.55:0/832962227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 msgr2=0x55e1e2964c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0891b0640 1 --2- 172.21.15.55:0/832962227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2964c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe0891b0640 1 -- 172.21.15.55:0/832962227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e288dc30 con 0x55e1e29659f0 2024-04-26T22:03:04.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.754+0000 7fe08a1b2640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0700a3c40 0x7fe0700a6100 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe080005fd0 tx=0x7fe080005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.755+0000 7fe0891b0640 1 --2- 172.21.15.55:0/832962227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2965e70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0a4fc0 tx=0x7fe07c09cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41aa7b0b387d8b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.755+0000 7fe06ffff640 1 -- 172.21.15.55:0/832962227 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0a54f0 con 0x55e1e29659f0 2024-04-26T22:03:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.755+0000 7fe06ffff640 1 -- 172.21.15.55:0/832962227 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe07c022dd0 con 0x55e1e29659f0 2024-04-26T22:03:04.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.755+0000 7fe06ffff640 1 -- 172.21.15.55:0/832962227 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0a5060 con 0x55e1e29659f0 2024-04-26T22:03:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 -- 172.21.15.55:0/832962227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 msgr2=0x55e1e2965e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 --2- 172.21.15.55:0/832962227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2965e70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe07c0a4fc0 tx=0x7fe07c09cf80 comp rx=0 tx=0).stop 2024-04-26T22:03:04.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 -- 172.21.15.55:0/832962227 shutdown_connections 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 --2- 172.21.15.55:0/832962227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e296cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 --2- 172.21.15.55:0/832962227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2965e70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 --2- 172.21.15.55:0/832962227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2964c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 -- 172.21.15.55:0/832962227 >> 172.21.15.55:0/832962227 conn(0x55e1e28a5910 msgr2=0x55e1e2960120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 -- 172.21.15.55:0/832962227 shutdown_connections 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 -- 172.21.15.55:0/832962227 wait complete. 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.756+0000 7fe08ea09740 1 -- start start 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe08ea09740 1 Processor -- start 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e2964820 0x55e1e296ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29659f0 0x55e1e296efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e29663b0 0x55e1e2976010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe08ea09740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e1e296d3c0 con 0x55e1e2964820 2024-04-26T22:03:04.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe08ea09740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e1e284fdc0 con 0x55e1e29663b0 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe08ea09740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e1e296d540 con 0x55e1e29659f0 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe0899b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e2964820 0x55e1e296ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe08ea09740 10 client.0 _ll_register_callbacks cb 0x55e1e286b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe0899b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e2964820 0x55e1e296ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33630/0 (socket says 172.21.15.55:33630) 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.757+0000 7fe0899b1640 1 -- 172.21.15.55:0/3000929648 learned_addr learned my addr 172.21.15.55:0/3000929648 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe0899b1640 1 -- 172.21.15.55:0/3000929648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29659f0 msgr2=0x55e1e296efd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe0899b1640 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29659f0 0x55e1e296efd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe0899b1640 1 -- 172.21.15.55:0/3000929648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e29663b0 msgr2=0x55e1e2976010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe0899b1640 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e29663b0 0x55e1e2976010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe0899b1640 1 -- 172.21.15.55:0/3000929648 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e25da3f0 con 0x55e1e2964820 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe0899b1640 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e2964820 0x55e1e296ea90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe07402a910 tx=0x7fe074004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac57169661d7ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe06d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3000929648 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074035040 con 0x55e1e2964820 2024-04-26T22:03:04.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe08ea09740 1 -- 172.21.15.55:0/3000929648 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe07c022dd0 con 0x55e1e2964820 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.758+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3000929648 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe074005510 con 0x55e1e2964820 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3000929648 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe0740046c0 con 0x55e1e2964820 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe08ea09740 10 client.17022 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe08ea09740 1 -- 172.21.15.55:0/3000929648 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e1e26a47d0 con 0x55e1e2964820 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3000929648 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe07c022dd0 con 0x55e1e2964820 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3000929648 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe074003b90 con 0x55e1e2964820 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe08ea09740 10 client.17022 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe08ea09740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe08ea09740 20 client.17022 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.759+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1e297a8b0 0x55e1e297cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.760+0000 7fe08ea09740 4 client.17022 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.760+0000 7fe08ea09740 1 -- 172.21.15.55:0/3000929648 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1e28d6990 con 0x55e1e297a8b0 2024-04-26T22:03:04.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.760+0000 7fe0891b0640 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1e297a8b0 0x55e1e297cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.762+0000 7fe0891b0640 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1e297a8b0 0x55e1e297cd70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2e8d3755b9a0c6 server_cookie=326dbfecbcf07889 in_seq=0 out_seq=0 2024-04-26T22:03:04.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.762+0000 7fe06d7fa640 10 client.17022 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.765+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3000929648 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1e28d6990 con 0x55e1e297a8b0 2024-04-26T22:03:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.765+0000 7fe06d7fa640 10 client.17022 handle_command_reply: tid=0 2024-04-26T22:03:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.765+0000 7fe08ea09740 1 client.17022 shutdown 2024-04-26T22:03:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.765+0000 7fe08ea09740 1 -- 172.21.15.55:0/3000929648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e2964820 msgr2=0x55e1e296ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.765+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e2964820 0x55e1e296ea90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe07402a910 tx=0x7fe074004460 comp rx=0 tx=0).stop 2024-04-26T22:03:04.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 1 -- 172.21.15.55:0/3000929648 shutdown_connections 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e29663b0 0x55e1e2976010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29659f0 0x55e1e296efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1e297a8b0 0x55e1e297cd70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3000929648 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e2964820 0x55e1e296ea90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe06d7fa640 0 client.17022 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 1 -- 172.21.15.55:0/3000929648 >> 172.21.15.55:0/3000929648 conn(0x55e1e28a5910 msgr2=0x55e1e295f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 1 -- 172.21.15.55:0/3000929648 shutdown_connections 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 1 -- 172.21.15.55:0/3000929648 wait complete. 2024-04-26T22:03:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.766+0000 7fe08ea09740 20 client.17022 trim_cache size 0 max 16384 2024-04-26T22:03:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.788+0000 7fe08ea09740 1 Processor -- start 2024-04-26T22:03:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.788+0000 7fe08ea09740 1 -- start start 2024-04-26T22:03:04.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.788+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e296e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.789+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2975f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.791+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e296e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.791+0000 7fe08ea09740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841b2e50 con 0x55e1e29659f0 2024-04-26T22:03:04.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.791+0000 7fe08ea09740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841b0720 con 0x55e1e2964820 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.791+0000 7fe08ea09740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841ae570 con 0x55e1e29663b0 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.791+0000 7fe0899b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e296e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.791+0000 7fe0899b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e296e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:38598/0 (socket says 172.21.15.55:38598) 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.791+0000 7fe0899b1640 1 -- 172.21.15.55:0/60206257 learned_addr learned my addr 172.21.15.55:0/60206257 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.792+0000 7fe08a1b2640 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e296e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.794+0000 7fe0899b1640 1 -- 172.21.15.55:0/60206257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 msgr2=0x55e1e296e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.794+0000 7fe0899b1640 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e296e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.794+0000 7fe0899b1640 1 -- 172.21.15.55:0/60206257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 msgr2=0x55e1e2975f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.800+0000 7fe0891b0640 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2975f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.800+0000 7fe0899b1640 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2975f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.800+0000 7fe0899b1640 1 -- 172.21.15.55:0/60206257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e2965470 con 0x55e1e2964820 2024-04-26T22:03:04.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.806+0000 7fe0899b1640 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e296e0b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe074009850 tx=0x7fe074001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6fb0aeabb29dfc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.835+0000 7fe06f7fe640 1 -- 172.21.15.55:0/60206257 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074035070 con 0x55e1e2964820 2024-04-26T22:03:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.835+0000 7fe06f7fe640 1 -- 172.21.15.55:0/60206257 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe07404a030 con 0x55e1e2964820 2024-04-26T22:03:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.835+0000 7fe06f7fe640 1 -- 172.21.15.55:0/60206257 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0740035f0 con 0x55e1e2964820 2024-04-26T22:03:04.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.837+0000 7fe08ea09740 1 -- 172.21.15.55:0/60206257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 msgr2=0x55e1e296e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.837 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.837+0000 7fe08ea09740 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e296e0b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe074009850 tx=0x7fe074001e70 comp rx=0 tx=0).stop 2024-04-26T22:03:04.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.837+0000 7fe08ea09740 1 -- 172.21.15.55:0/60206257 shutdown_connections 2024-04-26T22:03:04.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.838+0000 7fe08ea09740 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e296e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.838+0000 7fe08ea09740 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e2975f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.838+0000 7fe08ea09740 1 --2- 172.21.15.55:0/60206257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e296e0b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.838+0000 7fe08ea09740 1 -- 172.21.15.55:0/60206257 >> 172.21.15.55:0/60206257 conn(0x55e1e28a5910 msgr2=0x55e1e295f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.838+0000 7fe08ea09740 1 -- 172.21.15.55:0/60206257 shutdown_connections 2024-04-26T22:03:04.839 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.838+0000 7fe08ea09740 1 -- 172.21.15.55:0/60206257 wait complete. 2024-04-26T22:03:04.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.839+0000 7fe08ea09740 1 -- start start 2024-04-26T22:03:04.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.840+0000 7fe08ea09740 1 Processor -- start 2024-04-26T22:03:04.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.841+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2972e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.841+0000 7fe08ea09740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e29733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.844+0000 7fe0891b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e29733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.844+0000 7fe0891b0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e29733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33662/0 (socket says 172.21.15.55:33662) 2024-04-26T22:03:04.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.845+0000 7fe0899b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2972e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.845+0000 7fe0899b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2972e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:38606/0 (socket says 172.21.15.55:38606) 2024-04-26T22:03:04.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.845+0000 7fe0891b0640 1 -- 172.21.15.55:0/3522232817 learned_addr learned my addr 172.21.15.55:0/3522232817 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:04.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.845+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e29deea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.845+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841aebe0 con 0x55e1e29659f0 2024-04-26T22:03:04.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.846+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841b0050 con 0x55e1e2964820 2024-04-26T22:03:04.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.846+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841af2b0 con 0x55e1e29663b0 2024-04-26T22:03:04.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.846+0000 7fe08ea09740 10 client.0 _ll_register_callbacks cb 0x7fe0740040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:04.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.848+0000 7fe08a1b2640 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e29deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.848+0000 7fe0899b1640 1 -- 172.21.15.55:0/3522232817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 msgr2=0x55e1e29deea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.848+0000 7fe0899b1640 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e29deea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.848+0000 7fe0899b1640 1 -- 172.21.15.55:0/3522232817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 msgr2=0x55e1e29733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:04.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.848+0000 7fe0899b1640 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e29733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.848+0000 7fe0899b1640 1 -- 172.21.15.55:0/3522232817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e29df3e0 con 0x55e1e2964820 2024-04-26T22:03:04.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.851+0000 7fe0899b1640 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2972e90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe07402a910 tx=0x7fe0740041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272c0aba5c38f33e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:04.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.859+0000 7fe06d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:03:04.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.859+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3522232817 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07406e060 con 0x55e1e2964820 2024-04-26T22:03:04.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.859+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3522232817 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe074005510 con 0x55e1e2964820 2024-04-26T22:03:04.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.859+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe07404a030 con 0x55e1e2964820 2024-04-26T22:03:04.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.866+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3522232817 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe074035040 con 0x55e1e2964820 2024-04-26T22:03:04.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.868+0000 7fe08ea09740 10 client.26084 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:04.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.868+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e1e29e0580 con 0x55e1e2964820 2024-04-26T22:03:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.877+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3522232817 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe07404a030 con 0x55e1e2964820 2024-04-26T22:03:04.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.879+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3522232817 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe07403c440 con 0x55e1e2964820 2024-04-26T22:03:04.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.879+0000 7fe08ea09740 10 client.26084 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:04.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.879+0000 7fe08ea09740 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:04.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.879+0000 7fe08ea09740 20 client.26084 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:04.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.880+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1e29e0e60 0x55e1e29e3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:04.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.880+0000 7fe08ea09740 4 client.26084 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:03:04.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.880+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1e28d0800 con 0x55e1e29e0e60 2024-04-26T22:03:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.880+0000 7fe0891b0640 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1e29e0e60 0x55e1e29e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.881+0000 7fe0891b0640 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1e29e0e60 0x55e1e29e3250 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed193a9c7276da7 server_cookie=d53845ab6dee5146 in_seq=0 out_seq=0 2024-04-26T22:03:04.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.884+0000 7fe06d7fa640 10 client.26084 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:04.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.884+0000 7fe06d7fa640 1 -- 172.21.15.55:0/3522232817 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1e28d0800 con 0x55e1e29e0e60 2024-04-26T22:03:04.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.884+0000 7fe06d7fa640 10 client.26084 handle_command_reply: tid=0 2024-04-26T22:03:04.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.884+0000 7fe08ea09740 1 client.26084 shutdown 2024-04-26T22:03:04.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.887+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 msgr2=0x55e1e2972e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.887+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2972e90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe07402a910 tx=0x7fe0740041a0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.887+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 shutdown_connections 2024-04-26T22:03:04.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.887+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e1e29663b0 0x55e1e29deea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.887+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e1e29e0e60 0x55e1e29e3250 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.888+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e1e29659f0 0x55e1e29733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.888+0000 7fe08ea09740 1 --2- 172.21.15.55:0/3522232817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e1e2964820 0x55e1e2972e90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.888+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 >> 172.21.15.55:0/3522232817 conn(0x55e1e28a5910 msgr2=0x55e1e2962de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.888+0000 7fe06d7fa640 0 client.26084 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:04.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.888+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 shutdown_connections 2024-04-26T22:03:04.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.888+0000 7fe08ea09740 1 -- 172.21.15.55:0/3522232817 wait complete. 2024-04-26T22:03:04.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.889+0000 7fe08ea09740 20 client.26084 trim_cache size 0 max 16384 2024-04-26T22:03:04.894 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:03:04.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.900+0000 7fe08bc15640 1 -- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0700a3c40 msgr2=0x7fe0700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.900+0000 7fe08bc15640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0700a3c40 0x7fe0700a6100 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe080005fd0 tx=0x7fe080005cf0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.900+0000 7fe08bc15640 1 -- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe084103b50 msgr2=0x7fe08419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:04.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.900+0000 7fe08bc15640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe084103b50 0x7fe08419f700 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00db80 tx=0x7fe07c00b630 comp rx=0 tx=0).stop 2024-04-26T22:03:04.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.902+0000 7fe08bc15640 1 -- 172.21.15.55:0/2160134163 shutdown_connections 2024-04-26T22:03:04.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.902+0000 7fe08bc15640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe0700a3c40 0x7fe0700a6100 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.902+0000 7fe08bc15640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe084106500 0x7fe0841a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.902+0000 7fe08bc15640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe084103b50 0x7fe08419f700 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.902+0000 7fe08bc15640 1 --2- 172.21.15.55:0/2160134163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0841011e0 0x7fe08419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:04.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.903+0000 7fe08bc15640 1 -- 172.21.15.55:0/2160134163 >> 172.21.15.55:0/2160134163 conn(0x7fe0840fcde0 msgr2=0x7fe084104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:04.911 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.909+0000 7fe08bc15640 1 -- 172.21.15.55:0/2160134163 shutdown_connections 2024-04-26T22:03:04.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:04.912+0000 7fe08bc15640 1 -- 172.21.15.55:0/2160134163 wait complete. 2024-04-26T22:03:05.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:05 smithi055 ceph-mon[29221]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 870 MiB/s wr, 1.33k op/s 2024-04-26T22:03:05.972 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:03:05.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:03:05.972 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:03:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:05 smithi064 ceph-mon[32765]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 870 MiB/s wr, 1.33k op/s 2024-04-26T22:03:06.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:05 smithi165 ceph-mon[33296]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 870 MiB/s wr, 1.33k op/s 2024-04-26T22:03:06.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.401+0000 7f3620874640 1 -- 172.21.15.55:0/2416614282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f361c109930 msgr2=0x7f361c10bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.401+0000 7f3620874640 1 --2- 172.21.15.55:0/2416614282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f361c109930 0x7f361c10bd60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3604009900 tx=0x7f360402a400 comp rx=0 tx=0).stop 2024-04-26T22:03:06.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.404+0000 7f3620874640 1 -- 172.21.15.55:0/2416614282 shutdown_connections 2024-04-26T22:03:06.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.404+0000 7f3620874640 1 --2- 172.21.15.55:0/2416614282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361c10ec50 0x7f361c10f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.404+0000 7f3620874640 1 --2- 172.21.15.55:0/2416614282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f361c10c2a0 0x7f361c10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.404+0000 7f3620874640 1 --2- 172.21.15.55:0/2416614282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f361c109930 0x7f361c10bd60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.404+0000 7f3620874640 1 -- 172.21.15.55:0/2416614282 >> 172.21.15.55:0/2416614282 conn(0x7f361c06fb60 msgr2=0x7f361c071f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:06.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.406+0000 7f3620874640 1 -- 172.21.15.55:0/2416614282 shutdown_connections 2024-04-26T22:03:06.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.410+0000 7f3620874640 1 -- 172.21.15.55:0/2416614282 wait complete. 2024-04-26T22:03:06.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.411+0000 7f3620874640 1 Processor -- start 2024-04-26T22:03:06.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.414+0000 7f3620874640 1 -- start start 2024-04-26T22:03:06.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.414+0000 7f3620874640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f361c109930 0x7f361c1a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.414 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.414+0000 7f3620874640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361c10c2a0 0x7f361c1a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.414+0000 7f3620874640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f361c10ec50 0x7f361c1af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.414+0000 7f3620874640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f361c11d3a0 con 0x7f361c10ec50 2024-04-26T22:03:06.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.414+0000 7f3620874640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f361c11d220 con 0x7f361c10c2a0 2024-04-26T22:03:06.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.414+0000 7f3620874640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f361c11d520 con 0x7f361c109930 2024-04-26T22:03:06.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.419+0000 7f3619d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361c10c2a0 0x7f361c1a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.421 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.420+0000 7f3619d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361c10c2a0 0x7f361c1a8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:38616/0 (socket says 172.21.15.55:38616) 2024-04-26T22:03:06.422 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.420+0000 7f3619d9b640 1 -- 172.21.15.55:0/1817626704 learned_addr learned my addr 172.21.15.55:0/1817626704 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:06.424 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.423+0000 7f361ad9d640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f361c10ec50 0x7f361c1af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.424+0000 7f361a59c640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f361c109930 0x7f361c1a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.428+0000 7f3619d9b640 1 -- 172.21.15.55:0/1817626704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f361c109930 msgr2=0x7f361c1a7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.428+0000 7f3619d9b640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f361c109930 0x7f361c1a7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.428+0000 7f3619d9b640 1 -- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f361c10ec50 msgr2=0x7f361c1af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.428+0000 7f3619d9b640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f361c10ec50 0x7f361c1af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.431 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.428+0000 7f3619d9b640 1 -- 172.21.15.55:0/1817626704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c111190 con 0x7f361c10c2a0 2024-04-26T22:03:06.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.433+0000 7f3619d9b640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361c10c2a0 0x7f361c1a8070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f361400b4a0 tx=0x7f361400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.437+0000 7f36137fe640 1 -- 172.21.15.55:0/1817626704 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614014070 con 0x7f361c10c2a0 2024-04-26T22:03:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.437+0000 7f36137fe640 1 -- 172.21.15.55:0/1817626704 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3614010ac0 con 0x7f361c10c2a0 2024-04-26T22:03:06.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.437+0000 7f36137fe640 1 -- 172.21.15.55:0/1817626704 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614017be0 con 0x7f361c10c2a0 2024-04-26T22:03:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.441+0000 7f3620874640 1 -- 172.21.15.55:0/1817626704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c111480 con 0x7f361c10c2a0 2024-04-26T22:03:06.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.441+0000 7f3620874640 1 -- 172.21.15.55:0/1817626704 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c1119d0 con 0x7f361c10c2a0 2024-04-26T22:03:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.452+0000 7f362358b740 1 Processor -- start 2024-04-26T22:03:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.453+0000 7f36137fe640 1 -- 172.21.15.55:0/1817626704 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3614017070 con 0x7f361c10c2a0 2024-04-26T22:03:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.453+0000 7f362358b740 1 -- start start 2024-04-26T22:03:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.453+0000 7f362358b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b948ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.454+0000 7f362358b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 0x5641b9493da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.454+0000 7f362358b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.454+0000 7f362358b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5641b93f3b00 con 0x5641b9494410 2024-04-26T22:03:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.454+0000 7f362358b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5641b938f920 con 0x5641b948f2a0 2024-04-26T22:03:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.454+0000 7f362358b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641b93efac0 con 0x5641b948e8b0 2024-04-26T22:03:06.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.456+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b948ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.456+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b948ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36498/0 (socket says 172.21.15.55:36498) 2024-04-26T22:03:06.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.456+0000 7f361ad9d640 1 -- 172.21.15.55:0/4276927908 learned_addr learned my addr 172.21.15.55:0/4276927908 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:06.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.456+0000 7f361a59c640 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 0x5641b9493da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.457+0000 7f36137fe640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36080a3e40 0x7f36080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.458+0000 7f36137fe640 1 -- 172.21.15.55:0/1817626704 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f36140cf6a0 con 0x7f361c10c2a0 2024-04-26T22:03:06.460 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.459+0000 7f3619d9b640 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.459+0000 7f361ad9d640 1 -- 172.21.15.55:0/4276927908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 msgr2=0x5641b9493da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.459+0000 7f361ad9d640 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 0x5641b9493da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.459+0000 7f361ad9d640 1 -- 172.21.15.55:0/4276927908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 msgr2=0x5641b94968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.459+0000 7f361ad9d640 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.459+0000 7f361ad9d640 1 -- 172.21.15.55:0/4276927908 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b91cd7e0 con 0x5641b948e8b0 2024-04-26T22:03:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.459+0000 7f361ad9d640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36080a3e40 0x7f36080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.462+0000 7f361ad9d640 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b948ecd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f360c00a5d0 tx=0x7f360c00aa90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991c015096c6fb5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:06.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.462+0000 7f3610ff9640 1 -- 172.21.15.55:0/4276927908 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c016940 con 0x5641b948e8b0 2024-04-26T22:03:06.464 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.463+0000 7f3610ff9640 1 -- 172.21.15.55:0/4276927908 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f360c020480 con 0x5641b948e8b0 2024-04-26T22:03:06.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.464+0000 7f362358b740 1 -- 172.21.15.55:0/4276927908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 msgr2=0x5641b948ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.464+0000 7f362358b740 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b948ecd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f360c00a5d0 tx=0x7f360c00aa90 comp rx=0 tx=0).stop 2024-04-26T22:03:06.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.470+0000 7f362358b740 1 -- 172.21.15.55:0/4276927908 shutdown_connections 2024-04-26T22:03:06.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.470+0000 7f362358b740 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.470+0000 7f362358b740 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 0x5641b9493da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.470+0000 7f362358b740 1 --2- 172.21.15.55:0/4276927908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b948ecd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.470+0000 7f362358b740 1 -- 172.21.15.55:0/4276927908 >> 172.21.15.55:0/4276927908 conn(0x5641b9489f20 msgr2=0x5641b948a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:06.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.475+0000 7f361ad9d640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36080a3e40 0x7f36080a6300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f360c019000 tx=0x7f360c01b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:06.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.480+0000 7f362358b740 1 -- 172.21.15.55:0/4276927908 shutdown_connections 2024-04-26T22:03:06.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.486+0000 7f362358b740 1 -- 172.21.15.55:0/4276927908 wait complete. 2024-04-26T22:03:06.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.486+0000 7f362358b740 1 -- start start 2024-04-26T22:03:06.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.486+0000 7f362358b740 1 Processor -- start 2024-04-26T22:03:06.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f362358b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b9498d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f362358b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 0x5641b9499280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f362358b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f362358b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5641b9497410 con 0x5641b9494410 2024-04-26T22:03:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f362358b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5641b938d5d0 con 0x5641b948f2a0 2024-04-26T22:03:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f362358b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641b9497590 con 0x5641b948e8b0 2024-04-26T22:03:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f362358b740 10 client.0 _ll_register_callbacks cb 0x5641b937b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f361a59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b9498d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f361a59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b9498d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36506/0 (socket says 172.21.15.55:36506) 2024-04-26T22:03:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.487+0000 7f361a59c640 1 -- 172.21.15.55:0/1802993508 learned_addr learned my addr 172.21.15.55:0/1802993508 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.489+0000 7f361a59c640 1 -- 172.21.15.55:0/1802993508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 msgr2=0x5641b9499280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.489+0000 7f361a59c640 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 0x5641b9499280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.489+0000 7f361a59c640 1 -- 172.21.15.55:0/1802993508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 msgr2=0x5641b94a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.489+0000 7f361a59c640 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.489+0000 7f361a59c640 1 -- 172.21.15.55:0/1802993508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b922df70 con 0x5641b948e8b0 2024-04-26T22:03:06.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.492+0000 7f361a59c640 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b9498d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3604009900 tx=0x7f3604004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=463b267d91cca76a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:06.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.492+0000 7f3601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:03:06.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.494+0000 7f3601ffb640 1 -- 172.21.15.55:0/1802993508 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604035030 con 0x5641b948e8b0 2024-04-26T22:03:06.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.494+0000 7f362358b740 1 -- 172.21.15.55:0/1802993508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f360c020480 con 0x5641b948e8b0 2024-04-26T22:03:06.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.494+0000 7f3601ffb640 1 -- 172.21.15.55:0/1802993508 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3604003b00 con 0x5641b948e8b0 2024-04-26T22:03:06.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.494+0000 7f3601ffb640 1 -- 172.21.15.55:0/1802993508 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3604003d00 con 0x5641b948e8b0 2024-04-26T22:03:06.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.495+0000 7f362358b740 10 client.26095 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:06.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.495+0000 7f362358b740 1 -- 172.21.15.55:0/1802993508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5641b90c5170 con 0x5641b948e8b0 2024-04-26T22:03:06.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.497+0000 7f3601ffb640 1 -- 172.21.15.55:0/1802993508 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f360c020480 con 0x5641b948e8b0 2024-04-26T22:03:06.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.497+0000 7f3601ffb640 1 -- 172.21.15.55:0/1802993508 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f360403d780 con 0x5641b948e8b0 2024-04-26T22:03:06.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.497+0000 7f362358b740 10 client.26095 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:06.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.497+0000 7f362358b740 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:06.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.497+0000 7f362358b740 20 client.26095 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:06.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.497+0000 7f362358b740 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641b94a4c10 0x5641b94a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.497+0000 7f362358b740 4 client.26095 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:06.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.497+0000 7f362358b740 1 -- 172.21.15.55:0/1802993508 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b9401710 con 0x5641b94a4c10 2024-04-26T22:03:06.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.498+0000 7f3619d9b640 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641b94a4c10 0x5641b94a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.510+0000 7f3601ffb640 10 client.26095 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:06.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.510+0000 7f3619d9b640 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641b94a4c10 0x5641b94a70b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f890752bc5cff96 server_cookie=adcd533d4ef547bf in_seq=0 out_seq=0 2024-04-26T22:03:06.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.515+0000 7f3601ffb640 1 -- 172.21.15.55:0/1802993508 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641b9401710 con 0x5641b94a4c10 2024-04-26T22:03:06.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.515+0000 7f3601ffb640 10 client.26095 handle_command_reply: tid=0 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.516+0000 7f362358b740 1 client.26095 shutdown 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f362358b740 1 -- 172.21.15.55:0/1802993508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 msgr2=0x5641b9498d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f362358b740 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b9498d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3604009900 tx=0x7f3604004600 comp rx=0 tx=0).stop 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f362358b740 1 -- 172.21.15.55:0/1802993508 shutdown_connections 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f362358b740 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641b94a4c10 0x5641b94a70b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f362358b740 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f362358b740 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948f2a0 0x5641b9499280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f362358b740 1 --2- 172.21.15.55:0/1802993508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948e8b0 0x5641b9498d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f362358b740 1 -- 172.21.15.55:0/1802993508 >> 172.21.15.55:0/1802993508 conn(0x5641b9489f20 msgr2=0x5641b94890e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.517+0000 7f3601ffb640 0 client.26095 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:06.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.519+0000 7f362358b740 1 -- 172.21.15.55:0/1802993508 shutdown_connections 2024-04-26T22:03:06.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.519+0000 7f362358b740 1 -- 172.21.15.55:0/1802993508 wait complete. 2024-04-26T22:03:06.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.519+0000 7f362358b740 20 client.26095 trim_cache size 0 max 16384 2024-04-26T22:03:06.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.583+0000 7f362358b740 1 Processor -- start 2024-04-26T22:03:06.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.584+0000 7f362358b740 1 -- start start 2024-04-26T22:03:06.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.584+0000 7f362358b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 0x5641b9498360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.584+0000 7f362358b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b94a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.584+0000 7f362358b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94988a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.584+0000 7f362358b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1c40b0 con 0x5641b9494410 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.584+0000 7f362358b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1c1980 con 0x5641b948e8b0 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.584+0000 7f362358b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f361c118a00 con 0x5641b948f2a0 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.585+0000 7f3619d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b94a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.585+0000 7f3619d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b94a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36510/0 (socket says 172.21.15.55:36510) 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.585+0000 7f3619d9b640 1 -- 172.21.15.55:0/2611844076 learned_addr learned my addr 172.21.15.55:0/2611844076 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.585+0000 7f361ad9d640 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.585+0000 7f361a59c640 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 0x5641b9498360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.586+0000 7f3619d9b640 1 -- 172.21.15.55:0/2611844076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 msgr2=0x5641b9498360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.586+0000 7f3619d9b640 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 0x5641b9498360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.586+0000 7f3619d9b640 1 -- 172.21.15.55:0/2611844076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 msgr2=0x5641b94988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.586+0000 7f3619d9b640 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.586+0000 7f3619d9b640 1 -- 172.21.15.55:0/2611844076 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b94a31e0 con 0x5641b948f2a0 2024-04-26T22:03:06.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.587+0000 7f3619d9b640 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b94a0280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f361409b590 tx=0x7f3614006e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef14582284fee8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:06.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.589+0000 7f3603fff640 1 -- 172.21.15.55:0/2611844076 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614099b90 con 0x5641b948f2a0 2024-04-26T22:03:06.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.589+0000 7f3603fff640 1 -- 172.21.15.55:0/2611844076 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641b9401710 con 0x5641b948f2a0 2024-04-26T22:03:06.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.589+0000 7f3603fff640 1 -- 172.21.15.55:0/2611844076 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614010d90 con 0x5641b948f2a0 2024-04-26T22:03:06.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.590+0000 7f362358b740 1 -- 172.21.15.55:0/2611844076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 msgr2=0x5641b94a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:06.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.590+0000 7f362358b740 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b94a0280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f361409b590 tx=0x7f3614006e60 comp rx=0 tx=0).stop 2024-04-26T22:03:06.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.593+0000 7f362358b740 1 -- 172.21.15.55:0/2611844076 shutdown_connections 2024-04-26T22:03:06.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.593+0000 7f362358b740 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b94988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.593+0000 7f362358b740 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b94a0280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.593+0000 7f362358b740 1 --2- 172.21.15.55:0/2611844076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 0x5641b9498360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.593+0000 7f362358b740 1 -- 172.21.15.55:0/2611844076 >> 172.21.15.55:0/2611844076 conn(0x5641b9489f20 msgr2=0x5641b94890e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:06.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.600+0000 7f362358b740 1 -- 172.21.15.55:0/2611844076 shutdown_connections 2024-04-26T22:03:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.600+0000 7f362358b740 1 -- 172.21.15.55:0/2611844076 wait complete. 2024-04-26T22:03:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.601+0000 7f362358b740 1 -- start start 2024-04-26T22:03:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.601+0000 7f362358b740 1 Processor -- start 2024-04-26T22:03:06.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.601+0000 7f362358b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 0x5641b949ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.602+0000 7f362358b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b949e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.602+0000 7f362358b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b949f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.602+0000 7f362358b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f361c119070 con 0x5641b9494410 2024-04-26T22:03:06.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.602+0000 7f362358b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1c12b0 con 0x5641b948e8b0 2024-04-26T22:03:06.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.602+0000 7f362358b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1c0630 con 0x5641b948f2a0 2024-04-26T22:03:06.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.602+0000 7f362358b740 10 client.0 _ll_register_callbacks cb 0x7f3604004150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:06.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.603+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b949f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.603+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b949f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33708/0 (socket says 172.21.15.55:33708) 2024-04-26T22:03:06.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.603+0000 7f361ad9d640 1 -- 172.21.15.55:0/124464316 learned_addr learned my addr 172.21.15.55:0/124464316 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:06.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.605+0000 7f3619d9b640 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b949e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.605+0000 7f361a59c640 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 0x5641b949ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.610+0000 7f3619d9b640 1 -- 172.21.15.55:0/124464316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 msgr2=0x5641b949ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:06.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.610+0000 7f3619d9b640 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 0x5641b949ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.610+0000 7f3619d9b640 1 -- 172.21.15.55:0/124464316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 msgr2=0x5641b949f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:06.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.610+0000 7f3619d9b640 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b949f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:06.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.610+0000 7f3619d9b640 1 -- 172.21.15.55:0/124464316 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b949f930 con 0x5641b948f2a0 2024-04-26T22:03:06.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.614+0000 7f3619d9b640 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b949e330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3614098260 tx=0x7f361400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b96681510be8bac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:06.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.615+0000 7f3601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:03:06.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.618+0000 7f3601ffb640 1 -- 172.21.15.55:0/124464316 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614017360 con 0x5641b948f2a0 2024-04-26T22:03:06.618 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.618+0000 7f362358b740 1 -- 172.21.15.55:0/124464316 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641b9401710 con 0x5641b948f2a0 2024-04-26T22:03:06.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.618+0000 7f3601ffb640 1 -- 172.21.15.55:0/124464316 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3614097f00 con 0x5641b948f2a0 2024-04-26T22:03:06.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.630+0000 7f3601ffb640 1 -- 172.21.15.55:0/124464316 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f361409cbc0 con 0x5641b948f2a0 2024-04-26T22:03:06.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.630+0000 7f362358b740 10 client.26101 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:06.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.630+0000 7f362358b740 1 -- 172.21.15.55:0/124464316 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5641b950b700 con 0x5641b948f2a0 2024-04-26T22:03:06.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.631+0000 7f3601ffb640 1 -- 172.21.15.55:0/124464316 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36140b9030 con 0x5641b948f2a0 2024-04-26T22:03:06.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.631+0000 7f3601ffb640 1 -- 172.21.15.55:0/124464316 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f361409ae70 con 0x5641b948f2a0 2024-04-26T22:03:06.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.631+0000 7f362358b740 10 client.26101 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:06.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.631+0000 7f362358b740 10 client.26101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:06.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.631+0000 7f362358b740 20 client.26101 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:06.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.631+0000 7f362358b740 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641b950c010 0x5641b950e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:06.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.631+0000 7f362358b740 4 client.26101 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:03:06.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.631+0000 7f362358b740 1 -- 172.21.15.55:0/124464316 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641b93fb580 con 0x5641b950c010 2024-04-26T22:03:06.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.633+0000 7f361a59c640 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641b950c010 0x5641b950e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:06.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.637+0000 7f3601ffb640 10 client.26101 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:06.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:06.637+0000 7f361a59c640 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641b950c010 0x5641b950e430 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1460e506f9351f94 server_cookie=de21793056b44ddc in_seq=0 out_seq=0 2024-04-26T22:03:07.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.011+0000 7f3601ffb640 1 -- 172.21.15.55:0/124464316 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641b93fb580 con 0x5641b950c010 2024-04-26T22:03:07.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.011+0000 7f3601ffb640 10 client.26101 handle_command_reply: tid=0 2024-04-26T22:03:07.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.011+0000 7f362358b740 1 client.26101 shutdown 2024-04-26T22:03:07.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.011+0000 7f362358b740 1 -- 172.21.15.55:0/124464316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 msgr2=0x5641b949e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.012+0000 7f362358b740 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b949e330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3614098260 tx=0x7f361400bf90 comp rx=0 tx=0).stop 2024-04-26T22:03:07.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.012+0000 7f362358b740 1 -- 172.21.15.55:0/124464316 shutdown_connections 2024-04-26T22:03:07.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.012+0000 7f362358b740 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5641b950c010 0x5641b950e430 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.012+0000 7f362358b740 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5641b9494410 0x5641b949f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.012+0000 7f362358b740 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641b948f2a0 0x5641b949e330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.012+0000 7f362358b740 1 --2- 172.21.15.55:0/124464316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641b948e8b0 0x5641b949ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.012+0000 7f362358b740 1 -- 172.21.15.55:0/124464316 >> 172.21.15.55:0/124464316 conn(0x5641b9489f20 msgr2=0x5641b948cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.012+0000 7f3601ffb640 0 client.26101 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:07.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.013+0000 7f362358b740 1 -- 172.21.15.55:0/124464316 shutdown_connections 2024-04-26T22:03:07.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.014+0000 7f362358b740 1 -- 172.21.15.55:0/124464316 wait complete. 2024-04-26T22:03:07.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.014+0000 7f362358b740 20 client.26101 trim_cache size 0 max 16384 2024-04-26T22:03:07.014 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:03:07.015 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:03:07.015 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "29cb6850-47a9-4f83-ad2d-b49bf4166b8c", 2024-04-26T22:03:07.015 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:03:07.015 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 -- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36080a3e40 msgr2=0x7f36080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36080a3e40 0x7f36080a6300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f360c019000 tx=0x7f360c01b000 comp rx=0 tx=0).stop 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 -- 172.21.15.55:0/1817626704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361c10c2a0 msgr2=0x7f361c1a8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361c10c2a0 0x7f361c1a8070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f361400b4a0 tx=0x7f361400b960 comp rx=0 tx=0).stop 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 -- 172.21.15.55:0/1817626704 shutdown_connections 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f36080a3e40 0x7f36080a6300 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f361c10ec50 0x7f361c1af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361c10c2a0 0x7f361c1a8070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 --2- 172.21.15.55:0/1817626704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f361c109930 0x7f361c1a7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.015+0000 7f3620874640 1 -- 172.21.15.55:0/1817626704 >> 172.21.15.55:0/1817626704 conn(0x7f361c06fb60 msgr2=0x7f361c10cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.016+0000 7f3620874640 1 -- 172.21.15.55:0/1817626704 shutdown_connections 2024-04-26T22:03:07.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.016+0000 7f3620874640 1 -- 172.21.15.55:0/1817626704 wait complete. 2024-04-26T22:03:07.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:03:07.037 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:03:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.332+0000 7f0f54023640 1 -- 172.21.15.55:0/2033583756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 msgr2=0x7f0f4c101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.332+0000 7f0f54023640 1 --2- 172.21.15.55:0/2033583756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 0x7f0f4c101da0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009ca0 tx=0x7f0f4402a8f0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.332+0000 7f0f54023640 1 -- 172.21.15.55:0/2033583756 shutdown_connections 2024-04-26T22:03:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.332+0000 7f0f54023640 1 --2- 172.21.15.55:0/2033583756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c1022e0 0x7f0f4c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.332+0000 7f0f54023640 1 --2- 172.21.15.55:0/2033583756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 0x7f0f4c101da0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.332+0000 7f0f54023640 1 --2- 172.21.15.55:0/2033583756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4c100810 0x7f0f4c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.333+0000 7f0f54023640 1 -- 172.21.15.55:0/2033583756 >> 172.21.15.55:0/2033583756 conn(0x7f0f4c0fc130 msgr2=0x7f0f4c0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.338+0000 7f0f54023640 1 -- 172.21.15.55:0/2033583756 shutdown_connections 2024-04-26T22:03:07.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.339+0000 7f0f54023640 1 -- 172.21.15.55:0/2033583756 wait complete. 2024-04-26T22:03:07.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.340+0000 7f0f54023640 1 Processor -- start 2024-04-26T22:03:07.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.342+0000 7f0f54023640 1 -- start start 2024-04-26T22:03:07.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.342+0000 7f0f54023640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4c100810 0x7f0f4c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.342+0000 7f0f54023640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 0x7f0f4c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.342+0000 7f0f54023640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c1022e0 0x7f0f4c1a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.342+0000 7f0f54023640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c114be0 con 0x7f0f4c101940 2024-04-26T22:03:07.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.342+0000 7f0f54023640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c114a60 con 0x7f0f4c100810 2024-04-26T22:03:07.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.342+0000 7f0f54023640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c114d60 con 0x7f0f4c1022e0 2024-04-26T22:03:07.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.343+0000 7f0f515be640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 0x7f0f4c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.343+0000 7f0f515be640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 0x7f0f4c19f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33730/0 (socket says 172.21.15.55:33730) 2024-04-26T22:03:07.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.343+0000 7f0f515be640 1 -- 172.21.15.55:0/3450818573 learned_addr learned my addr 172.21.15.55:0/3450818573 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.345+0000 7f0f51dbf640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4c100810 0x7f0f4c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.346+0000 7f0f525c0640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c1022e0 0x7f0f4c1a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.352+0000 7f0f515be640 1 -- 172.21.15.55:0/3450818573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c1022e0 msgr2=0x7f0f4c1a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.352+0000 7f0f515be640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c1022e0 0x7f0f4c1a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.352+0000 7f0f515be640 1 -- 172.21.15.55:0/3450818573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4c100810 msgr2=0x7f0f4c19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.352+0000 7f0f515be640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4c100810 0x7f0f4c19f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.352+0000 7f0f515be640 1 -- 172.21.15.55:0/3450818573 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c1a6f80 con 0x7f0f4c101940 2024-04-26T22:03:07.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.353+0000 7f0f515be640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 0x7f0f4c19f8f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0f440098d0 tx=0x7f0f44003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:07.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.360+0000 7f0f42ffd640 1 -- 172.21.15.55:0/3450818573 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44004040 con 0x7f0f4c101940 2024-04-26T22:03:07.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.361+0000 7f0f42ffd640 1 -- 172.21.15.55:0/3450818573 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f4402d070 con 0x7f0f4c101940 2024-04-26T22:03:07.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.361+0000 7f0f42ffd640 1 -- 172.21.15.55:0/3450818573 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4402fc60 con 0x7f0f4c101940 2024-04-26T22:03:07.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.362+0000 7f0f54023640 1 -- 172.21.15.55:0/3450818573 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c1a7210 con 0x7f0f4c101940 2024-04-26T22:03:07.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.362+0000 7f0f54023640 1 -- 172.21.15.55:0/3450818573 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c1a7790 con 0x7f0f4c101940 2024-04-26T22:03:07.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.367+0000 7f0f56e1d740 1 Processor -- start 2024-04-26T22:03:07.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.369+0000 7f0f56e1d740 1 -- start start 2024-04-26T22:03:07.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.369+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a877300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.369+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 0x562f2a878530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.369+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 0x562f2a87f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.369+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562f2a7dbd80 con 0x562f2a876ee0 2024-04-26T22:03:07.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.369+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562f2a7958b0 con 0x562f2a8780b0 2024-04-26T22:03:07.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.369+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562f2a7d7d40 con 0x562f2a878a70 2024-04-26T22:03:07.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.371+0000 7f0f525c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a877300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.371+0000 7f0f525c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a877300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33738/0 (socket says 172.21.15.55:33738) 2024-04-26T22:03:07.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.371+0000 7f0f525c0640 1 -- 172.21.15.55:0/3939689470 learned_addr learned my addr 172.21.15.55:0/3939689470 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:07.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.371+0000 7f0f525c0640 1 -- 172.21.15.55:0/3939689470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 msgr2=0x562f2a87f300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:03:07.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.371+0000 7f0f525c0640 1 --2- 172.21.15.55:0/3939689470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 0x562f2a87f300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.371+0000 7f0f525c0640 1 -- 172.21.15.55:0/3939689470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 msgr2=0x562f2a878530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:07.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.371+0000 7f0f525c0640 1 --2- 172.21.15.55:0/3939689470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 0x562f2a878530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.371+0000 7f0f525c0640 1 -- 172.21.15.55:0/3939689470 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f2a766a40 con 0x562f2a876ee0 2024-04-26T22:03:07.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.372+0000 7f0f525c0640 1 --2- 172.21.15.55:0/3939689470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a877300 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0099c0 tx=0x7f0f3c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac4e4fb3e1a335 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.373+0000 7f0f37fff640 1 -- 172.21.15.55:0/3939689470 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c017070 con 0x562f2a876ee0 2024-04-26T22:03:07.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.374+0000 7f0f37fff640 1 -- 172.21.15.55:0/3939689470 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3c013470 con 0x562f2a876ee0 2024-04-26T22:03:07.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.375+0000 7f0f42ffd640 1 -- 172.21.15.55:0/3450818573 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f440041e0 con 0x7f0f4c101940 2024-04-26T22:03:07.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.375+0000 7f0f56e1d740 1 -- 172.21.15.55:0/3939689470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 msgr2=0x562f2a877300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.375+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/3939689470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a877300 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0099c0 tx=0x7f0f3c009e80 comp rx=0 tx=0).stop 2024-04-26T22:03:07.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.377+0000 7f0f42ffd640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f380a3e40 0x7f0f380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.377+0000 7f0f42ffd640 1 -- 172.21.15.55:0/3450818573 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0f440f1d40 con 0x7f0f4c101940 2024-04-26T22:03:07.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.378+0000 7f0f525c0640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f380a3e40 0x7f0f380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.380+0000 7f0f56e1d740 1 -- 172.21.15.55:0/3939689470 shutdown_connections 2024-04-26T22:03:07.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.380+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/3939689470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 0x562f2a87f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.380+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/3939689470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 0x562f2a878530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.380+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/3939689470 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a877300 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.380+0000 7f0f56e1d740 1 -- 172.21.15.55:0/3939689470 >> 172.21.15.55:0/3939689470 conn(0x562f2a8723d0 msgr2=0x562f2a8727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.381+0000 7f0f525c0640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f380a3e40 0x7f0f380a6300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009780 tx=0x7f0f3c00efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:07.382 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.382+0000 7f0f56e1d740 1 -- 172.21.15.55:0/3939689470 shutdown_connections 2024-04-26T22:03:07.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.382+0000 7f0f56e1d740 1 -- 172.21.15.55:0/3939689470 wait complete. 2024-04-26T22:03:07.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 1 -- start start 2024-04-26T22:03:07.383 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 1 Processor -- start 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a8923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 0x562f2a892920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 0x562f2a899960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x562f2a890ae0 con 0x562f2a876ee0 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562f2a762d10 con 0x562f2a8780b0 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562f2a890c60 con 0x562f2a878a70 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.383+0000 7f0f56e1d740 10 client.0 _ll_register_callbacks cb 0x562f2a78f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.384+0000 7f0f51dbf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a8923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.384+0000 7f0f51dbf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a8923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33746/0 (socket says 172.21.15.55:33746) 2024-04-26T22:03:07.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.384+0000 7f0f51dbf640 1 -- 172.21.15.55:0/82661878 learned_addr learned my addr 172.21.15.55:0/82661878 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:07.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.388+0000 7f0f525c0640 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 0x562f2a899960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.388+0000 7f0f515be640 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 0x562f2a892920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.389+0000 7f0f51dbf640 1 -- 172.21.15.55:0/82661878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 msgr2=0x562f2a899960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:07.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.389+0000 7f0f51dbf640 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 0x562f2a899960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.389+0000 7f0f51dbf640 1 -- 172.21.15.55:0/82661878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 msgr2=0x562f2a892920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:07.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.389+0000 7f0f51dbf640 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 0x562f2a892920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.389+0000 7f0f51dbf640 1 -- 172.21.15.55:0/82661878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f2a5b77d0 con 0x562f2a876ee0 2024-04-26T22:03:07.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.390+0000 7f0f51dbf640 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a8923e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e450 tx=0x7f0f4800e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6883608572254d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:07.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.390+0000 7f0f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:07.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.390+0000 7f0f357fa640 1 -- 172.21.15.55:0/82661878 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48017070 con 0x562f2a876ee0 2024-04-26T22:03:07.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.390+0000 7f0f56e1d740 1 -- 172.21.15.55:0/82661878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3c013470 con 0x562f2a876ee0 2024-04-26T22:03:07.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.391+0000 7f0f357fa640 1 -- 172.21.15.55:0/82661878 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0f48013680 con 0x562f2a876ee0 2024-04-26T22:03:07.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.391+0000 7f0f357fa640 1 -- 172.21.15.55:0/82661878 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0f48023050 con 0x562f2a876ee0 2024-04-26T22:03:07.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.391+0000 7f0f56e1d740 10 client.17052 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:07.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.391+0000 7f0f56e1d740 1 -- 172.21.15.55:0/82661878 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562f2a4ed3f0 con 0x562f2a876ee0 2024-04-26T22:03:07.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.392+0000 7f0f357fa640 1 -- 172.21.15.55:0/82661878 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3c013470 con 0x562f2a876ee0 2024-04-26T22:03:07.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.392+0000 7f0f357fa640 1 -- 172.21.15.55:0/82661878 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0f480129f0 con 0x562f2a876ee0 2024-04-26T22:03:07.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.392+0000 7f0f56e1d740 10 client.17052 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:07.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.392+0000 7f0f56e1d740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:07.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.392+0000 7f0f56e1d740 20 client.17052 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:07.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.392+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562f2a89e300 0x562f2a8a0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.392+0000 7f0f56e1d740 4 client.17052 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:07.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.392+0000 7f0f56e1d740 1 -- 172.21.15.55:0/82661878 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f2a7e9990 con 0x562f2a89e300 2024-04-26T22:03:07.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.407+0000 7f0f515be640 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562f2a89e300 0x562f2a8a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.415+0000 7f0f515be640 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562f2a89e300 0x562f2a8a0780 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1aa866afe3cc64 server_cookie=e6f8a8fe025e927a in_seq=0 out_seq=0 2024-04-26T22:03:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.415+0000 7f0f357fa640 10 client.17052 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:07.425 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.422+0000 7f0f357fa640 1 -- 172.21.15.55:0/82661878 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562f2a7e9990 con 0x562f2a89e300 2024-04-26T22:03:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.422+0000 7f0f357fa640 10 client.17052 handle_command_reply: tid=0 2024-04-26T22:03:07.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.422+0000 7f0f56e1d740 1 client.17052 shutdown 2024-04-26T22:03:07.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 -- 172.21.15.55:0/82661878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 msgr2=0x562f2a8923e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.427 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a8923e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e450 tx=0x7f0f4800e910 comp rx=0 tx=0).stop 2024-04-26T22:03:07.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 -- 172.21.15.55:0/82661878 shutdown_connections 2024-04-26T22:03:07.428 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a878a70 0x562f2a899960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a8780b0 0x562f2a892920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562f2a89e300 0x562f2a8a0780 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/82661878 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a876ee0 0x562f2a8923e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 -- 172.21.15.55:0/82661878 >> 172.21.15.55:0/82661878 conn(0x562f2a8723d0 msgr2=0x562f2a871550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f357fa640 0 client.17052 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:07.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 -- 172.21.15.55:0/82661878 shutdown_connections 2024-04-26T22:03:07.430 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.427+0000 7f0f56e1d740 1 -- 172.21.15.55:0/82661878 wait complete. 2024-04-26T22:03:07.432 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.432+0000 7f0f56e1d740 20 client.17052 trim_cache size 0 max 16384 2024-04-26T22:03:07.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.470+0000 7f0f56e1d740 1 Processor -- start 2024-04-26T22:03:07.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.470+0000 7f0f56e1d740 1 -- start start 2024-04-26T22:03:07.478 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.471+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a876ee0 0x562f2a87baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.471+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a8780b0 0x562f2a87c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.471+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a878a70 0x562f2a8998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.471+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0f30010fd0 con 0x562f2a8780b0 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.471+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000e8a0 con 0x562f2a878a70 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.471+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000c6f0 con 0x562f2a876ee0 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.471+0000 7f0f515be640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a8780b0 0x562f2a87c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.472+0000 7f0f515be640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a8780b0 0x562f2a87c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33758/0 (socket says 172.21.15.55:33758) 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.472+0000 7f0f515be640 1 -- 172.21.15.55:0/2585207836 learned_addr learned my addr 172.21.15.55:0/2585207836 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.472+0000 7f0f515be640 1 -- 172.21.15.55:0/2585207836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a876ee0 msgr2=0x562f2a87baf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.472+0000 7f0f515be640 1 --2- 172.21.15.55:0/2585207836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a876ee0 0x562f2a87baf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.479 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.472+0000 7f0f515be640 1 -- 172.21.15.55:0/2585207836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a878a70 msgr2=0x562f2a8998f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.472+0000 7f0f515be640 1 --2- 172.21.15.55:0/2585207836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a878a70 0x562f2a8998f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.472+0000 7f0f515be640 1 -- 172.21.15.55:0/2585207836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f2a4ad0f0 con 0x562f2a8780b0 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.473+0000 7f0f515be640 1 --2- 172.21.15.55:0/2585207836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a8780b0 0x562f2a87c030 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0f440ba390 tx=0x7f0f44003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27b59aaf68dd4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.473+0000 7f0f377fe640 1 -- 172.21.15.55:0/2585207836 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440bdf30 con 0x562f2a8780b0 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.473+0000 7f0f377fe640 1 -- 172.21.15.55:0/2585207836 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f44036030 con 0x562f2a8780b0 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.473+0000 7f0f377fe640 1 -- 172.21.15.55:0/2585207836 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f440bc2c0 con 0x562f2a8780b0 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.474+0000 7f0f56e1d740 1 -- 172.21.15.55:0/2585207836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a8780b0 msgr2=0x562f2a87c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.474+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/2585207836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a8780b0 0x562f2a87c030 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0f440ba390 tx=0x7f0f44003f90 comp rx=0 tx=0).stop 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.475+0000 7f0f56e1d740 1 -- 172.21.15.55:0/2585207836 shutdown_connections 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.475+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/2585207836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a878a70 0x562f2a8998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.475+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/2585207836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a8780b0 0x562f2a87c030 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.475+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/2585207836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a876ee0 0x562f2a87baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.475+0000 7f0f56e1d740 1 -- 172.21.15.55:0/2585207836 >> 172.21.15.55:0/2585207836 conn(0x562f2a8723d0 msgr2=0x562f2a8727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.475+0000 7f0f56e1d740 1 -- 172.21.15.55:0/2585207836 shutdown_connections 2024-04-26T22:03:07.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.475+0000 7f0f56e1d740 1 -- 172.21.15.55:0/2585207836 wait complete. 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.476+0000 7f0f56e1d740 1 -- start start 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.476+0000 7f0f56e1d740 1 Processor -- start 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a876ee0 0x562f2a896600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a8780b0 0x562f2a896b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f56e1d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a878a70 0x562f2a9028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000cd60 con 0x562f2a878a70 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000e1d0 con 0x562f2a876ee0 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f56e1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3000d430 con 0x562f2a8780b0 2024-04-26T22:03:07.481 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f56e1d740 10 client.0 _ll_register_callbacks cb 0x7f0f480093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f525c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a878a70 0x562f2a9028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f525c0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a878a70 0x562f2a9028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33766/0 (socket says 172.21.15.55:33766) 2024-04-26T22:03:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.477+0000 7f0f525c0640 1 -- 172.21.15.55:0/704960235 learned_addr learned my addr 172.21.15.55:0/704960235 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.478+0000 7f0f525c0640 1 -- 172.21.15.55:0/704960235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a8780b0 msgr2=0x562f2a896b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:07.483 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.478+0000 7f0f525c0640 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a8780b0 0x562f2a896b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.478+0000 7f0f525c0640 1 -- 172.21.15.55:0/704960235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a876ee0 msgr2=0x562f2a896600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.478+0000 7f0f525c0640 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a876ee0 0x562f2a896600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.478+0000 7f0f525c0640 1 -- 172.21.15.55:0/704960235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f2a79c4b0 con 0x562f2a878a70 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.478+0000 7f0f525c0640 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a878a70 0x562f2a9028f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c004540 tx=0x7f0f3c009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29bd6789fcf10da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.478+0000 7f0f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.479+0000 7f0f357fa640 1 -- 172.21.15.55:0/704960235 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c02e070 con 0x562f2a878a70 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.479+0000 7f0f357fa640 1 -- 172.21.15.55:0/704960235 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0f3c003890 con 0x562f2a878a70 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.479+0000 7f0f56e1d740 1 -- 172.21.15.55:0/704960235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f44036030 con 0x562f2a878a70 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.479+0000 7f0f357fa640 1 -- 172.21.15.55:0/704960235 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c017030 con 0x562f2a878a70 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.479+0000 7f0f56e1d740 10 client.17058 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.479+0000 7f0f56e1d740 1 -- 172.21.15.55:0/704960235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562f2a903cd0 con 0x562f2a878a70 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.480+0000 7f0f357fa640 1 -- 172.21.15.55:0/704960235 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f44036030 con 0x562f2a878a70 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.480+0000 7f0f357fa640 1 -- 172.21.15.55:0/704960235 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0f3c006080 con 0x562f2a878a70 2024-04-26T22:03:07.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.480+0000 7f0f56e1d740 10 client.17058 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.480+0000 7f0f56e1d740 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.480+0000 7f0f56e1d740 20 client.17058 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.480+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562f2a904440 0x562f2a906860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.480+0000 7f0f56e1d740 4 client.17058 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:03:07.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.480+0000 7f0f56e1d740 1 -- 172.21.15.55:0/704960235 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f2a7e3800 con 0x562f2a904440 2024-04-26T22:03:07.487 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.484+0000 7f0f51dbf640 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562f2a904440 0x562f2a906860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:07.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.490+0000 7f0f357fa640 10 client.17058 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:07.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.490+0000 7f0f51dbf640 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562f2a904440 0x562f2a906860 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26a99437f40cd8 server_cookie=aa5c0c1d58d154f7 in_seq=0 out_seq=0 2024-04-26T22:03:07.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.494+0000 7f0f357fa640 1 -- 172.21.15.55:0/704960235 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x562f2a7e3800 con 0x562f2a904440 2024-04-26T22:03:07.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f357fa640 10 client.17058 handle_command_reply: tid=0 2024-04-26T22:03:07.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 client.17058 shutdown 2024-04-26T22:03:07.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 -- 172.21.15.55:0/704960235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a878a70 msgr2=0x562f2a9028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a878a70 0x562f2a9028f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c004540 tx=0x7f0f3c009060 comp rx=0 tx=0).stop 2024-04-26T22:03:07.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 -- 172.21.15.55:0/704960235 shutdown_connections 2024-04-26T22:03:07.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x562f2a904440 0x562f2a906860 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f357fa640 0 client.17058 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:07.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562f2a878a70 0x562f2a9028f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562f2a8780b0 0x562f2a896b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 --2- 172.21.15.55:0/704960235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562f2a876ee0 0x562f2a896600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.495+0000 7f0f56e1d740 1 -- 172.21.15.55:0/704960235 >> 172.21.15.55:0/704960235 conn(0x562f2a8723d0 msgr2=0x562f2a87baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.496+0000 7f0f56e1d740 1 -- 172.21.15.55:0/704960235 shutdown_connections 2024-04-26T22:03:07.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.496+0000 7f0f56e1d740 1 -- 172.21.15.55:0/704960235 wait complete. 2024-04-26T22:03:07.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.496+0000 7f0f56e1d740 20 client.17058 trim_cache size 0 max 16384 2024-04-26T22:03:07.500 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:03:07.500 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (2 inodes in the stack)", 2024-04-26T22:03:07.500 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:03:07.500 INFO:teuthology.orchestra.run.smithi055.stdout: "29cb6850-47a9-4f83-ad2d-b49bf4166b8c": { 2024-04-26T22:03:07.500 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:03:07.501 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "29cb6850-47a9-4f83-ad2d-b49bf4166b8c", 2024-04-26T22:03:07.501 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:03:07.501 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:03:07.501 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:03:07.501 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:03:07.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 -- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f380a3e40 msgr2=0x7f0f380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f380a3e40 0x7f0f380a6300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009780 tx=0x7f0f3c00efd0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 -- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 msgr2=0x7f0f4c19f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 0x7f0f4c19f8f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0f440098d0 tx=0x7f0f44003ba0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 -- 172.21.15.55:0/3450818573 shutdown_connections 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0f380a3e40 0x7f0f380a6300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f4c1022e0 0x7f0f4c1a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f4c101940 0x7f0f4c19f8f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 --2- 172.21.15.55:0/3450818573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f4c100810 0x7f0f4c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.497+0000 7f0f54023640 1 -- 172.21.15.55:0/3450818573 >> 172.21.15.55:0/3450818573 conn(0x7f0f4c0fc130 msgr2=0x7f0f4c102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.498+0000 7f0f54023640 1 -- 172.21.15.55:0/3450818573 shutdown_connections 2024-04-26T22:03:07.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:07.498+0000 7f0f54023640 1 -- 172.21.15.55:0/3450818573 wait complete. 2024-04-26T22:03:07.521 INFO:tasks.cephfs.filesystem:scrub status for tag:29cb6850-47a9-4f83-ad2d-b49bf4166b8c - {'path': '/', 'tag': '29cb6850-47a9-4f83-ad2d-b49bf4166b8c', 'options': 'recursive,force'} 2024-04-26T22:03:07.878 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:07 smithi165 ceph-mon[33296]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 793 MiB/s wr, 1.29k op/s 2024-04-26T22:03:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:07 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:03:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:07 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:07.879 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:07 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:03:07.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:07 smithi055 ceph-mon[29221]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 793 MiB/s wr, 1.29k op/s 2024-04-26T22:03:07.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:07 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:03:07.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:07 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:07.939 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:07 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:03:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:07 smithi064 ceph-mon[32765]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 793 MiB/s wr, 1.29k op/s 2024-04-26T22:03:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:07 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:03:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:07 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:08.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:07 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:03:09.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:09 smithi055 ceph-mon[29221]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 867 MiB/s wr, 1.74k op/s 2024-04-26T22:03:09.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:09 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:03:09.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:09 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:10.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:09 smithi064 ceph-mon[32765]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 867 MiB/s wr, 1.74k op/s 2024-04-26T22:03:10.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:09 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:03:10.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:09 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:10.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:09 smithi165 ceph-mon[33296]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 867 MiB/s wr, 1.74k op/s 2024-04-26T22:03:10.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:09 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:03:10.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:09 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:11.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:11 smithi055 ceph-mon[29221]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 831 MiB/s wr, 1.93k op/s 2024-04-26T22:03:12.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:11 smithi064 ceph-mon[32765]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 831 MiB/s wr, 1.93k op/s 2024-04-26T22:03:12.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:11 smithi165 ceph-mon[33296]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 831 MiB/s wr, 1.93k op/s 2024-04-26T22:03:12.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:12 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:13.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:12 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:13.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:12 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:13.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:13 smithi055 ceph-mon[29221]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 889 MiB/s wr, 2.21k op/s 2024-04-26T22:03:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:13 smithi064 ceph-mon[32765]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 889 MiB/s wr, 2.21k op/s 2024-04-26T22:03:14.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:13 smithi165 ceph-mon[33296]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 889 MiB/s wr, 2.21k op/s 2024-04-26T22:03:15.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:15 smithi055 ceph-mon[29221]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 892 MiB/s wr, 2.32k op/s 2024-04-26T22:03:16.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:15 smithi064 ceph-mon[32765]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 892 MiB/s wr, 2.32k op/s 2024-04-26T22:03:16.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:15 smithi165 ceph-mon[33296]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 892 MiB/s wr, 2.32k op/s 2024-04-26T22:03:17.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:16 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:03:17.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:16 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:03:17.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:16 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:03:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:17 smithi064 ceph-mon[32765]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 771 MiB/s wr, 2.05k op/s 2024-04-26T22:03:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:17 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:03:18.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:17 smithi165 ceph-mon[33296]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 771 MiB/s wr, 2.05k op/s 2024-04-26T22:03:18.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:17 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:03:18.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:17 smithi055 ceph-mon[29221]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 771 MiB/s wr, 2.05k op/s 2024-04-26T22:03:18.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:17 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:03:20.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:19 smithi064 ceph-mon[32765]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 MiB/s wr, 2.14k op/s 2024-04-26T22:03:20.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:19 smithi165 ceph-mon[33296]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 MiB/s wr, 2.14k op/s 2024-04-26T22:03:20.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:19 smithi055 ceph-mon[29221]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 876 MiB/s wr, 2.14k op/s 2024-04-26T22:03:22.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:21 smithi064 ceph-mon[32765]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.83k op/s 2024-04-26T22:03:22.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:21 smithi165 ceph-mon[33296]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.83k op/s 2024-04-26T22:03:22.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:21 smithi055 ceph-mon[29221]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 1.83k op/s 2024-04-26T22:03:24.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:23 smithi064 ceph-mon[32765]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 866 MiB/s wr, 1.65k op/s 2024-04-26T22:03:24.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:23 smithi165 ceph-mon[33296]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 866 MiB/s wr, 1.65k op/s 2024-04-26T22:03:24.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:23 smithi055 ceph-mon[29221]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 866 MiB/s wr, 1.65k op/s 2024-04-26T22:03:26.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:25 smithi064 ceph-mon[32765]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 873 MiB/s wr, 1.46k op/s 2024-04-26T22:03:26.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:25 smithi165 ceph-mon[33296]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 873 MiB/s wr, 1.46k op/s 2024-04-26T22:03:26.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:25 smithi055 ceph-mon[29221]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 873 MiB/s wr, 1.46k op/s 2024-04-26T22:03:27.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:26 smithi165 ceph-mon[33296]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 770 MiB/s wr, 1.20k op/s 2024-04-26T22:03:27.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:26 smithi055 ceph-mon[29221]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 770 MiB/s wr, 1.20k op/s 2024-04-26T22:03:27.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:26 smithi064 ceph-mon[32765]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 770 MiB/s wr, 1.20k op/s 2024-04-26T22:03:29.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:28 smithi055 ceph-mon[29221]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 881 MiB/s wr, 1.36k op/s 2024-04-26T22:03:29.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:28 smithi064 ceph-mon[32765]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 881 MiB/s wr, 1.36k op/s 2024-04-26T22:03:29.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:28 smithi165 ceph-mon[33296]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 881 MiB/s wr, 1.36k op/s 2024-04-26T22:03:31.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:30 smithi055 ceph-mon[29221]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 857 MiB/s wr, 1.29k op/s 2024-04-26T22:03:31.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:30 smithi064 ceph-mon[32765]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 857 MiB/s wr, 1.29k op/s 2024-04-26T22:03:31.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:30 smithi165 ceph-mon[33296]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 857 MiB/s wr, 1.29k op/s 2024-04-26T22:03:33.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:32 smithi055 ceph-mon[29221]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 1.22k op/s 2024-04-26T22:03:33.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:32 smithi064 ceph-mon[32765]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 1.22k op/s 2024-04-26T22:03:33.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:32 smithi165 ceph-mon[33296]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 1.22k op/s 2024-04-26T22:03:35.188 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:34 smithi055 ceph-mon[29221]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 874 MiB/s wr, 1.15k op/s 2024-04-26T22:03:35.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:34 smithi064 ceph-mon[32765]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 874 MiB/s wr, 1.15k op/s 2024-04-26T22:03:35.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:34 smithi165 ceph-mon[33296]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 874 MiB/s wr, 1.15k op/s 2024-04-26T22:03:37.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:36 smithi064 ceph-mon[32765]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 770 MiB/s wr, 990 op/s 2024-04-26T22:03:37.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:36 smithi165 ceph-mon[33296]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 770 MiB/s wr, 990 op/s 2024-04-26T22:03:37.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:36 smithi055 ceph-mon[29221]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 770 MiB/s wr, 990 op/s 2024-04-26T22:03:37.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:03:37.522 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:03:37.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.891+0000 7fe65a4fd640 1 -- 172.21.15.55:0/2491842750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 msgr2=0x7fe65410a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:37.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.891+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/2491842750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 0x7fe65410a470 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe650009900 tx=0x7fe65002a440 comp rx=0 tx=0).stop 2024-04-26T22:03:37.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.899+0000 7fe65a4fd640 1 -- 172.21.15.55:0/2491842750 shutdown_connections 2024-04-26T22:03:37.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.899+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/2491842750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe65410a9b0 0x7fe65410f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.899+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/2491842750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 0x7fe65410a470 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.899+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/2491842750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe654073d70 0x7fe654074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.899+0000 7fe65a4fd640 1 -- 172.21.15.55:0/2491842750 >> 172.21.15.55:0/2491842750 conn(0x7fe65406f840 msgr2=0x7fe654071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:37.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.905+0000 7fe65a4fd640 1 -- 172.21.15.55:0/2491842750 shutdown_connections 2024-04-26T22:03:37.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.908+0000 7fe65a4fd640 1 -- 172.21.15.55:0/2491842750 wait complete. 2024-04-26T22:03:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.913+0000 7fe65a4fd640 1 Processor -- start 2024-04-26T22:03:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.914+0000 7fe65a4fd640 1 -- start start 2024-04-26T22:03:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.914+0000 7fe65a4fd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe654073d70 0x7fe6541a78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.914+0000 7fe65a4fd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 0x7fe6541a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.914+0000 7fe65a4fd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe65410a9b0 0x7fe6541aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.914+0000 7fe65a4fd640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe65411d200 con 0x7fe654073d70 2024-04-26T22:03:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.914+0000 7fe65a4fd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe65411d080 con 0x7fe65410a010 2024-04-26T22:03:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.914+0000 7fe65a4fd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe65411d380 con 0x7fe65410a9b0 2024-04-26T22:03:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.922+0000 7fe6594fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe654073d70 0x7fe6541a78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.922+0000 7fe6594fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe654073d70 0x7fe6541a78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50750/0 (socket says 172.21.15.55:50750) 2024-04-26T22:03:37.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.922+0000 7fe6594fb640 1 -- 172.21.15.55:0/4140917681 learned_addr learned my addr 172.21.15.55:0/4140917681 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.925+0000 7fe658cfa640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 0x7fe6541a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.925+0000 7fe659cfc640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe65410a9b0 0x7fe6541aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.928+0000 7fe658cfa640 1 -- 172.21.15.55:0/4140917681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe65410a9b0 msgr2=0x7fe6541aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:37.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.928+0000 7fe658cfa640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe65410a9b0 0x7fe6541aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.928+0000 7fe658cfa640 1 -- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe654073d70 msgr2=0x7fe6541a78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:37.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.928+0000 7fe658cfa640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe654073d70 0x7fe6541a78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.928+0000 7fe658cfa640 1 -- 172.21.15.55:0/4140917681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe654111190 con 0x7fe65410a010 2024-04-26T22:03:37.930 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.930+0000 7fe658cfa640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 0x7fe6541a7de0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe650009230 tx=0x7fe650003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:37.932 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.931+0000 7fe64a7fc640 1 -- 172.21.15.55:0/4140917681 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650040030 con 0x7fe65410a010 2024-04-26T22:03:37.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.932+0000 7fe65a4fd640 1 -- 172.21.15.55:0/4140917681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe654111420 con 0x7fe65410a010 2024-04-26T22:03:37.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.932+0000 7fe65a4fd640 1 -- 172.21.15.55:0/4140917681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6541119e0 con 0x7fe65410a010 2024-04-26T22:03:37.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.937+0000 7fe65e47a740 1 Processor -- start 2024-04-26T22:03:37.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.937+0000 7fe64a7fc640 1 -- 172.21.15.55:0/4140917681 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe650005160 con 0x7fe65410a010 2024-04-26T22:03:37.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.937+0000 7fe64a7fc640 1 -- 172.21.15.55:0/4140917681 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65003c7c0 con 0x7fe65410a010 2024-04-26T22:03:37.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.940+0000 7fe65e47a740 1 -- start start 2024-04-26T22:03:37.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.940+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.941+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.941+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 0x556e868be4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.941+0000 7fe65e47a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556e8681ad80 con 0x556e868b7280 2024-04-26T22:03:37.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.941+0000 7fe65e47a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556e867d48b0 con 0x556e868b60b0 2024-04-26T22:03:37.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.941+0000 7fe65e47a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556e86816d40 con 0x556e868b7c40 2024-04-26T22:03:37.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.946+0000 7fe64a7fc640 1 -- 172.21.15.55:0/4140917681 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe650043030 con 0x7fe65410a010 2024-04-26T22:03:37.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.946+0000 7fe659cfc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.948+0000 7fe64a7fc640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe6400a3d00 0x7fe6400a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.948+0000 7fe64a7fc640 1 -- 172.21.15.55:0/4140917681 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe6500f0b30 con 0x7fe65410a010 2024-04-26T22:03:37.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.951+0000 7fe6594fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.951+0000 7fe6594fb640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868b7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50766/0 (socket says 172.21.15.55:50766) 2024-04-26T22:03:37.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.951+0000 7fe6594fb640 1 -- 172.21.15.55:0/194119962 learned_addr learned my addr 172.21.15.55:0/194119962 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:37.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.951+0000 7fe658cfa640 1 --2- 172.21.15.55:0/194119962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 0x556e868be4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.960+0000 7fe6594fb640 1 -- 172.21.15.55:0/194119962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 msgr2=0x556e868be4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:37.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.960+0000 7fe6594fb640 1 --2- 172.21.15.55:0/194119962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 0x556e868be4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.960+0000 7fe6594fb640 1 -- 172.21.15.55:0/194119962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 msgr2=0x556e868b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:37.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.960+0000 7fe6594fb640 1 --2- 172.21.15.55:0/194119962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.960+0000 7fe6594fb640 1 -- 172.21.15.55:0/194119962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e867a5a40 con 0x556e868b7280 2024-04-26T22:03:37.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.961+0000 7fe659cfc640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe6400a3d00 0x7fe6400a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.967+0000 7fe659cfc640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe6400a3d00 0x7fe6400a61c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x556e868b7180 tx=0x7fe64c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:37.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.968+0000 7fe6594fb640 1 --2- 172.21.15.55:0/194119962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868b7700 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe644009b20 tx=0x7fe64400e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18f4f3768f9b3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:37.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.972+0000 7fe63f7fe640 1 -- 172.21.15.55:0/194119962 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644017070 con 0x556e868b7280 2024-04-26T22:03:37.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.972+0000 7fe63f7fe640 1 -- 172.21.15.55:0/194119962 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe64400ec10 con 0x556e868b7280 2024-04-26T22:03:37.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.972+0000 7fe63f7fe640 1 -- 172.21.15.55:0/194119962 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644012630 con 0x556e868b7280 2024-04-26T22:03:37.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.973+0000 7fe65e47a740 1 -- 172.21.15.55:0/194119962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 msgr2=0x556e868b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:37.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.973+0000 7fe65e47a740 1 --2- 172.21.15.55:0/194119962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868b7700 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe644009b20 tx=0x7fe64400e1d0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.975+0000 7fe65e47a740 1 -- 172.21.15.55:0/194119962 shutdown_connections 2024-04-26T22:03:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.975+0000 7fe65e47a740 1 --2- 172.21.15.55:0/194119962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 0x556e868be4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.975+0000 7fe65e47a740 1 --2- 172.21.15.55:0/194119962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868b7700 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.975+0000 7fe65e47a740 1 --2- 172.21.15.55:0/194119962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.975+0000 7fe65e47a740 1 -- 172.21.15.55:0/194119962 >> 172.21.15.55:0/194119962 conn(0x556e868b15a0 msgr2=0x556e868b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:37.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.981+0000 7fe65e47a740 1 -- 172.21.15.55:0/194119962 shutdown_connections 2024-04-26T22:03:37.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.983+0000 7fe65e47a740 1 -- 172.21.15.55:0/194119962 wait complete. 2024-04-26T22:03:37.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 1 -- start start 2024-04-26T22:03:37.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 1 Processor -- start 2024-04-26T22:03:37.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 0x556e868c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:37.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556e868bec50 con 0x556e868b7280 2024-04-26T22:03:37.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556e867a1d10 con 0x556e868b60b0 2024-04-26T22:03:37.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556e868bedd0 con 0x556e868b7c40 2024-04-26T22:03:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.984+0000 7fe65e47a740 10 client.0 _ll_register_callbacks cb 0x556e867ce210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.985+0000 7fe658cfa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.985+0000 7fe658cfa640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868c0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50774/0 (socket says 172.21.15.55:50774) 2024-04-26T22:03:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.985+0000 7fe658cfa640 1 -- 172.21.15.55:0/4075635510 learned_addr learned my addr 172.21.15.55:0/4075635510 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.986+0000 7fe6594fb640 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.987+0000 7fe658cfa640 1 -- 172.21.15.55:0/4075635510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 msgr2=0x556e868c7b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:03:37.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.987+0000 7fe658cfa640 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 0x556e868c7b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.987+0000 7fe658cfa640 1 -- 172.21.15.55:0/4075635510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 msgr2=0x556e868c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:37.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.987+0000 7fe658cfa640 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868c0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:37.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.987+0000 7fe658cfa640 1 -- 172.21.15.55:0/4075635510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e865f67d0 con 0x556e868b7280 2024-04-26T22:03:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.988+0000 7fe658cfa640 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868c0ad0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe6500b9180 tx=0x7fe6500bc680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce39d775b5fccc7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.988+0000 7fe63cff9640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.988+0000 7fe63cff9640 1 -- 172.21.15.55:0/4075635510 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65004a040 con 0x556e868b7280 2024-04-26T22:03:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.989+0000 7fe63cff9640 1 -- 172.21.15.55:0/4075635510 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe6500dd0c0 con 0x556e868b7280 2024-04-26T22:03:37.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:37.996+0000 7fe65e47a740 1 -- 172.21.15.55:0/4075635510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe64400ec10 con 0x556e868b7280 2024-04-26T22:03:38.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.007+0000 7fe63cff9640 1 -- 172.21.15.55:0/4075635510 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe6500dd2c0 con 0x556e868b7280 2024-04-26T22:03:38.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.008+0000 7fe65e47a740 10 client.17067 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:38.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.008+0000 7fe65e47a740 1 -- 172.21.15.55:0/4075635510 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556e8652c3f0 con 0x556e868b7280 2024-04-26T22:03:38.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.012+0000 7fe63cff9640 1 -- 172.21.15.55:0/4075635510 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe64400ec10 con 0x556e868b7280 2024-04-26T22:03:38.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.014+0000 7fe63cff9640 1 -- 172.21.15.55:0/4075635510 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe6500c5450 con 0x556e868b7280 2024-04-26T22:03:38.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.014+0000 7fe65e47a740 10 client.17067 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:38.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.014+0000 7fe65e47a740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:38.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.015+0000 7fe65e47a740 20 client.17067 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:38.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.015+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556e868cc050 0x556e868ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.015+0000 7fe65e47a740 4 client.17067 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:38.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.015+0000 7fe65e47a740 1 -- 172.21.15.55:0/4075635510 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e86828990 con 0x556e868cc050 2024-04-26T22:03:38.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.018+0000 7fe6594fb640 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556e868cc050 0x556e868ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.036+0000 7fe6594fb640 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556e868cc050 0x556e868ce4b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4e81916f01567c server_cookie=7cd8c92205985222 in_seq=0 out_seq=0 2024-04-26T22:03:38.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.037+0000 7fe63cff9640 10 client.17067 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:38.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.043+0000 7fe63cff9640 1 -- 172.21.15.55:0/4075635510 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e86828990 con 0x556e868cc050 2024-04-26T22:03:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.044+0000 7fe63cff9640 10 client.17067 handle_command_reply: tid=0 2024-04-26T22:03:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.044+0000 7fe65e47a740 1 client.17067 shutdown 2024-04-26T22:03:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.044+0000 7fe65e47a740 1 -- 172.21.15.55:0/4075635510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 msgr2=0x556e868c0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:38.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.045+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868c0ad0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe6500b9180 tx=0x7fe6500bc680 comp rx=0 tx=0).stop 2024-04-26T22:03:38.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.045+0000 7fe65e47a740 1 -- 172.21.15.55:0/4075635510 shutdown_connections 2024-04-26T22:03:38.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.045+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7c40 0x556e868c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.045+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556e868cc050 0x556e868ce4b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.045+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7280 0x556e868c0ad0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.045+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4075635510 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.045+0000 7fe65e47a740 1 -- 172.21.15.55:0/4075635510 >> 172.21.15.55:0/4075635510 conn(0x556e868b15a0 msgr2=0x556e868b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:38.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.045+0000 7fe63cff9640 0 client.17067 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:38.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.047+0000 7fe65e47a740 1 -- 172.21.15.55:0/4075635510 shutdown_connections 2024-04-26T22:03:38.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.047+0000 7fe65e47a740 1 -- 172.21.15.55:0/4075635510 wait complete. 2024-04-26T22:03:38.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.050+0000 7fe65e47a740 20 client.17067 trim_cache size 0 max 16384 2024-04-26T22:03:38.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe65e47a740 1 Processor -- start 2024-04-26T22:03:38.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe65e47a740 1 -- start start 2024-04-26T22:03:38.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7280 0x556e868c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe65e47a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe638010c80 con 0x556e868b7c40 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe65e47a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe63800e550 con 0x556e868b60b0 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe65e47a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe63800c450 con 0x556e868b7280 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.093+0000 7fe659cfc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe658cfa640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7280 0x556e868c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe658cfa640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7280 0x556e868c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45268/0 (socket says 172.21.15.55:45268) 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe658cfa640 1 -- 172.21.15.55:0/4227172386 learned_addr learned my addr 172.21.15.55:0/4227172386 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe6594fb640 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe659cfc640 1 -- 172.21.15.55:0/4227172386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7280 msgr2=0x556e868c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe659cfc640 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7280 0x556e868c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe659cfc640 1 -- 172.21.15.55:0/4227172386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 msgr2=0x556e868bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:38.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe659cfc640 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe659cfc640 1 -- 172.21.15.55:0/4227172386 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e864ec0f0 con 0x556e868b7c40 2024-04-26T22:03:38.096 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe658cfa640 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7280 0x556e868c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:38.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.094+0000 7fe659cfc640 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868afe20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe64c013400 tx=0x7fe64c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5160f1cea1c3eb43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:38.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.095+0000 7fe63effd640 1 -- 172.21.15.55:0/4227172386 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c01d630 con 0x556e868b7c40 2024-04-26T22:03:38.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.095+0000 7fe63effd640 1 -- 172.21.15.55:0/4227172386 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe64c024030 con 0x556e868b7c40 2024-04-26T22:03:38.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.095+0000 7fe63effd640 1 -- 172.21.15.55:0/4227172386 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c010a90 con 0x556e868b7c40 2024-04-26T22:03:38.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.096+0000 7fe65e47a740 1 -- 172.21.15.55:0/4227172386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 msgr2=0x556e868afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:38.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.096+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868afe20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe64c013400 tx=0x7fe64c019000 comp rx=0 tx=0).stop 2024-04-26T22:03:38.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.096+0000 7fe65e47a740 1 -- 172.21.15.55:0/4227172386 shutdown_connections 2024-04-26T22:03:38.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.096+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868afe20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.096+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b7280 0x556e868c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.096+0000 7fe65e47a740 1 --2- 172.21.15.55:0/4227172386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b60b0 0x556e868bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.096+0000 7fe65e47a740 1 -- 172.21.15.55:0/4227172386 >> 172.21.15.55:0/4227172386 conn(0x556e868b15a0 msgr2=0x556e868bf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:38.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.097+0000 7fe65e47a740 1 -- 172.21.15.55:0/4227172386 shutdown_connections 2024-04-26T22:03:38.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.097+0000 7fe65e47a740 1 -- 172.21.15.55:0/4227172386 wait complete. 2024-04-26T22:03:38.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.097+0000 7fe65e47a740 1 -- start start 2024-04-26T22:03:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.097+0000 7fe65e47a740 1 Processor -- start 2024-04-26T22:03:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.098+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b60b0 0x556e868c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.098+0000 7fe65e47a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b7280 0x556e868c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe6594fb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b60b0 0x556e868c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe6594fb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b60b0 0x556e868c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45278/0 (socket says 172.21.15.55:45278) 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe6594fb640 1 -- 172.21.15.55:0/912373830 learned_addr learned my addr 172.21.15.55:0/912373830 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe65e47a740 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe63800ca10 con 0x556e868b7c40 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe63800de80 con 0x556e868b7280 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe658cfa640 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b7280 0x556e868c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe659cfc640 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.099+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe63800d0e0 con 0x556e868b60b0 2024-04-26T22:03:38.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.100+0000 7fe6594fb640 1 -- 172.21.15.55:0/912373830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b7280 msgr2=0x556e868c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:38.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.100+0000 7fe6594fb640 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b7280 0x556e868c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.100+0000 7fe6594fb640 1 -- 172.21.15.55:0/912373830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 msgr2=0x556e868c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:38.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.100+0000 7fe6594fb640 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.100+0000 7fe6594fb640 1 -- 172.21.15.55:0/912373830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e867db4b0 con 0x556e868b60b0 2024-04-26T22:03:38.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.100+0000 7fe65e47a740 10 client.0 _ll_register_callbacks cb 0x7fe650003720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:38.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.100+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe64c024030 con 0x556e868b60b0 2024-04-26T22:03:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.107+0000 7fe6594fb640 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b60b0 0x556e868c4e80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe644009a70 tx=0x7fe6440046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f833d2aec3ce4451 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:38.107 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.107+0000 7fe63cff9640 10 client.26122 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:03:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.109+0000 7fe63cff9640 1 -- 172.21.15.55:0/912373830 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64402c060 con 0x556e868b60b0 2024-04-26T22:03:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.109+0000 7fe63cff9640 1 -- 172.21.15.55:0/912373830 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe644027450 con 0x556e868b60b0 2024-04-26T22:03:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.109+0000 7fe63cff9640 1 -- 172.21.15.55:0/912373830 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe644017040 con 0x556e868b60b0 2024-04-26T22:03:38.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.111+0000 7fe65e47a740 10 client.26122 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:38.111 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.111+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({fsmap=23}) -- 0x556e868c7a00 con 0x556e868b60b0 2024-04-26T22:03:38.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.120+0000 7fe63cff9640 1 -- 172.21.15.55:0/912373830 <== mon.1 v2:172.21.15.165:3300/0 4 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe644027690 con 0x556e868b60b0 2024-04-26T22:03:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.128+0000 7fe65e47a740 10 client.26122 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.128+0000 7fe65e47a740 10 client.26122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.128+0000 7fe65e47a740 20 client.26122 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.129+0000 7fe65e47a740 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556e86932630 0x556e86934a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.129+0000 7fe65e47a740 4 client.26122 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:03:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.129+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e86822800 con 0x556e86932630 2024-04-26T22:03:38.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.131+0000 7fe658cfa640 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556e86932630 0x556e86934a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.138+0000 7fe658cfa640 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556e86932630 0x556e86934a20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54110d4886bb4d49 server_cookie=9254434e6acf01d5 in_seq=0 out_seq=0 2024-04-26T22:03:38.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.138+0000 7fe63cff9640 10 client.26122 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:38.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.143+0000 7fe63cff9640 1 -- 172.21.15.55:0/912373830 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556e86822800 con 0x556e86932630 2024-04-26T22:03:38.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.143+0000 7fe63cff9640 10 client.26122 handle_command_reply: tid=0 2024-04-26T22:03:38.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.143+0000 7fe65e47a740 1 client.26122 shutdown 2024-04-26T22:03:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.148+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b60b0 msgr2=0x556e868c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.148+0000 7fe65e47a740 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b60b0 0x556e868c4e80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe644009a70 tx=0x7fe6440046c0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.153+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 shutdown_connections 2024-04-26T22:03:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.153+0000 7fe65e47a740 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556e86932630 0x556e86934a20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.153+0000 7fe65e47a740 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e868b7c40 0x556e868c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.153+0000 7fe65e47a740 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e868b7280 0x556e868c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.153+0000 7fe65e47a740 1 --2- 172.21.15.55:0/912373830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e868b60b0 0x556e868c4e80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.153+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 >> 172.21.15.55:0/912373830 conn(0x556e868b15a0 msgr2=0x556e868afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:38.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.153+0000 7fe63cff9640 0 client.26122 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:38.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.157+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 shutdown_connections 2024-04-26T22:03:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.162+0000 7fe65e47a740 1 -- 172.21.15.55:0/912373830 wait complete. 2024-04-26T22:03:38.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.167+0000 7fe65e47a740 20 client.26122 trim_cache size 0 max 16384 2024-04-26T22:03:38.170 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:03:38.170 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:03:38.171 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:03:38.171 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:03:38.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.178+0000 7fe65a4fd640 1 -- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe6400a3d00 msgr2=0x7fe6400a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:38.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.178+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe6400a3d00 0x7fe6400a61c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x556e868b7180 tx=0x7fe64c004ab0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.178+0000 7fe65a4fd640 1 -- 172.21.15.55:0/4140917681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 msgr2=0x7fe6541a7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:38.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.178+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 0x7fe6541a7de0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe650009230 tx=0x7fe650003e90 comp rx=0 tx=0).stop 2024-04-26T22:03:38.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.182+0000 7fe65a4fd640 1 -- 172.21.15.55:0/4140917681 shutdown_connections 2024-04-26T22:03:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.183+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe6400a3d00 0x7fe6400a61c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.183+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe65410a9b0 0x7fe6541aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.183+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe65410a010 0x7fe6541a7de0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.183+0000 7fe65a4fd640 1 --2- 172.21.15.55:0/4140917681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe654073d70 0x7fe6541a78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.183+0000 7fe65a4fd640 1 -- 172.21.15.55:0/4140917681 >> 172.21.15.55:0/4140917681 conn(0x7fe65406f840 msgr2=0x7fe6540714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.183+0000 7fe65a4fd640 1 -- 172.21.15.55:0/4140917681 shutdown_connections 2024-04-26T22:03:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.187+0000 7fe65a4fd640 1 -- 172.21.15.55:0/4140917681 wait complete. 2024-04-26T22:03:38.246 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:03:38.246 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:03:38.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.611+0000 7f587650f640 1 -- 172.21.15.55:0/3733080975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f587010bf10 msgr2=0x7f587010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:38.614 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.611+0000 7f587650f640 1 --2- 172.21.15.55:0/3733080975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f587010bf10 0x7f587010e3a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5868009ca0 tx=0x7f586802a8e0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.613+0000 7f587650f640 1 -- 172.21.15.55:0/3733080975 shutdown_connections 2024-04-26T22:03:38.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.613+0000 7f587650f640 1 --2- 172.21.15.55:0/3733080975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587010ea10 0x7f5870119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.613+0000 7f587650f640 1 --2- 172.21.15.55:0/3733080975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f587010bf10 0x7f587010e3a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.613+0000 7f587650f640 1 --2- 172.21.15.55:0/3733080975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5870074760 0x7f587010b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.613+0000 7f587650f640 1 -- 172.21.15.55:0/3733080975 >> 172.21.15.55:0/3733080975 conn(0x7f587006f690 msgr2=0x7f5870071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:38.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.615+0000 7f587650f640 1 -- 172.21.15.55:0/3733080975 shutdown_connections 2024-04-26T22:03:38.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.618+0000 7f587650f640 1 -- 172.21.15.55:0/3733080975 wait complete. 2024-04-26T22:03:38.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.619+0000 7f587650f640 1 Processor -- start 2024-04-26T22:03:38.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.620+0000 7f587650f640 1 -- start start 2024-04-26T22:03:38.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.620+0000 7f587650f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5870074760 0x7f58701160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.620+0000 7f587650f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587010bf10 0x7f5870116630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.620+0000 7f587650f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587010ea10 0x7f58701b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.620+0000 7f587650f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f587011d200 con 0x7f5870074760 2024-04-26T22:03:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.620+0000 7f587650f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f587011d080 con 0x7f587010ea10 2024-04-26T22:03:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.620+0000 7f587650f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f587011d380 con 0x7f587010bf10 2024-04-26T22:03:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.620+0000 7f587550d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5870074760 0x7f58701160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.621+0000 7f587550d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5870074760 0x7f58701160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50822/0 (socket says 172.21.15.55:50822) 2024-04-26T22:03:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.621+0000 7f587550d640 1 -- 172.21.15.55:0/1629950357 learned_addr learned my addr 172.21.15.55:0/1629950357 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:38.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.621+0000 7f587550d640 1 -- 172.21.15.55:0/1629950357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587010bf10 msgr2=0x7f5870116630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:38.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.621+0000 7f587550d640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587010bf10 0x7f5870116630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.621+0000 7f587550d640 1 -- 172.21.15.55:0/1629950357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587010ea10 msgr2=0x7f58701b7910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:38.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.621+0000 7f587550d640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587010ea10 0x7f58701b7910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:38.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.621+0000 7f587550d640 1 -- 172.21.15.55:0/1629950357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58701b7e50 con 0x7f5870074760 2024-04-26T22:03:38.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.622+0000 7f587550d640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5870074760 0x7f58701160f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f586c00b450 tx=0x7f586c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:38.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.623+0000 7f58667fc640 1 -- 172.21.15.55:0/1629950357 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c015020 con 0x7f5870074760 2024-04-26T22:03:38.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.623+0000 7f587650f640 1 -- 172.21.15.55:0/1629950357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58701b80e0 con 0x7f5870074760 2024-04-26T22:03:38.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.623+0000 7f587650f640 1 -- 172.21.15.55:0/1629950357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701b8680 con 0x7f5870074760 2024-04-26T22:03:38.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.624+0000 7f58667fc640 1 -- 172.21.15.55:0/1629950357 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586c011d40 con 0x7f5870074760 2024-04-26T22:03:38.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.626+0000 7f58667fc640 1 -- 172.21.15.55:0/1629950357 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c00f340 con 0x7f5870074760 2024-04-26T22:03:38.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.630+0000 7f587650f640 1 -- 172.21.15.55:0/1629950357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5870069670 con 0x7f5870074760 2024-04-26T22:03:38.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.630+0000 7f58667fc640 1 -- 172.21.15.55:0/1629950357 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f586c018020 con 0x7f5870074760 2024-04-26T22:03:38.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.631+0000 7f58667fc640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f585c0a3d20 0x7f585c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:38.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.632+0000 7f58667fc640 1 -- 172.21.15.55:0/1629950357 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f586c0cff00 con 0x7f5870074760 2024-04-26T22:03:38.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.632+0000 7f5874d0c640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f585c0a3d20 0x7f585c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:38.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.633+0000 7f5874d0c640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f585c0a3d20 0x7f585c0a61e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5868004af0 tx=0x7f5868000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:38.646+0000 7f58667fc640 1 -- 172.21.15.55:0/1629950357 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f586c01f020 con 0x7f5870074760 2024-04-26T22:03:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.122+0000 7f587650f640 1 -- 172.21.15.55:0/1629950357 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f587002de90 con 0x7f5870074760 2024-04-26T22:03:39.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.125+0000 7f58667fc640 1 -- 172.21.15.55:0/1629950357 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f586c00d040 con 0x7f5870074760 2024-04-26T22:03:39.130 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:03:39.130 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:03:39.131 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:03:39.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.142+0000 7f5853fff640 1 -- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f585c0a3d20 msgr2=0x7f585c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.143+0000 7f5853fff640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f585c0a3d20 0x7f585c0a61e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5868004af0 tx=0x7f5868000f40 comp rx=0 tx=0).stop 2024-04-26T22:03:39.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.143+0000 7f5853fff640 1 -- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5870074760 msgr2=0x7f58701160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.143+0000 7f5853fff640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5870074760 0x7f58701160f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f586c00b450 tx=0x7f586c00b910 comp rx=0 tx=0).stop 2024-04-26T22:03:39.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.145+0000 7f5853fff640 1 -- 172.21.15.55:0/1629950357 shutdown_connections 2024-04-26T22:03:39.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.145+0000 7f5853fff640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f585c0a3d20 0x7f585c0a61e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.145+0000 7f5853fff640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587010ea10 0x7f58701b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.145+0000 7f5853fff640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587010bf10 0x7f5870116630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.145+0000 7f5853fff640 1 --2- 172.21.15.55:0/1629950357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5870074760 0x7f58701160f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.145+0000 7f5853fff640 1 -- 172.21.15.55:0/1629950357 >> 172.21.15.55:0/1629950357 conn(0x7f587006f690 msgr2=0x7f5870109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:39.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.147+0000 7f5853fff640 1 -- 172.21.15.55:0/1629950357 shutdown_connections 2024-04-26T22:03:39.150 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.147+0000 7f5853fff640 1 -- 172.21.15.55:0/1629950357 wait complete. 2024-04-26T22:03:39.186 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:03:39.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:38 smithi064 ceph-mon[32765]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 885 MiB/s wr, 1.13k op/s 2024-04-26T22:03:39.276 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:38 smithi055 ceph-mon[29221]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 885 MiB/s wr, 1.13k op/s 2024-04-26T22:03:39.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:38 smithi165 ceph-mon[33296]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 885 MiB/s wr, 1.13k op/s 2024-04-26T22:03:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.482+0000 7fc1b4c39640 1 -- 172.21.15.55:0/3918219964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b010a8d0 msgr2=0x7fc1b010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.484 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.482+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/3918219964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b010a8d0 0x7fc1b010f240 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009900 tx=0x7fc1a402a440 comp rx=0 tx=0).stop 2024-04-26T22:03:39.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.490+0000 7fc1b4c39640 1 -- 172.21.15.55:0/3918219964 shutdown_connections 2024-04-26T22:03:39.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.490+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/3918219964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b010a8d0 0x7fc1b010f240 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.490+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/3918219964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b0109f10 0x7fc1b010a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.490+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/3918219964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1b0073c10 0x7fc1b0074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.490+0000 7fc1b4c39640 1 -- 172.21.15.55:0/3918219964 >> 172.21.15.55:0/3918219964 conn(0x7fc1b006f780 msgr2=0x7fc1b0071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:39.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.496+0000 7fc1b4c39640 1 -- 172.21.15.55:0/3918219964 shutdown_connections 2024-04-26T22:03:39.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.496+0000 7fc1b4c39640 1 -- 172.21.15.55:0/3918219964 wait complete. 2024-04-26T22:03:39.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.497+0000 7fc1b4c39640 1 Processor -- start 2024-04-26T22:03:39.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.499+0000 7fc1b4c39640 1 -- start start 2024-04-26T22:03:39.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.500+0000 7fc1b4c39640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1b0073c10 0x7fc1b0116000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.500+0000 7fc1b4c39640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b0109f10 0x7fc1b0116540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.500+0000 7fc1b4c39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b010a8d0 0x7fc1b01b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.500+0000 7fc1b4c39640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b011d140 con 0x7fc1b0109f10 2024-04-26T22:03:39.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.500+0000 7fc1b4c39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b011cfc0 con 0x7fc1b010a8d0 2024-04-26T22:03:39.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.500+0000 7fc1b4c39640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b011d2c0 con 0x7fc1b0073c10 2024-04-26T22:03:39.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.502+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1b0073c10 0x7fc1b0116000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.502+0000 7fc1affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b010a8d0 0x7fc1b01b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.502+0000 7fc1affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b010a8d0 0x7fc1b01b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:36852/0 (socket says 172.21.15.55:36852) 2024-04-26T22:03:39.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.502+0000 7fc1affff640 1 -- 172.21.15.55:0/610720546 learned_addr learned my addr 172.21.15.55:0/610720546 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:39.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.502+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1b0073c10 0x7fc1b0116000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45288/0 (socket says 172.21.15.55:45288) 2024-04-26T22:03:39.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.504+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b0109f10 0x7fc1b0116540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.503+0000 7fc1affff640 1 -- 172.21.15.55:0/610720546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1b0073c10 msgr2=0x7fc1b0116000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.504+0000 7fc1affff640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1b0073c10 0x7fc1b0116000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.504+0000 7fc1affff640 1 -- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b0109f10 msgr2=0x7fc1b0116540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.504+0000 7fc1affff640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b0109f10 0x7fc1b0116540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.504+0000 7fc1affff640 1 -- 172.21.15.55:0/610720546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b01b7d90 con 0x7fc1b010a8d0 2024-04-26T22:03:39.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.504+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b0109f10 0x7fc1b0116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:39.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.505+0000 7fc1affff640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b010a8d0 0x7fc1b01b7850 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009850 tx=0x7fc1a4003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.505+0000 7fc1acff9640 1 -- 172.21.15.55:0/610720546 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a403f070 con 0x7fc1b010a8d0 2024-04-26T22:03:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.506+0000 7fc1acff9640 1 -- 172.21.15.55:0/610720546 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a4031dd0 con 0x7fc1b010a8d0 2024-04-26T22:03:39.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.506+0000 7fc1acff9640 1 -- 172.21.15.55:0/610720546 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4035030 con 0x7fc1b010a8d0 2024-04-26T22:03:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.507+0000 7fc1b4c39640 1 -- 172.21.15.55:0/610720546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b01b7f60 con 0x7fc1b010a8d0 2024-04-26T22:03:39.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.507+0000 7fc1b4c39640 1 -- 172.21.15.55:0/610720546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b01b8310 con 0x7fc1b010a8d0 2024-04-26T22:03:39.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.513+0000 7fc1b8bb0740 1 Processor -- start 2024-04-26T22:03:39.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.519+0000 7fc1acff9640 1 -- 172.21.15.55:0/610720546 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc1a4042070 con 0x7fc1b010a8d0 2024-04-26T22:03:39.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.521+0000 7fc1acff9640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1980a3d00 0x7fc1980a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.521+0000 7fc1acff9640 1 -- 172.21.15.55:0/610720546 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1a40f0f60 con 0x7fc1b010a8d0 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.522+0000 7fc1b8bb0740 1 -- start start 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.522+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759ba30 0x55600759be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.522+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x55600759d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.522+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759d5c0 0x5560075a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.522+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5560074ffd80 con 0x55600759cc00 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.522+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5560074b9a20 con 0x55600759ba30 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.522+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5560074fbd40 con 0x55600759d5c0 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.522+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1980a3d00 0x7fc1980a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.523+0000 7fc1aeffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x55600759d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.524+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759ba30 0x55600759be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.524+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759ba30 0x55600759be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:36864/0 (socket says 172.21.15.55:36864) 2024-04-26T22:03:39.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.524+0000 7fc1af7fe640 1 -- 172.21.15.55:0/2646726261 learned_addr learned my addr 172.21.15.55:0/2646726261 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:39.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.526+0000 7fc1af7fe640 1 -- 172.21.15.55:0/2646726261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759d5c0 msgr2=0x5560075a3e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:39.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.526+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/2646726261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759d5c0 0x5560075a3e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.526+0000 7fc1af7fe640 1 -- 172.21.15.55:0/2646726261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 msgr2=0x55600759d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:39.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.526+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/2646726261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x55600759d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.526+0000 7fc1af7fe640 1 -- 172.21.15.55:0/2646726261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560074c4c30 con 0x55600759ba30 2024-04-26T22:03:39.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.527+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/2646726261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759ba30 0x55600759be50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009a40 tx=0x7fc1a0009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7923dfccdfe65cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.530+0000 7fc195ffb640 1 -- 172.21.15.55:0/2646726261 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0017070 con 0x55600759ba30 2024-04-26T22:03:39.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.530+0000 7fc195ffb640 1 -- 172.21.15.55:0/2646726261 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a0013470 con 0x55600759ba30 2024-04-26T22:03:39.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.532+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/2646726261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759ba30 msgr2=0x55600759be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.532+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/2646726261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759ba30 0x55600759be50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009a40 tx=0x7fc1a0009f00 comp rx=0 tx=0).stop 2024-04-26T22:03:39.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.533+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1980a3d00 0x7fc1980a61c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0117510 tx=0x7fc1a8008970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.533+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/2646726261 shutdown_connections 2024-04-26T22:03:39.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.533+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/2646726261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759d5c0 0x5560075a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.533+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/2646726261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x55600759d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.533+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/2646726261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759ba30 0x55600759be50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.533+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/2646726261 >> 172.21.15.55:0/2646726261 conn(0x5560074dc910 msgr2=0x556007597330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:39.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.536+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/2646726261 shutdown_connections 2024-04-26T22:03:39.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.536+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/2646726261 wait complete. 2024-04-26T22:03:39.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.537+0000 7fc1b8bb0740 1 -- start start 2024-04-26T22:03:39.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.538+0000 7fc1b8bb0740 1 Processor -- start 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.539+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.539+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 0x5560075a6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.539+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.539+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5560075a4620 con 0x55600759ba30 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.539+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556007486dc0 con 0x55600759d5c0 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.539+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5560075a47a0 con 0x55600759cc00 2024-04-26T22:03:39.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.539+0000 7fc1b8bb0740 10 client.0 _ll_register_callbacks cb 0x5560074a2600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:39.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.540+0000 7fc1affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.540+0000 7fc1affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ad270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:36878/0 (socket says 172.21.15.55:36878) 2024-04-26T22:03:39.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.540+0000 7fc1affff640 1 -- 172.21.15.55:0/1489816712 learned_addr learned my addr 172.21.15.55:0/1489816712 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:39.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.540+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 0x5560075a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.541+0000 7fc1affff640 1 -- 172.21.15.55:0/1489816712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 msgr2=0x5560075a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.541+0000 7fc1affff640 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 0x5560075a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.541+0000 7fc1affff640 1 -- 172.21.15.55:0/1489816712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 msgr2=0x5560075a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.541+0000 7fc1affff640 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.541+0000 7fc1affff640 1 -- 172.21.15.55:0/1489816712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560072113f0 con 0x55600759d5c0 2024-04-26T22:03:39.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.542+0000 7fc1affff640 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ad270 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40bd030 tx=0x7fc1a40bd7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b309e93dd33aeac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.556+0000 7fc18effd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:03:39.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.556+0000 7fc18effd640 1 -- 172.21.15.55:0/1489816712 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4049030 con 0x55600759d5c0 2024-04-26T22:03:39.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.557+0000 7fc18effd640 1 -- 172.21.15.55:0/1489816712 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1a40b9710 con 0x55600759d5c0 2024-04-26T22:03:39.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.557+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1489816712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a0013470 con 0x55600759d5c0 2024-04-26T22:03:39.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.558+0000 7fc18effd640 1 -- 172.21.15.55:0/1489816712 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1a40c1450 con 0x55600759d5c0 2024-04-26T22:03:39.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.559+0000 7fc1b8bb0740 10 client.26123 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:39.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.559+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1489816712 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5560072db7d0 con 0x55600759d5c0 2024-04-26T22:03:39.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.559+0000 7fc18effd640 1 -- 172.21.15.55:0/1489816712 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a0013470 con 0x55600759d5c0 2024-04-26T22:03:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.559+0000 7fc18effd640 1 -- 172.21.15.55:0/1489816712 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc1a403b460 con 0x55600759d5c0 2024-04-26T22:03:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.560+0000 7fc1b8bb0740 10 client.26123 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.560+0000 7fc1b8bb0740 10 client.26123 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:03:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.560+0000 7fc1b8bb0740 20 client.26123 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.560+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5560075b1cb0 0x5560075b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.560+0000 7fc1b8bb0740 4 client.26123 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:39.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.560+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1489816712 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55600750d990 con 0x5560075b1cb0 2024-04-26T22:03:39.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.560+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5560075b1cb0 0x5560075b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.563+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5560075b1cb0 0x5560075b4130 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32a55d9dc2b5a0ec server_cookie=1b254d056f327196 in_seq=0 out_seq=0 2024-04-26T22:03:39.565 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.564+0000 7fc18effd640 10 client.26123 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:03:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.565+0000 7fc18effd640 1 -- 172.21.15.55:0/1489816712 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55600750d990 con 0x5560075b1cb0 2024-04-26T22:03:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.565+0000 7fc18effd640 10 client.26123 handle_command_reply: tid=0 2024-04-26T22:03:39.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.565+0000 7fc1b8bb0740 1 client.26123 shutdown 2024-04-26T22:03:39.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1489816712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 msgr2=0x5560075ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ad270 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40bd030 tx=0x7fc1a40bd7a0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1489816712 shutdown_connections 2024-04-26T22:03:39.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x5560075b1cb0 0x5560075b4130 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ad270 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 0x5560075a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1489816712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc18effd640 0 client.26123 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:03:39.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.575+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1489816712 >> 172.21.15.55:0/1489816712 conn(0x5560074dc910 msgr2=0x5560075963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:39.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.576+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1489816712 shutdown_connections 2024-04-26T22:03:39.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.580+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1489816712 wait complete. 2024-04-26T22:03:39.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.580+0000 7fc1b8bb0740 20 client.26123 trim_cache size 0 max 16384 2024-04-26T22:03:39.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.630+0000 7fc1b8bb0740 1 Processor -- start 2024-04-26T22:03:39.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.632+0000 7fc1b8bb0740 1 -- start start 2024-04-26T22:03:39.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.632+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759ba30 0x5560075974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.632+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x5560075ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.632+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.632+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c010ee0 con 0x55600759cc00 2024-04-26T22:03:39.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.632+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c00e7b0 con 0x55600759d5c0 2024-04-26T22:03:39.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.632+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c00c600 con 0x55600759ba30 2024-04-26T22:03:39.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.634+0000 7fc1aeffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x5560075ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.634+0000 7fc1aeffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x5560075ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50878/0 (socket says 172.21.15.55:50878) 2024-04-26T22:03:39.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.634+0000 7fc1aeffd640 1 -- 172.21.15.55:0/1690806180 learned_addr learned my addr 172.21.15.55:0/1690806180 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:39.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.634+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759ba30 0x5560075974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.636+0000 7fc1affff640 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.637+0000 7fc1aeffd640 1 -- 172.21.15.55:0/1690806180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759ba30 msgr2=0x5560075974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:39.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.637+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759ba30 0x5560075974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.637+0000 7fc1aeffd640 1 -- 172.21.15.55:0/1690806180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 msgr2=0x5560075b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:39.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.637+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.637+0000 7fc1aeffd640 1 -- 172.21.15.55:0/1690806180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600759c680 con 0x55600759cc00 2024-04-26T22:03:39.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.639+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x5560075ad230 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8008ca0 tx=0x7fc1a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f987ba0e957505c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.641+0000 7fc1957fa640 1 -- 172.21.15.55:0/1690806180 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8023070 con 0x55600759cc00 2024-04-26T22:03:39.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.641+0000 7fc1957fa640 1 -- 172.21.15.55:0/1690806180 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a8012b00 con 0x55600759cc00 2024-04-26T22:03:39.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.641+0000 7fc1957fa640 1 -- 172.21.15.55:0/1690806180 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a801fb90 con 0x55600759cc00 2024-04-26T22:03:39.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.644+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1690806180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 msgr2=0x5560075ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.645 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.644+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x5560075ad230 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8008ca0 tx=0x7fc1a8017000 comp rx=0 tx=0).stop 2024-04-26T22:03:39.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.652+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1690806180 shutdown_connections 2024-04-26T22:03:39.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.652+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.652+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759cc00 0x5560075ad230 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.652+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1690806180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759ba30 0x5560075974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.652+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1690806180 >> 172.21.15.55:0/1690806180 conn(0x5560074dc910 msgr2=0x556007596db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:39.666 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.663+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1690806180 shutdown_connections 2024-04-26T22:03:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.664+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1690806180 wait complete. 2024-04-26T22:03:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.664+0000 7fc1b8bb0740 1 -- start start 2024-04-26T22:03:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.665+0000 7fc1b8bb0740 1 Processor -- start 2024-04-26T22:03:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.666+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.666+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 0x5560075ab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.666+0000 7fc1b8bb0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.666+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c00cc70 con 0x55600759ba30 2024-04-26T22:03:39.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.666+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c00e0e0 con 0x55600759d5c0 2024-04-26T22:03:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.666+0000 7fc1b8bb0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c00d340 con 0x55600759cc00 2024-04-26T22:03:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.666+0000 7fc1b8bb0740 10 client.0 _ll_register_callbacks cb 0x7fc1a4003ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.667+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.667+0000 7fc1af7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075aab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50890/0 (socket says 172.21.15.55:50890) 2024-04-26T22:03:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.667+0000 7fc1af7fe640 1 -- 172.21.15.55:0/1878235196 learned_addr learned my addr 172.21.15.55:0/1878235196 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.667+0000 7fc1af7fe640 1 -- 172.21.15.55:0/1878235196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 msgr2=0x5560075ab060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:39.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.667+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 0x5560075ab060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.667+0000 7fc1af7fe640 1 -- 172.21.15.55:0/1878235196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 msgr2=0x5560075ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:39.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.667+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.667+0000 7fc1af7fe640 1 -- 172.21.15.55:0/1878235196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560075ac610 con 0x55600759ba30 2024-04-26T22:03:39.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.668+0000 7fc1af7fe640 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075aab20 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009990 tx=0x7fc1a0006b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cc0522a79c7a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:39.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.668+0000 7fc18effd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:39.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.672+0000 7fc18effd640 1 -- 172.21.15.55:0/1878235196 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0027070 con 0x55600759ba30 2024-04-26T22:03:39.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.672+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1878235196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a8012b00 con 0x55600759ba30 2024-04-26T22:03:39.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.672+0000 7fc18effd640 1 -- 172.21.15.55:0/1878235196 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc1a0023d50 con 0x55600759ba30 2024-04-26T22:03:39.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.674+0000 7fc18effd640 1 -- 172.21.15.55:0/1878235196 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1a0017040 con 0x55600759ba30 2024-04-26T22:03:39.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.674+0000 7fc1b8bb0740 10 client.17091 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:39.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.674+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1878235196 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556007618480 con 0x55600759ba30 2024-04-26T22:03:39.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.675+0000 7fc18effd640 1 -- 172.21.15.55:0/1878235196 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a8012b00 con 0x55600759ba30 2024-04-26T22:03:39.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.677+0000 7fc18effd640 1 -- 172.21.15.55:0/1878235196 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc1a0023240 con 0x55600759ba30 2024-04-26T22:03:39.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.677+0000 7fc1b8bb0740 10 client.17091 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:39.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.677+0000 7fc1b8bb0740 10 client.17091 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:03:39.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.677+0000 7fc1b8bb0740 20 client.17091 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:39.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.677+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556007618e10 0x55600761b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:39.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.677+0000 7fc1b8bb0740 4 client.17091 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:03:39.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.677+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1878235196 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556007507800 con 0x556007618e10 2024-04-26T22:03:39.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.680+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556007618e10 0x55600761b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:39.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.692+0000 7fc1aeffd640 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556007618e10 0x55600761b230 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75ea0782577fb46b server_cookie=362b3fd54a2b6f14 in_seq=0 out_seq=0 2024-04-26T22:03:39.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.693+0000 7fc18effd640 10 client.17091 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:03:39.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.695+0000 7fc18effd640 1 -- 172.21.15.55:0/1878235196 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556007507800 con 0x556007618e10 2024-04-26T22:03:39.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.695+0000 7fc18effd640 10 client.17091 handle_command_reply: tid=0 2024-04-26T22:03:39.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.695+0000 7fc1b8bb0740 1 client.17091 shutdown 2024-04-26T22:03:39.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.695+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1878235196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 msgr2=0x5560075aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.695+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075aab20 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009990 tx=0x7fc1a0006b30 comp rx=0 tx=0).stop 2024-04-26T22:03:39.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.696+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1878235196 shutdown_connections 2024-04-26T22:03:39.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.696+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556007618e10 0x55600761b230 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.696+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55600759d5c0 0x5560075ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.696+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55600759cc00 0x5560075ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.696+0000 7fc1b8bb0740 1 --2- 172.21.15.55:0/1878235196 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55600759ba30 0x5560075aab20 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.696+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1878235196 >> 172.21.15.55:0/1878235196 conn(0x5560074dc910 msgr2=0x55600759a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:39.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.697+0000 7fc18effd640 0 client.17091 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:03:39.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.697+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1878235196 shutdown_connections 2024-04-26T22:03:39.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.697+0000 7fc1b8bb0740 1 -- 172.21.15.55:0/1878235196 wait complete. 2024-04-26T22:03:39.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.698+0000 7fc1b8bb0740 20 client.17091 trim_cache size 0 max 16384 2024-04-26T22:03:39.703 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:03:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.701+0000 7fc1b4c39640 1 -- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1980a3d00 msgr2=0x7fc1980a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.701+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1980a3d00 0x7fc1980a61c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0117510 tx=0x7fc1a8008970 comp rx=0 tx=0).stop 2024-04-26T22:03:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.701+0000 7fc1b4c39640 1 -- 172.21.15.55:0/610720546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b010a8d0 msgr2=0x7fc1b01b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.701+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b010a8d0 0x7fc1b01b7850 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009850 tx=0x7fc1a4003d60 comp rx=0 tx=0).stop 2024-04-26T22:03:39.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.702+0000 7fc1b4c39640 1 -- 172.21.15.55:0/610720546 shutdown_connections 2024-04-26T22:03:39.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.702+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc1980a3d00 0x7fc1980a61c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.702+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1b010a8d0 0x7fc1b01b7850 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.702+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1b0109f10 0x7fc1b0116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.702+0000 7fc1b4c39640 1 --2- 172.21.15.55:0/610720546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1b0073c10 0x7fc1b0116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:39.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.702+0000 7fc1b4c39640 1 -- 172.21.15.55:0/610720546 >> 172.21.15.55:0/610720546 conn(0x7fc1b006f780 msgr2=0x7fc1b00713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:39.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.703+0000 7fc1b4c39640 1 -- 172.21.15.55:0/610720546 shutdown_connections 2024-04-26T22:03:39.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:39.703+0000 7fc1b4c39640 1 -- 172.21.15.55:0/610720546 wait complete. 2024-04-26T22:03:39.731 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:03:40.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.055+0000 7f1002e0b640 1 -- 172.21.15.55:0/1694190286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ffc10a010 msgr2=0x7f0ffc10a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.055+0000 7f1002e0b640 1 --2- 172.21.15.55:0/1694190286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ffc10a010 0x7f0ffc10a470 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009860 tx=0x7f0fec02a380 comp rx=0 tx=0).stop 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.056+0000 7f1002e0b640 1 -- 172.21.15.55:0/1694190286 shutdown_connections 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.056+0000 7f1002e0b640 1 --2- 172.21.15.55:0/1694190286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc10a9b0 0x7f0ffc10f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.056+0000 7f1002e0b640 1 --2- 172.21.15.55:0/1694190286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ffc10a010 0x7f0ffc10a470 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.056+0000 7f1002e0b640 1 --2- 172.21.15.55:0/1694190286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ffc073d70 0x7f0ffc074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.056+0000 7f1002e0b640 1 -- 172.21.15.55:0/1694190286 >> 172.21.15.55:0/1694190286 conn(0x7f0ffc06f840 msgr2=0x7f0ffc071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.057+0000 7f1002e0b640 1 -- 172.21.15.55:0/1694190286 shutdown_connections 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.057+0000 7f1002e0b640 1 -- 172.21.15.55:0/1694190286 wait complete. 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.058+0000 7f1002e0b640 1 Processor -- start 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.058+0000 7f1002e0b640 1 -- start start 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.059+0000 7f1002e0b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ffc073d70 0x7f0ffc1160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.060+0000 7f1002e0b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc10a010 0x7f0ffc1165e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.060+0000 7f1002e0b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ffc10a9b0 0x7f0ffc1a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.060+0000 7f1002e0b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc11d180 con 0x7f0ffc10a010 2024-04-26T22:03:40.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.060+0000 7f1002e0b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc11d000 con 0x7f0ffc10a9b0 2024-04-26T22:03:40.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.060+0000 7f1002e0b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc11d300 con 0x7f0ffc073d70 2024-04-26T22:03:40.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.061+0000 7f1001608640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc10a010 0x7f0ffc1165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.061+0000 7f1001608640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc10a010 0x7f0ffc1165e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50916/0 (socket says 172.21.15.55:50916) 2024-04-26T22:03:40.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.061+0000 7f1001608640 1 -- 172.21.15.55:0/3486414447 learned_addr learned my addr 172.21.15.55:0/3486414447 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.062+0000 7f1001608640 1 -- 172.21.15.55:0/3486414447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ffc073d70 msgr2=0x7f0ffc1160a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:40.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.062+0000 7f1001608640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ffc073d70 0x7f0ffc1160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.062+0000 7f1001608640 1 -- 172.21.15.55:0/3486414447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ffc10a9b0 msgr2=0x7f0ffc1a6880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:03:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.062+0000 7f1001608640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ffc10a9b0 0x7f0ffc1a6880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.062+0000 7f1001608640 1 -- 172.21.15.55:0/3486414447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc1a6dc0 con 0x7f0ffc10a010 2024-04-26T22:03:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.063+0000 7f1001608640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc10a010 0x7f0ffc1165e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0fec000c00 tx=0x7f0fec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.065+0000 7f0ff2ffd640 1 -- 172.21.15.55:0/3486414447 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec040020 con 0x7f0ffc10a010 2024-04-26T22:03:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.066+0000 7f0ff2ffd640 1 -- 172.21.15.55:0/3486414447 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fec038040 con 0x7f0ffc10a010 2024-04-26T22:03:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.066+0000 7f0ff2ffd640 1 -- 172.21.15.55:0/3486414447 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec037070 con 0x7f0ffc10a010 2024-04-26T22:03:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.066+0000 7f1002e0b640 1 -- 172.21.15.55:0/3486414447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc1a7050 con 0x7f0ffc10a010 2024-04-26T22:03:40.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.066+0000 7f1002e0b640 1 -- 172.21.15.55:0/3486414447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a74f0 con 0x7f0ffc10a010 2024-04-26T22:03:40.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.071+0000 7f1006d85740 1 Processor -- start 2024-04-26T22:03:40.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.078+0000 7f1006d85740 1 -- start start 2024-04-26T22:03:40.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.079+0000 7f1006d85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fb4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.079+0000 7f1006d85740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 0x5583d1fb6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f1006d85740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 0x5583d1fbce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f1006d85740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5583d1f18d80 con 0x5583d1fb6590 2024-04-26T22:03:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f1006d85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5583d1ed2a20 con 0x5583d1fb4a00 2024-04-26T22:03:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f1006d85740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5583d1f14d40 con 0x5583d1fb5bd0 2024-04-26T22:03:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f100260a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fb4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f0ff2ffd640 1 -- 172.21.15.55:0/3486414447 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0fec043020 con 0x7f0ffc10a010 2024-04-26T22:03:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f100260a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fb4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:36930/0 (socket says 172.21.15.55:36930) 2024-04-26T22:03:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f100260a640 1 -- 172.21.15.55:0/1698767703 learned_addr learned my addr 172.21.15.55:0/1698767703 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f1001e09640 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 0x5583d1fb6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.080+0000 7f1001608640 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 0x5583d1fbce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.081+0000 7f1001e09640 1 -- 172.21.15.55:0/1698767703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 msgr2=0x5583d1fb4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.081+0000 7f1001e09640 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fb4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.081+0000 7f1001e09640 1 -- 172.21.15.55:0/1698767703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 msgr2=0x5583d1fbce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.081+0000 7f1001e09640 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 0x5583d1fbce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.081+0000 7f1001e09640 1 -- 172.21.15.55:0/1698767703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d1eddc30 con 0x5583d1fb5bd0 2024-04-26T22:03:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.081+0000 7f100260a640 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fb4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.082+0000 7f1001e09640 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 0x5583d1fb6050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009cc0 tx=0x7f0ff400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e9fcf92017e492 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.082+0000 7f0ff2ffd640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0fe80a3e40 0x7f0fe80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.082+0000 7f0fe7fff640 1 -- 172.21.15.55:0/1698767703 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4016070 con 0x5583d1fb5bd0 2024-04-26T22:03:40.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.083+0000 7f100260a640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0fe80a3e40 0x7f0fe80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.083+0000 7f0ff2ffd640 1 -- 172.21.15.55:0/3486414447 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0fec0f0560 con 0x7f0ffc10a010 2024-04-26T22:03:40.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.083+0000 7f0fe7fff640 1 -- 172.21.15.55:0/1698767703 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff4003970 con 0x5583d1fb5bd0 2024-04-26T22:03:40.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.083+0000 7f0fe7fff640 1 -- 172.21.15.55:0/1698767703 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4004600 con 0x5583d1fb5bd0 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.085+0000 7f1006d85740 1 -- 172.21.15.55:0/1698767703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 msgr2=0x5583d1fb6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.085+0000 7f100260a640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0fe80a3e40 0x7f0fe80a6300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5583d1fb5ad0 tx=0x7f0ff8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.085+0000 7f1006d85740 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 0x5583d1fb6050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009cc0 tx=0x7f0ff400e6c0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.087+0000 7f1006d85740 1 -- 172.21.15.55:0/1698767703 shutdown_connections 2024-04-26T22:03:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.087+0000 7f1006d85740 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 0x5583d1fbce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.088+0000 7f1006d85740 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 0x5583d1fb6050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.088+0000 7f1006d85740 1 --2- 172.21.15.55:0/1698767703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fb4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.088+0000 7f1006d85740 1 -- 172.21.15.55:0/1698767703 >> 172.21.15.55:0/1698767703 conn(0x5583d1ef5910 msgr2=0x5583d1fb0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.092+0000 7f1006d85740 1 -- 172.21.15.55:0/1698767703 shutdown_connections 2024-04-26T22:03:40.093 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.092+0000 7f1006d85740 1 -- 172.21.15.55:0/1698767703 wait complete. 2024-04-26T22:03:40.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.093+0000 7f1006d85740 1 -- start start 2024-04-26T22:03:40.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.093+0000 7f1006d85740 1 Processor -- start 2024-04-26T22:03:40.095 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.094+0000 7f1006d85740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb4a00 0x5583d1fbed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.094+0000 7f1006d85740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 0x5583d1fbf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.095+0000 7f1001e09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb4a00 0x5583d1fbed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.095+0000 7f1001e09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb4a00 0x5583d1fbed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50922/0 (socket says 172.21.15.55:50922) 2024-04-26T22:03:40.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.096+0000 7f1001e09640 1 -- 172.21.15.55:0/2155597472 learned_addr learned my addr 172.21.15.55:0/2155597472 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.096+0000 7f1006d85740 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb6590 0x5583d1fc6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.096+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5583d1fbd5a0 con 0x5583d1fb4a00 2024-04-26T22:03:40.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.096+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5583d1e9fdc0 con 0x5583d1fb6590 2024-04-26T22:03:40.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.096+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5583d1fbd720 con 0x5583d1fb5bd0 2024-04-26T22:03:40.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.097+0000 7f1006d85740 10 client.0 _ll_register_callbacks cb 0x5583d1ebb600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:40.100 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.099+0000 7f1001e09640 1 -- 172.21.15.55:0/2155597472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 msgr2=0x5583d1fbf240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:40.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.099+0000 7f1001e09640 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 0x5583d1fbf240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.099+0000 7f1001e09640 1 -- 172.21.15.55:0/2155597472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb6590 msgr2=0x5583d1fc6280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:40.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.099+0000 7f1001e09640 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb6590 0x5583d1fc6280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.099+0000 7f1001e09640 1 -- 172.21.15.55:0/2155597472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d1c2a3f0 con 0x5583d1fb4a00 2024-04-26T22:03:40.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.100+0000 7f1001e09640 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb4a00 0x5583d1fbed00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4000f80 tx=0x7f0ff4009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd4e6ae9d98b763 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.101 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.100+0000 7f0fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:40.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.101+0000 7f0fe57fa640 1 -- 172.21.15.55:0/2155597472 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4028060 con 0x5583d1fb4a00 2024-04-26T22:03:40.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.101+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff4003970 con 0x5583d1fb4a00 2024-04-26T22:03:40.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.101+0000 7f0fe57fa640 1 -- 172.21.15.55:0/2155597472 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0ff4004c50 con 0x5583d1fb4a00 2024-04-26T22:03:40.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.102+0000 7f0fe57fa640 1 -- 172.21.15.55:0/2155597472 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0ff4016040 con 0x5583d1fb4a00 2024-04-26T22:03:40.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.103+0000 7f1006d85740 10 client.17103 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:40.106 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.103+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5583d1cf47d0 con 0x5583d1fb4a00 2024-04-26T22:03:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.110+0000 7f0fe57fa640 1 -- 172.21.15.55:0/2155597472 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff4003970 con 0x5583d1fb4a00 2024-04-26T22:03:40.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.110+0000 7f0fe57fa640 1 -- 172.21.15.55:0/2155597472 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0ff400f040 con 0x5583d1fb4a00 2024-04-26T22:03:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.112+0000 7f1006d85740 10 client.17103 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:40.114 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.112+0000 7f1006d85740 10 client.17103 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:03:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.112+0000 7f1006d85740 20 client.17103 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.112+0000 7f1006d85740 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5583d1fca8a0 0x5583d1fccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.112+0000 7f1006d85740 4 client.17103 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:40.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.112+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d1f26990 con 0x5583d1fca8a0 2024-04-26T22:03:40.123 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.120+0000 7f1001608640 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5583d1fca8a0 0x5583d1fccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.139+0000 7f1001608640 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5583d1fca8a0 0x5583d1fccd10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=777ebf09e66719b5 server_cookie=2bf5030f15643fb9 in_seq=0 out_seq=0 2024-04-26T22:03:40.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.140+0000 7f0fe57fa640 10 client.17103 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:03:40.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.143+0000 7f0fe57fa640 1 -- 172.21.15.55:0/2155597472 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583d1f26990 con 0x5583d1fca8a0 2024-04-26T22:03:40.145 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.143+0000 7f0fe57fa640 10 client.17103 handle_command_reply: tid=0 2024-04-26T22:03:40.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.145+0000 7f1006d85740 1 client.17103 shutdown 2024-04-26T22:03:40.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.145+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb4a00 msgr2=0x5583d1fbed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.146 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.145+0000 7f1006d85740 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb4a00 0x5583d1fbed00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4000f80 tx=0x7f0ff4009fc0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.146+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 shutdown_connections 2024-04-26T22:03:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.146+0000 7f1006d85740 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5583d1fca8a0 0x5583d1fccd10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.146+0000 7f1006d85740 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb6590 0x5583d1fc6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.146+0000 7f1006d85740 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb5bd0 0x5583d1fbf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.146+0000 7f1006d85740 1 --2- 172.21.15.55:0/2155597472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb4a00 0x5583d1fbed00 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.146+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 >> 172.21.15.55:0/2155597472 conn(0x5583d1ef5910 msgr2=0x5583d1faf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.147+0000 7f0fe57fa640 0 client.17103 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:03:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.148+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 shutdown_connections 2024-04-26T22:03:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.148+0000 7f1006d85740 1 -- 172.21.15.55:0/2155597472 wait complete. 2024-04-26T22:03:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.149+0000 7f1006d85740 20 client.17103 trim_cache size 0 max 16384 2024-04-26T22:03:40.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.215+0000 7f1006d85740 1 Processor -- start 2024-04-26T22:03:40.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.215+0000 7f1006d85740 1 -- start start 2024-04-26T22:03:40.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1006d85740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb4a00 0x5583d1faed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1006d85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1006d85740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 0x5583d1faf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1006d85740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b2ce0 con 0x5583d1fb6590 2024-04-26T22:03:40.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1006d85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b05b0 con 0x5583d1fb5bd0 2024-04-26T22:03:40.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1006d85740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1ae400 con 0x5583d1fb4a00 2024-04-26T22:03:40.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1001608640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1001608640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:36956/0 (socket says 172.21.15.55:36956) 2024-04-26T22:03:40.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.216+0000 7f1001608640 1 -- 172.21.15.55:0/2004659600 learned_addr learned my addr 172.21.15.55:0/2004659600 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.217+0000 7f100260a640 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 0x5583d1faf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.217+0000 7f1001e09640 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb4a00 0x5583d1faed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.223 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.217+0000 7f1001608640 1 -- 172.21.15.55:0/2004659600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb4a00 msgr2=0x5583d1faed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.217+0000 7f1001608640 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb4a00 0x5583d1faed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.217+0000 7f1001608640 1 -- 172.21.15.55:0/2004659600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 msgr2=0x5583d1faf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.217+0000 7f1001608640 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 0x5583d1faf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.217+0000 7f1001608640 1 -- 172.21.15.55:0/2004659600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d1fa1bb0 con 0x5583d1fb5bd0 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.218+0000 7f1001608640 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0bc450 tx=0x7f0fec005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2b06b6e8e36ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.219+0000 7f0fe77fe640 1 -- 172.21.15.55:0/2004659600 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0bc0b0 con 0x5583d1fb5bd0 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.219+0000 7f0fe77fe640 1 -- 172.21.15.55:0/2004659600 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fec005580 con 0x5583d1fb5bd0 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.220+0000 7f1006d85740 1 -- 172.21.15.55:0/2004659600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb5bd0 msgr2=0x5583d1fc6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.220+0000 7f1006d85740 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0bc450 tx=0x7f0fec005f90 comp rx=0 tx=0).stop 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.220+0000 7f1006d85740 1 -- 172.21.15.55:0/2004659600 shutdown_connections 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.220+0000 7f1006d85740 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb6590 0x5583d1faf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.220+0000 7f1006d85740 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc6240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.220+0000 7f1006d85740 1 --2- 172.21.15.55:0/2004659600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb4a00 0x5583d1faed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.220+0000 7f1006d85740 1 -- 172.21.15.55:0/2004659600 >> 172.21.15.55:0/2004659600 conn(0x5583d1ef5910 msgr2=0x5583d1fbcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.221+0000 7f1006d85740 1 -- 172.21.15.55:0/2004659600 shutdown_connections 2024-04-26T22:03:40.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.221+0000 7f1006d85740 1 -- 172.21.15.55:0/2004659600 wait complete. 2024-04-26T22:03:40.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 1 -- start start 2024-04-26T22:03:40.225 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 1 Processor -- start 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fc31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb6590 0x5583d2050e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1aea70 con 0x5583d1fb5bd0 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1afee0 con 0x5583d1fb4a00 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1af140 con 0x5583d1fb6590 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.222+0000 7f1006d85740 10 client.0 _ll_register_callbacks cb 0x7f0fec0f3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.223+0000 7f1001608640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.223+0000 7f1001608640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50944/0 (socket says 172.21.15.55:50944) 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.223+0000 7f1001608640 1 -- 172.21.15.55:0/940932397 learned_addr learned my addr 172.21.15.55:0/940932397 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.224+0000 7f1001608640 1 -- 172.21.15.55:0/940932397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb6590 msgr2=0x5583d2050e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.224+0000 7f1001608640 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb6590 0x5583d2050e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.224+0000 7f1001608640 1 -- 172.21.15.55:0/940932397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 msgr2=0x5583d1fc31a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.224+0000 7f1001608640 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fc31a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.224+0000 7f1001608640 1 -- 172.21.15.55:0/940932397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d2051400 con 0x5583d1fb5bd0 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.224+0000 7f1001608640 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc36e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0ee000 tx=0x7f0fec02a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fbb2be693c9fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.225+0000 7f0fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.226+0000 7f0fe57fa640 1 -- 172.21.15.55:0/940932397 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0bc0b0 con 0x5583d1fb5bd0 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.226+0000 7f1006d85740 1 -- 172.21.15.55:0/940932397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583d1f20800 con 0x5583d1fb5bd0 2024-04-26T22:03:40.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.227+0000 7f0fe57fa640 1 -- 172.21.15.55:0/940932397 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0fec0c4f30 con 0x5583d1fb5bd0 2024-04-26T22:03:40.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.228+0000 7f0fe57fa640 1 -- 172.21.15.55:0/940932397 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0fec009440 con 0x5583d1fb5bd0 2024-04-26T22:03:40.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.228+0000 7f1006d85740 10 client.17109 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:40.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.228+0000 7f1006d85740 1 -- 172.21.15.55:0/940932397 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5583d2052770 con 0x5583d1fb5bd0 2024-04-26T22:03:40.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.229+0000 7f0fe57fa640 1 -- 172.21.15.55:0/940932397 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5583d1f20800 con 0x5583d1fb5bd0 2024-04-26T22:03:40.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.230+0000 7f0fe57fa640 1 -- 172.21.15.55:0/940932397 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f0fec0d4070 con 0x5583d1fb5bd0 2024-04-26T22:03:40.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.230+0000 7f1006d85740 10 client.17109 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:40.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.230+0000 7f1006d85740 10 client.17109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:03:40.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.230+0000 7f1006d85740 20 client.17109 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:40.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.230+0000 7f1006d85740 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5583d20531a0 0x5583d20555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.230+0000 7f1006d85740 4 client.17109 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:03:40.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.230+0000 7f1006d85740 1 -- 172.21.15.55:0/940932397 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583d1e9b610 con 0x5583d20531a0 2024-04-26T22:03:40.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.232+0000 7f1001e09640 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5583d20531a0 0x5583d20555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.242 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.240+0000 7f1001e09640 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5583d20531a0 0x5583d20555c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4de73abdd7e78b6e server_cookie=65173a8d9baf0557 in_seq=0 out_seq=0 2024-04-26T22:03:40.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.240+0000 7f0fe57fa640 10 client.17109 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:03:40.243 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.241+0000 7f0fe57fa640 1 -- 172.21.15.55:0/940932397 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583d1e9b610 con 0x5583d20531a0 2024-04-26T22:03:40.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.241+0000 7f0fe57fa640 10 client.17109 handle_command_reply: tid=0 2024-04-26T22:03:40.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.241+0000 7f1006d85740 1 client.17109 shutdown 2024-04-26T22:03:40.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.241+0000 7f1006d85740 1 -- 172.21.15.55:0/940932397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb5bd0 msgr2=0x5583d1fc36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.241+0000 7f1006d85740 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc36e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0ee000 tx=0x7f0fec02a380 comp rx=0 tx=0).stop 2024-04-26T22:03:40.244 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 1 -- 172.21.15.55:0/940932397 shutdown_connections 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5583d20531a0 0x5583d20555c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5583d1fb6590 0x5583d2050e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5583d1fb5bd0 0x5583d1fc36e0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 1 --2- 172.21.15.55:0/940932397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d1fb4a00 0x5583d1fc31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f0fe57fa640 0 client.17109 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 1 -- 172.21.15.55:0/940932397 >> 172.21.15.55:0/940932397 conn(0x5583d1ef5910 msgr2=0x5583d1faed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 1 -- 172.21.15.55:0/940932397 shutdown_connections 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 1 -- 172.21.15.55:0/940932397 wait complete. 2024-04-26T22:03:40.245 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.242+0000 7f1006d85740 20 client.17109 trim_cache size 0 max 16384 2024-04-26T22:03:40.246 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:03:40.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.243+0000 7f1002e0b640 1 -- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0fe80a3e40 msgr2=0x7f0fe80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.243+0000 7f1002e0b640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0fe80a3e40 0x7f0fe80a6300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5583d1fb5ad0 tx=0x7f0ff8009260 comp rx=0 tx=0).stop 2024-04-26T22:03:40.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.244+0000 7f1002e0b640 1 -- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc10a010 msgr2=0x7f0ffc1165e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.244+0000 7f1002e0b640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc10a010 0x7f0ffc1165e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0fec000c00 tx=0x7f0fec004270 comp rx=0 tx=0).stop 2024-04-26T22:03:40.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.244+0000 7f1002e0b640 1 -- 172.21.15.55:0/3486414447 shutdown_connections 2024-04-26T22:03:40.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.244+0000 7f1002e0b640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0fe80a3e40 0x7f0fe80a6300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.247 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.244+0000 7f1002e0b640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ffc10a9b0 0x7f0ffc1a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.244+0000 7f1002e0b640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ffc10a010 0x7f0ffc1165e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.244+0000 7f1002e0b640 1 --2- 172.21.15.55:0/3486414447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ffc073d70 0x7f0ffc1160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.244+0000 7f1002e0b640 1 -- 172.21.15.55:0/3486414447 >> 172.21.15.55:0/3486414447 conn(0x7f0ffc06f840 msgr2=0x7f0ffc071540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.245+0000 7f1002e0b640 1 -- 172.21.15.55:0/3486414447 shutdown_connections 2024-04-26T22:03:40.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.245+0000 7f1002e0b640 1 -- 172.21.15.55:0/3486414447 wait complete. 2024-04-26T22:03:40.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:40 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1629950357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:03:40.265 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:40 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1629950357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:03:40.267 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:03:40.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:40 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1629950357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 -- 172.21.15.55:0/3743645068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204108670 msgr2=0x7f520410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 --2- 172.21.15.55:0/3743645068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204108670 0x7f520410aac0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009ca0 tx=0x7f51fc02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 -- 172.21.15.55:0/3743645068 shutdown_connections 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 --2- 172.21.15.55:0/3743645068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204108670 0x7f520410aac0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 --2- 172.21.15.55:0/3743645068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5204105d40 0x7f5204108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 --2- 172.21.15.55:0/3743645068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5204103410 0x7f5204105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 -- 172.21.15.55:0/3743645068 >> 172.21.15.55:0/3743645068 conn(0x7f52040fce20 msgr2=0x7f52040ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 -- 172.21.15.55:0/3743645068 shutdown_connections 2024-04-26T22:03:40.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.583+0000 7f520a522640 1 -- 172.21.15.55:0/3743645068 wait complete. 2024-04-26T22:03:40.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.584+0000 7f520a522640 1 Processor -- start 2024-04-26T22:03:40.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.584+0000 7f520a522640 1 -- start start 2024-04-26T22:03:40.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.584+0000 7f520a522640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5204103410 0x7f5204102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.584+0000 7f520a522640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204105d40 0x7f52041013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.584+0000 7f520a522640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5204108670 0x7f5204101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.584+0000 7f520a522640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f520410e510 con 0x7f5204105d40 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.584+0000 7f520a522640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f520410e390 con 0x7f5204108670 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.584+0000 7f520a522640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f520410e690 con 0x7f5204103410 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204105d40 0x7f52041013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204105d40 0x7f52041013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50958/0 (socket says 172.21.15.55:50958) 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 -- 172.21.15.55:0/3699519930 learned_addr learned my addr 172.21.15.55:0/3699519930 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f5203fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5204103410 0x7f5204102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f5208abf640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5204108670 0x7f5204101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 -- 172.21.15.55:0/3699519930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5204103410 msgr2=0x7f5204102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5204103410 0x7f5204102d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 -- 172.21.15.55:0/3699519930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5204108670 msgr2=0x7f5204101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5204108670 0x7f5204101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 -- 172.21.15.55:0/3699519930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5204102190 con 0x7f5204105d40 2024-04-26T22:03:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.585+0000 7f51fb7fe640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204105d40 0x7f52041013c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00d770 tx=0x7f51ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.586+0000 7f5201ffb640 1 -- 172.21.15.55:0/3699519930 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec017070 con 0x7f5204105d40 2024-04-26T22:03:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.586+0000 7f520a522640 1 -- 172.21.15.55:0/3699519930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52041a2b20 con 0x7f5204105d40 2024-04-26T22:03:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.586+0000 7f520a522640 1 -- 172.21.15.55:0/3699519930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52041a3060 con 0x7f5204105d40 2024-04-26T22:03:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.586+0000 7f5201ffb640 1 -- 172.21.15.55:0/3699519930 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec009c30 con 0x7f5204105d40 2024-04-26T22:03:40.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.587+0000 7f5201ffb640 1 -- 172.21.15.55:0/3699519930 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec021060 con 0x7f5204105d40 2024-04-26T22:03:40.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f520d23d740 1 Processor -- start 2024-04-26T22:03:40.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f520d23d740 1 -- start start 2024-04-26T22:03:40.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f520d23d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f24c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f520d23d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f25e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f520d23d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 0x561187f2cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f520d23d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561187e88d80 con 0x561187f24820 2024-04-26T22:03:40.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f520d23d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561187e42a20 con 0x561187f259f0 2024-04-26T22:03:40.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f520d23d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561187e84d40 con 0x561187f263b0 2024-04-26T22:03:40.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f5208abf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f24c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f5208abf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f24c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50974/0 (socket says 172.21.15.55:50974) 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.590+0000 7f5208abf640 1 -- 172.21.15.55:0/1868036723 learned_addr learned my addr 172.21.15.55:0/1868036723 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5208abf640 1 -- 172.21.15.55:0/1868036723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 msgr2=0x561187f2cc40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5208abf640 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 0x561187f2cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5208abf640 1 -- 172.21.15.55:0/1868036723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 msgr2=0x561187f25e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5203fff640 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f25e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5208abf640 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f25e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5208abf640 1 -- 172.21.15.55:0/1868036723 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561187e4dc30 con 0x561187f24820 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5203fff640 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f25e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5201ffb640 1 -- 172.21.15.55:0/3699519930 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f51ec01a070 con 0x7f5204105d40 2024-04-26T22:03:40.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.591+0000 7f5208abf640 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f24c40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005e30 tx=0x7f51fc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3a2b91eb1d01f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.593+0000 7f5201ffb640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51f00a3d40 0x7f51f00a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.593+0000 7f5201ffb640 1 -- 172.21.15.55:0/3699519930 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f51ec0d00c0 con 0x7f5204105d40 2024-04-26T22:03:40.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.593+0000 7f51fa7fc640 1 -- 172.21.15.55:0/1868036723 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc031d60 con 0x561187f24820 2024-04-26T22:03:40.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.593+0000 7f51fa7fc640 1 -- 172.21.15.55:0/1868036723 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51fc036040 con 0x561187f24820 2024-04-26T22:03:40.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.593+0000 7f51fa7fc640 1 -- 172.21.15.55:0/1868036723 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0042a0 con 0x561187f24820 2024-04-26T22:03:40.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.593+0000 7f5203fff640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51f00a3d40 0x7f51f00a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.594+0000 7f5203fff640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51f00a3d40 0x7f51f00a6200 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f51f400bcb0 tx=0x7f51f400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.594+0000 7f520d23d740 1 -- 172.21.15.55:0/1868036723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 msgr2=0x561187f24c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.595+0000 7f520d23d740 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f24c40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f51fc005e30 tx=0x7f51fc009800 comp rx=0 tx=0).stop 2024-04-26T22:03:40.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.595+0000 7f520d23d740 1 -- 172.21.15.55:0/1868036723 shutdown_connections 2024-04-26T22:03:40.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.595+0000 7f520d23d740 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 0x561187f2cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.595+0000 7f520d23d740 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f25e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.595+0000 7f520d23d740 1 --2- 172.21.15.55:0/1868036723 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f24c40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.595+0000 7f520d23d740 1 -- 172.21.15.55:0/1868036723 >> 172.21.15.55:0/1868036723 conn(0x561187e65910 msgr2=0x561187f20120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.595+0000 7f520d23d740 1 -- 172.21.15.55:0/1868036723 shutdown_connections 2024-04-26T22:03:40.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.595+0000 7f520d23d740 1 -- 172.21.15.55:0/1868036723 wait complete. 2024-04-26T22:03:40.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.596+0000 7f520d23d740 1 -- start start 2024-04-26T22:03:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.596+0000 7f520d23d740 1 Processor -- start 2024-04-26T22:03:40.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.597+0000 7f520d23d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f24820 0x561187f2eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.597+0000 7f520d23d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f259f0 0x561187f2f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.597+0000 7f520d23d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 0x561187f36070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f5203fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f24820 0x561187f2eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f5203fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f24820 0x561187f2eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37000/0 (socket says 172.21.15.55:37000) 2024-04-26T22:03:40.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f5203fff640 1 -- 172.21.15.55:0/3479974072 learned_addr learned my addr 172.21.15.55:0/3479974072 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f5208abf640 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 0x561187f36070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561187f2d360 con 0x561187f259f0 2024-04-26T22:03:40.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561187e0fdc0 con 0x561187f24820 2024-04-26T22:03:40.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561187f2d4e0 con 0x561187f263b0 2024-04-26T22:03:40.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f520d23d740 10 client.0 _ll_register_callbacks cb 0x561187e2b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:40.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.598+0000 7f51fb7fe640 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f259f0 0x561187f2f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f5208abf640 1 -- 172.21.15.55:0/3479974072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f24820 msgr2=0x561187f2eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f5208abf640 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f24820 0x561187f2eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f5208abf640 1 -- 172.21.15.55:0/3479974072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f259f0 msgr2=0x561187f2f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f5208abf640 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f259f0 0x561187f2f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f5208abf640 1 -- 172.21.15.55:0/3479974072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561187b9a3f0 con 0x561187f263b0 2024-04-26T22:03:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f5203fff640 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f24820 0x561187f2eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:03:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f5208abf640 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 0x561187f36070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f51fc02aeb0 tx=0x7f51fc031e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=525679d08d5642d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f51e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:03:40.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f51e37fe640 1 -- 172.21.15.55:0/3479974072 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc052020 con 0x561187f263b0 2024-04-26T22:03:40.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.599+0000 7f51e37fe640 1 -- 172.21.15.55:0/3479974072 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f51fc031650 con 0x561187f263b0 2024-04-26T22:03:40.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.600+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51fc036040 con 0x561187f263b0 2024-04-26T22:03:40.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.601+0000 7f51e37fe640 1 -- 172.21.15.55:0/3479974072 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f51fc04a040 con 0x561187f263b0 2024-04-26T22:03:40.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.601+0000 7f520d23d740 10 client.26146 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:40.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.601+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561187c647d0 con 0x561187f263b0 2024-04-26T22:03:40.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.601+0000 7f51e37fe640 1 -- 172.21.15.55:0/3479974072 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51fc031850 con 0x561187f263b0 2024-04-26T22:03:40.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.601+0000 7f51e37fe640 1 -- 172.21.15.55:0/3479974072 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f51fc04d030 con 0x561187f263b0 2024-04-26T22:03:40.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.602+0000 7f520d23d740 10 client.26146 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:40.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.602+0000 7f520d23d740 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:40.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.602+0000 7f520d23d740 20 client.26146 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:40.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.602+0000 7f520d23d740 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561187f3a4b0 0x561187f3c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.602+0000 7f520d23d740 4 client.26146 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:40.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.602+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f51fc036040 con 0x561187f3a4b0 2024-04-26T22:03:40.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.603+0000 7f5203fff640 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561187f3a4b0 0x561187f3c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.603+0000 7f5203fff640 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561187f3a4b0 0x561187f3c8a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f5ebadd0b092e3 server_cookie=74d7550e76c6c1c9 in_seq=0 out_seq=0 2024-04-26T22:03:40.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.603+0000 7f51e37fe640 10 client.26146 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:40.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f51e37fe640 1 -- 172.21.15.55:0/3479974072 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f51fc036040 con 0x561187f3a4b0 2024-04-26T22:03:40.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f51e37fe640 10 client.26146 handle_command_reply: tid=0 2024-04-26T22:03:40.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 client.26146 shutdown 2024-04-26T22:03:40.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 msgr2=0x561187f36070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 0x561187f36070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f51fc02aeb0 tx=0x7f51fc031e10 comp rx=0 tx=0).stop 2024-04-26T22:03:40.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 shutdown_connections 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f263b0 0x561187f36070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561187f3a4b0 0x561187f3c8a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f51e37fe640 0 client.26146 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f259f0 0x561187f2f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 --2- 172.21.15.55:0/3479974072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f24820 0x561187f2eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 >> 172.21.15.55:0/3479974072 conn(0x561187e65910 msgr2=0x561187f1f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 shutdown_connections 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 1 -- 172.21.15.55:0/3479974072 wait complete. 2024-04-26T22:03:40.611 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.609+0000 7f520d23d740 20 client.26146 trim_cache size 0 max 16384 2024-04-26T22:03:40.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.631+0000 7f520d23d740 1 Processor -- start 2024-04-26T22:03:40.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.631+0000 7f520d23d740 1 -- start start 2024-04-26T22:03:40.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.631+0000 7f520d23d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f29350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.632+0000 7f520d23d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f259f0 0x561187f36030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.632+0000 7f520d23d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f263b0 0x561187f29890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.632+0000 7f520d23d740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f52041ae8f0 con 0x561187f24820 2024-04-26T22:03:40.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.633+0000 7f520d23d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52041ac1c0 con 0x561187f263b0 2024-04-26T22:03:40.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.633+0000 7f520d23d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52041aa010 con 0x561187f259f0 2024-04-26T22:03:40.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.633+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f259f0 0x561187f36030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.633+0000 7f5203fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f29350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.633+0000 7f5203fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f29350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50992/0 (socket says 172.21.15.55:50992) 2024-04-26T22:03:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.633+0000 7f5203fff640 1 -- 172.21.15.55:0/4028779412 learned_addr learned my addr 172.21.15.55:0/4028779412 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.635 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.633+0000 7f5208abf640 1 --2- 172.21.15.55:0/4028779412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f263b0 0x561187f29890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.635+0000 7f5208abf640 1 -- 172.21.15.55:0/4028779412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f259f0 msgr2=0x561187f36030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.635+0000 7f5208abf640 1 --2- 172.21.15.55:0/4028779412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f259f0 0x561187f36030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.635+0000 7f5208abf640 1 -- 172.21.15.55:0/4028779412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 msgr2=0x561187f29350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.635+0000 7f5208abf640 1 --2- 172.21.15.55:0/4028779412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f29350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.635+0000 7f5208abf640 1 -- 172.21.15.55:0/4028779412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561187f20dd0 con 0x561187f263b0 2024-04-26T22:03:40.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.636+0000 7f5208abf640 1 --2- 172.21.15.55:0/4028779412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f263b0 0x561187f29890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f51fc02ade0 tx=0x7f51fc030f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a42c0e26000579f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.648+0000 7f51f9ffb640 1 -- 172.21.15.55:0/4028779412 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc052040 con 0x561187f263b0 2024-04-26T22:03:40.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.648+0000 7f51f9ffb640 1 -- 172.21.15.55:0/4028779412 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51fc036040 con 0x561187f263b0 2024-04-26T22:03:40.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.648+0000 7f51f9ffb640 1 -- 172.21.15.55:0/4028779412 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc02f610 con 0x561187f263b0 2024-04-26T22:03:40.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.656+0000 7f520d23d740 1 -- 172.21.15.55:0/4028779412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f263b0 msgr2=0x561187f29890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.656+0000 7f520d23d740 1 --2- 172.21.15.55:0/4028779412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f263b0 0x561187f29890 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f51fc02ade0 tx=0x7f51fc030f90 comp rx=0 tx=0).stop 2024-04-26T22:03:40.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.658+0000 7f520d23d740 1 -- 172.21.15.55:0/4028779412 shutdown_connections 2024-04-26T22:03:40.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.658+0000 7f520d23d740 1 --2- 172.21.15.55:0/4028779412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f263b0 0x561187f29890 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.660 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.658+0000 7f520d23d740 1 --2- 172.21.15.55:0/4028779412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f259f0 0x561187f36030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.658+0000 7f520d23d740 1 --2- 172.21.15.55:0/4028779412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f24820 0x561187f29350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.661 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.658+0000 7f520d23d740 1 -- 172.21.15.55:0/4028779412 >> 172.21.15.55:0/4028779412 conn(0x561187e65910 msgr2=0x561187f1b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.666+0000 7f520d23d740 1 -- 172.21.15.55:0/4028779412 shutdown_connections 2024-04-26T22:03:40.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.667+0000 7f520d23d740 1 -- 172.21.15.55:0/4028779412 wait complete. 2024-04-26T22:03:40.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.668+0000 7f520d23d740 1 -- start start 2024-04-26T22:03:40.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.669+0000 7f520d23d740 1 Processor -- start 2024-04-26T22:03:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.669+0000 7f520d23d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f24820 0x561187f331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.670+0000 7f520d23d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.670+0000 7f520d23d740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f263b0 0x561187f346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f5203fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f24820 0x561187f331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f5203fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f24820 0x561187f331a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:45446/0 (socket says 172.21.15.55:45446) 2024-04-26T22:03:40.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f336e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:37012/0 (socket says 172.21.15.55:37012) 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f5203fff640 1 -- 172.21.15.55:0/3465660452 learned_addr learned my addr 172.21.15.55:0/3465660452 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f5208abf640 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f263b0 0x561187f346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f52041aa680 con 0x561187f263b0 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52041abaf0 con 0x561187f259f0 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f52041aad50 con 0x561187f24820 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.671+0000 7f520d23d740 10 client.0 _ll_register_callbacks cb 0x7f51fc009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.672+0000 7f5208abf640 1 -- 172.21.15.55:0/3465660452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f24820 msgr2=0x561187f331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.672+0000 7f5208abf640 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f24820 0x561187f331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.672+0000 7f5208abf640 1 -- 172.21.15.55:0/3465660452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 msgr2=0x561187f336e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:40.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.672+0000 7f5208abf640 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f336e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.672+0000 7f5208abf640 1 -- 172.21.15.55:0/3465660452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561187f34ce0 con 0x561187f263b0 2024-04-26T22:03:40.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.673+0000 7f5208abf640 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f263b0 0x561187f346f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f51fc02ae80 tx=0x7f51fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfe4a45e185a667 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:40.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.673+0000 7f51e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:40.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.675+0000 7f51e37fe640 1 -- 172.21.15.55:0/3465660452 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc030a70 con 0x561187f263b0 2024-04-26T22:03:40.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.675+0000 7f51e37fe640 1 -- 172.21.15.55:0/3465660452 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f51fc02f610 con 0x561187f263b0 2024-04-26T22:03:40.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.675+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51fc036040 con 0x561187f263b0 2024-04-26T22:03:40.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.677+0000 7f51e37fe640 1 -- 172.21.15.55:0/3465660452 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f51fc06f050 con 0x561187f263b0 2024-04-26T22:03:40.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.678+0000 7f520d23d740 10 client.17127 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:40.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.678+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561187f35fa0 con 0x561187f263b0 2024-04-26T22:03:40.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.680+0000 7f51e37fe640 1 -- 172.21.15.55:0/3465660452 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51fc064020 con 0x561187f263b0 2024-04-26T22:03:40.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.681+0000 7f51e37fe640 1 -- 172.21.15.55:0/3465660452 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f51fc04d040 con 0x561187f263b0 2024-04-26T22:03:40.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.682+0000 7f520d23d740 10 client.17127 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:40.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.682+0000 7f520d23d740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:40.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.682+0000 7f520d23d740 20 client.17127 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:40.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.682+0000 7f520d23d740 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561187f3a4b0 0x561187fa2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:40.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.683+0000 7f520d23d740 4 client.17127 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:03:40.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.683+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f51fc036040 con 0x561187f3a4b0 2024-04-26T22:03:40.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.685+0000 7f5203fff640 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561187f3a4b0 0x561187fa2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:40.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.707+0000 7f5203fff640 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561187f3a4b0 0x561187fa2950 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7398d45e9f0715e server_cookie=3f1eb0d7f35e6a7b in_seq=0 out_seq=0 2024-04-26T22:03:40.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.708+0000 7f51e37fe640 10 client.17127 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:40.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.711+0000 7f51e37fe640 1 -- 172.21.15.55:0/3465660452 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f51fc036040 con 0x561187f3a4b0 2024-04-26T22:03:40.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.711+0000 7f51e37fe640 10 client.17127 handle_command_reply: tid=0 2024-04-26T22:03:40.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.712+0000 7f520d23d740 1 client.17127 shutdown 2024-04-26T22:03:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.716+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f263b0 msgr2=0x561187f346f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.716+0000 7f520d23d740 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f263b0 0x561187f346f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f51fc02ae80 tx=0x7f51fc02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.722+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 shutdown_connections 2024-04-26T22:03:40.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.722+0000 7f520d23d740 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x561187f3a4b0 0x561187fa2950 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.723+0000 7f520d23d740 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561187f263b0 0x561187f346f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.723+0000 7f520d23d740 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561187f259f0 0x561187f336e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.723+0000 7f520d23d740 1 --2- 172.21.15.55:0/3465660452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561187f24820 0x561187f331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.723+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 >> 172.21.15.55:0/3465660452 conn(0x561187e65910 msgr2=0x561187f1e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.723+0000 7f51e37fe640 0 client.17127 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:40.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.734+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 shutdown_connections 2024-04-26T22:03:40.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.734+0000 7f520d23d740 1 -- 172.21.15.55:0/3465660452 wait complete. 2024-04-26T22:03:40.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.742+0000 7f520d23d740 20 client.17127 trim_cache size 0 max 16384 2024-04-26T22:03:40.754 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:03:40.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.753+0000 7f520a522640 1 -- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51f00a3d40 msgr2=0x7f51f00a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.754+0000 7f520a522640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51f00a3d40 0x7f51f00a6200 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f51f400bcb0 tx=0x7f51f400b390 comp rx=0 tx=0).stop 2024-04-26T22:03:40.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.754+0000 7f520a522640 1 -- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204105d40 msgr2=0x7f52041013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:40.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.754+0000 7f520a522640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204105d40 0x7f52041013c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00d770 tx=0x7f51ec00dc30 comp rx=0 tx=0).stop 2024-04-26T22:03:40.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.756+0000 7f520a522640 1 -- 172.21.15.55:0/3699519930 shutdown_connections 2024-04-26T22:03:40.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.756+0000 7f520a522640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f51f00a3d40 0x7f51f00a6200 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.756+0000 7f520a522640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5204108670 0x7f5204101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.756+0000 7f520a522640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5204105d40 0x7f52041013c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.756+0000 7f520a522640 1 --2- 172.21.15.55:0/3699519930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5204103410 0x7f5204102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:40.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.756+0000 7f520a522640 1 -- 172.21.15.55:0/3699519930 >> 172.21.15.55:0/3699519930 conn(0x7f52040fce20 msgr2=0x7f5204106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.767+0000 7f520a522640 1 -- 172.21.15.55:0/3699519930 shutdown_connections 2024-04-26T22:03:40.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:40.767+0000 7f520a522640 1 -- 172.21.15.55:0/3699519930 wait complete. 2024-04-26T22:03:41.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:41 smithi055 ceph-mon[29221]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 855 MiB/s wr, 1.09k op/s 2024-04-26T22:03:41.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:41 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:41.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:41 smithi165 ceph-mon[33296]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 855 MiB/s wr, 1.09k op/s 2024-04-26T22:03:41.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:41 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:41 smithi064 ceph-mon[32765]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 855 MiB/s wr, 1.09k op/s 2024-04-26T22:03:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:41 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:41.820 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:03:41.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:03:41.820 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:03:42.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.409+0000 7f302ec20640 1 -- 172.21.15.55:0/194920159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3028109930 msgr2=0x7f302810bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.409+0000 7f302ec20640 1 --2- 172.21.15.55:0/194920159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3028109930 0x7f302810bd60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f301c009860 tx=0x7f301c02a340 comp rx=0 tx=0).stop 2024-04-26T22:03:42.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.411+0000 7f302ec20640 1 -- 172.21.15.55:0/194920159 shutdown_connections 2024-04-26T22:03:42.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.411+0000 7f302ec20640 1 --2- 172.21.15.55:0/194920159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f302810ec70 0x7f302810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.411+0000 7f302ec20640 1 --2- 172.21.15.55:0/194920159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f302810c2a0 0x7f302810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.411+0000 7f302ec20640 1 --2- 172.21.15.55:0/194920159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3028109930 0x7f302810bd60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.415 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.411+0000 7f302ec20640 1 -- 172.21.15.55:0/194920159 >> 172.21.15.55:0/194920159 conn(0x7f302806f7c0 msgr2=0x7f3028071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:42.429 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.428+0000 7f302ec20640 1 -- 172.21.15.55:0/194920159 shutdown_connections 2024-04-26T22:03:42.434 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.434+0000 7f302ec20640 1 -- 172.21.15.55:0/194920159 wait complete. 2024-04-26T22:03:42.435 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.435+0000 7f302ec20640 1 Processor -- start 2024-04-26T22:03:42.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.436+0000 7f302ec20640 1 -- start start 2024-04-26T22:03:42.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.437+0000 7f302ec20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3028109930 0x7f30281a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.437+0000 7f302ec20640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f302810c2a0 0x7f30281a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.437+0000 7f302ec20640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f302810ec70 0x7f30281af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.437+0000 7f302ec20640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f302811d2d0 con 0x7f302810ec70 2024-04-26T22:03:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.437+0000 7f302ec20640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f302811d150 con 0x7f3028109930 2024-04-26T22:03:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.437+0000 7f302ec20640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f302811d450 con 0x7f302810c2a0 2024-04-26T22:03:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.438+0000 7f302d1bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f302810ec70 0x7f30281af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.438 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.438+0000 7f302d1bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f302810ec70 0x7f30281af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42082/0 (socket says 172.21.15.55:42082) 2024-04-26T22:03:42.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.438+0000 7f302d1bd640 1 -- 172.21.15.55:0/3945783317 learned_addr learned my addr 172.21.15.55:0/3945783317 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:42.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.440+0000 7f3027fff640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f302810c2a0 0x7f30281a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.440+0000 7f302c9bc640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3028109930 0x7f30281a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.441+0000 7f302d1bd640 1 -- 172.21.15.55:0/3945783317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f302810c2a0 msgr2=0x7f30281a7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.441+0000 7f302d1bd640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f302810c2a0 0x7f30281a7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.441+0000 7f302d1bd640 1 -- 172.21.15.55:0/3945783317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3028109930 msgr2=0x7f30281a7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.441+0000 7f302d1bd640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3028109930 0x7f30281a7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.441+0000 7f302d1bd640 1 -- 172.21.15.55:0/3945783317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30281af670 con 0x7f302810ec70 2024-04-26T22:03:42.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.448+0000 7f302d1bd640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f302810ec70 0x7f30281af000 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f302000eb40 tx=0x7f302000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.450+0000 7f3025ffb640 1 -- 172.21.15.55:0/3945783317 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020004b20 con 0x7f302810ec70 2024-04-26T22:03:42.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.450+0000 7f3025ffb640 1 -- 172.21.15.55:0/3945783317 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3020004cc0 con 0x7f302810ec70 2024-04-26T22:03:42.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.450+0000 7f3025ffb640 1 -- 172.21.15.55:0/3945783317 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020010720 con 0x7f302810ec70 2024-04-26T22:03:42.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.452+0000 7f302ec20640 1 -- 172.21.15.55:0/3945783317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30281af960 con 0x7f302810ec70 2024-04-26T22:03:42.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.453+0000 7f302ec20640 1 -- 172.21.15.55:0/3945783317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30281afe30 con 0x7f302810ec70 2024-04-26T22:03:42.462 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.461+0000 7f3031a15740 1 Processor -- start 2024-04-26T22:03:42.463 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.461+0000 7f3031a15740 1 -- start start 2024-04-26T22:03:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.464+0000 7f3031a15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17407cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.464+0000 7f3031a15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 0x563d1740cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.465+0000 7f3031a15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 0x563d1740f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.466 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.465+0000 7f3031a15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563d1736cb00 con 0x563d174078b0 2024-04-26T22:03:42.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.465+0000 7f3031a15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563d17308920 con 0x563d174082a0 2024-04-26T22:03:42.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.465+0000 7f3031a15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d17368ac0 con 0x563d1740d410 2024-04-26T22:03:42.467 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.466+0000 7f3027fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17407cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.466+0000 7f3027fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17407cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42090/0 (socket says 172.21.15.55:42090) 2024-04-26T22:03:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.466+0000 7f3027fff640 1 -- 172.21.15.55:0/4060147789 learned_addr learned my addr 172.21.15.55:0/4060147789 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.467+0000 7f302c9bc640 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 0x563d1740cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.469 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.467+0000 7f3027fff640 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 0x563d1740f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.469+0000 7f3027fff640 1 -- 172.21.15.55:0/4060147789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 msgr2=0x563d1740f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:42.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.469+0000 7f3027fff640 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 0x563d1740f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.469+0000 7f3027fff640 1 -- 172.21.15.55:0/4060147789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 msgr2=0x563d1740cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:42.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.469+0000 7f3027fff640 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 0x563d1740cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.470 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.469+0000 7f3027fff640 1 -- 172.21.15.55:0/4060147789 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d171467e0 con 0x563d174078b0 2024-04-26T22:03:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.471+0000 7f3025ffb640 1 -- 172.21.15.55:0/3945783317 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3020005560 con 0x7f302810ec70 2024-04-26T22:03:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.473+0000 7f3025ffb640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30100a3dc0 0x7f30100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.473+0000 7f3025ffb640 1 -- 172.21.15.55:0/3945783317 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f30200d1130 con 0x7f302810ec70 2024-04-26T22:03:42.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.480+0000 7f302c9bc640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30100a3dc0 0x7f30100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.480 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.480+0000 7f3027fff640 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17407cd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3018009510 tx=0x7f30180096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97eb3fec4bb8e73d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.481+0000 7f300effd640 1 -- 172.21.15.55:0/4060147789 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018010690 con 0x563d174078b0 2024-04-26T22:03:42.485 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.485+0000 7f300effd640 1 -- 172.21.15.55:0/4060147789 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3018010ce0 con 0x563d174078b0 2024-04-26T22:03:42.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.487+0000 7f3031a15740 1 -- 172.21.15.55:0/4060147789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 msgr2=0x563d17407cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.487+0000 7f3031a15740 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17407cd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3018009510 tx=0x7f30180096c0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.487+0000 7f3031a15740 1 -- 172.21.15.55:0/4060147789 shutdown_connections 2024-04-26T22:03:42.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.487+0000 7f3031a15740 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 0x563d1740f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.487+0000 7f3031a15740 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 0x563d1740cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.488 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.487+0000 7f3031a15740 1 --2- 172.21.15.55:0/4060147789 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17407cd0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.487+0000 7f3031a15740 1 -- 172.21.15.55:0/4060147789 >> 172.21.15.55:0/4060147789 conn(0x563d17402f20 msgr2=0x563d17403320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:42.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.488+0000 7f302c9bc640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30100a3dc0 0x7f30100a6280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f301c009780 tx=0x7f301c030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.489 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.488+0000 7f3031a15740 1 -- 172.21.15.55:0/4060147789 shutdown_connections 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.488+0000 7f3031a15740 1 -- 172.21.15.55:0/4060147789 wait complete. 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.488+0000 7f3031a15740 1 -- start start 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.489+0000 7f3031a15740 1 Processor -- start 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.489+0000 7f3031a15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.490+0000 7f3031a15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 0x563d17412270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.490+0000 7f3031a15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 0x563d174192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.490+0000 7f3031a15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563d17410460 con 0x563d174078b0 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.490+0000 7f3031a15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563d173065d0 con 0x563d174082a0 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.490+0000 7f3031a15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d174105e0 con 0x563d1740d410 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.490+0000 7f3031a15740 10 client.0 _ll_register_callbacks cb 0x563d172f4080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:42.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.491+0000 7f302c9bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.491+0000 7f302c9bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42098/0 (socket says 172.21.15.55:42098) 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.491+0000 7f302c9bc640 1 -- 172.21.15.55:0/3797865731 learned_addr learned my addr 172.21.15.55:0/3797865731 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.492+0000 7f302c9bc640 1 -- 172.21.15.55:0/3797865731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 msgr2=0x563d174192b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.492+0000 7f302c9bc640 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 0x563d174192b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.492+0000 7f302c9bc640 1 -- 172.21.15.55:0/3797865731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 msgr2=0x563d17412270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.492+0000 7f302c9bc640 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 0x563d17412270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.492+0000 7f302c9bc640 1 -- 172.21.15.55:0/3797865731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d171a6f70 con 0x563d174078b0 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.493+0000 7f302c9bc640 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411d30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f301c009860 tx=0x7f301c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8640f2bc0e92f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.493+0000 7f3007fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.494+0000 7f3007fff640 1 -- 172.21.15.55:0/3797865731 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c037040 con 0x563d174078b0 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.494+0000 7f3007fff640 1 -- 172.21.15.55:0/3797865731 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f301c004ad0 con 0x563d174078b0 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.494+0000 7f3031a15740 1 -- 172.21.15.55:0/3797865731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3018010ce0 con 0x563d174078b0 2024-04-26T22:03:42.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.495+0000 7f3007fff640 1 -- 172.21.15.55:0/3797865731 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f301c004cd0 con 0x563d174078b0 2024-04-26T22:03:42.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.495+0000 7f3031a15740 10 client.17139 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:42.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.495+0000 7f3031a15740 1 -- 172.21.15.55:0/3797865731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563d1703e170 con 0x563d174078b0 2024-04-26T22:03:42.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.498+0000 7f3007fff640 1 -- 172.21.15.55:0/3797865731 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3018010ce0 con 0x563d174078b0 2024-04-26T22:03:42.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.498+0000 7f3007fff640 1 -- 172.21.15.55:0/3797865731 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f301c030550 con 0x563d174078b0 2024-04-26T22:03:42.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.498+0000 7f3031a15740 10 client.17139 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:42.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.498+0000 7f3031a15740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:42.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.498+0000 7f3031a15740 20 client.17139 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:42.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.498+0000 7f3031a15740 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563d1741d910 0x563d1741fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.499+0000 7f3031a15740 4 client.17139 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:42.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.499+0000 7f3031a15740 1 -- 172.21.15.55:0/3797865731 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d1737a710 con 0x563d1741d910 2024-04-26T22:03:42.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.500+0000 7f3027fff640 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563d1741d910 0x563d1741fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.503+0000 7f3027fff640 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563d1741d910 0x563d1741fdd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb8e1dc2906e76b server_cookie=5aa64bae4ccd8f86 in_seq=0 out_seq=0 2024-04-26T22:03:42.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.503+0000 7f3007fff640 10 client.17139 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:42.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.507+0000 7f3007fff640 1 -- 172.21.15.55:0/3797865731 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d1737a710 con 0x563d1741d910 2024-04-26T22:03:42.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.507+0000 7f3007fff640 10 client.17139 handle_command_reply: tid=0 2024-04-26T22:03:42.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.508+0000 7f3031a15740 1 client.17139 shutdown 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.508+0000 7f3031a15740 1 -- 172.21.15.55:0/3797865731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 msgr2=0x563d17411d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.508+0000 7f3031a15740 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411d30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f301c009860 tx=0x7f301c02a340 comp rx=0 tx=0).stop 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.509+0000 7f3031a15740 1 -- 172.21.15.55:0/3797865731 shutdown_connections 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.509+0000 7f3031a15740 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d1740d410 0x563d174192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.509+0000 7f3031a15740 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d174082a0 0x563d17412270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.509+0000 7f3031a15740 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563d1741d910 0x563d1741fdd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.509+0000 7f3007fff640 0 client.17139 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.509+0000 7f3031a15740 1 --2- 172.21.15.55:0/3797865731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411d30 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.509+0000 7f3031a15740 1 -- 172.21.15.55:0/3797865731 >> 172.21.15.55:0/3797865731 conn(0x563d17402f20 msgr2=0x563d174020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:42.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.511+0000 7f3031a15740 1 -- 172.21.15.55:0/3797865731 shutdown_connections 2024-04-26T22:03:42.514 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.514+0000 7f3031a15740 1 -- 172.21.15.55:0/3797865731 wait complete. 2024-04-26T22:03:42.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.516+0000 7f3031a15740 20 client.17139 trim_cache size 0 max 16384 2024-04-26T22:03:42.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.575+0000 7f3031a15740 1 Processor -- start 2024-04-26T22:03:42.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.575+0000 7f3031a15740 1 -- start start 2024-04-26T22:03:42.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.575+0000 7f3031a15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f3031a15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d174082a0 0x563d17419270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f3031a15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 0x563d17411260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f302c9bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f302c9bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42102/0 (socket says 172.21.15.55:42102) 2024-04-26T22:03:42.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f302c9bc640 1 -- 172.21.15.55:0/1701018489 learned_addr learned my addr 172.21.15.55:0/1701018489 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:42.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f3031a15740 1 -- 172.21.15.55:0/1701018489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30281bb660 con 0x563d174078b0 2024-04-26T22:03:42.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f302d1bd640 1 --2- 172.21.15.55:0/1701018489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 0x563d17411260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f3031a15740 1 -- 172.21.15.55:0/1701018489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30281b8f30 con 0x563d1740d410 2024-04-26T22:03:42.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.576+0000 7f3031a15740 1 -- 172.21.15.55:0/1701018489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30281b6d80 con 0x563d174082a0 2024-04-26T22:03:42.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.577+0000 7f302c9bc640 1 -- 172.21.15.55:0/1701018489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d174082a0 msgr2=0x563d17419270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:42.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.577+0000 7f302c9bc640 1 --2- 172.21.15.55:0/1701018489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d174082a0 0x563d17419270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.577+0000 7f302c9bc640 1 -- 172.21.15.55:0/1701018489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 msgr2=0x563d17411260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:42.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.577+0000 7f302c9bc640 1 --2- 172.21.15.55:0/1701018489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 0x563d17411260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.578 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.577+0000 7f302c9bc640 1 -- 172.21.15.55:0/1701018489 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d17403bf0 con 0x563d174078b0 2024-04-26T22:03:42.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.578+0000 7f302c9bc640 1 --2- 172.21.15.55:0/1701018489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411fb0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f301c0309c0 tx=0x7f301c004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7c532c50821aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.579+0000 7f300e7fc640 1 -- 172.21.15.55:0/1701018489 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c037070 con 0x563d174078b0 2024-04-26T22:03:42.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.579+0000 7f300e7fc640 1 -- 172.21.15.55:0/1701018489 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f301c046040 con 0x563d174078b0 2024-04-26T22:03:42.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.579+0000 7f300e7fc640 1 -- 172.21.15.55:0/1701018489 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c003d00 con 0x563d174078b0 2024-04-26T22:03:42.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.581+0000 7f3031a15740 1 -- 172.21.15.55:0/1701018489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 msgr2=0x563d17411fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.581+0000 7f3031a15740 1 --2- 172.21.15.55:0/1701018489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411fb0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f301c0309c0 tx=0x7f301c004e70 comp rx=0 tx=0).stop 2024-04-26T22:03:42.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.581+0000 7f3031a15740 1 -- 172.21.15.55:0/1701018489 shutdown_connections 2024-04-26T22:03:42.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.581+0000 7f3031a15740 1 --2- 172.21.15.55:0/1701018489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 0x563d17411260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.581+0000 7f3031a15740 1 --2- 172.21.15.55:0/1701018489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d174082a0 0x563d17419270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.581+0000 7f3031a15740 1 --2- 172.21.15.55:0/1701018489 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17411fb0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.581+0000 7f3031a15740 1 -- 172.21.15.55:0/1701018489 >> 172.21.15.55:0/1701018489 conn(0x563d17402f20 msgr2=0x563d17405f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:42.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.581+0000 7f3031a15740 1 -- 172.21.15.55:0/1701018489 shutdown_connections 2024-04-26T22:03:42.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.582+0000 7f3031a15740 1 -- 172.21.15.55:0/1701018489 wait complete. 2024-04-26T22:03:42.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.582+0000 7f3031a15740 1 -- start start 2024-04-26T22:03:42.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.582+0000 7f3031a15740 1 Processor -- start 2024-04-26T22:03:42.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f3031a15740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17416ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f3031a15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d174082a0 0x563d17417410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f3031a15740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 0x563d17418420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f3031a15740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30281b73f0 con 0x563d174078b0 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f302c9bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17416ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f302c9bc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17416ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42116/0 (socket says 172.21.15.55:42116) 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f3031a15740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30281b8860 con 0x563d1740d410 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30281b7ac0 con 0x563d174082a0 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.583+0000 7f302c9bc640 1 -- 172.21.15.55:0/4143232422 learned_addr learned my addr 172.21.15.55:0/4143232422 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.584+0000 7f3031a15740 10 client.0 _ll_register_callbacks cb 0x7f301c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.584+0000 7f302c9bc640 1 -- 172.21.15.55:0/4143232422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d174082a0 msgr2=0x563d17417410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:42.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.584+0000 7f302c9bc640 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d174082a0 0x563d17417410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.584+0000 7f302c9bc640 1 -- 172.21.15.55:0/4143232422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 msgr2=0x563d17418420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:03:42.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.584+0000 7f302c9bc640 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 0x563d17418420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.584+0000 7f302c9bc640 1 -- 172.21.15.55:0/4143232422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d17418a10 con 0x563d174078b0 2024-04-26T22:03:42.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.584+0000 7f302c9bc640 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17416ed0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f301c030a40 tx=0x7f301c030de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a621ac61410d503 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:42.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.584+0000 7f3007fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:03:42.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.585+0000 7f3007fff640 1 -- 172.21.15.55:0/4143232422 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c037040 con 0x563d174078b0 2024-04-26T22:03:42.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.585+0000 7f3007fff640 1 -- 172.21.15.55:0/4143232422 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f301c003a00 con 0x563d174078b0 2024-04-26T22:03:42.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.585+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f301c046040 con 0x563d174078b0 2024-04-26T22:03:42.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.585+0000 7f3007fff640 1 -- 172.21.15.55:0/4143232422 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f301c06a050 con 0x563d174078b0 2024-04-26T22:03:42.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.586+0000 7f3031a15740 10 client.17145 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:42.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.586+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563d174843e0 con 0x563d174078b0 2024-04-26T22:03:42.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.586+0000 7f3007fff640 1 -- 172.21.15.55:0/4143232422 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f301c046040 con 0x563d174078b0 2024-04-26T22:03:42.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.586+0000 7f3007fff640 1 -- 172.21.15.55:0/4143232422 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f301c032c20 con 0x563d174078b0 2024-04-26T22:03:42.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.586+0000 7f3031a15740 10 client.17145 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:42.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.586+0000 7f3031a15740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:42.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.586+0000 7f3031a15740 20 client.17145 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:42.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.586+0000 7f3031a15740 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563d1741d910 0x563d17486d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:42.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.587+0000 7f3031a15740 4 client.17145 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:03:42.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.587+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563d17374580 con 0x563d1741d910 2024-04-26T22:03:42.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.588+0000 7f3027fff640 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563d1741d910 0x563d17486d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:42.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.593+0000 7f3027fff640 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563d1741d910 0x563d17486d00 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51fec289115b3c6 server_cookie=a17b0b85a47957eb in_seq=0 out_seq=0 2024-04-26T22:03:42.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.593+0000 7f3007fff640 10 client.17145 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:42.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.883+0000 7f3007fff640 1 -- 172.21.15.55:0/4143232422 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563d17374580 con 0x563d1741d910 2024-04-26T22:03:42.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.883+0000 7f3007fff640 10 client.17145 handle_command_reply: tid=0 2024-04-26T22:03:42.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.883+0000 7f3031a15740 1 client.17145 shutdown 2024-04-26T22:03:42.891 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:03:42.891 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:03:42.891 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "4b1b761d-8cfc-410e-b596-71ab70bc18f6", 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 msgr2=0x563d17416ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17416ed0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f301c030a40 tx=0x7f301c030de0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 shutdown_connections 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d1740d410 0x563d17418420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d174082a0 0x563d17417410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563d1741d910 0x563d17486d00 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 --2- 172.21.15.55:0/4143232422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563d174078b0 0x563d17416ed0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 >> 172.21.15.55:0/4143232422 conn(0x563d17402f20 msgr2=0x563d174020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:42.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3007fff640 0 client.17145 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:42.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 shutdown_connections 2024-04-26T22:03:42.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 1 -- 172.21.15.55:0/4143232422 wait complete. 2024-04-26T22:03:42.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.888+0000 7f3031a15740 20 client.17145 trim_cache size 0 max 16384 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 -- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30100a3dc0 msgr2=0x7f30100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30100a3dc0 0x7f30100a6280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f301c009780 tx=0x7f301c030450 comp rx=0 tx=0).stop 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 -- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f302810ec70 msgr2=0x7f30281af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f302810ec70 0x7f30281af000 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f302000eb40 tx=0x7f302000c2d0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 -- 172.21.15.55:0/3945783317 shutdown_connections 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30100a3dc0 0x7f30100a6280 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f302810ec70 0x7f30281af000 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f302810c2a0 0x7f30281a7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 --2- 172.21.15.55:0/3945783317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3028109930 0x7f30281a7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:42.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 -- 172.21.15.55:0/3945783317 >> 172.21.15.55:0/3945783317 conn(0x7f302806f7c0 msgr2=0x7f302810c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:42.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 -- 172.21.15.55:0/3945783317 shutdown_connections 2024-04-26T22:03:42.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:42.905+0000 7f302ec20640 1 -- 172.21.15.55:0/3945783317 wait complete. 2024-04-26T22:03:42.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:03:42.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:03:43.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:43 smithi165 ceph-mon[33296]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 870 MiB/s wr, 1.14k op/s 2024-04-26T22:03:43.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:43 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:03:43.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:43 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:43.341 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:43 smithi055 ceph-mon[29221]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 870 MiB/s wr, 1.14k op/s 2024-04-26T22:03:43.341 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:43 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:03:43.341 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:43 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:43.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.342+0000 7fe3b7fff640 1 -- 172.21.15.55:0/277054582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b810a9b0 msgr2=0x7fe3b810f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:43.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.342+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/277054582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b810a9b0 0x7fe3b810f270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009ca0 tx=0x7fe3b002a8d0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.342+0000 7fe3b7fff640 1 -- 172.21.15.55:0/277054582 shutdown_connections 2024-04-26T22:03:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.342+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/277054582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b810a9b0 0x7fe3b810f270 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.342+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/277054582 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b810a010 0x7fe3b810a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.342+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/277054582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b8073d70 0x7fe3b8074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.342+0000 7fe3b7fff640 1 -- 172.21.15.55:0/277054582 >> 172.21.15.55:0/277054582 conn(0x7fe3b806f840 msgr2=0x7fe3b8071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:43.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.352+0000 7fe3b7fff640 1 -- 172.21.15.55:0/277054582 shutdown_connections 2024-04-26T22:03:43.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.357+0000 7fe3b7fff640 1 -- 172.21.15.55:0/277054582 wait complete. 2024-04-26T22:03:43.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.357+0000 7fe3b7fff640 1 Processor -- start 2024-04-26T22:03:43.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.358+0000 7fe3b7fff640 1 -- start start 2024-04-26T22:03:43.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.358+0000 7fe3b7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b8073d70 0x7fe3b8116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.358+0000 7fe3b7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b810a010 0x7fe3b81165a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.358+0000 7fe3b7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b810a9b0 0x7fe3b81b7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.358+0000 7fe3b7fff640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b811d170 con 0x7fe3b810a9b0 2024-04-26T22:03:43.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.358+0000 7fe3b7fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b811cff0 con 0x7fe3b810a010 2024-04-26T22:03:43.359 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.358+0000 7fe3b7fff640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b811d2f0 con 0x7fe3b8073d70 2024-04-26T22:03:43.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.359+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b8073d70 0x7fe3b8116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.359+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b8073d70 0x7fe3b8116060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:41948/0 (socket says 172.21.15.55:41948) 2024-04-26T22:03:43.362 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.359+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/2293340486 learned_addr learned my addr 172.21.15.55:0/2293340486 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:43.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.359+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/2293340486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b810a010 msgr2=0x7fe3b81165a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:03:43.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.359+0000 7fe3b6ffd640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b810a010 0x7fe3b81165a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.359+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b810a9b0 msgr2=0x7fe3b81b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:43.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.359+0000 7fe3b6ffd640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b810a9b0 0x7fe3b81b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.359+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/2293340486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b81b7d80 con 0x7fe3b8073d70 2024-04-26T22:03:43.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.360+0000 7fe3b6ffd640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b8073d70 0x7fe3b8116060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00d950 tx=0x7fe3ac00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:43.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.365+0000 7fe39ffff640 1 -- 172.21.15.55:0/2293340486 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac017070 con 0x7fe3b8073d70 2024-04-26T22:03:43.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.365+0000 7fe3b7fff640 1 -- 172.21.15.55:0/2293340486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b81b7f50 con 0x7fe3b8073d70 2024-04-26T22:03:43.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.365+0000 7fe3b7fff640 1 -- 172.21.15.55:0/2293340486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b81b8380 con 0x7fe3b8073d70 2024-04-26T22:03:43.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.365+0000 7fe39ffff640 1 -- 172.21.15.55:0/2293340486 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ac009c30 con 0x7fe3b8073d70 2024-04-26T22:03:43.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.365+0000 7fe39ffff640 1 -- 172.21.15.55:0/2293340486 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00f620 con 0x7fe3b8073d70 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.371+0000 7fe39ffff640 1 -- 172.21.15.55:0/2293340486 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe3ac00f7c0 con 0x7fe3b8073d70 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.373+0000 7fe39ffff640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a00a3e40 0x7fe3a00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.373+0000 7fe39ffff640 1 -- 172.21.15.55:0/2293340486 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe3ac0d0190 con 0x7fe3b8073d70 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.374+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a00a3e40 0x7fe3a00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.376+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a00a3e40 0x7fe3a00a6300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8117520 tx=0x7fe3a800b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.377+0000 7fe3c0424740 1 Processor -- start 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.378+0000 7fe3c0424740 1 -- start start 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.378+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed461260b0 0x55ed461264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.378+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127280 0x55ed46127700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.378+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed4612e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.378+0000 7fe3c0424740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ed4608ad80 con 0x55ed46127c40 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.378+0000 7fe3c0424740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ed460448b0 con 0x55ed46127280 2024-04-26T22:03:43.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.378+0000 7fe3c0424740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed46086d40 con 0x55ed461260b0 2024-04-26T22:03:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.379+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed461260b0 0x55ed461264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.379+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed461260b0 0x55ed461264d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:41954/0 (socket says 172.21.15.55:41954) 2024-04-26T22:03:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.379+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/2515123827 learned_addr learned my addr 172.21.15.55:0/2515123827 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:43.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.380+0000 7fe3b77fe640 1 --2- 172.21.15.55:0/2515123827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed4612e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.380+0000 7fe3b77fe640 1 -- 172.21.15.55:0/2515123827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed461260b0 msgr2=0x55ed461264d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.380+0000 7fe3b77fe640 1 --2- 172.21.15.55:0/2515123827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed461260b0 0x55ed461264d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.380+0000 7fe3b77fe640 1 -- 172.21.15.55:0/2515123827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127280 msgr2=0x55ed46127700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.380+0000 7fe3b77fe640 1 --2- 172.21.15.55:0/2515123827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127280 0x55ed46127700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.380+0000 7fe3b77fe640 1 -- 172.21.15.55:0/2515123827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed46015a40 con 0x55ed46127c40 2024-04-26T22:03:43.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.381+0000 7fe3b77fe640 1 --2- 172.21.15.55:0/2515123827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed4612e4d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009bf0 tx=0x7fe3b002f320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b951abf25cacc56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:43.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.385+0000 7fe39d7fa640 1 -- 172.21.15.55:0/2515123827 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0031460 con 0x55ed46127c40 2024-04-26T22:03:43.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.386+0000 7fe39d7fa640 1 -- 172.21.15.55:0/2515123827 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b003f070 con 0x55ed46127c40 2024-04-26T22:03:43.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.386+0000 7fe39d7fa640 1 -- 172.21.15.55:0/2515123827 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00043a0 con 0x55ed46127c40 2024-04-26T22:03:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.387+0000 7fe3c0424740 1 -- 172.21.15.55:0/2515123827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 msgr2=0x55ed4612e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.387+0000 7fe3c0424740 1 --2- 172.21.15.55:0/2515123827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed4612e4d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009bf0 tx=0x7fe3b002f320 comp rx=0 tx=0).stop 2024-04-26T22:03:43.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.391+0000 7fe3c0424740 1 -- 172.21.15.55:0/2515123827 shutdown_connections 2024-04-26T22:03:43.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.391+0000 7fe3c0424740 1 --2- 172.21.15.55:0/2515123827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed4612e4d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.391+0000 7fe3c0424740 1 --2- 172.21.15.55:0/2515123827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127280 0x55ed46127700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.391+0000 7fe3c0424740 1 --2- 172.21.15.55:0/2515123827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed461260b0 0x55ed461264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.391+0000 7fe3c0424740 1 -- 172.21.15.55:0/2515123827 >> 172.21.15.55:0/2515123827 conn(0x55ed461215a0 msgr2=0x55ed461219a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:43.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.392+0000 7fe3c0424740 1 -- 172.21.15.55:0/2515123827 shutdown_connections 2024-04-26T22:03:43.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.392+0000 7fe3c0424740 1 -- 172.21.15.55:0/2515123827 wait complete. 2024-04-26T22:03:43.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 1 -- start start 2024-04-26T22:03:43.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 1 Processor -- start 2024-04-26T22:03:43.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed461260b0 0x55ed46130390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127280 0x55ed461308d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127c40 0x55ed46137910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55ed4612ec40 con 0x55ed461260b0 2024-04-26T22:03:43.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ed46011d10 con 0x55ed46127c40 2024-04-26T22:03:43.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ed4612edc0 con 0x55ed46127280 2024-04-26T22:03:43.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.393+0000 7fe3c0424740 10 client.0 _ll_register_callbacks cb 0x55ed4603e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:43.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.394+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed461260b0 0x55ed46130390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.394+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed461260b0 0x55ed46130390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42158/0 (socket says 172.21.15.55:42158) 2024-04-26T22:03:43.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.394+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/992160889 learned_addr learned my addr 172.21.15.55:0/992160889 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:43.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.394+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127280 0x55ed461308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.394+0000 7fe3b77fe640 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127c40 0x55ed46137910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.395+0000 7fe3b67fc640 1 -- 172.21.15.55:0/992160889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127c40 msgr2=0x55ed46137910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:43.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.395+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127c40 0x55ed46137910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.395+0000 7fe3b67fc640 1 -- 172.21.15.55:0/992160889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed461260b0 msgr2=0x55ed46130390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:43.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.395+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed461260b0 0x55ed46130390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.395+0000 7fe3b67fc640 1 -- 172.21.15.55:0/992160889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed45e667d0 con 0x55ed46127280 2024-04-26T22:03:43.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.395+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127280 0x55ed461308d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800d6b0 tx=0x7fe3a8010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a38b5b599631259 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:43.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.401+0000 7fe39aff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:03:43.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.401+0000 7fe39aff5640 1 -- 172.21.15.55:0/992160889 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8021020 con 0x55ed46127280 2024-04-26T22:03:43.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.401+0000 7fe39aff5640 1 -- 172.21.15.55:0/992160889 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe3a8010da0 con 0x55ed46127280 2024-04-26T22:03:43.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.401+0000 7fe3c0424740 1 -- 172.21.15.55:0/992160889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b003f070 con 0x55ed46127280 2024-04-26T22:03:43.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.402+0000 7fe39aff5640 1 -- 172.21.15.55:0/992160889 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe3a801d440 con 0x55ed46127280 2024-04-26T22:03:43.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.402+0000 7fe3c0424740 10 client.26170 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:43.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.402+0000 7fe3c0424740 1 -- 172.21.15.55:0/992160889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ed45d9c3f0 con 0x55ed46127280 2024-04-26T22:03:43.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.405+0000 7fe39aff5640 1 -- 172.21.15.55:0/992160889 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b003f070 con 0x55ed46127280 2024-04-26T22:03:43.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.406+0000 7fe39aff5640 1 -- 172.21.15.55:0/992160889 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe3a8026810 con 0x55ed46127280 2024-04-26T22:03:43.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.407+0000 7fe3c0424740 10 client.26170 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:43.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.407+0000 7fe3c0424740 10 client.26170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:43.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.407+0000 7fe3c0424740 20 client.26170 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:43.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.407+0000 7fe3c0424740 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ed4613c300 0x55ed4613e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.407+0000 7fe3c0424740 4 client.26170 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:03:43.407 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.407+0000 7fe3c0424740 1 -- 172.21.15.55:0/992160889 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed46098990 con 0x55ed4613c300 2024-04-26T22:03:43.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.412+0000 7fe3b6ffd640 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ed4613c300 0x55ed4613e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.426+0000 7fe3b6ffd640 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ed4613c300 0x55ed4613e760 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f0853ea1b8ea7a server_cookie=8714bd0ca8be6137 in_seq=0 out_seq=0 2024-04-26T22:03:43.426 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.426+0000 7fe39aff5640 10 client.26170 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.436+0000 7fe39aff5640 1 -- 172.21.15.55:0/992160889 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed46098990 con 0x55ed4613c300 2024-04-26T22:03:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.436+0000 7fe39aff5640 10 client.26170 handle_command_reply: tid=0 2024-04-26T22:03:43.437 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.437+0000 7fe3c0424740 1 client.26170 shutdown 2024-04-26T22:03:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.442+0000 7fe3c0424740 1 -- 172.21.15.55:0/992160889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127280 msgr2=0x55ed461308d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:43.442 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.442+0000 7fe3c0424740 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127280 0x55ed461308d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800d6b0 tx=0x7fe3a8010830 comp rx=0 tx=0).stop 2024-04-26T22:03:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.442+0000 7fe3c0424740 1 -- 172.21.15.55:0/992160889 shutdown_connections 2024-04-26T22:03:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.442+0000 7fe3c0424740 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed46127c40 0x55ed46137910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.442+0000 7fe3c0424740 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127280 0x55ed461308d0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800d6b0 tx=0x7fe3a8010830 comp rx=0 tx=0).stop 2024-04-26T22:03:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.442+0000 7fe3c0424740 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ed4613c300 0x55ed4613e760 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.442+0000 7fe3c0424740 1 --2- 172.21.15.55:0/992160889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed461260b0 0x55ed46130390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.442+0000 7fe3c0424740 1 -- 172.21.15.55:0/992160889 >> 172.21.15.55:0/992160889 conn(0x55ed461215a0 msgr2=0x55ed461206b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:43.446 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.445+0000 7fe39aff5640 0 client.26170 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.447+0000 7fe3c0424740 1 -- 172.21.15.55:0/992160889 shutdown_connections 2024-04-26T22:03:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.447+0000 7fe3c0424740 1 -- 172.21.15.55:0/992160889 wait complete. 2024-04-26T22:03:43.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.448+0000 7fe3c0424740 20 client.26170 trim_cache size 0 max 16384 2024-04-26T22:03:43.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.495+0000 7fe3c0424740 1 Processor -- start 2024-04-26T22:03:43.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.495+0000 7fe3c0424740 1 -- start start 2024-04-26T22:03:43.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.496+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed4612e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.496+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127c40 0x55ed461378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.496+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed4612b250 0x55ed4611fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.496+0000 7fe3c0424740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4010e30 con 0x55ed4612b250 2024-04-26T22:03:43.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.496+0000 7fe3b77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed4612b250 0x55ed4611fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.497+0000 7fe3b77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed4612b250 0x55ed4611fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42172/0 (socket says 172.21.15.55:42172) 2024-04-26T22:03:43.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.497+0000 7fe3b77fe640 1 -- 172.21.15.55:0/3583127295 learned_addr learned my addr 172.21.15.55:0/3583127295 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:43.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.497+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed4612e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.497+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/3583127295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127c40 0x55ed461378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.497+0000 7fe3c0424740 1 -- 172.21.15.55:0/3583127295 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a400e700 con 0x55ed461260b0 2024-04-26T22:03:43.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.497+0000 7fe3c0424740 1 -- 172.21.15.55:0/3583127295 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a400c600 con 0x55ed46127c40 2024-04-26T22:03:43.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.499+0000 7fe3b67fc640 1 -- 172.21.15.55:0/3583127295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 msgr2=0x55ed4612e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:43.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.499+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/3583127295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed4612e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.499+0000 7fe3b67fc640 1 -- 172.21.15.55:0/3583127295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed4612b250 msgr2=0x55ed4611fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:43.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.499+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/3583127295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed4612b250 0x55ed4611fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.499+0000 7fe3b67fc640 1 -- 172.21.15.55:0/3583127295 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed45d5c0f0 con 0x55ed46127c40 2024-04-26T22:03:43.503 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.501+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/3583127295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127c40 0x55ed461378d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8016000 tx=0x7fe3a8005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49632550f2e4ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:43.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.505+0000 7fe39cff9640 1 -- 172.21.15.55:0/3583127295 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8036020 con 0x55ed46127c40 2024-04-26T22:03:43.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.505+0000 7fe39cff9640 1 -- 172.21.15.55:0/3583127295 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a8016030 con 0x55ed46127c40 2024-04-26T22:03:43.507 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.505+0000 7fe39cff9640 1 -- 172.21.15.55:0/3583127295 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8021040 con 0x55ed46127c40 2024-04-26T22:03:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:43 smithi064 ceph-mon[32765]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 870 MiB/s wr, 1.14k op/s 2024-04-26T22:03:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:43 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:03:43.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:43 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:43.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.509+0000 7fe3c0424740 1 -- 172.21.15.55:0/3583127295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127c40 msgr2=0x55ed461378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:43.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.509+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3583127295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127c40 0x55ed461378d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8016000 tx=0x7fe3a8005980 comp rx=0 tx=0).stop 2024-04-26T22:03:43.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.511+0000 7fe3c0424740 1 -- 172.21.15.55:0/3583127295 shutdown_connections 2024-04-26T22:03:43.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.511+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3583127295 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed4612b250 0x55ed4611fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.511+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3583127295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46127c40 0x55ed461378d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.511+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3583127295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed4612e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.511+0000 7fe3c0424740 1 -- 172.21.15.55:0/3583127295 >> 172.21.15.55:0/3583127295 conn(0x55ed461215a0 msgr2=0x55ed4612ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:43.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.518+0000 7fe3c0424740 1 -- 172.21.15.55:0/3583127295 shutdown_connections 2024-04-26T22:03:43.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.518+0000 7fe3c0424740 1 -- 172.21.15.55:0/3583127295 wait complete. 2024-04-26T22:03:43.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.519+0000 7fe3c0424740 1 -- start start 2024-04-26T22:03:43.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.519+0000 7fe3c0424740 1 Processor -- start 2024-04-26T22:03:43.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.520+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed46134ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.520+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed46135530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3c0424740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46136540 0x55ed461369c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed46134ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3c0424740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a400cbc0 con 0x55ed46127c40 2024-04-26T22:03:43.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3c0424740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a400e030 con 0x55ed461260b0 2024-04-26T22:03:43.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3c0424740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a400d290 con 0x55ed46136540 2024-04-26T22:03:43.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed46134ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43814/0 (socket says 172.21.15.55:43814) 2024-04-26T22:03:43.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/3539567697 learned_addr learned my addr 172.21.15.55:0/3539567697 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:03:43.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3c0424740 10 client.0 _ll_register_callbacks cb 0x7fe3a8005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:03:43.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed46135530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.521+0000 7fe3b77fe640 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46136540 0x55ed461369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.522+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/3539567697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46136540 msgr2=0x55ed461369c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:43.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.522+0000 7fe3b6ffd640 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46136540 0x55ed461369c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.522+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/3539567697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 msgr2=0x55ed46135530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:03:43.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.522+0000 7fe3b6ffd640 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed46135530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.522+0000 7fe3b6ffd640 1 -- 172.21.15.55:0/3539567697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4604b4b0 con 0x55ed461260b0 2024-04-26T22:03:43.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.522+0000 7fe3b6ffd640 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed46134ff0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac005b60 tx=0x7fe3ac0a4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e022edeaa9db7d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:03:43.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.522+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed46135530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:03:43.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.524+0000 7fe39aff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:03:43.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.524+0000 7fe39aff5640 1 -- 172.21.15.55:0/3539567697 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac020040 con 0x55ed461260b0 2024-04-26T22:03:43.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.525+0000 7fe39aff5640 1 -- 172.21.15.55:0/3539567697 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe3ac0981f0 con 0x55ed461260b0 2024-04-26T22:03:43.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.527+0000 7fe3c0424740 1 -- 172.21.15.55:0/3539567697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3a8016030 con 0x55ed461260b0 2024-04-26T22:03:43.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.531+0000 7fe39aff5640 1 -- 172.21.15.55:0/3539567697 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe3ac0983f0 con 0x55ed461260b0 2024-04-26T22:03:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.533+0000 7fe3c0424740 10 client.26168 fetch_fsmap learned FSMap version 23 2024-04-26T22:03:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.533+0000 7fe3c0424740 1 -- 172.21.15.55:0/3539567697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ed461a2ad0 con 0x55ed461260b0 2024-04-26T22:03:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.536+0000 7fe39aff5640 1 -- 172.21.15.55:0/3539567697 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a8016030 con 0x55ed461260b0 2024-04-26T22:03:43.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.536+0000 7fe39aff5640 1 -- 172.21.15.55:0/3539567697 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe3ac0a0120 con 0x55ed461260b0 2024-04-26T22:03:43.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.537+0000 7fe3c0424740 10 client.26168 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:03:43.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.537+0000 7fe3c0424740 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:03:43.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.537+0000 7fe3c0424740 20 client.26168 populate_metadata read hostname 'smithi055' 2024-04-26T22:03:43.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.537+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ed461a3480 0x55ed461a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:03:43.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.537+0000 7fe3c0424740 4 client.26168 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:03:43.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.537+0000 7fe3c0424740 1 -- 172.21.15.55:0/3539567697 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed46092800 con 0x55ed461a3480 2024-04-26T22:03:43.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.538+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ed461a3480 0x55ed461a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:03:43.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.541+0000 7fe3b67fc640 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ed461a3480 0x55ed461a58a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e755067660e5ff88 server_cookie=e3b5ee236b767ad3 in_seq=0 out_seq=0 2024-04-26T22:03:43.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.541+0000 7fe39aff5640 10 client.26168 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:43.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.545+0000 7fe39aff5640 1 -- 172.21.15.55:0/3539567697 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55ed46092800 con 0x55ed461a3480 2024-04-26T22:03:43.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.545+0000 7fe39aff5640 10 client.26168 handle_command_reply: tid=0 2024-04-26T22:03:43.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.546+0000 7fe3c0424740 1 client.26168 shutdown 2024-04-26T22:03:43.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.555+0000 7fe3c0424740 1 -- 172.21.15.55:0/3539567697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 msgr2=0x55ed46134ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:43.556 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.555+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed46134ff0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac005b60 tx=0x7fe3ac0a4040 comp rx=0 tx=0).stop 2024-04-26T22:03:43.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.566+0000 7fe3c0424740 1 -- 172.21.15.55:0/3539567697 shutdown_connections 2024-04-26T22:03:43.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.566+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ed46136540 0x55ed461369c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.566+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55ed461a3480 0x55ed461a58a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.566 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.566+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ed46127c40 0x55ed46135530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.566+0000 7fe3c0424740 1 --2- 172.21.15.55:0/3539567697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed461260b0 0x55ed46134ff0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.566+0000 7fe3c0424740 1 -- 172.21.15.55:0/3539567697 >> 172.21.15.55:0/3539567697 conn(0x55ed461215a0 msgr2=0x55ed4611fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:43.567 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.566+0000 7fe39aff5640 0 client.26168 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:03:43.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.568+0000 7fe3c0424740 1 -- 172.21.15.55:0/3539567697 shutdown_connections 2024-04-26T22:03:43.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.568+0000 7fe3c0424740 1 -- 172.21.15.55:0/3539567697 wait complete. 2024-04-26T22:03:43.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.573+0000 7fe3c0424740 20 client.26168 trim_cache size 0 max 16384 2024-04-26T22:03:43.577 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:03:43.577 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (3 inodes in the stack)", 2024-04-26T22:03:43.577 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:03:43.578 INFO:teuthology.orchestra.run.smithi055.stdout: "4b1b761d-8cfc-410e-b596-71ab70bc18f6": { 2024-04-26T22:03:43.578 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:03:43.578 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "4b1b761d-8cfc-410e-b596-71ab70bc18f6", 2024-04-26T22:03:43.578 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:03:43.578 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:03:43.578 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:03:43.578 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:03:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.581+0000 7fe3b7fff640 1 -- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a00a3e40 msgr2=0x7fe3a00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.581+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a00a3e40 0x7fe3a00a6300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8117520 tx=0x7fe3a800b370 comp rx=0 tx=0).stop 2024-04-26T22:03:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.581+0000 7fe3b7fff640 1 -- 172.21.15.55:0/2293340486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b8073d70 msgr2=0x7fe3b8116060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:03:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.581+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b8073d70 0x7fe3b8116060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00d950 tx=0x7fe3ac00de10 comp rx=0 tx=0).stop 2024-04-26T22:03:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.599+0000 7fe3b7fff640 1 -- 172.21.15.55:0/2293340486 shutdown_connections 2024-04-26T22:03:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.599+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a00a3e40 0x7fe3a00a6300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.599+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3b810a9b0 0x7fe3b81b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.599+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3b810a010 0x7fe3b81165a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.599+0000 7fe3b7fff640 1 --2- 172.21.15.55:0/2293340486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b8073d70 0x7fe3b8116060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:03:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.599+0000 7fe3b7fff640 1 -- 172.21.15.55:0/2293340486 >> 172.21.15.55:0/2293340486 conn(0x7fe3b806f840 msgr2=0x7fe3b80701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:03:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.599+0000 7fe3b7fff640 1 -- 172.21.15.55:0/2293340486 shutdown_connections 2024-04-26T22:03:43.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:03:43.600+0000 7fe3b7fff640 1 -- 172.21.15.55:0/2293340486 wait complete. 2024-04-26T22:03:43.643 INFO:tasks.cephfs.filesystem:scrub status for tag:4b1b761d-8cfc-410e-b596-71ab70bc18f6 - {'path': '/', 'tag': '4b1b761d-8cfc-410e-b596-71ab70bc18f6', 'options': 'recursive,force'} 2024-04-26T22:03:44.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:44 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:03:44.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:44 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:44.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:44 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:03:44.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:44 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:44 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:44 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:44 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:44 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:03:44.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:44 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:44 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:03:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:44 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:44 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:44 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:44 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:44 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:03:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:44 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:44 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:03:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:44 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:44 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:44 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:44.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:44 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:45.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:45 smithi165 ceph-mon[33296]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.27k op/s 2024-04-26T22:03:45.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:45 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:45.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:45 smithi055 ceph-mon[29221]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.27k op/s 2024-04-26T22:03:45.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:45 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:45.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:45 smithi064 ceph-mon[32765]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.27k op/s 2024-04-26T22:03:45.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:45 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:03:47.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:47 smithi165 ceph-mon[33296]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 772 MiB/s wr, 1.20k op/s 2024-04-26T22:03:47.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:47 smithi055 ceph-mon[29221]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 772 MiB/s wr, 1.20k op/s 2024-04-26T22:03:47.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:47 smithi064 ceph-mon[32765]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 772 MiB/s wr, 1.20k op/s 2024-04-26T22:03:49.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:48 smithi055 ceph-mon[29221]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 883 MiB/s wr, 1.52k op/s 2024-04-26T22:03:49.264 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:49.265 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:48 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:49.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:48 smithi064 ceph-mon[32765]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 883 MiB/s wr, 1.52k op/s 2024-04-26T22:03:49.265 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:49.266 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:48 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:49.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:48 smithi165 ceph-mon[33296]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 883 MiB/s wr, 1.52k op/s 2024-04-26T22:03:49.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:49.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:48 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:51.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:50 smithi055 ceph-mon[29221]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.49k op/s 2024-04-26T22:03:51.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:51.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:51.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:50 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:50 smithi064 ceph-mon[32765]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.49k op/s 2024-04-26T22:03:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:51.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:50 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:51.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:50 smithi165 ceph-mon[33296]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 844 MiB/s wr, 1.49k op/s 2024-04-26T22:03:51.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:51.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:51.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:50 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:03:52.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:51 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:03:52.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:51 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:03:52.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:51 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:03:53.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:52 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:03:53.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:52 smithi064 ceph-mon[32765]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 1.54k op/s 2024-04-26T22:03:53.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:52 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:03:53.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:52 smithi165 ceph-mon[33296]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 1.54k op/s 2024-04-26T22:03:53.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:52 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:03:53.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:52 smithi055 ceph-mon[29221]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 859 MiB/s wr, 1.54k op/s 2024-04-26T22:03:55.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:54 smithi064 ceph-mon[32765]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 844 MiB/s wr, 1.68k op/s 2024-04-26T22:03:55.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:54 smithi165 ceph-mon[33296]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 844 MiB/s wr, 1.68k op/s 2024-04-26T22:03:55.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:54 smithi055 ceph-mon[29221]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 844 MiB/s wr, 1.68k op/s 2024-04-26T22:03:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:56 smithi064 ceph-mon[32765]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 745 MiB/s wr, 1.54k op/s 2024-04-26T22:03:57.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:56 smithi165 ceph-mon[33296]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 745 MiB/s wr, 1.54k op/s 2024-04-26T22:03:57.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:56 smithi055 ceph-mon[29221]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 745 MiB/s wr, 1.54k op/s 2024-04-26T22:03:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:03:58 smithi064 ceph-mon[32765]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 885 MiB/s wr, 1.89k op/s 2024-04-26T22:03:59.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:03:58 smithi165 ceph-mon[33296]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 885 MiB/s wr, 1.89k op/s 2024-04-26T22:03:59.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:03:58 smithi055 ceph-mon[29221]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 885 MiB/s wr, 1.89k op/s 2024-04-26T22:04:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:00 smithi064 ceph-mon[32765]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 854 MiB/s wr, 1.62k op/s 2024-04-26T22:04:01.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:00 smithi165 ceph-mon[33296]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 854 MiB/s wr, 1.62k op/s 2024-04-26T22:04:01.444 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:00 smithi055 ceph-mon[29221]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 854 MiB/s wr, 1.62k op/s 2024-04-26T22:04:03.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:02 smithi064 ceph-mon[32765]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 873 MiB/s wr, 1.63k op/s 2024-04-26T22:04:03.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:02 smithi165 ceph-mon[33296]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 873 MiB/s wr, 1.63k op/s 2024-04-26T22:04:03.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:02 smithi055 ceph-mon[29221]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 873 MiB/s wr, 1.63k op/s 2024-04-26T22:04:05.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:05 smithi064 ceph-mon[32765]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 844 MiB/s wr, 1.67k op/s 2024-04-26T22:04:05.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:05 smithi165 ceph-mon[33296]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 844 MiB/s wr, 1.67k op/s 2024-04-26T22:04:05.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:05 smithi055 ceph-mon[29221]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 844 MiB/s wr, 1.67k op/s 2024-04-26T22:04:07.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:07 smithi165 ceph-mon[33296]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 761 MiB/s wr, 1.47k op/s 2024-04-26T22:04:07.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:07 smithi055 ceph-mon[29221]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 761 MiB/s wr, 1.47k op/s 2024-04-26T22:04:07.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:07 smithi064 ceph-mon[32765]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 761 MiB/s wr, 1.47k op/s 2024-04-26T22:04:09.325 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:09 smithi165 ceph-mon[33296]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 1.72k op/s 2024-04-26T22:04:09.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:09 smithi055 ceph-mon[29221]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 1.72k op/s 2024-04-26T22:04:09.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:09 smithi064 ceph-mon[32765]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 908 MiB/s wr, 1.72k op/s 2024-04-26T22:04:11.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:11 smithi165 ceph-mon[33296]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 853 MiB/s wr, 1.39k op/s 2024-04-26T22:04:11.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:11 smithi055 ceph-mon[29221]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 853 MiB/s wr, 1.39k op/s 2024-04-26T22:04:11.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:11 smithi064 ceph-mon[32765]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 853 MiB/s wr, 1.39k op/s 2024-04-26T22:04:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:13 smithi165 ceph-mon[33296]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.53k op/s 2024-04-26T22:04:13.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:13 smithi055 ceph-mon[29221]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.53k op/s 2024-04-26T22:04:13.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:13 smithi064 ceph-mon[32765]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 874 MiB/s wr, 1.53k op/s 2024-04-26T22:04:13.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:04:13.644 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:04:13.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.954+0000 7f6f21372640 1 -- 172.21.15.55:0/3855577934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c105d10 msgr2=0x7f6f1c108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:13.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.954+0000 7f6f21372640 1 --2- 172.21.15.55:0/3855577934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c105d10 0x7f6f1c108100 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009860 tx=0x7f6f1002a360 comp rx=0 tx=0).stop 2024-04-26T22:04:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.958+0000 7f6f21372640 1 -- 172.21.15.55:0/3855577934 shutdown_connections 2024-04-26T22:04:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.958+0000 7f6f21372640 1 --2- 172.21.15.55:0/3855577934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f1c108640 0x7f6f1c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.958+0000 7f6f21372640 1 --2- 172.21.15.55:0/3855577934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c105d10 0x7f6f1c108100 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.958+0000 7f6f21372640 1 --2- 172.21.15.55:0/3855577934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c1033e0 0x7f6f1c1057d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.958+0000 7f6f21372640 1 -- 172.21.15.55:0/3855577934 >> 172.21.15.55:0/3855577934 conn(0x7f6f1c0fcdf0 msgr2=0x7f6f1c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:13.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.961+0000 7f6f21372640 1 -- 172.21.15.55:0/3855577934 shutdown_connections 2024-04-26T22:04:13.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.963+0000 7f6f21372640 1 -- 172.21.15.55:0/3855577934 wait complete. 2024-04-26T22:04:13.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.964+0000 7f6f21372640 1 Processor -- start 2024-04-26T22:04:13.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.966+0000 7f6f21372640 1 -- start start 2024-04-26T22:04:13.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.967+0000 7f6f21372640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c1033e0 0x7f6f1c0721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:13.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.967+0000 7f6f21372640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c105d10 0x7f6f1c0726f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:13.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.967+0000 7f6f21372640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f1c108640 0x7f6f1c073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:13.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.967+0000 7f6f21372640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c06c800 con 0x7f6f1c1033e0 2024-04-26T22:04:13.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.967+0000 7f6f21372640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c06c680 con 0x7f6f1c105d10 2024-04-26T22:04:13.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.967+0000 7f6f21372640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c06c980 con 0x7f6f1c108640 2024-04-26T22:04:13.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.968+0000 7f6f1affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c1033e0 0x7f6f1c0721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:13.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.968+0000 7f6f1affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c1033e0 0x7f6f1c0721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49430/0 (socket says 172.21.15.55:49430) 2024-04-26T22:04:13.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.968+0000 7f6f1affd640 1 -- 172.21.15.55:0/2101195499 learned_addr learned my addr 172.21.15.55:0/2101195499 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.972+0000 7f6f1affd640 1 -- 172.21.15.55:0/2101195499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f1c108640 msgr2=0x7f6f1c073730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.972+0000 7f6f1affd640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f1c108640 0x7f6f1c073730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.972+0000 7f6f1affd640 1 -- 172.21.15.55:0/2101195499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c105d10 msgr2=0x7f6f1c0726f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.972+0000 7f6f1affd640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c105d10 0x7f6f1c0726f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.972+0000 7f6f1affd640 1 -- 172.21.15.55:0/2101195499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c10f2d0 con 0x7f6f1c1033e0 2024-04-26T22:04:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.974+0000 7f6f1affd640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c1033e0 0x7f6f1c0721b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d950 tx=0x7f6f0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.975+0000 7f6f03fff640 1 -- 172.21.15.55:0/2101195499 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c017070 con 0x7f6f1c1033e0 2024-04-26T22:04:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.975+0000 7f6f21372640 1 -- 172.21.15.55:0/2101195499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c073cd0 con 0x7f6f1c1033e0 2024-04-26T22:04:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.975+0000 7f6f21372640 1 -- 172.21.15.55:0/2101195499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c0741f0 con 0x7f6f1c1033e0 2024-04-26T22:04:13.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.975+0000 7f6f03fff640 1 -- 172.21.15.55:0/2101195499 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0c009c30 con 0x7f6f1c1033e0 2024-04-26T22:04:13.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.975+0000 7f6f03fff640 1 -- 172.21.15.55:0/2101195499 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00f620 con 0x7f6f1c1033e0 2024-04-26T22:04:13.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.981+0000 7f6f03fff640 1 -- 172.21.15.55:0/2101195499 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6f0c00f7c0 con 0x7f6f1c1033e0 2024-04-26T22:04:13.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.983+0000 7f6f03fff640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f040a3e40 0x7f6f040a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:13.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.984+0000 7f6f03fff640 1 -- 172.21.15.55:0/2101195499 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f6f0c0d11a0 con 0x7f6f1c1033e0 2024-04-26T22:04:13.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.984+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f040a3e40 0x7f6f040a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:13.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.985+0000 7f6f24085740 1 Processor -- start 2024-04-26T22:04:13.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.985+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f040a3e40 0x7f6f040a6300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009780 tx=0x7f6f10000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:13.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.986+0000 7f6f24085740 1 -- start start 2024-04-26T22:04:13.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.986+0000 7f6f24085740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a3eee0 0x55fb11a3f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:13.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.986+0000 7f6f24085740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a40530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:13.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.986+0000 7f6f24085740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 0x55fb11a47300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:13.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.986+0000 7f6f24085740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fb119a3d80 con 0x55fb11a400b0 2024-04-26T22:04:13.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.986+0000 7f6f24085740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fb1195d8b0 con 0x55fb11a3eee0 2024-04-26T22:04:13.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.986+0000 7f6f24085740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fb1199fd40 con 0x55fb11a40a70 2024-04-26T22:04:13.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.987+0000 7f6f1a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a40530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:13.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.988+0000 7f6f1a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a40530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49446/0 (socket says 172.21.15.55:49446) 2024-04-26T22:04:13.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.988+0000 7f6f1a7fc640 1 -- 172.21.15.55:0/505552543 learned_addr learned my addr 172.21.15.55:0/505552543 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:13.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.989+0000 7f6f1a7fc640 1 -- 172.21.15.55:0/505552543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 msgr2=0x55fb11a47300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:13.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.989+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/505552543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 0x55fb11a47300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.989+0000 7f6f1a7fc640 1 -- 172.21.15.55:0/505552543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a3eee0 msgr2=0x55fb11a3f300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:13.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.989+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/505552543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a3eee0 0x55fb11a3f300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.989+0000 7f6f1a7fc640 1 -- 172.21.15.55:0/505552543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb1192ea40 con 0x55fb11a400b0 2024-04-26T22:04:13.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.990+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/505552543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a40530 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6f10034000 tx=0x7f6f10039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d97b59b0ffcec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:13.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.993+0000 7f6f017fa640 1 -- 172.21.15.55:0/505552543 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10041050 con 0x55fb11a400b0 2024-04-26T22:04:13.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.993+0000 7f6f017fa640 1 -- 172.21.15.55:0/505552543 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1004b070 con 0x55fb11a400b0 2024-04-26T22:04:13.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.995+0000 7f6f24085740 1 -- 172.21.15.55:0/505552543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 msgr2=0x55fb11a40530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:13.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.995+0000 7f6f24085740 1 --2- 172.21.15.55:0/505552543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a40530 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6f10034000 tx=0x7f6f10039000 comp rx=0 tx=0).stop 2024-04-26T22:04:13.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.998+0000 7f6f24085740 1 -- 172.21.15.55:0/505552543 shutdown_connections 2024-04-26T22:04:13.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.998+0000 7f6f24085740 1 --2- 172.21.15.55:0/505552543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 0x55fb11a47300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.998+0000 7f6f24085740 1 --2- 172.21.15.55:0/505552543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a40530 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.998+0000 7f6f24085740 1 --2- 172.21.15.55:0/505552543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a3eee0 0x55fb11a3f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:13.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:13.998+0000 7f6f24085740 1 -- 172.21.15.55:0/505552543 >> 172.21.15.55:0/505552543 conn(0x55fb11a3a3d0 msgr2=0x55fb11a3a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:14.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.002+0000 7f6f24085740 1 -- 172.21.15.55:0/505552543 shutdown_connections 2024-04-26T22:04:14.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.007+0000 7f6f24085740 1 -- 172.21.15.55:0/505552543 wait complete. 2024-04-26T22:04:14.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.008+0000 7f6f24085740 1 -- start start 2024-04-26T22:04:14.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.008+0000 7f6f24085740 1 Processor -- start 2024-04-26T22:04:14.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f24085740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a3eee0 0x55fb11a49640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f24085740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a49b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f24085740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 0x55fb11a50bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f24085740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fb11a47a70 con 0x55fb11a400b0 2024-04-26T22:04:14.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f24085740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fb1192ad10 con 0x55fb11a3eee0 2024-04-26T22:04:14.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f24085740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fb11a47bf0 con 0x55fb11a40a70 2024-04-26T22:04:14.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f24085740 10 client.0 _ll_register_callbacks cb 0x55fb11957210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:14.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f1a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a49b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f1a7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a49b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49452/0 (socket says 172.21.15.55:49452) 2024-04-26T22:04:14.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.009+0000 7f6f1a7fc640 1 -- 172.21.15.55:0/2876737946 learned_addr learned my addr 172.21.15.55:0/2876737946 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:14.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.012+0000 7f6f1b7fe640 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 0x55fb11a50bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.013+0000 7f6f1a7fc640 1 -- 172.21.15.55:0/2876737946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 msgr2=0x55fb11a50bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:14.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.013+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 0x55fb11a50bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.013+0000 7f6f1a7fc640 1 -- 172.21.15.55:0/2876737946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a3eee0 msgr2=0x55fb11a49640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:14.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.013+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a3eee0 0x55fb11a49640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.014 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.013+0000 7f6f1a7fc640 1 -- 172.21.15.55:0/2876737946 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1177f7d0 con 0x55fb11a400b0 2024-04-26T22:04:14.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.019+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a49b80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009860 tx=0x7f6f100044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49600360d77cad41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:14.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.020+0000 7f6efeff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:14.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.023+0000 7f6efeff5640 1 -- 172.21.15.55:0/2876737946 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10053070 con 0x55fb11a400b0 2024-04-26T22:04:14.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.023+0000 7f6f24085740 1 -- 172.21.15.55:0/2876737946 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fb119b1990 con 0x55fb11a400b0 2024-04-26T22:04:14.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.023+0000 7f6efeff5640 1 -- 172.21.15.55:0/2876737946 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f6f10046b30 con 0x55fb11a400b0 2024-04-26T22:04:14.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.026+0000 7f6efeff5640 1 -- 172.21.15.55:0/2876737946 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6f10046d30 con 0x55fb11a400b0 2024-04-26T22:04:14.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.026+0000 7f6f24085740 10 client.17172 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:14.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.027+0000 7f6f24085740 1 -- 172.21.15.55:0/2876737946 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fb116b53f0 con 0x55fb11a400b0 2024-04-26T22:04:14.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.029+0000 7f6efeff5640 1 -- 172.21.15.55:0/2876737946 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fb119b1990 con 0x55fb11a400b0 2024-04-26T22:04:14.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.029+0000 7f6efeff5640 1 -- 172.21.15.55:0/2876737946 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6f1003a030 con 0x55fb11a400b0 2024-04-26T22:04:14.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.030+0000 7f6f24085740 10 client.17172 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:14.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.030+0000 7f6f24085740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:14.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.030+0000 7f6f24085740 20 client.17172 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:14.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.030+0000 7f6f24085740 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fb11a552a0 0x55fb11a57700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.030+0000 7f6f24085740 4 client.17172 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:14.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.030+0000 7f6f24085740 1 -- 172.21.15.55:0/2876737946 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb119ab800 con 0x55fb11a552a0 2024-04-26T22:04:14.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.034+0000 7f6f1affd640 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fb11a552a0 0x55fb11a57700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.058+0000 7f6efeff5640 10 client.17172 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.058+0000 7f6f1affd640 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fb11a552a0 0x55fb11a57700 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd1bc0c71db215a server_cookie=5c3cca8a4538d0bf in_seq=0 out_seq=0 2024-04-26T22:04:14.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.063+0000 7f6efeff5640 1 -- 172.21.15.55:0/2876737946 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb119ab800 con 0x55fb11a552a0 2024-04-26T22:04:14.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.063+0000 7f6efeff5640 10 client.17172 handle_command_reply: tid=0 2024-04-26T22:04:14.067 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.063+0000 7f6f24085740 1 client.17172 shutdown 2024-04-26T22:04:14.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.067+0000 7f6f24085740 1 -- 172.21.15.55:0/2876737946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 msgr2=0x55fb11a49b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.067+0000 7f6f24085740 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a49b80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009860 tx=0x7f6f100044c0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.068+0000 7f6f24085740 1 -- 172.21.15.55:0/2876737946 shutdown_connections 2024-04-26T22:04:14.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.068+0000 7f6f24085740 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a40a70 0x55fb11a50bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.068+0000 7f6f24085740 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fb11a552a0 0x55fb11a57700 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.068+0000 7f6f24085740 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a400b0 0x55fb11a49b80 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.068+0000 7f6f24085740 1 --2- 172.21.15.55:0/2876737946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a3eee0 0x55fb11a49640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.069 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.068+0000 7f6f24085740 1 -- 172.21.15.55:0/2876737946 >> 172.21.15.55:0/2876737946 conn(0x55fb11a3a3d0 msgr2=0x55fb11a39550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:14.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.069+0000 7f6efeff5640 0 client.17172 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:14.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.075+0000 7f6f24085740 1 -- 172.21.15.55:0/2876737946 shutdown_connections 2024-04-26T22:04:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.079+0000 7f6f24085740 1 -- 172.21.15.55:0/2876737946 wait complete. 2024-04-26T22:04:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.079+0000 7f6f24085740 20 client.17172 trim_cache size 0 max 16384 2024-04-26T22:04:14.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.137+0000 7f6f24085740 1 Processor -- start 2024-04-26T22:04:14.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.138+0000 7f6f24085740 1 -- start start 2024-04-26T22:04:14.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.138+0000 7f6f24085740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a49020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.139+0000 7f6f24085740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 0x55fb11a50b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.139+0000 7f6f24085740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11a3d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.139+0000 7f6f1affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a49020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.140+0000 7f6f24085740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08010fd0 con 0x55fb11a40a70 2024-04-26T22:04:14.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.140+0000 7f6f24085740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0800e8a0 con 0x55fb11a400b0 2024-04-26T22:04:14.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.140+0000 7f6f24085740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0800c6f0 con 0x55fb11a3eee0 2024-04-26T22:04:14.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.140+0000 7f6f1affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a49020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:41988/0 (socket says 172.21.15.55:41988) 2024-04-26T22:04:14.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.140+0000 7f6f1affd640 1 -- 172.21.15.55:0/3201348862 learned_addr learned my addr 172.21.15.55:0/3201348862 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:14.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.140+0000 7f6f1a7fc640 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 0x55fb11a50b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.141+0000 7f6f1b7fe640 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11a3d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.141+0000 7f6f1affd640 1 -- 172.21.15.55:0/3201348862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 msgr2=0x55fb11a50b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:14.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.141+0000 7f6f1affd640 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 0x55fb11a50b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.141+0000 7f6f1affd640 1 -- 172.21.15.55:0/3201348862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 msgr2=0x55fb11a3d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:14.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.141+0000 7f6f1affd640 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11a3d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.141+0000 7f6f1affd640 1 -- 172.21.15.55:0/3201348862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb116750f0 con 0x55fb11a3eee0 2024-04-26T22:04:14.144 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.142+0000 7f6f1affd640 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a49020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c099c50 tx=0x7f6f0c00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d37615728e2a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:14.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.151+0000 7f6f00ff9640 1 -- 172.21.15.55:0/3201348862 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c020040 con 0x55fb11a3eee0 2024-04-26T22:04:14.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.151+0000 7f6f00ff9640 1 -- 172.21.15.55:0/3201348862 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0c0a5ef0 con 0x55fb11a3eee0 2024-04-26T22:04:14.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.151+0000 7f6f00ff9640 1 -- 172.21.15.55:0/3201348862 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c09c6d0 con 0x55fb11a3eee0 2024-04-26T22:04:14.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.153+0000 7f6f24085740 1 -- 172.21.15.55:0/3201348862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 msgr2=0x55fb11a49020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.153 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.153+0000 7f6f24085740 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a49020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c099c50 tx=0x7f6f0c00bf60 comp rx=0 tx=0).stop 2024-04-26T22:04:14.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.153+0000 7f6f24085740 1 -- 172.21.15.55:0/3201348862 shutdown_connections 2024-04-26T22:04:14.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.153+0000 7f6f24085740 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11a3d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.153+0000 7f6f24085740 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 0x55fb11a50b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.153+0000 7f6f24085740 1 --2- 172.21.15.55:0/3201348862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a49020 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.153+0000 7f6f24085740 1 -- 172.21.15.55:0/3201348862 >> 172.21.15.55:0/3201348862 conn(0x55fb11a3a3d0 msgr2=0x55fb11a48910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:14.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.155+0000 7f6f24085740 1 -- 172.21.15.55:0/3201348862 shutdown_connections 2024-04-26T22:04:14.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.155+0000 7f6f24085740 1 -- 172.21.15.55:0/3201348862 wait complete. 2024-04-26T22:04:14.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 1 -- start start 2024-04-26T22:04:14.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 1 Processor -- start 2024-04-26T22:04:14.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 0x55fb11a4bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11ab9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0800cd60 con 0x55fb11a40a70 2024-04-26T22:04:14.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0800e1d0 con 0x55fb11a400b0 2024-04-26T22:04:14.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0800d430 con 0x55fb11a3eee0 2024-04-26T22:04:14.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.156+0000 7f6f24085740 10 client.0 _ll_register_callbacks cb 0x7f6f100096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.158+0000 7f6f1b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11ab9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.158+0000 7f6f1b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11ab9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49470/0 (socket says 172.21.15.55:49470) 2024-04-26T22:04:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.158+0000 7f6f1b7fe640 1 -- 172.21.15.55:0/2808461251 learned_addr learned my addr 172.21.15.55:0/2808461251 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:14.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.159+0000 7f6f1b7fe640 1 -- 172.21.15.55:0/2808461251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 msgr2=0x55fb11a4b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:14.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.159+0000 7f6f1b7fe640 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.159+0000 7f6f1b7fe640 1 -- 172.21.15.55:0/2808461251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 msgr2=0x55fb11a4bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:14.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.159+0000 7f6f1b7fe640 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 0x55fb11a4bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.159+0000 7f6f1b7fe640 1 -- 172.21.15.55:0/2808461251 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb119644b0 con 0x55fb11a40a70 2024-04-26T22:04:14.160 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.160+0000 7f6f1b7fe640 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11ab9830 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6f140107b0 tx=0x7f6f14010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4d6ac1b8048be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:14.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.161+0000 7f6efeff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:14.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.161+0000 7f6efeff5640 1 -- 172.21.15.55:0/2808461251 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1401e060 con 0x55fb11a40a70 2024-04-26T22:04:14.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.161+0000 7f6efeff5640 1 -- 172.21.15.55:0/2808461251 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f6f1400eae0 con 0x55fb11a40a70 2024-04-26T22:04:14.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.162+0000 7f6f24085740 1 -- 172.21.15.55:0/2808461251 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0c0a5ef0 con 0x55fb11a40a70 2024-04-26T22:04:14.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.164+0000 7f6efeff5640 1 -- 172.21.15.55:0/2808461251 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6f1400ece0 con 0x55fb11a40a70 2024-04-26T22:04:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.165+0000 7f6f24085740 10 client.17178 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.165+0000 7f6f24085740 1 -- 172.21.15.55:0/2808461251 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fb11abaeb0 con 0x55fb11a40a70 2024-04-26T22:04:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.169+0000 7f6efeff5640 1 -- 172.21.15.55:0/2808461251 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0c0a5ef0 con 0x55fb11a40a70 2024-04-26T22:04:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.169+0000 7f6efeff5640 1 -- 172.21.15.55:0/2808461251 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6f1400b040 con 0x55fb11a40a70 2024-04-26T22:04:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.169+0000 7f6f24085740 10 client.17178 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.169+0000 7f6f24085740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.169+0000 7f6f24085740 20 client.17178 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:14.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.169+0000 7f6f24085740 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fb11abb840 0x55fb11abdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.169+0000 7f6f24085740 4 client.17178 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:04:14.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.169+0000 7f6f24085740 1 -- 172.21.15.55:0/2808461251 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb119264a0 con 0x55fb11abb840 2024-04-26T22:04:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.171+0000 7f6f1affd640 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fb11abb840 0x55fb11abdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.173+0000 7f6f1affd640 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fb11abb840 0x55fb11abdca0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82032be934653d41 server_cookie=7bf8a79c13a7d38 in_seq=0 out_seq=0 2024-04-26T22:04:14.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.175+0000 7f6efeff5640 10 client.17178 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:14.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.176+0000 7f6efeff5640 1 -- 172.21.15.55:0/2808461251 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fb119264a0 con 0x55fb11abb840 2024-04-26T22:04:14.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.176+0000 7f6efeff5640 10 client.17178 handle_command_reply: tid=0 2024-04-26T22:04:14.177 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.176+0000 7f6f24085740 1 client.17178 shutdown 2024-04-26T22:04:14.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.178+0000 7f6f24085740 1 -- 172.21.15.55:0/2808461251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 msgr2=0x55fb11ab9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.178+0000 7f6f24085740 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11ab9830 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6f140107b0 tx=0x7f6f14010c70 comp rx=0 tx=0).stop 2024-04-26T22:04:14.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.182+0000 7f6f24085740 1 -- 172.21.15.55:0/2808461251 shutdown_connections 2024-04-26T22:04:14.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.182+0000 7f6f24085740 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55fb11abb840 0x55fb11abdca0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.182+0000 7f6f24085740 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fb11a40a70 0x55fb11ab9830 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.182+0000 7f6f24085740 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fb11a400b0 0x55fb11a4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.182+0000 7f6f24085740 1 --2- 172.21.15.55:0/2808461251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb11a3eee0 0x55fb11a4b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.182+0000 7f6f24085740 1 -- 172.21.15.55:0/2808461251 >> 172.21.15.55:0/2808461251 conn(0x55fb11a3a3d0 msgr2=0x55fb11a38cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.187+0000 7f6efeff5640 0 client.17178 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.188+0000 7f6f24085740 1 -- 172.21.15.55:0/2808461251 shutdown_connections 2024-04-26T22:04:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.188+0000 7f6f24085740 1 -- 172.21.15.55:0/2808461251 wait complete. 2024-04-26T22:04:14.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.190+0000 7f6f24085740 20 client.17178 trim_cache size 0 max 16384 2024-04-26T22:04:14.191 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:04:14.191 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:04:14.192 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:04:14.192 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:04:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.192+0000 7f6f21372640 1 -- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f040a3e40 msgr2=0x7f6f040a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.192+0000 7f6f21372640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f040a3e40 0x7f6f040a6300 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009780 tx=0x7f6f10000bc0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.192+0000 7f6f21372640 1 -- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c1033e0 msgr2=0x7f6f1c0721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.193+0000 7f6f21372640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c1033e0 0x7f6f1c0721b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d950 tx=0x7f6f0c00de10 comp rx=0 tx=0).stop 2024-04-26T22:04:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.194+0000 7f6f21372640 1 -- 172.21.15.55:0/2101195499 shutdown_connections 2024-04-26T22:04:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.194+0000 7f6f21372640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f6f040a3e40 0x7f6f040a6300 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.194+0000 7f6f21372640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f1c108640 0x7f6f1c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.194+0000 7f6f21372640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f1c105d10 0x7f6f1c0726f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.194+0000 7f6f21372640 1 --2- 172.21.15.55:0/2101195499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f1c1033e0 0x7f6f1c0721b0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.194+0000 7f6f21372640 1 -- 172.21.15.55:0/2101195499 >> 172.21.15.55:0/2101195499 conn(0x7f6f1c0fcdf0 msgr2=0x7f6f1c103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.195+0000 7f6f21372640 1 -- 172.21.15.55:0/2101195499 shutdown_connections 2024-04-26T22:04:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.195+0000 7f6f21372640 1 -- 172.21.15.55:0/2101195499 wait complete. 2024-04-26T22:04:14.238 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:04:14.239 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:14.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.687+0000 7fb814987640 1 -- 172.21.15.55:0/215021946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 msgr2=0x7fb81010f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.687+0000 7fb814987640 1 --2- 172.21.15.55:0/215021946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 0x7fb81010f0d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb808009ca0 tx=0x7fb80802a8f0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.688+0000 7fb814987640 1 -- 172.21.15.55:0/215021946 shutdown_connections 2024-04-26T22:04:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.688+0000 7fb814987640 1 --2- 172.21.15.55:0/215021946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 0x7fb81010f0d0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.688+0000 7fb814987640 1 --2- 172.21.15.55:0/215021946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb81010c2a0 0x7fb81010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.688+0000 7fb814987640 1 --2- 172.21.15.55:0/215021946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb810109930 0x7fb81010bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.688+0000 7fb814987640 1 -- 172.21.15.55:0/215021946 >> 172.21.15.55:0/215021946 conn(0x7fb81006f6c0 msgr2=0x7fb810071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:14.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.689+0000 7fb814987640 1 -- 172.21.15.55:0/215021946 shutdown_connections 2024-04-26T22:04:14.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.689+0000 7fb814987640 1 -- 172.21.15.55:0/215021946 wait complete. 2024-04-26T22:04:14.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.690+0000 7fb814987640 1 Processor -- start 2024-04-26T22:04:14.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.690+0000 7fb814987640 1 -- start start 2024-04-26T22:04:14.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.690+0000 7fb814987640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb810109930 0x7fb8101a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.690+0000 7fb814987640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81010c2a0 0x7fb8101a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.690+0000 7fb814987640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 0x7fb81007ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.691+0000 7fb814987640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb81011d1b0 con 0x7fb81010ec50 2024-04-26T22:04:14.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.691+0000 7fb814987640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb81011d030 con 0x7fb810109930 2024-04-26T22:04:14.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.691+0000 7fb814987640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb81011d330 con 0x7fb81010c2a0 2024-04-26T22:04:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.693+0000 7fb80e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb810109930 0x7fb8101a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.694+0000 7fb80ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 0x7fb81007ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.694+0000 7fb80ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 0x7fb81007ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49496/0 (socket says 172.21.15.55:49496) 2024-04-26T22:04:14.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.694+0000 7fb80ed9d640 1 -- 172.21.15.55:0/2593856438 learned_addr learned my addr 172.21.15.55:0/2593856438 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:14.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.695+0000 7fb80ed9d640 1 -- 172.21.15.55:0/2593856438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81010c2a0 msgr2=0x7fb8101a7ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:14.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.695+0000 7fb80ed9d640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81010c2a0 0x7fb8101a7ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.695+0000 7fb80ed9d640 1 -- 172.21.15.55:0/2593856438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb810109930 msgr2=0x7fb8101a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:14.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.695+0000 7fb80ed9d640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb810109930 0x7fb8101a7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:14.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.695+0000 7fb80ed9d640 1 -- 172.21.15.55:0/2593856438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81007b230 con 0x7fb81010ec50 2024-04-26T22:04:14.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.696+0000 7fb80ed9d640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 0x7fb81007ab90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb808006010 tx=0x7fb80802fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.697+0000 7fb7ff7fe640 1 -- 172.21.15.55:0/2593856438 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808030a00 con 0x7fb81010ec50 2024-04-26T22:04:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.697+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81007b4c0 con 0x7fb81010ec50 2024-04-26T22:04:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.697+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81007ba10 con 0x7fb81010ec50 2024-04-26T22:04:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.698+0000 7fb7ff7fe640 1 -- 172.21.15.55:0/2593856438 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb80802d070 con 0x7fb81010ec50 2024-04-26T22:04:14.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.698+0000 7fb7ff7fe640 1 -- 172.21.15.55:0/2593856438 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80803a400 con 0x7fb81010ec50 2024-04-26T22:04:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.701+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8100721c0 con 0x7fb81010ec50 2024-04-26T22:04:14.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.704+0000 7fb7ff7fe640 1 -- 172.21.15.55:0/2593856438 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb80803a5a0 con 0x7fb81010ec50 2024-04-26T22:04:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.706+0000 7fb7ff7fe640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7f80a3dc0 0x7fb7f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:14.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.711+0000 7fb7ff7fe640 1 -- 172.21.15.55:0/2593856438 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb808031c40 con 0x7fb81010ec50 2024-04-26T22:04:14.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.719+0000 7fb80e59c640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7f80a3dc0 0x7fb7f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:14.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.722+0000 7fb7ff7fe640 1 -- 172.21.15.55:0/2593856438 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb808038310 con 0x7fb81010ec50 2024-04-26T22:04:14.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:14.737+0000 7fb80e59c640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7f80a3dc0 0x7fb7f80a6280 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb8101a8cb0 tx=0x7fb800008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:15.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:15 smithi165 ceph-mon[33296]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.61k op/s 2024-04-26T22:04:15.422 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:15 smithi055 ceph-mon[29221]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.61k op/s 2024-04-26T22:04:15.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.487+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb810063530 con 0x7fb81010ec50 2024-04-26T22:04:15.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.492+0000 7fb7ff7fe640 1 -- 172.21.15.55:0/2593856438 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fb8080bef00 con 0x7fb81010ec50 2024-04-26T22:04:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:04:15.501 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:04:15.501 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:04:15.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:15 smithi064 ceph-mon[32765]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.61k op/s 2024-04-26T22:04:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.505+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7f80a3dc0 msgr2=0x7fb7f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.505+0000 7fb814987640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7f80a3dc0 0x7fb7f80a6280 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb8101a8cb0 tx=0x7fb800008040 comp rx=0 tx=0).stop 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.505+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 msgr2=0x7fb81007ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.505+0000 7fb814987640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 0x7fb81007ab90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb808006010 tx=0x7fb80802fe80 comp rx=0 tx=0).stop 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.506+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 shutdown_connections 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.506+0000 7fb814987640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb7f80a3dc0 0x7fb7f80a6280 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.506+0000 7fb814987640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb81010ec50 0x7fb81007ab90 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.506+0000 7fb814987640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb81010c2a0 0x7fb8101a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.506+0000 7fb814987640 1 --2- 172.21.15.55:0/2593856438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb810109930 0x7fb8101a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.506+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 >> 172.21.15.55:0/2593856438 conn(0x7fb81006f6c0 msgr2=0x7fb81010ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.507+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 shutdown_connections 2024-04-26T22:04:15.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.507+0000 7fb814987640 1 -- 172.21.15.55:0/2593856438 wait complete. 2024-04-26T22:04:15.552 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:04:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.937+0000 7f041759e640 1 -- 172.21.15.55:0/3385747010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0418109930 msgr2=0x7f041810bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.937+0000 7f041759e640 1 --2- 172.21.15.55:0/3385747010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0418109930 0x7f041810bd60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0410009ca0 tx=0x7f041002a980 comp rx=0 tx=0).stop 2024-04-26T22:04:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.937+0000 7f041759e640 1 -- 172.21.15.55:0/3385747010 shutdown_connections 2024-04-26T22:04:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.937+0000 7f041759e640 1 --2- 172.21.15.55:0/3385747010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041810ec50 0x7f041810f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.937+0000 7f041759e640 1 --2- 172.21.15.55:0/3385747010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041810c2a0 0x7f041810e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.937+0000 7f041759e640 1 --2- 172.21.15.55:0/3385747010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0418109930 0x7f041810bd60 secure :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0410009ca0 tx=0x7f041002a980 comp rx=0 tx=0).stop 2024-04-26T22:04:15.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.937+0000 7f041759e640 1 -- 172.21.15.55:0/3385747010 >> 172.21.15.55:0/3385747010 conn(0x7f04180706d0 msgr2=0x7f0418072b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:15.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.943+0000 7f041759e640 1 -- 172.21.15.55:0/3385747010 shutdown_connections 2024-04-26T22:04:15.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.943+0000 7f041759e640 1 -- 172.21.15.55:0/3385747010 wait complete. 2024-04-26T22:04:15.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.944+0000 7f041759e640 1 Processor -- start 2024-04-26T22:04:15.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.945+0000 7f041759e640 1 -- start start 2024-04-26T22:04:15.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.945+0000 7f041759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041810c2a0 0x7f04181a7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.945+0000 7f041759e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041810ec50 0x7f04181a8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.945+0000 7f041759e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04181ad180 0x7f04181af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.946+0000 7f041659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041810c2a0 0x7f04181a7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.946+0000 7f041659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041810c2a0 0x7f04181a7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48524/0 (socket says 172.21.15.55:48524) 2024-04-26T22:04:15.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.946+0000 7f041659c640 1 -- 172.21.15.55:0/584620707 learned_addr learned my addr 172.21.15.55:0/584620707 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:15.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.946+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f041811d290 con 0x7f04181ad180 2024-04-26T22:04:15.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.946+0000 7f0415d9b640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041810ec50 0x7f04181a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.946+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f041811d110 con 0x7f041810c2a0 2024-04-26T22:04:15.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.946+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f041811d410 con 0x7f041810ec50 2024-04-26T22:04:15.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.948+0000 7f0416d9d640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04181ad180 0x7f04181af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.949+0000 7f0416d9d640 1 -- 172.21.15.55:0/584620707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041810ec50 msgr2=0x7f04181a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.949+0000 7f0416d9d640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041810ec50 0x7f04181a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.949+0000 7f0416d9d640 1 -- 172.21.15.55:0/584620707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041810c2a0 msgr2=0x7f04181a7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.949+0000 7f0416d9d640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041810c2a0 0x7f04181a7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.949+0000 7f0416d9d640 1 -- 172.21.15.55:0/584620707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181afbe0 con 0x7f04181ad180 2024-04-26T22:04:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.950+0000 7f0416d9d640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04181ad180 0x7f04181af570 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f040c00b410 tx=0x7f040c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.950+0000 7f04077fe640 1 -- 172.21.15.55:0/584620707 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c015020 con 0x7f04181ad180 2024-04-26T22:04:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.951+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181afed0 con 0x7f04181ad180 2024-04-26T22:04:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.951+0000 7f04077fe640 1 -- 172.21.15.55:0/584620707 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040c011ce0 con 0x7f04181ad180 2024-04-26T22:04:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.951+0000 7f04077fe640 1 -- 172.21.15.55:0/584620707 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c010e00 con 0x7f04181ad180 2024-04-26T22:04:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.951+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181b0410 con 0x7f04181ad180 2024-04-26T22:04:15.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.957+0000 7f041f620740 1 Processor -- start 2024-04-26T22:04:15.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.963+0000 7f041f620740 1 -- start start 2024-04-26T22:04:15.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.964+0000 7f041f620740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a1ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.964+0000 7f041f620740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1dbd0 0x556e63a1e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.964+0000 7f041f620740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a24e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.965+0000 7f041f620740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556e63980d80 con 0x556e63a1e590 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.965+0000 7f041f620740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556e6393aa20 con 0x556e63a1dbd0 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.965+0000 7f041f620740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556e6397cd40 con 0x556e63a1ca00 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.965+0000 7f04077fe640 1 -- 172.21.15.55:0/584620707 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f040c018020 con 0x7f04181ad180 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.966+0000 7f041659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1dbd0 0x556e63a1e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.966+0000 7f0415d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a24e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.966+0000 7f0415d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a24e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49532/0 (socket says 172.21.15.55:49532) 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.966+0000 7f0415d9b640 1 -- 172.21.15.55:0/3839943515 learned_addr learned my addr 172.21.15.55:0/3839943515 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.966+0000 7f0415d9b640 1 --2- 172.21.15.55:0/3839943515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a1ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.967+0000 7f0415d9b640 1 -- 172.21.15.55:0/3839943515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 msgr2=0x556e63a1ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.967+0000 7f0415d9b640 1 --2- 172.21.15.55:0/3839943515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a1ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.967+0000 7f0415d9b640 1 -- 172.21.15.55:0/3839943515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1dbd0 msgr2=0x556e63a1e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.967+0000 7f0415d9b640 1 --2- 172.21.15.55:0/3839943515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1dbd0 0x556e63a1e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.967+0000 7f0415d9b640 1 -- 172.21.15.55:0/3839943515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e63945c30 con 0x556e63a1e590 2024-04-26T22:04:15.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.967+0000 7f0415d9b640 1 --2- 172.21.15.55:0/3839943515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a24e20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0408015360 tx=0x7f0408009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfca0af9140f3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:15.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.968+0000 7f04077fe640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04000a3d80 0x7f04000a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.969+0000 7f041659c640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04000a3d80 0x7f04000a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.969+0000 7f04077fe640 1 -- 172.21.15.55:0/584620707 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f040c0cfce0 con 0x7f04181ad180 2024-04-26T22:04:15.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.971+0000 7f041659c640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04000a3d80 0x7f04000a6240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0410000f80 tx=0x7f0410009910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:15.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.971+0000 7f0404ff9640 1 -- 172.21.15.55:0/3839943515 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408015ea0 con 0x556e63a1e590 2024-04-26T22:04:15.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.971+0000 7f0404ff9640 1 -- 172.21.15.55:0/3839943515 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040801c070 con 0x556e63a1e590 2024-04-26T22:04:15.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.973+0000 7f041f620740 1 -- 172.21.15.55:0/3839943515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 msgr2=0x556e63a24e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.973+0000 7f041f620740 1 --2- 172.21.15.55:0/3839943515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a24e20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0408015360 tx=0x7f0408009520 comp rx=0 tx=0).stop 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.975+0000 7f041f620740 1 -- 172.21.15.55:0/3839943515 shutdown_connections 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.975+0000 7f041f620740 1 --2- 172.21.15.55:0/3839943515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a24e20 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.975+0000 7f041f620740 1 --2- 172.21.15.55:0/3839943515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1dbd0 0x556e63a1e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.975+0000 7f041f620740 1 --2- 172.21.15.55:0/3839943515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a1ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.975+0000 7f041f620740 1 -- 172.21.15.55:0/3839943515 >> 172.21.15.55:0/3839943515 conn(0x556e6395d910 msgr2=0x556e63a182b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:15.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.982+0000 7f041f620740 1 -- 172.21.15.55:0/3839943515 shutdown_connections 2024-04-26T22:04:15.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.983+0000 7f041f620740 1 -- 172.21.15.55:0/3839943515 wait complete. 2024-04-26T22:04:15.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.983+0000 7f041f620740 1 -- start start 2024-04-26T22:04:15.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.984+0000 7f041f620740 1 Processor -- start 2024-04-26T22:04:15.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.984+0000 7f041f620740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1ca00 0x556e63a26c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.984+0000 7f041f620740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 0x556e63a271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.984+0000 7f041f620740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 0x556e63a2e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:15.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.984+0000 7f041f620740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556e63a255f0 con 0x556e63a1ca00 2024-04-26T22:04:15.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.984+0000 7f041f620740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556e63907dc0 con 0x556e63a1e590 2024-04-26T22:04:15.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.984+0000 7f041f620740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556e63a25770 con 0x556e63a1dbd0 2024-04-26T22:04:15.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.984+0000 7f041f620740 10 client.0 _ll_register_callbacks cb 0x556e63923600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.988+0000 7f041659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1ca00 0x556e63a26c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.988+0000 7f041659c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1ca00 0x556e63a26c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49540/0 (socket says 172.21.15.55:49540) 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.988+0000 7f041659c640 1 -- 172.21.15.55:0/3378133836 learned_addr learned my addr 172.21.15.55:0/3378133836 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:15.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.988+0000 7f0415d9b640 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 0x556e63a271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.989+0000 7f0416d9d640 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 0x556e63a2e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:15.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.991+0000 7f041659c640 1 -- 172.21.15.55:0/3378133836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 msgr2=0x556e63a271d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:15.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.991+0000 7f041659c640 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 0x556e63a271d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.991+0000 7f041659c640 1 -- 172.21.15.55:0/3378133836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 msgr2=0x556e63a2e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:15.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.991+0000 7f041659c640 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 0x556e63a2e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:15.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:15.991+0000 7f041659c640 1 -- 172.21.15.55:0/3378133836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e636923f0 con 0x556e63a1ca00 2024-04-26T22:04:16.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.002+0000 7f041659c640 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1ca00 0x556e63a26c90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0410009bf0 tx=0x7f04100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c8bb6fb1f2eed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.002+0000 7f03fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:16.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.006+0000 7f03fdffb640 1 -- 172.21.15.55:0/3378133836 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041002fe80 con 0x556e63a1ca00 2024-04-26T22:04:16.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.006+0000 7f041f620740 1 -- 172.21.15.55:0/3378133836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040801c070 con 0x556e63a1ca00 2024-04-26T22:04:16.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.007+0000 7f03fdffb640 1 -- 172.21.15.55:0/3378133836 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0410003fc0 con 0x556e63a1ca00 2024-04-26T22:04:16.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.009+0000 7f03fdffb640 1 -- 172.21.15.55:0/3378133836 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0410004200 con 0x556e63a1ca00 2024-04-26T22:04:16.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.009+0000 7f041f620740 10 client.17196 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:16.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.009+0000 7f041f620740 1 -- 172.21.15.55:0/3378133836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556e6375c7d0 con 0x556e63a1ca00 2024-04-26T22:04:16.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.017+0000 7f03fdffb640 1 -- 172.21.15.55:0/3378133836 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040801c070 con 0x556e63a1ca00 2024-04-26T22:04:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.017+0000 7f03fdffb640 1 -- 172.21.15.55:0/3378133836 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f041004f070 con 0x556e63a1ca00 2024-04-26T22:04:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.019+0000 7f041f620740 10 client.17196 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.019+0000 7f041f620740 10 client.17196 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:04:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.019+0000 7f041f620740 20 client.17196 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.019+0000 7f041f620740 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556e63a32ae0 0x556e63a34f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.019+0000 7f041f620740 4 client.17196 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.019+0000 7f041f620740 1 -- 172.21.15.55:0/3378133836 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e6398e990 con 0x556e63a32ae0 2024-04-26T22:04:16.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.027+0000 7f0415d9b640 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556e63a32ae0 0x556e63a34f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.052+0000 7f03fdffb640 10 client.17196 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:04:16.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.052+0000 7f0415d9b640 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556e63a32ae0 0x556e63a34f60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e805c16103a2492c server_cookie=74d0e5aba9eae624 in_seq=0 out_seq=0 2024-04-26T22:04:16.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.058+0000 7f03fdffb640 1 -- 172.21.15.55:0/3378133836 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e6398e990 con 0x556e63a32ae0 2024-04-26T22:04:16.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.058+0000 7f03fdffb640 10 client.17196 handle_command_reply: tid=0 2024-04-26T22:04:16.063 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.060+0000 7f041f620740 1 client.17196 shutdown 2024-04-26T22:04:16.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.072+0000 7f041f620740 1 -- 172.21.15.55:0/3378133836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1ca00 msgr2=0x556e63a26c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.072+0000 7f041f620740 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1ca00 0x556e63a26c90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f0410009bf0 tx=0x7f04100095c0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.081+0000 7f041f620740 1 -- 172.21.15.55:0/3378133836 shutdown_connections 2024-04-26T22:04:16.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.081+0000 7f041f620740 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556e63a32ae0 0x556e63a34f60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.081+0000 7f041f620740 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 0x556e63a2e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.081+0000 7f041f620740 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 0x556e63a271d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.081+0000 7f041f620740 1 --2- 172.21.15.55:0/3378133836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1ca00 0x556e63a26c90 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.081+0000 7f03fdffb640 0 client.17196 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:04:16.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.081+0000 7f041f620740 1 -- 172.21.15.55:0/3378133836 >> 172.21.15.55:0/3378133836 conn(0x556e6395d910 msgr2=0x556e63a17450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.084+0000 7f041f620740 1 -- 172.21.15.55:0/3378133836 shutdown_connections 2024-04-26T22:04:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.086+0000 7f041f620740 1 -- 172.21.15.55:0/3378133836 wait complete. 2024-04-26T22:04:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.086+0000 7f041f620740 20 client.17196 trim_cache size 0 max 16384 2024-04-26T22:04:16.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.139+0000 7f041f620740 1 Processor -- start 2024-04-26T22:04:16.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.139+0000 7f041f620740 1 -- start start 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.140+0000 7f041f620740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a17450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.140+0000 7f041f620740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1dbd0 0x556e63a2e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.140+0000 7f041f620740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 0x556e63a39820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.140+0000 7f041f620740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04181bbb00 con 0x556e63a1dbd0 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.141+0000 7f041f620740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b93d0 con 0x556e63a1e590 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.141+0000 7f041f620740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b7220 con 0x556e63a1ca00 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.141+0000 7f041659c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a17450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.141+0000 7f041659c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a17450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42070/0 (socket says 172.21.15.55:42070) 2024-04-26T22:04:16.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.141+0000 7f041659c640 1 -- 172.21.15.55:0/2896438868 learned_addr learned my addr 172.21.15.55:0/2896438868 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:16.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.142+0000 7f041659c640 1 -- 172.21.15.55:0/2896438868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 msgr2=0x556e63a39820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.142+0000 7f041659c640 1 --2- 172.21.15.55:0/2896438868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 0x556e63a39820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.142+0000 7f041659c640 1 -- 172.21.15.55:0/2896438868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1dbd0 msgr2=0x556e63a2e1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:16.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.142+0000 7f041659c640 1 --2- 172.21.15.55:0/2896438868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1dbd0 0x556e63a2e1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.142 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.142+0000 7f041659c640 1 -- 172.21.15.55:0/2896438868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e63a1d960 con 0x556e63a1ca00 2024-04-26T22:04:16.143 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.142+0000 7f041659c640 1 --2- 172.21.15.55:0/2896438868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a17450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f041002a9c0 tx=0x7f041002f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365c5275f4e2f32b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.152+0000 7f03fffff640 1 -- 172.21.15.55:0/2896438868 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410004040 con 0x556e63a1ca00 2024-04-26T22:04:16.152 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.152+0000 7f03fffff640 1 -- 172.21.15.55:0/2896438868 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0410004850 con 0x556e63a1ca00 2024-04-26T22:04:16.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.154+0000 7f041f620740 1 -- 172.21.15.55:0/2896438868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 msgr2=0x556e63a17450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.154 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.154+0000 7f041f620740 1 --2- 172.21.15.55:0/2896438868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a17450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f041002a9c0 tx=0x7f041002f7f0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.156+0000 7f041f620740 1 -- 172.21.15.55:0/2896438868 shutdown_connections 2024-04-26T22:04:16.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.157+0000 7f041f620740 1 --2- 172.21.15.55:0/2896438868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1e590 0x556e63a39820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.157+0000 7f041f620740 1 --2- 172.21.15.55:0/2896438868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1dbd0 0x556e63a2e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.157+0000 7f041f620740 1 --2- 172.21.15.55:0/2896438868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1ca00 0x556e63a17450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.157+0000 7f041f620740 1 -- 172.21.15.55:0/2896438868 >> 172.21.15.55:0/2896438868 conn(0x556e6395d910 msgr2=0x556e63a26f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.158+0000 7f041f620740 1 -- 172.21.15.55:0/2896438868 shutdown_connections 2024-04-26T22:04:16.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.162+0000 7f041f620740 1 -- 172.21.15.55:0/2896438868 wait complete. 2024-04-26T22:04:16.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.162+0000 7f041f620740 1 -- start start 2024-04-26T22:04:16.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.162+0000 7f041f620740 1 Processor -- start 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.163+0000 7f041f620740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1ca00 0x556e63a2bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.163+0000 7f041f620740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 0x556e63a2c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.163+0000 7f041f620740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a2d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.163+0000 7f041f620740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b7890 con 0x556e63a1e590 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.163+0000 7f041f620740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b8d00 con 0x556e63a1ca00 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.163+0000 7f041f620740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04181b7f60 con 0x556e63a1dbd0 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.163+0000 7f041f620740 10 client.0 _ll_register_callbacks cb 0x7f0410009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.164+0000 7f041659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1ca00 0x556e63a2bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.164+0000 7f041659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1ca00 0x556e63a2bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48576/0 (socket says 172.21.15.55:48576) 2024-04-26T22:04:16.165 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.164+0000 7f041659c640 1 -- 172.21.15.55:0/1635735666 learned_addr learned my addr 172.21.15.55:0/1635735666 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:16.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.167+0000 7f0416d9d640 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a2d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.171 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.167+0000 7f041659c640 1 -- 172.21.15.55:0/1635735666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 msgr2=0x556e63a2c4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:16.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.167+0000 7f041659c640 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 0x556e63a2c4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.167+0000 7f041659c640 1 -- 172.21.15.55:0/1635735666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 msgr2=0x556e63a2d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.167+0000 7f041659c640 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a2d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.167+0000 7f041659c640 1 -- 172.21.15.55:0/1635735666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e63a183c0 con 0x556e63a1ca00 2024-04-26T22:04:16.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.174+0000 7f041659c640 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1ca00 0x556e63a2bfb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f041005bd60 tx=0x7f041002ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631a2690074c8386 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.174+0000 7f03fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:04:16.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.175+0000 7f03fdffb640 1 -- 172.21.15.55:0/1635735666 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041006a060 con 0x556e63a1ca00 2024-04-26T22:04:16.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.175+0000 7f041f620740 1 -- 172.21.15.55:0/1635735666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0410004850 con 0x556e63a1ca00 2024-04-26T22:04:16.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.175+0000 7f03fdffb640 1 -- 172.21.15.55:0/1635735666 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0410051440 con 0x556e63a1ca00 2024-04-26T22:04:16.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.175+0000 7f03fdffb640 1 -- 172.21.15.55:0/1635735666 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f041004b040 con 0x556e63a1ca00 2024-04-26T22:04:16.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.177+0000 7f041f620740 10 client.26186 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:16.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.177+0000 7f041f620740 1 -- 172.21.15.55:0/1635735666 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556e63a99560 con 0x556e63a1ca00 2024-04-26T22:04:16.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.178+0000 7f03fdffb640 1 -- 172.21.15.55:0/1635735666 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0410004850 con 0x556e63a1ca00 2024-04-26T22:04:16.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.178+0000 7f03fdffb640 1 -- 172.21.15.55:0/1635735666 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f041004f040 con 0x556e63a1ca00 2024-04-26T22:04:16.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.178+0000 7f041f620740 10 client.26186 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:16.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.178+0000 7f041f620740 10 client.26186 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:04:16.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.178+0000 7f041f620740 20 client.26186 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:16.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.178+0000 7f041f620740 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556e63a99e40 0x556e63a9c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.178+0000 7f041f620740 4 client.26186 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:04:16.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.178+0000 7f041f620740 1 -- 172.21.15.55:0/1635735666 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e63988800 con 0x556e63a99e40 2024-04-26T22:04:16.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.179+0000 7f0415d9b640 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556e63a99e40 0x556e63a9c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.187+0000 7f0415d9b640 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556e63a99e40 0x556e63a9c260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea68907544296366 server_cookie=c58d2c13d8601343 in_seq=0 out_seq=0 2024-04-26T22:04:16.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.187+0000 7f03fdffb640 10 client.26186 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:04:16.190 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:16 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2593856438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:04:16.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f03fdffb640 1 -- 172.21.15.55:0/1635735666 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e63988800 con 0x556e63a99e40 2024-04-26T22:04:16.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f03fdffb640 10 client.26186 handle_command_reply: tid=0 2024-04-26T22:04:16.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 client.26186 shutdown 2024-04-26T22:04:16.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 -- 172.21.15.55:0/1635735666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1ca00 msgr2=0x556e63a2bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1ca00 0x556e63a2bfb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f041005bd60 tx=0x7f041002ff90 comp rx=0 tx=0).stop 2024-04-26T22:04:16.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 -- 172.21.15.55:0/1635735666 shutdown_connections 2024-04-26T22:04:16.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x556e63a99e40 0x556e63a9c260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556e63a1e590 0x556e63a2d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556e63a1dbd0 0x556e63a2c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 --2- 172.21.15.55:0/1635735666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e63a1ca00 0x556e63a2bfb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.190+0000 7f041f620740 1 -- 172.21.15.55:0/1635735666 >> 172.21.15.55:0/1635735666 conn(0x556e6395d910 msgr2=0x556e63a17450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.193+0000 7f03fdffb640 0 client.26186 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:04:16.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.193+0000 7f041f620740 1 -- 172.21.15.55:0/1635735666 shutdown_connections 2024-04-26T22:04:16.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.199+0000 7f041f620740 1 -- 172.21.15.55:0/1635735666 wait complete. 2024-04-26T22:04:16.202 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.202+0000 7f041f620740 20 client.26186 trim_cache size 0 max 16384 2024-04-26T22:04:16.208 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:04:16.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.224+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04000a3d80 msgr2=0x7f04000a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.224+0000 7f041759e640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04000a3d80 0x7f04000a6240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0410000f80 tx=0x7f0410009910 comp rx=0 tx=0).stop 2024-04-26T22:04:16.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.224+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04181ad180 msgr2=0x7f04181af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.224 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.224+0000 7f041759e640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04181ad180 0x7f04181af570 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f040c00b410 tx=0x7f040c00b810 comp rx=0 tx=0).stop 2024-04-26T22:04:16.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.229+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 shutdown_connections 2024-04-26T22:04:16.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.229+0000 7f041759e640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04000a3d80 0x7f04000a6240 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.229+0000 7f041759e640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04181ad180 0x7f04181af570 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.229+0000 7f041759e640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f041810ec50 0x7f04181a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.229+0000 7f041759e640 1 --2- 172.21.15.55:0/584620707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f041810c2a0 0x7f04181a7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.229+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 >> 172.21.15.55:0/584620707 conn(0x7f04180706d0 msgr2=0x7f041810ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.229+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 shutdown_connections 2024-04-26T22:04:16.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.229+0000 7f041759e640 1 -- 172.21.15.55:0/584620707 wait complete. 2024-04-26T22:04:16.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:04:16.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:16 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2593856438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:04:16.579 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:16 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2593856438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:04:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.685+0000 7f72c0977640 1 -- 172.21.15.55:0/929503096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc103970 msgr2=0x7f72bc105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.685+0000 7f72c0977640 1 --2- 172.21.15.55:0/929503096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc103970 0x7f72bc105de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009900 tx=0x7f72ac02a410 comp rx=0 tx=0).stop 2024-04-26T22:04:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.689+0000 7f72c0977640 1 -- 172.21.15.55:0/929503096 shutdown_connections 2024-04-26T22:04:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.689+0000 7f72c0977640 1 --2- 172.21.15.55:0/929503096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72bc106320 0x7f72bc1067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.689+0000 7f72c0977640 1 --2- 172.21.15.55:0/929503096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc103970 0x7f72bc105de0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.689+0000 7f72c0977640 1 --2- 172.21.15.55:0/929503096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc101000 0x7f72bc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.689+0000 7f72c0977640 1 -- 172.21.15.55:0/929503096 >> 172.21.15.55:0/929503096 conn(0x7f72bc0fcc40 msgr2=0x7f72bc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.696+0000 7f72c0977640 1 -- 172.21.15.55:0/929503096 shutdown_connections 2024-04-26T22:04:16.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.699+0000 7f72c0977640 1 -- 172.21.15.55:0/929503096 wait complete. 2024-04-26T22:04:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.700+0000 7f72c0977640 1 Processor -- start 2024-04-26T22:04:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.701+0000 7f72c0977640 1 -- start start 2024-04-26T22:04:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.701+0000 7f72c0977640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72bc101000 0x7f72bc071280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.701+0000 7f72c0977640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc103970 0x7f72bc0717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.701+0000 7f72c0977640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc106320 0x7f72bc19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.701+0000 7f72c0977640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc114960 con 0x7f72bc101000 2024-04-26T22:04:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.701+0000 7f72c0977640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1147e0 con 0x7f72bc106320 2024-04-26T22:04:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.701+0000 7f72c0977640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc114ae0 con 0x7f72bc103970 2024-04-26T22:04:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.704+0000 7f72bad9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72bc101000 0x7f72bc071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.704+0000 7f72bb59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc106320 0x7f72bc19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.704+0000 7f72bb59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc106320 0x7f72bc19e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48590/0 (socket says 172.21.15.55:48590) 2024-04-26T22:04:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.704+0000 7f72bb59e640 1 -- 172.21.15.55:0/1767701724 learned_addr learned my addr 172.21.15.55:0/1767701724 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:16.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.706+0000 7f72ba59c640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc103970 0x7f72bc0717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.708+0000 7f72bb59e640 1 -- 172.21.15.55:0/1767701724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc103970 msgr2=0x7f72bc0717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.708+0000 7f72bb59e640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc103970 0x7f72bc0717c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.708+0000 7f72bb59e640 1 -- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72bc101000 msgr2=0x7f72bc071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.708+0000 7f72bb59e640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72bc101000 0x7f72bc071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.708+0000 7f72bb59e640 1 -- 172.21.15.55:0/1767701724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72bc19e5a0 con 0x7f72bc106320 2024-04-26T22:04:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.715+0000 7f72bb59e640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc106320 0x7f72bc19e060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009fd0 tx=0x7f72b000b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.721+0000 7f72a3fff640 1 -- 172.21.15.55:0/1767701724 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0011830 con 0x7f72bc106320 2024-04-26T22:04:16.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.721+0000 7f72a3fff640 1 -- 172.21.15.55:0/1767701724 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b0016070 con 0x7f72bc106320 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.721+0000 7f72a3fff640 1 -- 172.21.15.55:0/1767701724 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0010de0 con 0x7f72bc106320 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.723+0000 7f72c0977640 1 -- 172.21.15.55:0/1767701724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72bc19e890 con 0x7f72bc106320 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.723+0000 7f72c0977640 1 -- 172.21.15.55:0/1767701724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72bc19ed90 con 0x7f72bc106320 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.729+0000 7f72c3f6f740 1 Processor -- start 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.729+0000 7f72c3f6f740 1 -- start start 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.729+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.729+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b7c00 0x55fd964b8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.729+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964bee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.729+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fd9641ad80 con 0x55fd964b85c0 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.729+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fd963d4a20 con 0x55fd964b7c00 2024-04-26T22:04:16.732 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.729+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fd96416d40 con 0x55fd964b6a30 2024-04-26T22:04:16.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.734+0000 7f72bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b7c00 0x55fd964b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.734+0000 7f72ba59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964bee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.734+0000 7f72ba59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964bee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49574/0 (socket says 172.21.15.55:49574) 2024-04-26T22:04:16.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.734+0000 7f72ba59c640 1 -- 172.21.15.55:0/3275210405 learned_addr learned my addr 172.21.15.55:0/3275210405 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:16.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.734+0000 7f72ba59c640 1 --2- 172.21.15.55:0/3275210405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.736+0000 7f72ba59c640 1 -- 172.21.15.55:0/3275210405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 msgr2=0x55fd964b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.736+0000 7f72ba59c640 1 --2- 172.21.15.55:0/3275210405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.736+0000 7f72ba59c640 1 -- 172.21.15.55:0/3275210405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b7c00 msgr2=0x55fd964b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.736+0000 7f72ba59c640 1 --2- 172.21.15.55:0/3275210405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b7c00 0x55fd964b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.737+0000 7f72ba59c640 1 -- 172.21.15.55:0/3275210405 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd963dfc30 con 0x55fd964b85c0 2024-04-26T22:04:16.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.740+0000 7f72ba59c640 1 --2- 172.21.15.55:0/3275210405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964bee50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f72ac000c00 tx=0x7f72ac02a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ee76033152a42f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.743+0000 7f72a3fff640 1 -- 172.21.15.55:0/1767701724 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f72b00119d0 con 0x7f72bc106320 2024-04-26T22:04:16.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.745+0000 7f72a3fff640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72a40a3e40 0x7f72a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.746+0000 7f72a3fff640 1 -- 172.21.15.55:0/1767701724 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f72b00cf4d0 con 0x7f72bc106320 2024-04-26T22:04:16.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.750+0000 7f72a17fa640 1 -- 172.21.15.55:0/3275210405 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac035070 con 0x55fd964b85c0 2024-04-26T22:04:16.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.750+0000 7f72a17fa640 1 -- 172.21.15.55:0/3275210405 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ac045070 con 0x55fd964b85c0 2024-04-26T22:04:16.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.752+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3275210405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 msgr2=0x55fd964bee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.752+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3275210405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964bee50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f72ac000c00 tx=0x7f72ac02a910 comp rx=0 tx=0).stop 2024-04-26T22:04:16.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.753+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3275210405 shutdown_connections 2024-04-26T22:04:16.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.753+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3275210405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964bee50 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.753+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3275210405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b7c00 0x55fd964b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.753+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3275210405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.753+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3275210405 >> 172.21.15.55:0/3275210405 conn(0x55fd963f7910 msgr2=0x55fd964b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.758+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3275210405 shutdown_connections 2024-04-26T22:04:16.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.758+0000 7f72bad9d640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72a40a3e40 0x7f72a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.759+0000 7f72bad9d640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72a40a3e40 0x7f72a40a6300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55fd964b31e0 tx=0x7f72b4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.761+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3275210405 wait complete. 2024-04-26T22:04:16.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.762+0000 7f72c3f6f740 1 -- start start 2024-04-26T22:04:16.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.763+0000 7f72c3f6f740 1 Processor -- start 2024-04-26T22:04:16.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.763+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.763+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b7c00 0x55fd964c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.763+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964c8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.763+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fd964bf5c0 con 0x55fd964b85c0 2024-04-26T22:04:16.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.763+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fd963a1dc0 con 0x55fd964b6a30 2024-04-26T22:04:16.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.764+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fd964bf740 con 0x55fd964b7c00 2024-04-26T22:04:16.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.764+0000 7f72c3f6f740 10 client.0 _ll_register_callbacks cb 0x55fd963bd600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:16.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.765+0000 7f72bb59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.765+0000 7f72bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.765+0000 7f72bad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48598/0 (socket says 172.21.15.55:48598) 2024-04-26T22:04:16.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.765+0000 7f72bb59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49578/0 (socket says 172.21.15.55:49578) 2024-04-26T22:04:16.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.765+0000 7f72bad9d640 1 -- 172.21.15.55:0/3802113458 learned_addr learned my addr 172.21.15.55:0/3802113458 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:16.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.767+0000 7f72bb59e640 1 -- 172.21.15.55:0/3802113458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b7c00 msgr2=0x55fd964c1130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:16.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.767+0000 7f72bb59e640 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b7c00 0x55fd964c1130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.767+0000 7f72bb59e640 1 -- 172.21.15.55:0/3802113458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 msgr2=0x55fd964c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.767+0000 7f72bb59e640 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.767+0000 7f72bb59e640 1 -- 172.21.15.55:0/3802113458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd9612c3f0 con 0x55fd964b85c0 2024-04-26T22:04:16.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.771+0000 7f72bb59e640 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964c8170 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f72b00ccef0 tx=0x7f72b009be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99256d1e6baa8ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.773+0000 7f729eff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:16.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.773+0000 7f729eff5640 1 -- 172.21.15.55:0/3802113458 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b009fb00 con 0x55fd964b85c0 2024-04-26T22:04:16.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.773+0000 7f729eff5640 1 -- 172.21.15.55:0/3802113458 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f72b0097dc0 con 0x55fd964b85c0 2024-04-26T22:04:16.774 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.773+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3802113458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ac045070 con 0x55fd964b85c0 2024-04-26T22:04:16.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.774+0000 7f729eff5640 1 -- 172.21.15.55:0/3802113458 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f72b00a4330 con 0x55fd964b85c0 2024-04-26T22:04:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.776+0000 7f72c3f6f740 10 client.17208 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.776+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3802113458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fd961f67d0 con 0x55fd964b85c0 2024-04-26T22:04:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.779+0000 7f729eff5640 1 -- 172.21.15.55:0/3802113458 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ac045070 con 0x55fd964b85c0 2024-04-26T22:04:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.780+0000 7f729eff5640 1 -- 172.21.15.55:0/3802113458 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f72b00974a0 con 0x55fd964b85c0 2024-04-26T22:04:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.780+0000 7f72c3f6f740 10 client.17208 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.780+0000 7f72c3f6f740 10 client.17208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:04:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.780+0000 7f72c3f6f740 20 client.17208 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.780+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fd964ccde0 0x55fd964cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.780+0000 7f72c3f6f740 4 client.17208 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.780+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3802113458 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd96428990 con 0x55fd964ccde0 2024-04-26T22:04:16.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.782+0000 7f72bad9d640 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fd964ccde0 0x55fd964cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.789 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.789+0000 7f72bad9d640 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fd964ccde0 0x55fd964cf260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5690a21b88c44dd7 server_cookie=f55bdfad70083e77 in_seq=0 out_seq=0 2024-04-26T22:04:16.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.789+0000 7f729eff5640 10 client.17208 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:04:16.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.792+0000 7f729eff5640 1 -- 172.21.15.55:0/3802113458 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd96428990 con 0x55fd964ccde0 2024-04-26T22:04:16.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.792+0000 7f729eff5640 10 client.17208 handle_command_reply: tid=0 2024-04-26T22:04:16.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.792+0000 7f72c3f6f740 1 client.17208 shutdown 2024-04-26T22:04:16.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.794+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3802113458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 msgr2=0x55fd964c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.794+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964c8170 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f72b00ccef0 tx=0x7f72b009be50 comp rx=0 tx=0).stop 2024-04-26T22:04:16.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.794+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3802113458 shutdown_connections 2024-04-26T22:04:16.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.794+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fd964ccde0 0x55fd964cf260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.794+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b85c0 0x55fd964c8170 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.794+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b7c00 0x55fd964c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.794+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3802113458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.794+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3802113458 >> 172.21.15.55:0/3802113458 conn(0x55fd963f7910 msgr2=0x55fd964b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.795+0000 7f729eff5640 0 client.17208 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:04:16.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.799+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3802113458 shutdown_connections 2024-04-26T22:04:16.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.799+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3802113458 wait complete. 2024-04-26T22:04:16.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.801+0000 7f72c3f6f740 20 client.17208 trim_cache size 0 max 16384 2024-04-26T22:04:16.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.852+0000 7f72c3f6f740 1 Processor -- start 2024-04-26T22:04:16.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.853+0000 7f72c3f6f740 1 -- start start 2024-04-26T22:04:16.853 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.853+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.853+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.854 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.853+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b85c0 0x55fd964bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72ba59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72bad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72ba59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49580/0 (socket says 172.21.15.55:49580) 2024-04-26T22:04:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72bad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964c0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42124/0 (socket says 172.21.15.55:42124) 2024-04-26T22:04:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72ba59c640 1 -- 172.21.15.55:0/1971501530 learned_addr learned my addr 172.21.15.55:0/1971501530 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72c3f6f740 1 -- 172.21.15.55:0/1971501530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1aa350 con 0x55fd964b7c00 2024-04-26T22:04:16.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72c3f6f740 1 -- 172.21.15.55:0/1971501530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1a7c20 con 0x55fd964b85c0 2024-04-26T22:04:16.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72c3f6f740 1 -- 172.21.15.55:0/1971501530 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1a5a70 con 0x55fd964b6a30 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.858+0000 7f72bb59e640 1 --2- 172.21.15.55:0/1971501530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b85c0 0x55fd964bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.859+0000 7f72ba59c640 1 -- 172.21.15.55:0/1971501530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 msgr2=0x55fd964c0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.859+0000 7f72ba59c640 1 --2- 172.21.15.55:0/1971501530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964c0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.859+0000 7f72ba59c640 1 -- 172.21.15.55:0/1971501530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b85c0 msgr2=0x55fd964bb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.859+0000 7f72ba59c640 1 --2- 172.21.15.55:0/1971501530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b85c0 0x55fd964bb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.859+0000 7f72ba59c640 1 -- 172.21.15.55:0/1971501530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd964b28d0 con 0x55fd964b7c00 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.859+0000 7f72ba59c640 1 --2- 172.21.15.55:0/1971501530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c8130 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009900 tx=0x7f72ac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1126311b802b97b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.861+0000 7f72a0ff9640 1 -- 172.21.15.55:0/1971501530 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac04e070 con 0x55fd964b7c00 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.861+0000 7f72a0ff9640 1 -- 172.21.15.55:0/1971501530 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ac045040 con 0x55fd964b7c00 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.862+0000 7f72c3f6f740 1 -- 172.21.15.55:0/1971501530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 msgr2=0x55fd964c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.862+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/1971501530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c8130 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009900 tx=0x7f72ac02a410 comp rx=0 tx=0).stop 2024-04-26T22:04:16.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.863+0000 7f72c3f6f740 1 -- 172.21.15.55:0/1971501530 shutdown_connections 2024-04-26T22:04:16.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.863+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/1971501530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b85c0 0x55fd964bb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.863+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/1971501530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c8130 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.863+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/1971501530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b6a30 0x55fd964c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.863+0000 7f72c3f6f740 1 -- 172.21.15.55:0/1971501530 >> 172.21.15.55:0/1971501530 conn(0x55fd963f7910 msgr2=0x55fd964c0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.864+0000 7f72c3f6f740 1 -- 172.21.15.55:0/1971501530 shutdown_connections 2024-04-26T22:04:16.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.864+0000 7f72c3f6f740 1 -- 172.21.15.55:0/1971501530 wait complete. 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 1 -- start start 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 1 Processor -- start 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b85c0 0x55fd965313d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1a60e0 con 0x55fd964b7c00 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1a7550 con 0x55fd964b6a30 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72bc1a67b0 con 0x55fd964b85c0 2024-04-26T22:04:16.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.865+0000 7f72c3f6f740 10 client.0 _ll_register_callbacks cb 0x7f72b00973b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:16.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.866+0000 7f72ba59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.866+0000 7f72bb59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b85c0 0x55fd965313d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.866+0000 7f72bb59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b85c0 0x55fd965313d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42128/0 (socket says 172.21.15.55:42128) 2024-04-26T22:04:16.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.866+0000 7f72bb59e640 1 -- 172.21.15.55:0/3161079665 learned_addr learned my addr 172.21.15.55:0/3161079665 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:16.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.866+0000 7f72bad9d640 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.867+0000 7f72bb59e640 1 -- 172.21.15.55:0/3161079665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 msgr2=0x55fd964c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.867+0000 7f72bb59e640 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.867+0000 7f72bb59e640 1 -- 172.21.15.55:0/3161079665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 msgr2=0x55fd964c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:16.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.867+0000 7f72bb59e640 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.867+0000 7f72bb59e640 1 -- 172.21.15.55:0/3161079665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd964c5ac0 con 0x55fd964b85c0 2024-04-26T22:04:16.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.868+0000 7f72bb59e640 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b85c0 0x55fd965313d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72b009b910 tx=0x7f72b0011ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c8099f2abf4488b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:16.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.868+0000 7f729eff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:04:16.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.868+0000 7f729eff5640 1 -- 172.21.15.55:0/3161079665 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00a8020 con 0x55fd964b85c0 2024-04-26T22:04:16.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.868+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3161079665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ac045040 con 0x55fd964b85c0 2024-04-26T22:04:16.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.869+0000 7f729eff5640 1 -- 172.21.15.55:0/3161079665 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f72b0097dc0 con 0x55fd964b85c0 2024-04-26T22:04:16.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.871+0000 7f729eff5640 1 -- 172.21.15.55:0/3161079665 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f72b009b290 con 0x55fd964b85c0 2024-04-26T22:04:16.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.873+0000 7f72c3f6f740 10 client.26212 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:16.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.873+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3161079665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fd96543ac0 con 0x55fd964b85c0 2024-04-26T22:04:16.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.881+0000 7f729eff5640 1 -- 172.21.15.55:0/3161079665 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ac045040 con 0x55fd964b85c0 2024-04-26T22:04:16.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.882+0000 7f729eff5640 1 -- 172.21.15.55:0/3161079665 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f72b009a660 con 0x55fd964b85c0 2024-04-26T22:04:16.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.882+0000 7f72c3f6f740 10 client.26212 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:16.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.882+0000 7f72c3f6f740 10 client.26212 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:04:16.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.882+0000 7f72c3f6f740 20 client.26212 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:16.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.882+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fd965444f0 0x55fd96546910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:16.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.882+0000 7f72c3f6f740 4 client.26212 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:04:16.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.882+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3161079665 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd964c09e0 con 0x55fd965444f0 2024-04-26T22:04:16.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.885+0000 7f72bad9d640 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fd965444f0 0x55fd96546910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:16.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.890+0000 7f72bad9d640 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fd965444f0 0x55fd96546910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb6bfba96f0148ef server_cookie=b786df2a91320b3a in_seq=0 out_seq=0 2024-04-26T22:04:16.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.890+0000 7f729eff5640 10 client.26212 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:04:16.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.893+0000 7f729eff5640 1 -- 172.21.15.55:0/3161079665 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd964c09e0 con 0x55fd965444f0 2024-04-26T22:04:16.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.893+0000 7f729eff5640 10 client.26212 handle_command_reply: tid=0 2024-04-26T22:04:16.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.893+0000 7f72c3f6f740 1 client.26212 shutdown 2024-04-26T22:04:16.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3161079665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b85c0 msgr2=0x55fd965313d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b85c0 0x55fd965313d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72b009b910 tx=0x7f72b0011ef0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3161079665 shutdown_connections 2024-04-26T22:04:16.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55fd965444f0 0x55fd96546910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fd964b85c0 0x55fd965313d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fd964b7c00 0x55fd964c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f72c3f6f740 1 --2- 172.21.15.55:0/3161079665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fd964b6a30 0x55fd964c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3161079665 >> 172.21.15.55:0/3161079665 conn(0x55fd963f7910 msgr2=0x55fd964c0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.896+0000 7f729eff5640 0 client.26212 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:04:16.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.898+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3161079665 shutdown_connections 2024-04-26T22:04:16.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.898+0000 7f72c3f6f740 1 -- 172.21.15.55:0/3161079665 wait complete. 2024-04-26T22:04:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.902+0000 7f72c3f6f740 20 client.26212 trim_cache size 0 max 16384 2024-04-26T22:04:16.903 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:04:16.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 -- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72a40a3e40 msgr2=0x7f72a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72a40a3e40 0x7f72a40a6300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55fd964b31e0 tx=0x7f72b4009500 comp rx=0 tx=0).stop 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 -- 172.21.15.55:0/1767701724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc106320 msgr2=0x7f72bc19e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc106320 0x7f72bc19e060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009fd0 tx=0x7f72b000b8e0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 -- 172.21.15.55:0/1767701724 shutdown_connections 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f72a40a3e40 0x7f72a40a6300 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72bc106320 0x7f72bc19e060 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72bc103970 0x7f72bc0717c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 --2- 172.21.15.55:0/1767701724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72bc101000 0x7f72bc071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.906+0000 7f72c0977640 1 -- 172.21.15.55:0/1767701724 >> 172.21.15.55:0/1767701724 conn(0x7f72bc0fcc40 msgr2=0x7f72bc1018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:16.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.908+0000 7f72c0977640 1 -- 172.21.15.55:0/1767701724 shutdown_connections 2024-04-26T22:04:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:16.909+0000 7f72c0977640 1 -- 172.21.15.55:0/1767701724 wait complete. 2024-04-26T22:04:16.952 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:04:17.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.315+0000 7f85193c8640 1 -- 172.21.15.55:0/563482671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8514109f10 msgr2=0x7f851410a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:17.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.315+0000 7f85193c8640 1 --2- 172.21.15.55:0/563482671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8514109f10 0x7f851410a390 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8508009900 tx=0x7f850802a440 comp rx=0 tx=0).stop 2024-04-26T22:04:17.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.325+0000 7f85193c8640 1 -- 172.21.15.55:0/563482671 shutdown_connections 2024-04-26T22:04:17.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.325+0000 7f85193c8640 1 --2- 172.21.15.55:0/563482671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f851410a8d0 0x7f851410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.325+0000 7f85193c8640 1 --2- 172.21.15.55:0/563482671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8514109f10 0x7f851410a390 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.325+0000 7f85193c8640 1 --2- 172.21.15.55:0/563482671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514073c10 0x7f8514074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.325+0000 7f85193c8640 1 -- 172.21.15.55:0/563482671 >> 172.21.15.55:0/563482671 conn(0x7f851406f780 msgr2=0x7f8514071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:17.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.328+0000 7f85193c8640 1 -- 172.21.15.55:0/563482671 shutdown_connections 2024-04-26T22:04:17.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.332+0000 7f85193c8640 1 -- 172.21.15.55:0/563482671 wait complete. 2024-04-26T22:04:17.333 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.333+0000 7f85193c8640 1 Processor -- start 2024-04-26T22:04:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.338+0000 7f85193c8640 1 -- start start 2024-04-26T22:04:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.338+0000 7f85193c8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8514073c10 0x7f8514116000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.338+0000 7f85193c8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8514109f10 0x7f8514116540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.338+0000 7f85193c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851410a8d0 0x7f85141b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.338+0000 7f85193c8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f851411d230 con 0x7f8514073c10 2024-04-26T22:04:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.338+0000 7f85193c8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f851411d0b0 con 0x7f851410a8d0 2024-04-26T22:04:17.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.338+0000 7f85193c8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f851411d3b0 con 0x7f8514109f10 2024-04-26T22:04:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.339+0000 7f85137fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8514109f10 0x7f8514116540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.339+0000 7f85137fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8514109f10 0x7f8514116540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42150/0 (socket says 172.21.15.55:42150) 2024-04-26T22:04:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.339+0000 7f85137fe640 1 -- 172.21.15.55:0/1323321707 learned_addr learned my addr 172.21.15.55:0/1323321707 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.340+0000 7f85137fe640 1 -- 172.21.15.55:0/1323321707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851410a8d0 msgr2=0x7f85141b7940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.340+0000 7f85137fe640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851410a8d0 0x7f85141b7940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.340+0000 7f85137fe640 1 -- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8514073c10 msgr2=0x7f8514116000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T22:04:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.340+0000 7f85137fe640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8514073c10 0x7f8514116000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.340+0000 7f85137fe640 1 -- 172.21.15.55:0/1323321707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85141b7e80 con 0x7f8514109f10 2024-04-26T22:04:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.341+0000 7f85137fe640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8514109f10 0x7f8514116540 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f850802aa30 tx=0x7f8508030bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.341+0000 7f85117fa640 1 -- 172.21.15.55:0/1323321707 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508040070 con 0x7f8514109f10 2024-04-26T22:04:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.342+0000 7f85193c8640 1 -- 172.21.15.55:0/1323321707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85141b8050 con 0x7f8514109f10 2024-04-26T22:04:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.342+0000 7f85193c8640 1 -- 172.21.15.55:0/1323321707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85141b84d0 con 0x7f8514109f10 2024-04-26T22:04:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.342+0000 7f85117fa640 1 -- 172.21.15.55:0/1323321707 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8508004220 con 0x7f8514109f10 2024-04-26T22:04:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.342+0000 7f85117fa640 1 -- 172.21.15.55:0/1323321707 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508035030 con 0x7f8514109f10 2024-04-26T22:04:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.350+0000 7f85117fa640 1 -- 172.21.15.55:0/1323321707 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f85080035f0 con 0x7f8514109f10 2024-04-26T22:04:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.356+0000 7f85117fa640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f84fc0a3e40 0x7f84fc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.356+0000 7f85117fa640 1 -- 172.21.15.55:0/1323321707 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f85080f1650 con 0x7f8514109f10 2024-04-26T22:04:17.358 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.358+0000 7f851d421740 1 Processor -- start 2024-04-26T22:04:17.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.359+0000 7f851d421740 1 -- start start 2024-04-26T22:04:17.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.359+0000 7f851d421740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.359+0000 7f851d421740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 0x558f486ba080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.359+0000 7f851d421740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f486c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.359+0000 7f851d421740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558f4861cd80 con 0x558f486b8a30 2024-04-26T22:04:17.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.359+0000 7f851d421740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f485d6a20 con 0x558f486ba5c0 2024-04-26T22:04:17.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.359+0000 7f851d421740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f48618d40 con 0x558f486b9c00 2024-04-26T22:04:17.360 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.359+0000 7f8513fff640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f84fc0a3e40 0x7f84fc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.362+0000 7f8518bc7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f486c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.362+0000 7f85137fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 0x558f486ba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.362+0000 7f8513fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.362+0000 7f8513fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486b8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49588/0 (socket says 172.21.15.55:49588) 2024-04-26T22:04:17.363 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.362+0000 7f8513fff640 1 -- 172.21.15.55:0/69081902 learned_addr learned my addr 172.21.15.55:0/69081902 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:17.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.368+0000 7f8513fff640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f84fc0a3e40 0x7f84fc0a6300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f850c00ad30 tx=0x7f850c00a670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:17.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.369+0000 7f8513fff640 1 -- 172.21.15.55:0/69081902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 msgr2=0x558f486ba080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:17.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.369+0000 7f8513fff640 1 --2- 172.21.15.55:0/69081902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 0x558f486ba080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.369+0000 7f8513fff640 1 -- 172.21.15.55:0/69081902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 msgr2=0x558f486c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:17.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.369+0000 7f8513fff640 1 --2- 172.21.15.55:0/69081902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f486c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.369+0000 7f8513fff640 1 -- 172.21.15.55:0/69081902 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f485e1c30 con 0x558f486b8a30 2024-04-26T22:04:17.381 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.381+0000 7f8513fff640 1 --2- 172.21.15.55:0/69081902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486b8e50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f850c00efd0 tx=0x7f850c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64439760120190d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:17.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.385+0000 7f84fa7fc640 1 -- 172.21.15.55:0/69081902 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c00d1d0 con 0x558f486b8a30 2024-04-26T22:04:17.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.385+0000 7f84fa7fc640 1 -- 172.21.15.55:0/69081902 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c021910 con 0x558f486b8a30 2024-04-26T22:04:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.387+0000 7f851d421740 1 -- 172.21.15.55:0/69081902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 msgr2=0x558f486b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.387+0000 7f851d421740 1 --2- 172.21.15.55:0/69081902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486b8e50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f850c00efd0 tx=0x7f850c017000 comp rx=0 tx=0).stop 2024-04-26T22:04:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.388+0000 7f851d421740 1 -- 172.21.15.55:0/69081902 shutdown_connections 2024-04-26T22:04:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.388+0000 7f851d421740 1 --2- 172.21.15.55:0/69081902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f486c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.388+0000 7f851d421740 1 --2- 172.21.15.55:0/69081902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 0x558f486ba080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.388+0000 7f851d421740 1 --2- 172.21.15.55:0/69081902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486b8e50 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.388+0000 7f851d421740 1 -- 172.21.15.55:0/69081902 >> 172.21.15.55:0/69081902 conn(0x558f485f9910 msgr2=0x558f486b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:17.392 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.392+0000 7f851d421740 1 -- 172.21.15.55:0/69081902 shutdown_connections 2024-04-26T22:04:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.395+0000 7f851d421740 1 -- 172.21.15.55:0/69081902 wait complete. 2024-04-26T22:04:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.395+0000 7f851d421740 1 -- start start 2024-04-26T22:04:17.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.396+0000 7f851d421740 1 Processor -- start 2024-04-26T22:04:17.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.399+0000 7f851d421740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b8a30 0x558f486c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.399+0000 7f851d421740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b9c00 0x558f486c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.399+0000 7f851d421740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486ba5c0 0x558f486ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.399+0000 7f851d421740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558f486c1620 con 0x558f486b9c00 2024-04-26T22:04:17.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.399+0000 7f851d421740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f485a3dc0 con 0x558f486b8a30 2024-04-26T22:04:17.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.399+0000 7f851d421740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f486c17a0 con 0x558f486ba5c0 2024-04-26T22:04:17.400 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.399+0000 7f851d421740 10 client.0 _ll_register_callbacks cb 0x558f485bf600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:17.406 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.406+0000 7f8518bc7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486ba5c0 0x558f486ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.407+0000 7f8518bc7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486ba5c0 0x558f486ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42172/0 (socket says 172.21.15.55:42172) 2024-04-26T22:04:17.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.407+0000 7f8518bc7640 1 -- 172.21.15.55:0/2759352805 learned_addr learned my addr 172.21.15.55:0/2759352805 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.408+0000 7f85137fe640 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b9c00 0x558f486c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.409+0000 7f8513fff640 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b8a30 0x558f486c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.410+0000 7f8518bc7640 1 -- 172.21.15.55:0/2759352805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b8a30 msgr2=0x558f486c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.410+0000 7f8518bc7640 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b8a30 0x558f486c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.410+0000 7f8518bc7640 1 -- 172.21.15.55:0/2759352805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b9c00 msgr2=0x558f486c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.410+0000 7f8518bc7640 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b9c00 0x558f486c3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.410+0000 7f8518bc7640 1 -- 172.21.15.55:0/2759352805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f4832e3f0 con 0x558f486ba5c0 2024-04-26T22:04:17.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:17 smithi055 ceph-mon[29221]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 800 MiB/s wr, 1.47k op/s 2024-04-26T22:04:17.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:17 smithi064 ceph-mon[32765]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 800 MiB/s wr, 1.47k op/s 2024-04-26T22:04:17.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:17 smithi165 ceph-mon[33296]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 800 MiB/s wr, 1.47k op/s 2024-04-26T22:04:17.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.620+0000 7f8518bc7640 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486ba5c0 0x558f486ca4d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f850400e200 tx=0x7f850400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=448f962dcb5bfe93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:17.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.620+0000 7f84f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:04:17.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.621+0000 7f84f37fe640 1 -- 172.21.15.55:0/2759352805 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504017070 con 0x558f486ba5c0 2024-04-26T22:04:17.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.621+0000 7f84f37fe640 1 -- 172.21.15.55:0/2759352805 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8504013680 con 0x558f486ba5c0 2024-04-26T22:04:17.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.622+0000 7f851d421740 1 -- 172.21.15.55:0/2759352805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f4862a990 con 0x558f486ba5c0 2024-04-26T22:04:17.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.625+0000 7f84f37fe640 1 -- 172.21.15.55:0/2759352805 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8504023050 con 0x558f486ba5c0 2024-04-26T22:04:17.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.625+0000 7f851d421740 10 client.26221 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:17.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.625+0000 7f851d421740 1 -- 172.21.15.55:0/2759352805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558f483f87d0 con 0x558f486ba5c0 2024-04-26T22:04:17.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.627+0000 7f84f37fe640 1 -- 172.21.15.55:0/2759352805 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558f4862a990 con 0x558f486ba5c0 2024-04-26T22:04:17.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.627+0000 7f84f37fe640 1 -- 172.21.15.55:0/2759352805 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f85040128d0 con 0x558f486ba5c0 2024-04-26T22:04:17.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.627+0000 7f851d421740 10 client.26221 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:17.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.627+0000 7f851d421740 10 client.26221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:17.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.627+0000 7f851d421740 20 client.26221 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:17.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.627+0000 7f851d421740 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558f486ceef0 0x558f486d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.627+0000 7f851d421740 4 client.26221 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:17.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.627+0000 7f851d421740 1 -- 172.21.15.55:0/2759352805 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f48624800 con 0x558f486ceef0 2024-04-26T22:04:17.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.628+0000 7f8513fff640 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558f486ceef0 0x558f486d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.629+0000 7f8513fff640 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558f486ceef0 0x558f486d1370 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94cfa38b44b80d03 server_cookie=45771f918997e269 in_seq=0 out_seq=0 2024-04-26T22:04:17.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.630+0000 7f84f37fe640 10 client.26221 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:17.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.634+0000 7f84f37fe640 1 -- 172.21.15.55:0/2759352805 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f48624800 con 0x558f486ceef0 2024-04-26T22:04:17.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.634+0000 7f84f37fe640 10 client.26221 handle_command_reply: tid=0 2024-04-26T22:04:17.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.634+0000 7f851d421740 1 client.26221 shutdown 2024-04-26T22:04:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f851d421740 1 -- 172.21.15.55:0/2759352805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486ba5c0 msgr2=0x558f486ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:17.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f851d421740 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486ba5c0 0x558f486ca4d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f850400e200 tx=0x7f850400e6c0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f851d421740 1 -- 172.21.15.55:0/2759352805 shutdown_connections 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f851d421740 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486ba5c0 0x558f486ca4d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f851d421740 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558f486ceef0 0x558f486d1370 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f851d421740 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b9c00 0x558f486c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f851d421740 1 --2- 172.21.15.55:0/2759352805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b8a30 0x558f486c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f851d421740 1 -- 172.21.15.55:0/2759352805 >> 172.21.15.55:0/2759352805 conn(0x558f485f9910 msgr2=0x558f486b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.636+0000 7f84f37fe640 0 client.26221 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.637+0000 7f851d421740 1 -- 172.21.15.55:0/2759352805 shutdown_connections 2024-04-26T22:04:17.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.637+0000 7f851d421740 1 -- 172.21.15.55:0/2759352805 wait complete. 2024-04-26T22:04:17.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.638+0000 7f851d421740 20 client.26221 trim_cache size 0 max 16384 2024-04-26T22:04:17.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.679+0000 7f851d421740 1 Processor -- start 2024-04-26T22:04:17.680 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.680+0000 7f851d421740 1 -- start start 2024-04-26T22:04:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.680+0000 7f851d421740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b8a30 0x558f486c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.681+0000 7f851d421740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b9c00 0x558f486ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.681+0000 7f851d421740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486ba5c0 0x558f486b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.681+0000 7f851d421740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8500010fd0 con 0x558f486ba5c0 2024-04-26T22:04:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.681+0000 7f851d421740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f850000e8a0 con 0x558f486b9c00 2024-04-26T22:04:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.681+0000 7f851d421740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f850000c6f0 con 0x558f486b8a30 2024-04-26T22:04:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.682+0000 7f8513fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b8a30 0x558f486c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.682+0000 7f8513fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b8a30 0x558f486c2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:42186/0 (socket says 172.21.15.55:42186) 2024-04-26T22:04:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.682+0000 7f8513fff640 1 -- 172.21.15.55:0/3732673961 learned_addr learned my addr 172.21.15.55:0/3732673961 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.682+0000 7f85137fe640 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b9c00 0x558f486ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.682+0000 7f8518bc7640 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486ba5c0 0x558f486b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.683+0000 7f8518bc7640 1 -- 172.21.15.55:0/3732673961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b8a30 msgr2=0x558f486c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.683+0000 7f8518bc7640 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b8a30 0x558f486c2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.683+0000 7f8518bc7640 1 -- 172.21.15.55:0/3732673961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b9c00 msgr2=0x558f486ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.683+0000 7f8518bc7640 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b9c00 0x558f486ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.683+0000 7f8518bc7640 1 -- 172.21.15.55:0/3732673961 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f486afcd0 con 0x558f486ba5c0 2024-04-26T22:04:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.683+0000 7f85137fe640 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b9c00 0x558f486ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:04:17.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.683+0000 7f8518bc7640 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486ba5c0 0x558f486b7090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8504000ea0 tx=0x7f8504009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec755c346832c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:17.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.684+0000 7f84f9ffb640 1 -- 172.21.15.55:0/3732673961 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850402c070 con 0x558f486ba5c0 2024-04-26T22:04:17.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.684+0000 7f84f9ffb640 1 -- 172.21.15.55:0/3732673961 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8504020030 con 0x558f486ba5c0 2024-04-26T22:04:17.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.684+0000 7f84f9ffb640 1 -- 172.21.15.55:0/3732673961 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504017030 con 0x558f486ba5c0 2024-04-26T22:04:17.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.686+0000 7f851d421740 1 -- 172.21.15.55:0/3732673961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486ba5c0 msgr2=0x558f486b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:17.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.686+0000 7f851d421740 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486ba5c0 0x558f486b7090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8504000ea0 tx=0x7f8504009be0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.686+0000 7f851d421740 1 -- 172.21.15.55:0/3732673961 shutdown_connections 2024-04-26T22:04:17.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.686+0000 7f851d421740 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486ba5c0 0x558f486b7090 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.686+0000 7f851d421740 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486b9c00 0x558f486ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.686+0000 7f851d421740 1 --2- 172.21.15.55:0/3732673961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b8a30 0x558f486c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.686+0000 7f851d421740 1 -- 172.21.15.55:0/3732673961 >> 172.21.15.55:0/3732673961 conn(0x558f485f9910 msgr2=0x558f486c1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:17.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.692+0000 7f851d421740 1 -- 172.21.15.55:0/3732673961 shutdown_connections 2024-04-26T22:04:17.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.692+0000 7f851d421740 1 -- 172.21.15.55:0/3732673961 wait complete. 2024-04-26T22:04:17.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.693+0000 7f851d421740 1 -- start start 2024-04-26T22:04:17.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.693+0000 7f851d421740 1 Processor -- start 2024-04-26T22:04:17.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.694+0000 7f851d421740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.694+0000 7f851d421740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 0x558f486c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.694+0000 7f851d421740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f487334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.695+0000 7f851d421740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f850000cd60 con 0x558f486b8a30 2024-04-26T22:04:17.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.695+0000 7f851d421740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f850000e1d0 con 0x558f486ba5c0 2024-04-26T22:04:17.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.695+0000 7f851d421740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f850000d430 con 0x558f486b9c00 2024-04-26T22:04:17.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.695+0000 7f851d421740 10 client.0 _ll_register_callbacks cb 0x7f8504009300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:17.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.695+0000 7f8518bc7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f487334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.695+0000 7f8518bc7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f487334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:48670/0 (socket says 172.21.15.55:48670) 2024-04-26T22:04:17.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.695+0000 7f8518bc7640 1 -- 172.21.15.55:0/3567353969 learned_addr learned my addr 172.21.15.55:0/3567353969 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:17.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.696+0000 7f8513fff640 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.696+0000 7f85137fe640 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 0x558f486c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.696+0000 7f8518bc7640 1 -- 172.21.15.55:0/3567353969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 msgr2=0x558f486c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:17.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.696+0000 7f8518bc7640 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 0x558f486c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.696+0000 7f8518bc7640 1 -- 172.21.15.55:0/3567353969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 msgr2=0x558f486c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:17.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.696+0000 7f8518bc7640 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.696+0000 7f8518bc7640 1 -- 172.21.15.55:0/3567353969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f487339e0 con 0x558f486ba5c0 2024-04-26T22:04:17.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.697+0000 7f8518bc7640 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f487334a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8504000c00 tx=0x7f8504012600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a6040c54104e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:17.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.698+0000 7f84f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:04:17.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.699+0000 7f84f37fe640 1 -- 172.21.15.55:0/3567353969 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850402c070 con 0x558f486ba5c0 2024-04-26T22:04:17.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.699+0000 7f851d421740 1 -- 172.21.15.55:0/3567353969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8504020030 con 0x558f486ba5c0 2024-04-26T22:04:17.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.699+0000 7f84f37fe640 1 -- 172.21.15.55:0/3567353969 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8504013680 con 0x558f486ba5c0 2024-04-26T22:04:17.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.702+0000 7f84f37fe640 1 -- 172.21.15.55:0/3567353969 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8504017050 con 0x558f486ba5c0 2024-04-26T22:04:17.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.702+0000 7f851d421740 10 client.26216 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.702+0000 7f851d421740 1 -- 172.21.15.55:0/3567353969 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558f48734ce0 con 0x558f486ba5c0 2024-04-26T22:04:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.704+0000 7f84f37fe640 1 -- 172.21.15.55:0/3567353969 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8504020030 con 0x558f486ba5c0 2024-04-26T22:04:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.705+0000 7f84f37fe640 1 -- 172.21.15.55:0/3567353969 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f850401c910 con 0x558f486ba5c0 2024-04-26T22:04:17.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.706+0000 7f851d421740 10 client.26216 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:17.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.706+0000 7f851d421740 10 client.26216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:17.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.706+0000 7f851d421740 20 client.26216 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:17.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.706+0000 7f851d421740 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558f48735590 0x558f487379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:17.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.706+0000 7f851d421740 4 client.26216 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:04:17.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.706+0000 7f851d421740 1 -- 172.21.15.55:0/3567353969 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f4859f610 con 0x558f48735590 2024-04-26T22:04:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.709+0000 7f8513fff640 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558f48735590 0x558f487379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:17.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.709+0000 7f8513fff640 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558f48735590 0x558f487379b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6ef191e7b142b3 server_cookie=21e63d329de17d1 in_seq=0 out_seq=0 2024-04-26T22:04:17.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.709+0000 7f84f37fe640 10 client.26216 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:17.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.713+0000 7f84f37fe640 1 -- 172.21.15.55:0/3567353969 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f4859f610 con 0x558f48735590 2024-04-26T22:04:17.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.714+0000 7f84f37fe640 10 client.26216 handle_command_reply: tid=0 2024-04-26T22:04:17.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.714+0000 7f851d421740 1 client.26216 shutdown 2024-04-26T22:04:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.720+0000 7f851d421740 1 -- 172.21.15.55:0/3567353969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 msgr2=0x558f487334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:17.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.720+0000 7f851d421740 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f487334a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8504000c00 tx=0x7f8504012600 comp rx=0 tx=0).stop 2024-04-26T22:04:17.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.724+0000 7f851d421740 1 -- 172.21.15.55:0/3567353969 shutdown_connections 2024-04-26T22:04:17.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.724+0000 7f851d421740 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f486ba5c0 0x558f487334a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.724+0000 7f851d421740 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f486b9c00 0x558f486c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.724+0000 7f851d421740 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558f48735590 0x558f487379b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.724+0000 7f851d421740 1 --2- 172.21.15.55:0/3567353969 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558f486b8a30 0x558f486c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.724+0000 7f851d421740 1 -- 172.21.15.55:0/3567353969 >> 172.21.15.55:0/3567353969 conn(0x558f485f9910 msgr2=0x558f486b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:17.725 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.724+0000 7f84f37fe640 0 client.26216 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:17.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.725+0000 7f851d421740 1 -- 172.21.15.55:0/3567353969 shutdown_connections 2024-04-26T22:04:17.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.726+0000 7f851d421740 1 -- 172.21.15.55:0/3567353969 wait complete. 2024-04-26T22:04:17.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.728+0000 7f851d421740 20 client.26216 trim_cache size 0 max 16384 2024-04-26T22:04:17.729 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:04:17.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.737+0000 7f85193c8640 1 -- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f84fc0a3e40 msgr2=0x7f84fc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:17.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.737+0000 7f85193c8640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f84fc0a3e40 0x7f84fc0a6300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f850c00ad30 tx=0x7f850c00a670 comp rx=0 tx=0).stop 2024-04-26T22:04:17.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.737+0000 7f85193c8640 1 -- 172.21.15.55:0/1323321707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8514109f10 msgr2=0x7f8514116540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:17.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.737+0000 7f85193c8640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8514109f10 0x7f8514116540 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f850802aa30 tx=0x7f8508030bc0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.740+0000 7f85193c8640 1 -- 172.21.15.55:0/1323321707 shutdown_connections 2024-04-26T22:04:17.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.740+0000 7f85193c8640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f84fc0a3e40 0x7f84fc0a6300 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.740+0000 7f85193c8640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f851410a8d0 0x7f85141b7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.740+0000 7f85193c8640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8514109f10 0x7f8514116540 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.740+0000 7f85193c8640 1 --2- 172.21.15.55:0/1323321707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8514073c10 0x7f8514116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:17.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.740+0000 7f85193c8640 1 -- 172.21.15.55:0/1323321707 >> 172.21.15.55:0/1323321707 conn(0x7f851406f780 msgr2=0x7f85140713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.742+0000 7f85193c8640 1 -- 172.21.15.55:0/1323321707 shutdown_connections 2024-04-26T22:04:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:17.743+0000 7f85193c8640 1 -- 172.21.15.55:0/1323321707 wait complete. 2024-04-26T22:04:18.781 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:04:18.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:04:18.781 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:04:19.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.263+0000 7f2e661af640 1 -- 172.21.15.55:0/4021096766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e60109930 msgr2=0x7f2e6010bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:19.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.263+0000 7f2e661af640 1 --2- 172.21.15.55:0/4021096766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e60109930 0x7f2e6010bd60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009860 tx=0x7f2e5402a380 comp rx=0 tx=0).stop 2024-04-26T22:04:19.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.269+0000 7f2e661af640 1 -- 172.21.15.55:0/4021096766 shutdown_connections 2024-04-26T22:04:19.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.269+0000 7f2e661af640 1 --2- 172.21.15.55:0/4021096766 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6010ec70 0x7f2e6010f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.269+0000 7f2e661af640 1 --2- 172.21.15.55:0/4021096766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6010c2a0 0x7f2e6010e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.269+0000 7f2e661af640 1 --2- 172.21.15.55:0/4021096766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e60109930 0x7f2e6010bd60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.269+0000 7f2e661af640 1 -- 172.21.15.55:0/4021096766 >> 172.21.15.55:0/4021096766 conn(0x7f2e6006f7c0 msgr2=0x7f2e60071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:19.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.274+0000 7f2e661af640 1 -- 172.21.15.55:0/4021096766 shutdown_connections 2024-04-26T22:04:19.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.277+0000 7f2e661af640 1 -- 172.21.15.55:0/4021096766 wait complete. 2024-04-26T22:04:19.280 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.278+0000 7f2e661af640 1 Processor -- start 2024-04-26T22:04:19.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.282+0000 7f2e661af640 1 -- start start 2024-04-26T22:04:19.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.282+0000 7f2e661af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e60109930 0x7f2e6019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.282+0000 7f2e661af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6010c2a0 0x7f2e6019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.282+0000 7f2e661af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6010ec70 0x7f2e601a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.282+0000 7f2e661af640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60112af0 con 0x7f2e6010c2a0 2024-04-26T22:04:19.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.282+0000 7f2e661af640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60112970 con 0x7f2e60109930 2024-04-26T22:04:19.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.282+0000 7f2e661af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60112c70 con 0x7f2e6010ec70 2024-04-26T22:04:19.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.286+0000 7f2e5f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e60109930 0x7f2e6019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.286+0000 7f2e5effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6010c2a0 0x7f2e6019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.286+0000 7f2e5effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6010c2a0 0x7f2e6019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49638/0 (socket says 172.21.15.55:49638) 2024-04-26T22:04:19.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.286+0000 7f2e5effd640 1 -- 172.21.15.55:0/2494514601 learned_addr learned my addr 172.21.15.55:0/2494514601 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:19.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.286+0000 7f2e5ffff640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6010ec70 0x7f2e601a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.293+0000 7f2e5effd640 1 -- 172.21.15.55:0/2494514601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6010ec70 msgr2=0x7f2e601a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:19.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.293+0000 7f2e5effd640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6010ec70 0x7f2e601a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.293+0000 7f2e5effd640 1 -- 172.21.15.55:0/2494514601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e60109930 msgr2=0x7f2e6019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:19.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.293+0000 7f2e5effd640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e60109930 0x7f2e6019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.293+0000 7f2e5effd640 1 -- 172.21.15.55:0/2494514601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e601a6e30 con 0x7f2e6010c2a0 2024-04-26T22:04:19.302 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.300+0000 7f2e5effd640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6010c2a0 0x7f2e6019f750 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c970 tx=0x7f2e5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:19.306 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.306+0000 7f2e5cff9640 1 -- 172.21.15.55:0/2494514601 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50004220 con 0x7f2e6010c2a0 2024-04-26T22:04:19.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.306+0000 7f2e661af640 1 -- 172.21.15.55:0/2494514601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e601a7120 con 0x7f2e6010c2a0 2024-04-26T22:04:19.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.307+0000 7f2e661af640 1 -- 172.21.15.55:0/2494514601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e601a7670 con 0x7f2e6010c2a0 2024-04-26T22:04:19.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.307+0000 7f2e5cff9640 1 -- 172.21.15.55:0/2494514601 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e500043c0 con 0x7f2e6010c2a0 2024-04-26T22:04:19.307 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.307+0000 7f2e5cff9640 1 -- 172.21.15.55:0/2494514601 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5000f450 con 0x7f2e6010c2a0 2024-04-26T22:04:19.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.316+0000 7f2e5cff9640 1 -- 172.21.15.55:0/2494514601 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2e5000f5f0 con 0x7f2e6010c2a0 2024-04-26T22:04:19.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.316+0000 7f2e68fa5740 1 Processor -- start 2024-04-26T22:04:19.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.316+0000 7f2e68fa5740 1 -- start start 2024-04-26T22:04:19.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.317+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.317+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f32a0 0x563c776f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.317 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.317+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f8410 0x563c776fa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.317+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563c77657b00 con 0x563c776f28b0 2024-04-26T22:04:19.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.317+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563c775f3920 con 0x563c776f8410 2024-04-26T22:04:19.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.317+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563c77653ac0 con 0x563c776f32a0 2024-04-26T22:04:19.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.319+0000 7f2e5cff9640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2e480a3e40 0x7f2e480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.320+0000 7f2e5cff9640 1 -- 172.21.15.55:0/2494514601 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2e500d10e0 con 0x7f2e6010c2a0 2024-04-26T22:04:19.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.320+0000 7f2e5f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f32a0 0x563c776f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.321+0000 7f2e5ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.321+0000 7f2e5ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49644/0 (socket says 172.21.15.55:49644) 2024-04-26T22:04:19.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.321+0000 7f2e5ffff640 1 -- 172.21.15.55:0/4188820143 learned_addr learned my addr 172.21.15.55:0/4188820143 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:19.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.321+0000 7f2e5effd640 1 --2- 172.21.15.55:0/4188820143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f8410 0x563c776fa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.324+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/4188820143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f8410 msgr2=0x563c776fa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:19.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.324+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/4188820143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f8410 0x563c776fa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.324+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/4188820143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 msgr2=0x563c776f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:19.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.324+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/4188820143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.324+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/4188820143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c774317e0 con 0x563c776f32a0 2024-04-26T22:04:19.327 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.327+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/4188820143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f32a0 0x563c776f7da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e540303b0 tx=0x7f2e54009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8536a81779fd832 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:19.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.327+0000 7f2e5ffff640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2e480a3e40 0x7f2e480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.334+0000 7f2e45ffb640 1 -- 172.21.15.55:0/4188820143 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54043070 con 0x563c776f32a0 2024-04-26T22:04:19.335 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.334+0000 7f2e45ffb640 1 -- 172.21.15.55:0/4188820143 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e54038040 con 0x563c776f32a0 2024-04-26T22:04:19.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.336+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4188820143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f32a0 msgr2=0x563c776f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:19.337 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.336+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4188820143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f32a0 0x563c776f7da0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e540303b0 tx=0x7f2e54009560 comp rx=0 tx=0).stop 2024-04-26T22:04:19.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.344+0000 7f2e5ffff640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2e480a3e40 0x7f2e480a6300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x563c776f3b00 tx=0x7f2e5801a5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:19.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.344+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4188820143 shutdown_connections 2024-04-26T22:04:19.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.344+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4188820143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f8410 0x563c776fa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.344+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4188820143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f32a0 0x563c776f7da0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.344+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4188820143 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.345 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.344+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4188820143 >> 172.21.15.55:0/4188820143 conn(0x563c776edf20 msgr2=0x563c776ee320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:19.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.344+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4188820143 shutdown_connections 2024-04-26T22:04:19.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.346+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4188820143 wait complete. 2024-04-26T22:04:19.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.347+0000 7f2e68fa5740 1 -- start start 2024-04-26T22:04:19.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.347+0000 7f2e68fa5740 1 Processor -- start 2024-04-26T22:04:19.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.352+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.352+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 0x563c776fd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.352+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c77704210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.352+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563c776fb3b0 con 0x563c776f28b0 2024-04-26T22:04:19.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.352+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563c775f15d0 con 0x563c776f32a0 2024-04-26T22:04:19.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.352+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563c776fb530 con 0x563c776f8410 2024-04-26T22:04:19.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.352+0000 7f2e68fa5740 10 client.0 _ll_register_callbacks cb 0x563c775df080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:19.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.355+0000 7f2e5ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c77704210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.356 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.355+0000 7f2e5effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 0x563c776fd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.359+0000 7f2e5f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.359+0000 7f2e5f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776fcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49648/0 (socket says 172.21.15.55:49648) 2024-04-26T22:04:19.361 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.359+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/3653505964 learned_addr learned my addr 172.21.15.55:0/3653505964 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:19.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.364+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/3653505964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 msgr2=0x563c77704210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:19.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.364+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c77704210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.364+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/3653505964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 msgr2=0x563c776fd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:19.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.364+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 0x563c776fd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.365 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.364+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/3653505964 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c77491f70 con 0x563c776f28b0 2024-04-26T22:04:19.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.366+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776fcc90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2e5402a970 tx=0x7f2e54030fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310fc1263e653e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:19.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.366+0000 7f2e3effd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:19.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.368+0000 7f2e3effd640 1 -- 172.21.15.55:0/3653505964 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5404c070 con 0x563c776f28b0 2024-04-26T22:04:19.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.368+0000 7f2e68fa5740 1 -- 172.21.15.55:0/3653505964 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e54038040 con 0x563c776f28b0 2024-04-26T22:04:19.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.368+0000 7f2e3effd640 1 -- 172.21.15.55:0/3653505964 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2e54030780 con 0x563c776f28b0 2024-04-26T22:04:19.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.369+0000 7f2e3effd640 1 -- 172.21.15.55:0/3653505964 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2e54058050 con 0x563c776f28b0 2024-04-26T22:04:19.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.369+0000 7f2e68fa5740 10 client.17241 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:19.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.369+0000 7f2e68fa5740 1 -- 172.21.15.55:0/3653505964 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563c77329170 con 0x563c776f28b0 2024-04-26T22:04:19.373 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.372+0000 7f2e3effd640 1 -- 172.21.15.55:0/3653505964 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e54038040 con 0x563c776f28b0 2024-04-26T22:04:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.374+0000 7f2e3effd640 1 -- 172.21.15.55:0/3653505964 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2e540487d0 con 0x563c776f28b0 2024-04-26T22:04:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.374+0000 7f2e68fa5740 10 client.17241 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.374+0000 7f2e68fa5740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.374+0000 7f2e68fa5740 20 client.17241 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.374+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563c77708830 0x563c7770ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.374+0000 7f2e68fa5740 4 client.17241 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.374+0000 7f2e68fa5740 1 -- 172.21.15.55:0/3653505964 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c77665710 con 0x563c77708830 2024-04-26T22:04:19.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.377+0000 7f2e5effd640 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563c77708830 0x563c7770ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.387+0000 7f2e5effd640 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563c77708830 0x563c7770ac50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e510b3424292542d server_cookie=249f0d73f9dce865 in_seq=0 out_seq=0 2024-04-26T22:04:19.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.387+0000 7f2e3effd640 10 client.17241 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:19.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.392+0000 7f2e3effd640 1 -- 172.21.15.55:0/3653505964 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c77665710 con 0x563c77708830 2024-04-26T22:04:19.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.392+0000 7f2e3effd640 10 client.17241 handle_command_reply: tid=0 2024-04-26T22:04:19.396 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.392+0000 7f2e68fa5740 1 client.17241 shutdown 2024-04-26T22:04:19.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.395+0000 7f2e68fa5740 1 -- 172.21.15.55:0/3653505964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 msgr2=0x563c776fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:19.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.395+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776fcc90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2e5402a970 tx=0x7f2e54030fb0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.397 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.396+0000 7f2e68fa5740 1 -- 172.21.15.55:0/3653505964 shutdown_connections 2024-04-26T22:04:19.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.396+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c77704210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.396+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 0x563c776fd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.396+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563c77708830 0x563c7770ac50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.396+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/3653505964 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776fcc90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.396+0000 7f2e3effd640 0 client.17241 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:19.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.396+0000 7f2e68fa5740 1 -- 172.21.15.55:0/3653505964 >> 172.21.15.55:0/3653505964 conn(0x563c776edf20 msgr2=0x563c776ed0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:19.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.403+0000 7f2e68fa5740 1 -- 172.21.15.55:0/3653505964 shutdown_connections 2024-04-26T22:04:19.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.403+0000 7f2e68fa5740 1 -- 172.21.15.55:0/3653505964 wait complete. 2024-04-26T22:04:19.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.403+0000 7f2e68fa5740 20 client.17241 trim_cache size 0 max 16384 2024-04-26T22:04:19.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:19 smithi055 ceph-mon[29221]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 926 MiB/s wr, 1.58k op/s 2024-04-26T22:04:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.490+0000 7f2e68fa5740 1 Processor -- start 2024-04-26T22:04:19.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.494+0000 7f2e68fa5740 1 -- start start 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.495+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.495+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 0x563c777041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.495+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c776f1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.495+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c010e40 con 0x563c776f28b0 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.495+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c00e710 con 0x563c776f32a0 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.495+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c00c610 con 0x563c776f8410 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.496+0000 7f2e5f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.496+0000 7f2e5f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49662/0 (socket says 172.21.15.55:49662) 2024-04-26T22:04:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.496+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/1199337033 learned_addr learned my addr 172.21.15.55:0/1199337033 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.497+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/1199337033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 msgr2=0x563c776f1850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.497+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/1199337033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c776f1850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.497+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/1199337033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 msgr2=0x563c777041d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.497+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/1199337033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 0x563c777041d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.497+0000 7f2e5f7fe640 1 -- 172.21.15.55:0/1199337033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c77653a50 con 0x563c776f28b0 2024-04-26T22:04:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.498+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/1199337033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f1310 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2e540303b0 tx=0x7f2e540096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0fe878cc90915e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:19.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.500+0000 7f2e457fa640 1 -- 172.21.15.55:0/1199337033 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54055020 con 0x563c776f28b0 2024-04-26T22:04:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.500+0000 7f2e457fa640 1 -- 172.21.15.55:0/1199337033 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e54030780 con 0x563c776f28b0 2024-04-26T22:04:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.500+0000 7f2e457fa640 1 -- 172.21.15.55:0/1199337033 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5404c040 con 0x563c776f28b0 2024-04-26T22:04:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.501+0000 7f2e68fa5740 1 -- 172.21.15.55:0/1199337033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 msgr2=0x563c776f1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.501+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/1199337033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f1310 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2e540303b0 tx=0x7f2e540096b0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:19 smithi064 ceph-mon[32765]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 926 MiB/s wr, 1.58k op/s 2024-04-26T22:04:19.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.509+0000 7f2e68fa5740 1 -- 172.21.15.55:0/1199337033 shutdown_connections 2024-04-26T22:04:19.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.509+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/1199337033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c776f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.509+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/1199337033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f32a0 0x563c777041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.509+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/1199337033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f28b0 0x563c776f1310 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.509+0000 7f2e68fa5740 1 -- 172.21.15.55:0/1199337033 >> 172.21.15.55:0/1199337033 conn(0x563c776edf20 msgr2=0x563c776f0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:19.525 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.524+0000 7f2e68fa5740 1 -- 172.21.15.55:0/1199337033 shutdown_connections 2024-04-26T22:04:19.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.528+0000 7f2e68fa5740 1 -- 172.21.15.55:0/1199337033 wait complete. 2024-04-26T22:04:19.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.528+0000 7f2e68fa5740 1 -- start start 2024-04-26T22:04:19.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.529+0000 7f2e68fa5740 1 Processor -- start 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.530+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f28b0 0x563c77701a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.530+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f32a0 0x563c77701fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.530+0000 7f2e68fa5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c77702fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.530+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c00cbd0 con 0x563c776f32a0 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.530+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c00e040 con 0x563c776f28b0 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.530+0000 7f2e68fa5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c00d2a0 con 0x563c776f8410 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.530+0000 7f2e68fa5740 10 client.0 _ll_register_callbacks cb 0x7f2e54005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.530+0000 7f2e5effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f32a0 0x563c77701fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.531+0000 7f2e5effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f32a0 0x563c77701fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49668/0 (socket says 172.21.15.55:49668) 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.531+0000 7f2e5effd640 1 -- 172.21.15.55:0/4146798149 learned_addr learned my addr 172.21.15.55:0/4146798149 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.531+0000 7f2e5effd640 1 -- 172.21.15.55:0/4146798149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 msgr2=0x563c77702fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.531+0000 7f2e5effd640 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c77702fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.531+0000 7f2e5effd640 1 -- 172.21.15.55:0/4146798149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f28b0 msgr2=0x563c77701a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:19.534 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.531+0000 7f2e5effd640 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f28b0 0x563c77701a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:19.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.531+0000 7f2e5effd640 1 -- 172.21.15.55:0/4146798149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c777035a0 con 0x563c776f32a0 2024-04-26T22:04:19.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.531+0000 7f2e5effd640 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f32a0 0x563c77701fa0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2e500996d0 tx=0x7f2e500cef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258bde616d3ad67f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:19.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.532+0000 7f2e3effd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:19.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.534+0000 7f2e3effd640 1 -- 172.21.15.55:0/4146798149 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5001d040 con 0x563c776f32a0 2024-04-26T22:04:19.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.534+0000 7f2e3effd640 1 -- 172.21.15.55:0/4146798149 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2e500998e0 con 0x563c776f32a0 2024-04-26T22:04:19.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.534+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4146798149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e54030780 con 0x563c776f32a0 2024-04-26T22:04:19.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.536+0000 7f2e3effd640 1 -- 172.21.15.55:0/4146798149 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2e500a1150 con 0x563c776f32a0 2024-04-26T22:04:19.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.536+0000 7f2e68fa5740 10 client.17247 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:19.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.536+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4146798149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563c77791000 con 0x563c776f32a0 2024-04-26T22:04:19.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.537+0000 7f2e3effd640 1 -- 172.21.15.55:0/4146798149 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e54030780 con 0x563c776f32a0 2024-04-26T22:04:19.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.537+0000 7f2e3effd640 1 -- 172.21.15.55:0/4146798149 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2e50098fc0 con 0x563c776f32a0 2024-04-26T22:04:19.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.537+0000 7f2e68fa5740 10 client.17247 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:19.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.537+0000 7f2e68fa5740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:19.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.537+0000 7f2e68fa5740 20 client.17247 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:19.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.538+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563c77791a30 0x563c77793e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:19.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.538+0000 7f2e68fa5740 4 client.17247 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:04:19.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.538+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4146798149 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c7765f580 con 0x563c77791a30 2024-04-26T22:04:19.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.540+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563c77791a30 0x563c77793e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:19.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.542+0000 7f2e5f7fe640 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563c77791a30 0x563c77793e50 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bfd495c73cf1d8 server_cookie=422bdb1b91cdb2ec in_seq=0 out_seq=0 2024-04-26T22:04:19.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:19.542+0000 7f2e3effd640 10 client.17247 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:19.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:19 smithi165 ceph-mon[33296]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 926 MiB/s wr, 1.58k op/s 2024-04-26T22:04:20.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.492+0000 7f2e3effd640 1 -- 172.21.15.55:0/4146798149 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c7765f580 con 0x563c77791a30 2024-04-26T22:04:20.492 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.492+0000 7f2e3effd640 10 client.17247 handle_command_reply: tid=0 2024-04-26T22:04:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.492+0000 7f2e68fa5740 1 client.17247 shutdown 2024-04-26T22:04:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.492+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4146798149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f32a0 msgr2=0x563c77701fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.492+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f32a0 0x563c77701fa0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2e500996d0 tx=0x7f2e500cef20 comp rx=0 tx=0).stop 2024-04-26T22:04:20.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.494+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4146798149 shutdown_connections 2024-04-26T22:04:20.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.494+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c776f8410 0x563c77702fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.494+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x563c77791a30 0x563c77793e50 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.494+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563c776f32a0 0x563c77701fa0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.494+0000 7f2e68fa5740 1 --2- 172.21.15.55:0/4146798149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c776f28b0 0x563c77701a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.494+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4146798149 >> 172.21.15.55:0/4146798149 conn(0x563c776edf20 msgr2=0x563c776f1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.494+0000 7f2e3effd640 0 client.17247 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.495+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4146798149 shutdown_connections 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.495+0000 7f2e68fa5740 1 -- 172.21.15.55:0/4146798149 wait complete. 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.495+0000 7f2e68fa5740 20 client.17247 trim_cache size 0 max 16384 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:04:20.499 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "371918ca-63e3-45ed-805c-e29e707d8118", 2024-04-26T22:04:20.500 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:04:20.500 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 -- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2e480a3e40 msgr2=0x7f2e480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2e480a3e40 0x7f2e480a6300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x563c776f3b00 tx=0x7f2e5801a5f0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 -- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6010c2a0 msgr2=0x7f2e6019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6010c2a0 0x7f2e6019f750 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c970 tx=0x7f2e5000ce30 comp rx=0 tx=0).stop 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 -- 172.21.15.55:0/2494514601 shutdown_connections 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2e480a3e40 0x7f2e480a6300 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e6010ec70 0x7f2e601a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e6010c2a0 0x7f2e6019f750 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 --2- 172.21.15.55:0/2494514601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e60109930 0x7f2e6019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.504 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.502+0000 7f2e661af640 1 -- 172.21.15.55:0/2494514601 >> 172.21.15.55:0/2494514601 conn(0x7f2e6006f7c0 msgr2=0x7f2e6010a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.503+0000 7f2e661af640 1 -- 172.21.15.55:0/2494514601 shutdown_connections 2024-04-26T22:04:20.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.503+0000 7f2e661af640 1 -- 172.21.15.55:0/2494514601 wait complete. 2024-04-26T22:04:20.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:04:20.555 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:04:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 -- 172.21.15.55:0/3229343262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 msgr2=0x7f44a0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 --2- 172.21.15.55:0/3229343262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 0x7f44a0101f30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4494009900 tx=0x7f449402a400 comp rx=0 tx=0).stop 2024-04-26T22:04:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 -- 172.21.15.55:0/3229343262 shutdown_connections 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 --2- 172.21.15.55:0/3229343262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44a0102470 0x7f44a0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 --2- 172.21.15.55:0/3229343262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 0x7f44a0101f30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 --2- 172.21.15.55:0/3229343262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44a0102ef0 0x7f44a0103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 -- 172.21.15.55:0/3229343262 >> 172.21.15.55:0/3229343262 conn(0x7f44a00fc560 msgr2=0x7f44a00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 -- 172.21.15.55:0/3229343262 shutdown_connections 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.808+0000 7f44a62ea640 1 -- 172.21.15.55:0/3229343262 wait complete. 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.809+0000 7f44a62ea640 1 Processor -- start 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.809+0000 7f44a62ea640 1 -- start start 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.809+0000 7f44a62ea640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 0x7f44a0198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.809+0000 7f44a62ea640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44a0102470 0x7f44a0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.809+0000 7f44a62ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44a0102ef0 0x7f44a01a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f44a62ea640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d490 con 0x7f44a0101ab0 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f44a62ea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d310 con 0x7f44a0102ef0 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f44a62ea640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d610 con 0x7f44a0102470 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 0x7f44a0198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 0x7f44a0198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35064/0 (socket says 172.21.15.55:35064) 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 -- 172.21.15.55:0/871484055 learned_addr learned my addr 172.21.15.55:0/871484055 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449f7fe640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44a0102470 0x7f44a0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 -- 172.21.15.55:0/871484055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44a0102470 msgr2=0x7f44a0199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44a0102470 0x7f44a0199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 -- 172.21.15.55:0/871484055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44a0102ef0 msgr2=0x7f44a01a0380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44a0102ef0 0x7f44a01a0380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 -- 172.21.15.55:0/871484055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a01a09f0 con 0x7f44a0101ab0 2024-04-26T22:04:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.810+0000 7f449ffff640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 0x7f44a0198dd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f449000b4a0 tx=0x7f449000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.811+0000 7f449d7fa640 1 -- 172.21.15.55:0/871484055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490014070 con 0x7f44a0101ab0 2024-04-26T22:04:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.811+0000 7f449d7fa640 1 -- 172.21.15.55:0/871484055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4490010ac0 con 0x7f44a0101ab0 2024-04-26T22:04:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.811+0000 7f449d7fa640 1 -- 172.21.15.55:0/871484055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490017c30 con 0x7f44a0101ab0 2024-04-26T22:04:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.811+0000 7f44a62ea640 1 -- 172.21.15.55:0/871484055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a01a0ce0 con 0x7f44a0101ab0 2024-04-26T22:04:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.811+0000 7f44a62ea640 1 -- 172.21.15.55:0/871484055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a0103f40 con 0x7f44a0101ab0 2024-04-26T22:04:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.814+0000 7f449d7fa640 1 -- 172.21.15.55:0/871484055 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4490017070 con 0x7f44a0101ab0 2024-04-26T22:04:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.815+0000 7f449d7fa640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f44880a3cc0 0x7f44880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.816+0000 7f449f7fe640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f44880a3cc0 0x7f44880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.816+0000 7f44a9004740 1 Processor -- start 2024-04-26T22:04:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.816+0000 7f449d7fa640 1 -- 172.21.15.55:0/871484055 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f44900cfbe0 con 0x7f44a0101ab0 2024-04-26T22:04:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.817+0000 7f44a9004740 1 -- start start 2024-04-26T22:04:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.817+0000 7f449f7fe640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f44880a3cc0 0x7f44880a6180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4494000c00 tx=0x7f44940314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.817+0000 7f44a9004740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5be8ee0 0x5615e5be9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.817+0000 7f44a9004740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5bea0b0 0x5615e5bea530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.817+0000 7f44a9004740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5beaa70 0x5615e5bf1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.818+0000 7f44a4887640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5beaa70 0x5615e5bf1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.818+0000 7f44a4887640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5beaa70 0x5615e5bf1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35078/0 (socket says 172.21.15.55:35078) 2024-04-26T22:04:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.818+0000 7f44a4887640 1 -- 172.21.15.55:0/3416930440 learned_addr learned my addr 172.21.15.55:0/3416930440 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.818+0000 7f44a9004740 1 -- 172.21.15.55:0/3416930440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5615e5b4dd80 con 0x5615e5beaa70 2024-04-26T22:04:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.818+0000 7f44a9004740 1 -- 172.21.15.55:0/3416930440 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5615e5b078b0 con 0x5615e5be8ee0 2024-04-26T22:04:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.818+0000 7f44a9004740 1 -- 172.21.15.55:0/3416930440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5615e5b49d40 con 0x5615e5bea0b0 2024-04-26T22:04:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.818+0000 7f449f7fe640 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5bea0b0 0x5615e5bea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.819+0000 7f449f7fe640 1 -- 172.21.15.55:0/3416930440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5be8ee0 msgr2=0x5615e5be9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:20.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.819+0000 7f449f7fe640 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5be8ee0 0x5615e5be9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.819+0000 7f449f7fe640 1 -- 172.21.15.55:0/3416930440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5beaa70 msgr2=0x5615e5bf1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:20.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.819+0000 7f449f7fe640 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5beaa70 0x5615e5bf1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.819+0000 7f449f7fe640 1 -- 172.21.15.55:0/3416930440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e5ad8a40 con 0x5615e5bea0b0 2024-04-26T22:04:20.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.819+0000 7f44a4887640 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5beaa70 0x5615e5bf1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:20.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.819+0000 7f449f7fe640 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5bea0b0 0x5615e5bea530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4494006010 tx=0x7f449403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b4f79c0306c91f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:20.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.820+0000 7f44867fc640 1 -- 172.21.15.55:0/3416930440 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940035f0 con 0x5615e5bea0b0 2024-04-26T22:04:20.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.820+0000 7f44867fc640 1 -- 172.21.15.55:0/3416930440 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449404e070 con 0x5615e5bea0b0 2024-04-26T22:04:20.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.820+0000 7f44867fc640 1 -- 172.21.15.55:0/3416930440 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494049b60 con 0x5615e5bea0b0 2024-04-26T22:04:20.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.822+0000 7f44a9004740 1 -- 172.21.15.55:0/3416930440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5bea0b0 msgr2=0x5615e5bea530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.822+0000 7f44a9004740 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5bea0b0 0x5615e5bea530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4494006010 tx=0x7f449403d000 comp rx=0 tx=0).stop 2024-04-26T22:04:20.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.822+0000 7f44a9004740 1 -- 172.21.15.55:0/3416930440 shutdown_connections 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.822+0000 7f44a9004740 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5beaa70 0x5615e5bf1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.822+0000 7f44a9004740 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5bea0b0 0x5615e5bea530 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.823+0000 7f44a9004740 1 --2- 172.21.15.55:0/3416930440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5be8ee0 0x5615e5be9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.823+0000 7f44a9004740 1 -- 172.21.15.55:0/3416930440 >> 172.21.15.55:0/3416930440 conn(0x5615e5be43d0 msgr2=0x5615e5be47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.823+0000 7f44a9004740 1 -- 172.21.15.55:0/3416930440 shutdown_connections 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.823+0000 7f44a9004740 1 -- 172.21.15.55:0/3416930440 wait complete. 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.824+0000 7f44a9004740 1 -- start start 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.824+0000 7f44a9004740 1 Processor -- start 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f44a9004740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f44a9004740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 0x5615e5bf3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f44a9004740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5bfa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f44a9004740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5615e5bf1a20 con 0x5615e5be8ee0 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f449ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f44a9004740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5615e5ad4d10 con 0x5615e5bea0b0 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f44a9004740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5615e5bf1ba0 con 0x5615e5beaa70 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f449ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35094/0 (socket says 172.21.15.55:35094) 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f449ffff640 1 -- 172.21.15.55:0/1647980434 learned_addr learned my addr 172.21.15.55:0/1647980434 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.825+0000 7f44a9004740 10 client.0 _ll_register_callbacks cb 0x5615e5b01210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.826+0000 7f449ffff640 1 -- 172.21.15.55:0/1647980434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 msgr2=0x5615e5bfa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.826+0000 7f449ffff640 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5bfa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.826+0000 7f449ffff640 1 -- 172.21.15.55:0/1647980434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 msgr2=0x5615e5bf3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.826+0000 7f449ffff640 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 0x5615e5bf3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.826+0000 7f449ffff640 1 -- 172.21.15.55:0/1647980434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e59297d0 con 0x5615e5be8ee0 2024-04-26T22:04:20.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.826+0000 7f449ffff640 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3220 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f449009bdc0 tx=0x7f4490017360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857b9e68d7cd982b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:20.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.826+0000 7f447f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:20.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.827+0000 7f447f7fe640 1 -- 172.21.15.55:0/1647980434 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449001e070 con 0x5615e5be8ee0 2024-04-26T22:04:20.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.827+0000 7f447f7fe640 1 -- 172.21.15.55:0/1647980434 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4490010c70 con 0x5615e5be8ee0 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.827+0000 7f44a9004740 1 -- 172.21.15.55:0/1647980434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449404e070 con 0x5615e5be8ee0 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.827+0000 7f447f7fe640 1 -- 172.21.15.55:0/1647980434 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f449009be30 con 0x5615e5be8ee0 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.827+0000 7f44a9004740 10 client.17259 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.827+0000 7f44a9004740 1 -- 172.21.15.55:0/1647980434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5615e585f3f0 con 0x5615e5be8ee0 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.828+0000 7f447f7fe640 1 -- 172.21.15.55:0/1647980434 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f449404e070 con 0x5615e5be8ee0 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.828+0000 7f447f7fe640 1 -- 172.21.15.55:0/1647980434 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f44900a4610 con 0x5615e5be8ee0 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.828+0000 7f44a9004740 10 client.17259 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.828+0000 7f44a9004740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:20.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.828+0000 7f44a9004740 20 client.17259 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:20.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.828+0000 7f44a9004740 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5615e5bfecd0 0x5615e5c01190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.828+0000 7f44a9004740 4 client.17259 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:20.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.828+0000 7f44a9004740 1 -- 172.21.15.55:0/1647980434 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615e5b5b990 con 0x5615e5bfecd0 2024-04-26T22:04:20.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.829+0000 7f449f7fe640 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5615e5bfecd0 0x5615e5c01190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.834+0000 7f449f7fe640 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5615e5bfecd0 0x5615e5c01190 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3a3ceba4026fd76 server_cookie=327ae664fb6cff46 in_seq=0 out_seq=0 2024-04-26T22:04:20.834 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.834+0000 7f447f7fe640 10 client.17259 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:20.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.837+0000 7f447f7fe640 1 -- 172.21.15.55:0/1647980434 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615e5b5b990 con 0x5615e5bfecd0 2024-04-26T22:04:20.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.837+0000 7f447f7fe640 10 client.17259 handle_command_reply: tid=0 2024-04-26T22:04:20.838 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.837+0000 7f44a9004740 1 client.17259 shutdown 2024-04-26T22:04:20.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.839+0000 7f44a9004740 1 -- 172.21.15.55:0/1647980434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 msgr2=0x5615e5bf3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.839+0000 7f44a9004740 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3220 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f449009bdc0 tx=0x7f4490017360 comp rx=0 tx=0).stop 2024-04-26T22:04:20.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.839+0000 7f44a9004740 1 -- 172.21.15.55:0/1647980434 shutdown_connections 2024-04-26T22:04:20.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.839+0000 7f44a9004740 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5bfa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.840+0000 7f44a9004740 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 0x5615e5bf3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.840+0000 7f44a9004740 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5615e5bfecd0 0x5615e5c01190 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.840+0000 7f44a9004740 1 --2- 172.21.15.55:0/1647980434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3220 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.841 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.840+0000 7f44a9004740 1 -- 172.21.15.55:0/1647980434 >> 172.21.15.55:0/1647980434 conn(0x5615e5be43d0 msgr2=0x5615e5be3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.840+0000 7f447f7fe640 0 client.17259 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:20.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.840+0000 7f44a9004740 1 -- 172.21.15.55:0/1647980434 shutdown_connections 2024-04-26T22:04:20.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.840+0000 7f44a9004740 1 -- 172.21.15.55:0/1647980434 wait complete. 2024-04-26T22:04:20.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.840+0000 7f44a9004740 20 client.17259 trim_cache size 0 max 16384 2024-04-26T22:04:20.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.877+0000 7f44a9004740 1 Processor -- start 2024-04-26T22:04:20.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.877+0000 7f44a9004740 1 -- start start 2024-04-26T22:04:20.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.877+0000 7f44a9004740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.877+0000 7f44a9004740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 0x5615e5bfa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.877+0000 7f44a9004740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5be72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.877+0000 7f44a9004740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01aeba0 con 0x5615e5be8ee0 2024-04-26T22:04:20.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.877+0000 7f44a9004740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01ac470 con 0x5615e5bea0b0 2024-04-26T22:04:20.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.877+0000 7f44a9004740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01aa2c0 con 0x5615e5beaa70 2024-04-26T22:04:20.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35108/0 (socket says 172.21.15.55:35108) 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449ffff640 1 -- 172.21.15.55:0/483626602 learned_addr learned my addr 172.21.15.55:0/483626602 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f44a4887640 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5be72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449f7fe640 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 0x5615e5bfa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449ffff640 1 -- 172.21.15.55:0/483626602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 msgr2=0x5615e5be72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449ffff640 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5be72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449ffff640 1 -- 172.21.15.55:0/483626602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 msgr2=0x5615e5bfa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449ffff640 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 0x5615e5bfa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.878+0000 7f449ffff640 1 -- 172.21.15.55:0/483626602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e581f0f0 con 0x5615e5be8ee0 2024-04-26T22:04:20.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.879+0000 7f449ffff640 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3540 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f449009bb30 tx=0x7f449009a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13decf2ec2aa0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:20.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.879+0000 7f4485ffb640 1 -- 172.21.15.55:0/483626602 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449001e070 con 0x5615e5be8ee0 2024-04-26T22:04:20.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.879+0000 7f4485ffb640 1 -- 172.21.15.55:0/483626602 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44900a4aa0 con 0x5615e5be8ee0 2024-04-26T22:04:20.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.880+0000 7f4485ffb640 1 -- 172.21.15.55:0/483626602 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44900a4790 con 0x5615e5be8ee0 2024-04-26T22:04:20.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 -- 172.21.15.55:0/483626602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 msgr2=0x5615e5bf3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3540 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f449009bb30 tx=0x7f449009a340 comp rx=0 tx=0).stop 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 -- 172.21.15.55:0/483626602 shutdown_connections 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5be72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5bea0b0 0x5615e5bfa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 --2- 172.21.15.55:0/483626602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5be8ee0 0x5615e5bf3540 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 -- 172.21.15.55:0/483626602 >> 172.21.15.55:0/483626602 conn(0x5615e5be43d0 msgr2=0x5615e5bf2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 -- 172.21.15.55:0/483626602 shutdown_connections 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 -- 172.21.15.55:0/483626602 wait complete. 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.881+0000 7f44a9004740 1 -- start start 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.882+0000 7f44a9004740 1 Processor -- start 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.882+0000 7f44a9004740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5be8ee0 0x5615e5bf5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.882+0000 7f44a9004740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5bea0b0 0x5615e5bf5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.882+0000 7f44a9004740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5bf8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.882+0000 7f44a9004740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01aa930 con 0x5615e5bea0b0 2024-04-26T22:04:20.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.882+0000 7f44a9004740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01abda0 con 0x5615e5be8ee0 2024-04-26T22:04:20.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.882+0000 7f44a9004740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01ab000 con 0x5615e5beaa70 2024-04-26T22:04:20.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.882+0000 7f44a9004740 10 client.0 _ll_register_callbacks cb 0x7f44900092d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:20.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.883+0000 7f449f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5bea0b0 0x5615e5bf5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.883+0000 7f449f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5bea0b0 0x5615e5bf5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35122/0 (socket says 172.21.15.55:35122) 2024-04-26T22:04:20.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.883+0000 7f449f7fe640 1 -- 172.21.15.55:0/1820689453 learned_addr learned my addr 172.21.15.55:0/1820689453 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:20.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.884+0000 7f449f7fe640 1 -- 172.21.15.55:0/1820689453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 msgr2=0x5615e5bf8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:20.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.884+0000 7f449f7fe640 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5bf8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.884+0000 7f449f7fe640 1 -- 172.21.15.55:0/1820689453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5be8ee0 msgr2=0x5615e5bf5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:20.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.884+0000 7f449f7fe640 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5be8ee0 0x5615e5bf5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.884+0000 7f449f7fe640 1 -- 172.21.15.55:0/1820689453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e5b0e4b0 con 0x5615e5bea0b0 2024-04-26T22:04:20.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.884+0000 7f449f7fe640 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5bea0b0 0x5615e5bf5eb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4494004650 tx=0x7f4494031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e1e75078aecaa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:20.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.884+0000 7f447f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:20.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.885+0000 7f447f7fe640 1 -- 172.21.15.55:0/1820689453 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494040040 con 0x5615e5bea0b0 2024-04-26T22:04:20.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.885+0000 7f447f7fe640 1 -- 172.21.15.55:0/1820689453 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4494005250 con 0x5615e5bea0b0 2024-04-26T22:04:20.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.885+0000 7f44a9004740 1 -- 172.21.15.55:0/1820689453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44900a4aa0 con 0x5615e5bea0b0 2024-04-26T22:04:20.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.886+0000 7f447f7fe640 1 -- 172.21.15.55:0/1820689453 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4494005450 con 0x5615e5bea0b0 2024-04-26T22:04:20.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.886+0000 7f44a9004740 10 client.17265 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:20.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.887+0000 7f44a9004740 1 -- 172.21.15.55:0/1820689453 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5615e5bfa5b0 con 0x5615e5bea0b0 2024-04-26T22:04:20.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.888+0000 7f447f7fe640 1 -- 172.21.15.55:0/1820689453 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44900a4aa0 con 0x5615e5bea0b0 2024-04-26T22:04:20.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.888+0000 7f447f7fe640 1 -- 172.21.15.55:0/1820689453 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4494004ab0 con 0x5615e5bea0b0 2024-04-26T22:04:20.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.888+0000 7f44a9004740 10 client.17265 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:20.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.888+0000 7f44a9004740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:20.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.888+0000 7f44a9004740 20 client.17265 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:20.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.888+0000 7f44a9004740 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5615e5c7eb40 0x5615e5c80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:20.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.888+0000 7f44a9004740 4 client.17265 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:04:20.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.888+0000 7f44a9004740 1 -- 172.21.15.55:0/1820689453 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615e5b55800 con 0x5615e5c7eb40 2024-04-26T22:04:20.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.889+0000 7f449ffff640 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5615e5c7eb40 0x5615e5c80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:20.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.890+0000 7f447f7fe640 10 client.17265 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:20.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.892+0000 7f449ffff640 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5615e5c7eb40 0x5615e5c80f30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a0f2a7f2b4d29f server_cookie=b608711d705592f0 in_seq=0 out_seq=0 2024-04-26T22:04:20.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f447f7fe640 1 -- 172.21.15.55:0/1820689453 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x5615e5b55800 con 0x5615e5c7eb40 2024-04-26T22:04:20.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f447f7fe640 10 client.17265 handle_command_reply: tid=0 2024-04-26T22:04:20.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 client.17265 shutdown 2024-04-26T22:04:20.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 -- 172.21.15.55:0/1820689453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5bea0b0 msgr2=0x5615e5bf5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5bea0b0 0x5615e5bf5eb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4494004650 tx=0x7f4494031d20 comp rx=0 tx=0).stop 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 -- 172.21.15.55:0/1820689453 shutdown_connections 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615e5beaa70 0x5615e5bf8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5615e5c7eb40 0x5615e5c80f30 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5615e5bea0b0 0x5615e5bf5eb0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 --2- 172.21.15.55:0/1820689453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5615e5be8ee0 0x5615e5bf5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f447f7fe640 0 client.17265 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 -- 172.21.15.55:0/1820689453 >> 172.21.15.55:0/1820689453 conn(0x5615e5be43d0 msgr2=0x5615e5be2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 -- 172.21.15.55:0/1820689453 shutdown_connections 2024-04-26T22:04:20.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.896+0000 7f44a9004740 1 -- 172.21.15.55:0/1820689453 wait complete. 2024-04-26T22:04:20.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a9004740 20 client.17265 trim_cache size 0 max 16384 2024-04-26T22:04:20.899 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:04:20.899 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (2 inodes in the stack)", 2024-04-26T22:04:20.899 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:04:20.899 INFO:teuthology.orchestra.run.smithi055.stdout: "371918ca-63e3-45ed-805c-e29e707d8118": { 2024-04-26T22:04:20.899 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:04:20.900 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "371918ca-63e3-45ed-805c-e29e707d8118", 2024-04-26T22:04:20.900 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:04:20.900 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:04:20.900 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:04:20.900 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:04:20.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 -- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f44880a3cc0 msgr2=0x7f44880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f44880a3cc0 0x7f44880a6180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4494000c00 tx=0x7f44940314a0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 -- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 msgr2=0x7f44a0198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:20.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 0x7f44a0198dd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f449000b4a0 tx=0x7f449000b960 comp rx=0 tx=0).stop 2024-04-26T22:04:20.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 -- 172.21.15.55:0/871484055 shutdown_connections 2024-04-26T22:04:20.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f44880a3cc0 0x7f44880a6180 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44a0102ef0 0x7f44a01a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44a0102470 0x7f44a0199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 --2- 172.21.15.55:0/871484055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a0101ab0 0x7f44a0198dd0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:20.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.897+0000 7f44a62ea640 1 -- 172.21.15.55:0/871484055 >> 172.21.15.55:0/871484055 conn(0x7f44a00fc560 msgr2=0x7f44a0100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:20.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.898+0000 7f44a62ea640 1 -- 172.21.15.55:0/871484055 shutdown_connections 2024-04-26T22:04:20.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:20.898+0000 7f44a62ea640 1 -- 172.21.15.55:0/871484055 wait complete. 2024-04-26T22:04:20.961 INFO:tasks.cephfs.filesystem:scrub status for tag:371918ca-63e3-45ed-805c-e29e707d8118 - {'path': '/', 'tag': '371918ca-63e3-45ed-805c-e29e707d8118', 'options': 'recursive,force'} 2024-04-26T22:04:21.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:21 smithi165 ceph-mon[33296]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 880 MiB/s wr, 1.41k op/s 2024-04-26T22:04:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:21 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:04:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:21 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:21 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:04:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:21 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:04:21.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:21 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:21 smithi055 ceph-mon[29221]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 880 MiB/s wr, 1.41k op/s 2024-04-26T22:04:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:21 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:04:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:21 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:21 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:04:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:21 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:04:21.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:21 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:21 smithi064 ceph-mon[32765]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 880 MiB/s wr, 1.41k op/s 2024-04-26T22:04:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:21 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:04:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:21 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:21 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:04:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:21 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:04:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:21 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:23.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:23 smithi055 ceph-mon[29221]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 1.50k op/s 2024-04-26T22:04:23.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:23 smithi064 ceph-mon[32765]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 1.50k op/s 2024-04-26T22:04:23.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:23 smithi165 ceph-mon[33296]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 1.50k op/s 2024-04-26T22:04:25.443 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:25 smithi055 ceph-mon[29221]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.42k op/s 2024-04-26T22:04:25.443 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:25 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:25.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:25 smithi064 ceph-mon[32765]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.42k op/s 2024-04-26T22:04:25.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:25 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:25.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:25 smithi165 ceph-mon[33296]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.42k op/s 2024-04-26T22:04:25.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:25 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:25.818 INFO:tasks.workunit.client.0.smithi055.stdout:stopping iogen 2024-04-26T22:04:25.818 INFO:tasks.workunit.client.0.smithi055.stderr:+ echo 'stopping iogen' 2024-04-26T22:04:25.819 INFO:tasks.workunit.client.0.smithi055.stderr:+ ./iogen -k 2024-04-26T22:04:27.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:27 smithi055 ceph-mon[29221]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 789 MiB/s wr, 1.25k op/s 2024-04-26T22:04:27.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:27 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:04:27.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:27 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:04:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:27 smithi064 ceph-mon[32765]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 789 MiB/s wr, 1.25k op/s 2024-04-26T22:04:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:27 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:04:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:27 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:04:27.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:27 smithi165 ceph-mon[33296]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 789 MiB/s wr, 1.25k op/s 2024-04-26T22:04:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:27 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:04:27.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:27 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:04:29.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:29 smithi064 ceph-mon[32765]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 MiB/s wr, 1.60k op/s 2024-04-26T22:04:29.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:29 smithi165 ceph-mon[33296]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 MiB/s wr, 1.60k op/s 2024-04-26T22:04:29.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:29 smithi055 ceph-mon[29221]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 MiB/s wr, 1.60k op/s 2024-04-26T22:04:31.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:31 smithi064 ceph-mon[32765]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 MiB/s wr, 1.77k op/s 2024-04-26T22:04:31.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:31 smithi165 ceph-mon[33296]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 MiB/s wr, 1.77k op/s 2024-04-26T22:04:31.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:31 smithi055 ceph-mon[29221]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 893 MiB/s wr, 1.77k op/s 2024-04-26T22:04:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:33 smithi064 ceph-mon[32765]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 884 MiB/s wr, 1.67k op/s 2024-04-26T22:04:33.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:33 smithi165 ceph-mon[33296]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 884 MiB/s wr, 1.67k op/s 2024-04-26T22:04:33.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:33 smithi055 ceph-mon[29221]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 884 MiB/s wr, 1.67k op/s 2024-04-26T22:04:35.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:35 smithi064 ceph-mon[32765]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 809 MiB/s wr, 1.54k op/s 2024-04-26T22:04:35.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:35 smithi165 ceph-mon[33296]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 809 MiB/s wr, 1.54k op/s 2024-04-26T22:04:35.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:35 smithi055 ceph-mon[29221]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 809 MiB/s wr, 1.54k op/s 2024-04-26T22:04:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:37 smithi064 ceph-mon[32765]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 MiB/s wr, 1.49k op/s 2024-04-26T22:04:37.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:37 smithi165 ceph-mon[33296]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 MiB/s wr, 1.49k op/s 2024-04-26T22:04:37.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:37 smithi055 ceph-mon[29221]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 MiB/s wr, 1.49k op/s 2024-04-26T22:04:39.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:39 smithi064 ceph-mon[32765]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 901 MiB/s wr, 1.70k op/s 2024-04-26T22:04:39.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:39 smithi165 ceph-mon[33296]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 901 MiB/s wr, 1.70k op/s 2024-04-26T22:04:39.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:39 smithi055 ceph-mon[29221]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 901 MiB/s wr, 1.70k op/s 2024-04-26T22:04:41.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:41 smithi064 ceph-mon[32765]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 904 MiB/s wr, 1.46k op/s 2024-04-26T22:04:41.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:41 smithi165 ceph-mon[33296]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 904 MiB/s wr, 1.46k op/s 2024-04-26T22:04:41.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:41 smithi055 ceph-mon[29221]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 904 MiB/s wr, 1.46k op/s 2024-04-26T22:04:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:43 smithi165 ceph-mon[33296]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 881 MiB/s wr, 1.27k op/s 2024-04-26T22:04:43.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:43 smithi055 ceph-mon[29221]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 881 MiB/s wr, 1.27k op/s 2024-04-26T22:04:43.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:43 smithi064 ceph-mon[32765]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 881 MiB/s wr, 1.27k op/s 2024-04-26T22:04:45.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:45 smithi165 ceph-mon[33296]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 814 MiB/s wr, 1.26k op/s 2024-04-26T22:04:45.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:45 smithi055 ceph-mon[29221]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 814 MiB/s wr, 1.26k op/s 2024-04-26T22:04:45.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:45 smithi064 ceph-mon[32765]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 814 MiB/s wr, 1.26k op/s 2024-04-26T22:04:47.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:47 smithi165 ceph-mon[33296]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 824 MiB/s wr, 1.29k op/s 2024-04-26T22:04:47.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:47 smithi055 ceph-mon[29221]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 824 MiB/s wr, 1.29k op/s 2024-04-26T22:04:47.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:47 smithi064 ceph-mon[32765]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 824 MiB/s wr, 1.29k op/s 2024-04-26T22:04:49.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:49 smithi165 ceph-mon[33296]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 906 MiB/s wr, 1.40k op/s 2024-04-26T22:04:49.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:49 smithi055 ceph-mon[29221]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 906 MiB/s wr, 1.40k op/s 2024-04-26T22:04:49.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:49 smithi064 ceph-mon[32765]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 906 MiB/s wr, 1.40k op/s 2024-04-26T22:04:50.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:04:50.962 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:04:51.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:51 smithi055 ceph-mon[29221]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.28k op/s 2024-04-26T22:04:51.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:51 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:51.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.489+0000 7fc2655a4640 1 -- 172.21.15.55:0/4262173677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26010ec70 msgr2=0x7fc26010f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.490 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.489+0000 7fc2655a4640 1 --2- 172.21.15.55:0/4262173677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26010ec70 0x7fc26010f120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc258009ca0 tx=0x7fc25802a8f0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.494+0000 7fc2655a4640 1 -- 172.21.15.55:0/4262173677 shutdown_connections 2024-04-26T22:04:51.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.494+0000 7fc2655a4640 1 --2- 172.21.15.55:0/4262173677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26010ec70 0x7fc26010f120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.494+0000 7fc2655a4640 1 --2- 172.21.15.55:0/4262173677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc26010c2a0 0x7fc26010e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.494+0000 7fc2655a4640 1 --2- 172.21.15.55:0/4262173677 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc260109930 0x7fc26010bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.494+0000 7fc2655a4640 1 -- 172.21.15.55:0/4262173677 >> 172.21.15.55:0/4262173677 conn(0x7fc26006f7c0 msgr2=0x7fc260071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.495+0000 7fc2655a4640 1 -- 172.21.15.55:0/4262173677 shutdown_connections 2024-04-26T22:04:51.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.495+0000 7fc2655a4640 1 -- 172.21.15.55:0/4262173677 wait complete. 2024-04-26T22:04:51.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.496+0000 7fc2655a4640 1 Processor -- start 2024-04-26T22:04:51.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.497+0000 7fc2655a4640 1 -- start start 2024-04-26T22:04:51.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.497+0000 7fc2655a4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260109930 0x7fc2601a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.497+0000 7fc2655a4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc26010c2a0 0x7fc2601a7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.497+0000 7fc2655a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26010ec70 0x7fc2601aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.497+0000 7fc2655a4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc26011d2d0 con 0x7fc26010c2a0 2024-04-26T22:04:51.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.497+0000 7fc2655a4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc26011d150 con 0x7fc26010ec70 2024-04-26T22:04:51.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.497+0000 7fc2655a4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc26011d450 con 0x7fc260109930 2024-04-26T22:04:51.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.499+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc26010c2a0 0x7fc2601a7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.499+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc26010c2a0 0x7fc2601a7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52086/0 (socket says 172.21.15.55:52086) 2024-04-26T22:04:51.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.499+0000 7fc25e7fc640 1 -- 172.21.15.55:0/893206967 learned_addr learned my addr 172.21.15.55:0/893206967 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:51.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.499+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26010ec70 0x7fc2601aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.500+0000 7fc25effd640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260109930 0x7fc2601a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.504+0000 7fc25e7fc640 1 -- 172.21.15.55:0/893206967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260109930 msgr2=0x7fc2601a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.504+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260109930 0x7fc2601a7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.504+0000 7fc25e7fc640 1 -- 172.21.15.55:0/893206967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26010ec70 msgr2=0x7fc2601aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.504+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26010ec70 0x7fc2601aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.504+0000 7fc25e7fc640 1 -- 172.21.15.55:0/893206967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2601af5e0 con 0x7fc26010c2a0 2024-04-26T22:04:51.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.516+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc26010c2a0 0x7fc2601a7f50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc25000d770 tx=0x7fc25000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.518+0000 7fc247fff640 1 -- 172.21.15.55:0/893206967 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250017070 con 0x7fc26010c2a0 2024-04-26T22:04:51.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.518+0000 7fc247fff640 1 -- 172.21.15.55:0/893206967 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc250009c30 con 0x7fc26010c2a0 2024-04-26T22:04:51.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.518+0000 7fc247fff640 1 -- 172.21.15.55:0/893206967 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250003cc0 con 0x7fc26010c2a0 2024-04-26T22:04:51.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.518+0000 7fc2655a4640 1 -- 172.21.15.55:0/893206967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2601af8d0 con 0x7fc26010c2a0 2024-04-26T22:04:51.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.519+0000 7fc2655a4640 1 -- 172.21.15.55:0/893206967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2601afd10 con 0x7fc26010c2a0 2024-04-26T22:04:51.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.525+0000 7fc247fff640 1 -- 172.21.15.55:0/893206967 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc250010460 con 0x7fc26010c2a0 2024-04-26T22:04:51.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.526+0000 7fc268396740 1 Processor -- start 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.535+0000 7fc268396740 1 -- start start 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.535+0000 7fc268396740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 0x55eb4d32a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.535+0000 7fc268396740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d32b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.535+0000 7fc268396740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d332300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.535+0000 7fc268396740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55eb4d28ed80 con 0x55eb4d32b0b0 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.535+0000 7fc268396740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55eb4d2488b0 con 0x55eb4d329ee0 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.535+0000 7fc268396740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55eb4d28ad40 con 0x55eb4d32ba70 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.536+0000 7fc247fff640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2480a3d90 0x7fc2480a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.536+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d332300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.536+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d332300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36132/0 (socket says 172.21.15.55:36132) 2024-04-26T22:04:51.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.536+0000 7fc25e7fc640 1 -- 172.21.15.55:0/1860566881 learned_addr learned my addr 172.21.15.55:0/1860566881 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:51.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.536+0000 7fc247fff640 1 -- 172.21.15.55:0/893206967 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2500d0e00 con 0x7fc26010c2a0 2024-04-26T22:04:51.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.537+0000 7fc25e7fc640 1 -- 172.21.15.55:0/1860566881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 msgr2=0x55eb4d32a300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:51.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.537+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/1860566881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 0x55eb4d32a300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.537+0000 7fc25e7fc640 1 -- 172.21.15.55:0/1860566881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 msgr2=0x55eb4d32b530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:51.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.537+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/1860566881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d32b530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.539 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.537+0000 7fc25e7fc640 1 -- 172.21.15.55:0/1860566881 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb4d219a40 con 0x55eb4d32ba70 2024-04-26T22:04:51.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.539+0000 7fc25f7fe640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2480a3d90 0x7fc2480a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.540+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/1860566881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d332300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc2500993a0 tx=0x7fc25006fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=902cef8f2b95cbc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.546+0000 7fc2457fa640 1 -- 172.21.15.55:0/1860566881 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250020070 con 0x55eb4d32ba70 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.546+0000 7fc2457fa640 1 -- 172.21.15.55:0/1860566881 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc250009de0 con 0x55eb4d32ba70 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.546+0000 7fc2457fa640 1 -- 172.21.15.55:0/1860566881 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250023d40 con 0x55eb4d32ba70 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.548+0000 7fc268396740 1 -- 172.21.15.55:0/1860566881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 msgr2=0x55eb4d332300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.548+0000 7fc268396740 1 --2- 172.21.15.55:0/1860566881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d332300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc2500993a0 tx=0x7fc25006fb00 comp rx=0 tx=0).stop 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.548+0000 7fc268396740 1 -- 172.21.15.55:0/1860566881 shutdown_connections 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.548+0000 7fc268396740 1 --2- 172.21.15.55:0/1860566881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d332300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.548+0000 7fc268396740 1 --2- 172.21.15.55:0/1860566881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d32b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.548+0000 7fc268396740 1 --2- 172.21.15.55:0/1860566881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 0x55eb4d32a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.548+0000 7fc268396740 1 -- 172.21.15.55:0/1860566881 >> 172.21.15.55:0/1860566881 conn(0x55eb4d3253d0 msgr2=0x55eb4d3257d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.551+0000 7fc268396740 1 -- 172.21.15.55:0/1860566881 shutdown_connections 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.551+0000 7fc268396740 1 -- 172.21.15.55:0/1860566881 wait complete. 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.551+0000 7fc268396740 1 -- start start 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.552+0000 7fc268396740 1 Processor -- start 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.552+0000 7fc268396740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d329ee0 0x55eb4d334430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.552+0000 7fc268396740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32b0b0 0x55eb4d334970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.552+0000 7fc268396740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32ba70 0x55eb4d33b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.552+0000 7fc268396740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55eb4d332a80 con 0x55eb4d329ee0 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.552+0000 7fc268396740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55eb4d215d10 con 0x55eb4d32ba70 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.552+0000 7fc268396740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55eb4d332c00 con 0x55eb4d32b0b0 2024-04-26T22:04:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.552+0000 7fc268396740 10 client.0 _ll_register_callbacks cb 0x55eb4d242210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.553+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32b0b0 0x55eb4d334970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.553+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32b0b0 0x55eb4d334970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36148/0 (socket says 172.21.15.55:36148) 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.553+0000 7fc25e7fc640 1 -- 172.21.15.55:0/3282892972 learned_addr learned my addr 172.21.15.55:0/3282892972 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.553+0000 7fc25f7fe640 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32ba70 0x55eb4d33b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.554+0000 7fc25e7fc640 1 -- 172.21.15.55:0/3282892972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32ba70 msgr2=0x55eb4d33b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.554+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32ba70 0x55eb4d33b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.554+0000 7fc25e7fc640 1 -- 172.21.15.55:0/3282892972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d329ee0 msgr2=0x55eb4d334430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.554+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d329ee0 0x55eb4d334430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.554+0000 7fc25e7fc640 1 -- 172.21.15.55:0/3282892972 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb4d06a7d0 con 0x55eb4d32b0b0 2024-04-26T22:04:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.556+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32b0b0 0x55eb4d334970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc2500ce7f0 tx=0x7fc2500cebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea1e785e446aa774 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.560+0000 7fc242ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:04:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.560+0000 7fc242ff5640 1 -- 172.21.15.55:0/3282892972 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2500b5020 con 0x55eb4d32b0b0 2024-04-26T22:04:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.560+0000 7fc268396740 1 -- 172.21.15.55:0/3282892972 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb4d29c990 con 0x55eb4d32b0b0 2024-04-26T22:04:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.560+0000 7fc242ff5640 1 -- 172.21.15.55:0/3282892972 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2500995b0 con 0x55eb4d32b0b0 2024-04-26T22:04:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.562+0000 7fc242ff5640 1 -- 172.21.15.55:0/3282892972 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc250020040 con 0x55eb4d32b0b0 2024-04-26T22:04:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.564+0000 7fc268396740 10 client.26257 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.564+0000 7fc268396740 1 -- 172.21.15.55:0/3282892972 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55eb4cfa03f0 con 0x55eb4d32b0b0 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.565+0000 7fc242ff5640 1 -- 172.21.15.55:0/3282892972 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55eb4d29c990 con 0x55eb4d32b0b0 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.565+0000 7fc242ff5640 1 -- 172.21.15.55:0/3282892972 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc2500a0070 con 0x55eb4d32b0b0 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.566+0000 7fc268396740 10 client.26257 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.566+0000 7fc268396740 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.566+0000 7fc268396740 20 client.26257 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.566+0000 7fc268396740 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55eb4d33ffa0 0x55eb4d3423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.566+0000 7fc268396740 4 client.26257 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.566+0000 7fc268396740 1 -- 172.21.15.55:0/3282892972 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb4d296800 con 0x55eb4d33ffa0 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.568+0000 7fc25effd640 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55eb4d33ffa0 0x55eb4d3423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.568+0000 7fc25f7fe640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2480a3d90 0x7fc2480a6250 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc258006010 tx=0x7fc2580098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:51 smithi165 ceph-mon[33296]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.28k op/s 2024-04-26T22:04:51.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:51 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:51.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.573+0000 7fc25effd640 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55eb4d33ffa0 0x55eb4d3423e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708c3040c538f17b server_cookie=31a872200d18137c in_seq=0 out_seq=0 2024-04-26T22:04:51.577 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.573+0000 7fc242ff5640 10 client.26257 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:51.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.583+0000 7fc242ff5640 1 -- 172.21.15.55:0/3282892972 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb4d296800 con 0x55eb4d33ffa0 2024-04-26T22:04:51.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.583+0000 7fc242ff5640 10 client.26257 handle_command_reply: tid=0 2024-04-26T22:04:51.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.583+0000 7fc268396740 1 client.26257 shutdown 2024-04-26T22:04:51.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.583+0000 7fc268396740 1 -- 172.21.15.55:0/3282892972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32b0b0 msgr2=0x55eb4d334970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.585 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.583+0000 7fc268396740 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32b0b0 0x55eb4d334970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc2500ce7f0 tx=0x7fc2500cebf0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.587+0000 7fc268396740 1 -- 172.21.15.55:0/3282892972 shutdown_connections 2024-04-26T22:04:51.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.587+0000 7fc268396740 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32ba70 0x55eb4d33b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.587+0000 7fc268396740 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32b0b0 0x55eb4d334970 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.587+0000 7fc268396740 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55eb4d33ffa0 0x55eb4d3423e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.587+0000 7fc268396740 1 --2- 172.21.15.55:0/3282892972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d329ee0 0x55eb4d334430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.587+0000 7fc268396740 1 -- 172.21.15.55:0/3282892972 >> 172.21.15.55:0/3282892972 conn(0x55eb4d3253d0 msgr2=0x55eb4d324550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.588 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.587+0000 7fc242ff5640 0 client.26257 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:51.598 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.598+0000 7fc268396740 1 -- 172.21.15.55:0/3282892972 shutdown_connections 2024-04-26T22:04:51.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.604+0000 7fc268396740 1 -- 172.21.15.55:0/3282892972 wait complete. 2024-04-26T22:04:51.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.604+0000 7fc268396740 20 client.26257 trim_cache size 0 max 16384 2024-04-26T22:04:51.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.684+0000 7fc268396740 1 Processor -- start 2024-04-26T22:04:51.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.695+0000 7fc268396740 1 -- start start 2024-04-26T22:04:51.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.695+0000 7fc268396740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d329ee0 0x55eb4d328ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.695+0000 7fc268396740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32b0b0 0x55eb4d33b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.696+0000 7fc268396740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32ba70 0x55eb4d346ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.696+0000 7fc268396740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c010ef0 con 0x55eb4d32ba70 2024-04-26T22:04:51.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.696+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32ba70 0x55eb4d346ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.696+0000 7fc25f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32ba70 0x55eb4d346ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52100/0 (socket says 172.21.15.55:52100) 2024-04-26T22:04:51.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.696+0000 7fc25f7fe640 1 -- 172.21.15.55:0/1182475242 learned_addr learned my addr 172.21.15.55:0/1182475242 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:51.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.696+0000 7fc268396740 1 -- 172.21.15.55:0/1182475242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00e7c0 con 0x55eb4d32b0b0 2024-04-26T22:04:51.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.696+0000 7fc268396740 1 -- 172.21.15.55:0/1182475242 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00c610 con 0x55eb4d329ee0 2024-04-26T22:04:51.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.702+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32b0b0 0x55eb4d33b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.702+0000 7fc25effd640 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d329ee0 0x55eb4d328ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.703+0000 7fc25f7fe640 1 -- 172.21.15.55:0/1182475242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d329ee0 msgr2=0x55eb4d328ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.703+0000 7fc25f7fe640 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d329ee0 0x55eb4d328ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.703+0000 7fc25f7fe640 1 -- 172.21.15.55:0/1182475242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32b0b0 msgr2=0x55eb4d33b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.703+0000 7fc25f7fe640 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32b0b0 0x55eb4d33b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.703+0000 7fc25f7fe640 1 -- 172.21.15.55:0/1182475242 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb4cf600f0 con 0x55eb4d32ba70 2024-04-26T22:04:51.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.708+0000 7fc25f7fe640 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32ba70 0x55eb4d346ce0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc258035000 tx=0x7fc258043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be23535509f26d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.709+0000 7fc244ff9640 1 -- 172.21.15.55:0/1182475242 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258050070 con 0x55eb4d32ba70 2024-04-26T22:04:51.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.710+0000 7fc244ff9640 1 -- 172.21.15.55:0/1182475242 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc258036040 con 0x55eb4d32ba70 2024-04-26T22:04:51.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.711+0000 7fc268396740 1 -- 172.21.15.55:0/1182475242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32ba70 msgr2=0x55eb4d346ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.711+0000 7fc268396740 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32ba70 0x55eb4d346ce0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc258035000 tx=0x7fc258043000 comp rx=0 tx=0).stop 2024-04-26T22:04:51.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.712+0000 7fc268396740 1 -- 172.21.15.55:0/1182475242 shutdown_connections 2024-04-26T22:04:51.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.712+0000 7fc268396740 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32ba70 0x55eb4d346ce0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.712+0000 7fc268396740 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d32b0b0 0x55eb4d33b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.712+0000 7fc268396740 1 --2- 172.21.15.55:0/1182475242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d329ee0 0x55eb4d328ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.713 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.712+0000 7fc268396740 1 -- 172.21.15.55:0/1182475242 >> 172.21.15.55:0/1182475242 conn(0x55eb4d3253d0 msgr2=0x55eb4d3283c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.716+0000 7fc268396740 1 -- 172.21.15.55:0/1182475242 shutdown_connections 2024-04-26T22:04:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.718+0000 7fc268396740 1 -- 172.21.15.55:0/1182475242 wait complete. 2024-04-26T22:04:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.718+0000 7fc268396740 1 -- start start 2024-04-26T22:04:51.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.719+0000 7fc268396740 1 Processor -- start 2024-04-26T22:04:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.720+0000 7fc268396740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 0x55eb4d338b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.720+0000 7fc268396740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d3390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.720+0000 7fc268396740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d33a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.720+0000 7fc268396740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00cc80 con 0x55eb4d32b0b0 2024-04-26T22:04:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.720+0000 7fc268396740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00e0f0 con 0x55eb4d329ee0 2024-04-26T22:04:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.720+0000 7fc268396740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c00d350 con 0x55eb4d32ba70 2024-04-26T22:04:51.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.720+0000 7fc268396740 10 client.0 _ll_register_callbacks cb 0x7fc25009cb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:51.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.721+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d3390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.721+0000 7fc25e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d3390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52108/0 (socket says 172.21.15.55:52108) 2024-04-26T22:04:51.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.721+0000 7fc25e7fc640 1 -- 172.21.15.55:0/898962402 learned_addr learned my addr 172.21.15.55:0/898962402 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:51.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.721+0000 7fc25f7fe640 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d33a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.722+0000 7fc25effd640 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 0x55eb4d338b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.722+0000 7fc25e7fc640 1 -- 172.21.15.55:0/898962402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 msgr2=0x55eb4d33a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.722+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d33a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.722+0000 7fc25e7fc640 1 -- 172.21.15.55:0/898962402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 msgr2=0x55eb4d338b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:51.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.722+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 0x55eb4d338b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.722+0000 7fc25e7fc640 1 -- 172.21.15.55:0/898962402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb4d24f4b0 con 0x55eb4d32b0b0 2024-04-26T22:04:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.723+0000 7fc25e7fc640 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d3390b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc25009b1f0 tx=0x7fc2500ce730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eafca54d37c780 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:51.726 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.723+0000 7fc242ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:51.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.737+0000 7fc242ff5640 1 -- 172.21.15.55:0/898962402 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2500b5070 con 0x55eb4d32b0b0 2024-04-26T22:04:51.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.737+0000 7fc242ff5640 1 -- 172.21.15.55:0/898962402 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fc2500995b0 con 0x55eb4d32b0b0 2024-04-26T22:04:51.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.738+0000 7fc268396740 1 -- 172.21.15.55:0/898962402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc258036040 con 0x55eb4d32b0b0 2024-04-26T22:04:51.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.744+0000 7fc242ff5640 1 -- 172.21.15.55:0/898962402 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc25009cd10 con 0x55eb4d32b0b0 2024-04-26T22:04:51.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.746+0000 7fc268396740 10 client.17277 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:51.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.746+0000 7fc268396740 1 -- 172.21.15.55:0/898962402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55eb4d33b7f0 con 0x55eb4d32b0b0 2024-04-26T22:04:51.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.756+0000 7fc242ff5640 1 -- 172.21.15.55:0/898962402 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc258036040 con 0x55eb4d32b0b0 2024-04-26T22:04:51.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.756+0000 7fc242ff5640 1 -- 172.21.15.55:0/898962402 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc2500a0070 con 0x55eb4d32b0b0 2024-04-26T22:04:51.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.756+0000 7fc268396740 10 client.17277 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:51.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.756+0000 7fc268396740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:51.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.756+0000 7fc268396740 20 client.17277 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:51.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.756+0000 7fc268396740 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55eb4d3a6770 0x55eb4d3a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:51.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.756+0000 7fc268396740 4 client.17277 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:04:51.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.756+0000 7fc268396740 1 -- 172.21.15.55:0/898962402 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb4d334220 con 0x55eb4d3a6770 2024-04-26T22:04:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:51 smithi064 ceph-mon[32765]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.28k op/s 2024-04-26T22:04:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:51 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:51.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.760+0000 7fc25effd640 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55eb4d3a6770 0x55eb4d3a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:51.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.763+0000 7fc25effd640 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55eb4d3a6770 0x55eb4d3a8b60 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8d67cf5d00f87c server_cookie=df93816c9be1a655 in_seq=0 out_seq=0 2024-04-26T22:04:51.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.764+0000 7fc242ff5640 10 client.17277 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.770+0000 7fc242ff5640 1 -- 172.21.15.55:0/898962402 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eb4d334220 con 0x55eb4d3a6770 2024-04-26T22:04:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.770+0000 7fc242ff5640 10 client.17277 handle_command_reply: tid=0 2024-04-26T22:04:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.770+0000 7fc268396740 1 client.17277 shutdown 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.771+0000 7fc268396740 1 -- 172.21.15.55:0/898962402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 msgr2=0x55eb4d3390b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.772+0000 7fc268396740 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d3390b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc25009b1f0 tx=0x7fc2500ce730 comp rx=0 tx=0).stop 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.773+0000 7fc268396740 1 -- 172.21.15.55:0/898962402 shutdown_connections 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.773+0000 7fc268396740 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb4d32ba70 0x55eb4d33a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.773+0000 7fc268396740 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55eb4d3a6770 0x55eb4d3a8b60 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.773+0000 7fc268396740 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55eb4d32b0b0 0x55eb4d3390b0 secure :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc25009b1f0 tx=0x7fc2500ce730 comp rx=0 tx=0).stop 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.773+0000 7fc268396740 1 --2- 172.21.15.55:0/898962402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55eb4d329ee0 0x55eb4d338b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.773+0000 7fc268396740 1 -- 172.21.15.55:0/898962402 >> 172.21.15.55:0/898962402 conn(0x55eb4d3253d0 msgr2=0x55eb4d328c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.774+0000 7fc242ff5640 0 client.17277 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:51.781 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.778+0000 7fc268396740 1 -- 172.21.15.55:0/898962402 shutdown_connections 2024-04-26T22:04:51.782 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.779+0000 7fc268396740 1 -- 172.21.15.55:0/898962402 wait complete. 2024-04-26T22:04:51.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.782+0000 7fc268396740 20 client.17277 trim_cache size 0 max 16384 2024-04-26T22:04:51.784 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:04:51.784 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:04:51.784 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:04:51.785 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:04:51.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.793+0000 7fc2655a4640 1 -- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2480a3d90 msgr2=0x7fc2480a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.793+0000 7fc2655a4640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2480a3d90 0x7fc2480a6250 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc258006010 tx=0x7fc2580098a0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.793+0000 7fc2655a4640 1 -- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc26010c2a0 msgr2=0x7fc2601a7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:51.797 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.793+0000 7fc2655a4640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc26010c2a0 0x7fc2601a7f50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc25000d770 tx=0x7fc25000dc30 comp rx=0 tx=0).stop 2024-04-26T22:04:51.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.798+0000 7fc2655a4640 1 -- 172.21.15.55:0/893206967 shutdown_connections 2024-04-26T22:04:51.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.798+0000 7fc2655a4640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fc2480a3d90 0x7fc2480a6250 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.798+0000 7fc2655a4640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26010ec70 0x7fc2601aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.801 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.798+0000 7fc2655a4640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc26010c2a0 0x7fc2601a7f50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.798+0000 7fc2655a4640 1 --2- 172.21.15.55:0/893206967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260109930 0x7fc2601a7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:51.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.798+0000 7fc2655a4640 1 -- 172.21.15.55:0/893206967 >> 172.21.15.55:0/893206967 conn(0x7fc26006f7c0 msgr2=0x7fc26010af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:51.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.803+0000 7fc2655a4640 1 -- 172.21.15.55:0/893206967 shutdown_connections 2024-04-26T22:04:51.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:51.806+0000 7fc2655a4640 1 -- 172.21.15.55:0/893206967 wait complete. 2024-04-26T22:04:51.859 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:04:51.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:04:52.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.169+0000 7f0a5730a640 1 -- 172.21.15.55:0/1354256765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 msgr2=0x7f0a5010bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:52.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.169+0000 7f0a5730a640 1 --2- 172.21.15.55:0/1354256765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 0x7f0a5010bd60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0a4001a910 tx=0x7f0a4003b470 comp rx=0 tx=0).stop 2024-04-26T22:04:52.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.177+0000 7f0a5730a640 1 -- 172.21.15.55:0/1354256765 shutdown_connections 2024-04-26T22:04:52.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.177+0000 7f0a5730a640 1 --2- 172.21.15.55:0/1354256765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5010ec50 0x7f0a5010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.177+0000 7f0a5730a640 1 --2- 172.21.15.55:0/1354256765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a5010c2a0 0x7f0a5010e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.177+0000 7f0a5730a640 1 --2- 172.21.15.55:0/1354256765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 0x7f0a5010bd60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.177+0000 7f0a5730a640 1 -- 172.21.15.55:0/1354256765 >> 172.21.15.55:0/1354256765 conn(0x7f0a5006f6c0 msgr2=0x7f0a50071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:52.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.183+0000 7f0a5730a640 1 -- 172.21.15.55:0/1354256765 shutdown_connections 2024-04-26T22:04:52.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.187+0000 7f0a5730a640 1 -- 172.21.15.55:0/1354256765 wait complete. 2024-04-26T22:04:52.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.188+0000 7f0a5730a640 1 Processor -- start 2024-04-26T22:04:52.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.194+0000 7f0a5730a640 1 -- start start 2024-04-26T22:04:52.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.195+0000 7f0a5730a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 0x7f0a5019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:52.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.195+0000 7f0a5730a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5010c2a0 0x7f0a5019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.195+0000 7f0a5730a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a5010ec50 0x7f0a501a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.195+0000 7f0a550a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 0x7f0a5019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.196+0000 7f0a550a6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 0x7f0a5019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35330/0 (socket says 172.21.15.55:35330) 2024-04-26T22:04:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.196+0000 7f0a548a5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5010c2a0 0x7f0a5019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.196+0000 7f0a548a5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5010c2a0 0x7f0a5019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36180/0 (socket says 172.21.15.55:36180) 2024-04-26T22:04:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.196+0000 7f0a550a6640 1 -- 172.21.15.55:0/229076170 learned_addr learned my addr 172.21.15.55:0/229076170 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.196+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50112a00 con 0x7f0a5010ec50 2024-04-26T22:04:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.196+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50112880 con 0x7f0a50109930 2024-04-26T22:04:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.196+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50112b80 con 0x7f0a5010c2a0 2024-04-26T22:04:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.197+0000 7f0a550a6640 1 -- 172.21.15.55:0/229076170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5010c2a0 msgr2=0x7f0a5019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.197+0000 7f0a550a6640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5010c2a0 0x7f0a5019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.197+0000 7f0a550a6640 1 -- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a5010ec50 msgr2=0x7f0a501a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.197+0000 7f0a550a6640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a5010ec50 0x7f0a501a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.197+0000 7f0a550a6640 1 -- 172.21.15.55:0/229076170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a501a6db0 con 0x7f0a50109930 2024-04-26T22:04:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.198+0000 7f0a550a6640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 0x7f0a5019f190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a40004a30 tx=0x7f0a40004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:52.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.199+0000 7f0a467fc640 1 -- 172.21.15.55:0/229076170 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4004f070 con 0x7f0a50109930 2024-04-26T22:04:52.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.199+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a501a7040 con 0x7f0a50109930 2024-04-26T22:04:52.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.199+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a501a75b0 con 0x7f0a50109930 2024-04-26T22:04:52.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.201+0000 7f0a467fc640 1 -- 172.21.15.55:0/229076170 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40040bd0 con 0x7f0a50109930 2024-04-26T22:04:52.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.201+0000 7f0a467fc640 1 -- 172.21.15.55:0/229076170 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40046030 con 0x7f0a50109930 2024-04-26T22:04:52.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.203+0000 7f0a467fc640 1 -- 172.21.15.55:0/229076170 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0a40052070 con 0x7f0a50109930 2024-04-26T22:04:52.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.204+0000 7f0a467fc640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0a3c0a3dc0 0x7f0a3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:52.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.205+0000 7f0a548a5640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0a3c0a3dc0 0x7f0a3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:52.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.205+0000 7f0a467fc640 1 -- 172.21.15.55:0/229076170 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0a40101ac0 con 0x7f0a50109930 2024-04-26T22:04:52.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.205+0000 7f0a3bfff640 1 -- 172.21.15.55:0/229076170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a340052c0 con 0x7f0a50109930 2024-04-26T22:04:52.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.214+0000 7f0a467fc640 1 -- 172.21.15.55:0/229076170 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0a400ce300 con 0x7f0a50109930 2024-04-26T22:04:52.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.230+0000 7f0a548a5640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0a3c0a3dc0 0x7f0a3c0a6280 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0a501a06a0 tx=0x7f0a4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:52.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.870+0000 7f0a3bfff640 1 -- 172.21.15.55:0/229076170 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a34005880 con 0x7f0a50109930 2024-04-26T22:04:52.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.877+0000 7f0a467fc640 1 -- 172.21.15.55:0/229076170 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f0a400cdc00 con 0x7f0a50109930 2024-04-26T22:04:52.882 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:04:52.882 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:04:52.882 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:04:52.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.887+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0a3c0a3dc0 msgr2=0x7f0a3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:52.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.887+0000 7f0a5730a640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0a3c0a3dc0 0x7f0a3c0a6280 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0a501a06a0 tx=0x7f0a4c008040 comp rx=0 tx=0).stop 2024-04-26T22:04:52.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.887+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 msgr2=0x7f0a5019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:52.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.887+0000 7f0a5730a640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 0x7f0a5019f190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a40004a30 tx=0x7f0a40004a60 comp rx=0 tx=0).stop 2024-04-26T22:04:52.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.891+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 shutdown_connections 2024-04-26T22:04:52.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.891+0000 7f0a5730a640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0a3c0a3dc0 0x7f0a3c0a6280 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.891+0000 7f0a5730a640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a5010ec50 0x7f0a501a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.891+0000 7f0a5730a640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5010c2a0 0x7f0a5019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.891+0000 7f0a5730a640 1 --2- 172.21.15.55:0/229076170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a50109930 0x7f0a5019f190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:52.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.891+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 >> 172.21.15.55:0/229076170 conn(0x7f0a5006f6c0 msgr2=0x7f0a5010a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:52.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.895+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 shutdown_connections 2024-04-26T22:04:52.898 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:52.898+0000 7f0a5730a640 1 -- 172.21.15.55:0/229076170 wait complete. 2024-04-26T22:04:52.956 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:04:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:52 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.056 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.056 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.056 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.056 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:52 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:52 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:53.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.327+0000 7f2bada78640 1 -- 172.21.15.55:0/2719412670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 msgr2=0x7f2ba810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.330 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.327+0000 7f2bada78640 1 --2- 172.21.15.55:0/2719412670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 0x7f2ba810a470 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009900 tx=0x7f2b9c02a440 comp rx=0 tx=0).stop 2024-04-26T22:04:53.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.328+0000 7f2bada78640 1 -- 172.21.15.55:0/2719412670 shutdown_connections 2024-04-26T22:04:53.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.328+0000 7f2bada78640 1 --2- 172.21.15.55:0/2719412670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba810a9b0 0x7f2ba810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.331 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.328+0000 7f2bada78640 1 --2- 172.21.15.55:0/2719412670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 0x7f2ba810a470 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.328+0000 7f2bada78640 1 --2- 172.21.15.55:0/2719412670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ba8073d70 0x7f2ba8074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.328+0000 7f2bada78640 1 -- 172.21.15.55:0/2719412670 >> 172.21.15.55:0/2719412670 conn(0x7f2ba806f840 msgr2=0x7f2ba8071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:53.332 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.330+0000 7f2bada78640 1 -- 172.21.15.55:0/2719412670 shutdown_connections 2024-04-26T22:04:53.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.336+0000 7f2bada78640 1 -- 172.21.15.55:0/2719412670 wait complete. 2024-04-26T22:04:53.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.337+0000 7f2bada78640 1 Processor -- start 2024-04-26T22:04:53.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.337+0000 7f2bada78640 1 -- start start 2024-04-26T22:04:53.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.337+0000 7f2bada78640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8073d70 0x7f2ba81a7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.337+0000 7f2bada78640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 0x7f2ba81a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.337+0000 7f2bada78640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ba810a9b0 0x7f2ba807ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.337+0000 7f2bada78640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba811d2a0 con 0x7f2ba8073d70 2024-04-26T22:04:53.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.337+0000 7f2bada78640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba811d120 con 0x7f2ba810a9b0 2024-04-26T22:04:53.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.337+0000 7f2bada78640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba811d420 con 0x7f2ba810a010 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.338+0000 7f2ba7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 0x7f2ba81a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.338+0000 7f2ba7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 0x7f2ba81a7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36194/0 (socket says 172.21.15.55:36194) 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.338+0000 7f2ba7fff640 1 -- 172.21.15.55:0/2974151091 learned_addr learned my addr 172.21.15.55:0/2974151091 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.339+0000 7f2ba7fff640 1 -- 172.21.15.55:0/2974151091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ba810a9b0 msgr2=0x7f2ba807ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.339+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ba810a9b0 0x7f2ba807ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.339+0000 7f2ba7fff640 1 -- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8073d70 msgr2=0x7f2ba81a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.339+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8073d70 0x7f2ba81a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.339+0000 7f2ba7fff640 1 -- 172.21.15.55:0/2974151091 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba807b230 con 0x7f2ba810a010 2024-04-26T22:04:53.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.340+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 0x7f2ba81a7e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c031470 tx=0x7f2b9c003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:53.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.345+0000 7f2ba5ffb640 1 -- 172.21.15.55:0/2974151091 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c03f070 con 0x7f2ba810a010 2024-04-26T22:04:53.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.345+0000 7f2bada78640 1 -- 172.21.15.55:0/2974151091 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba807b4c0 con 0x7f2ba810a010 2024-04-26T22:04:53.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.346+0000 7f2bada78640 1 -- 172.21.15.55:0/2974151091 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba807ba30 con 0x7f2ba810a010 2024-04-26T22:04:53.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.346+0000 7f2ba5ffb640 1 -- 172.21.15.55:0/2974151091 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c031dd0 con 0x7f2ba810a010 2024-04-26T22:04:53.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.346+0000 7f2ba5ffb640 1 -- 172.21.15.55:0/2974151091 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c035030 con 0x7f2ba810a010 2024-04-26T22:04:53.357 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.356+0000 7f2ba5ffb640 1 -- 172.21.15.55:0/2974151091 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2b9c042070 con 0x7f2ba810a010 2024-04-26T22:04:53.376 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.373+0000 7f2bb19fa740 1 Processor -- start 2024-04-26T22:04:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.376+0000 7f2bb19fa740 1 -- start start 2024-04-26T22:04:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.376+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2aa00 0x55fad9f2ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.376+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f2c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.376+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f32e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.376+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fad9e8ed80 con 0x55fad9f2aa00 2024-04-26T22:04:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.376+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fad9e48a20 con 0x55fad9f2c590 2024-04-26T22:04:53.377 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.376+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fad9e8ad40 con 0x55fad9f2bbd0 2024-04-26T22:04:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.377+0000 7f2ba7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f32e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.377+0000 7f2ba7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f32e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35358/0 (socket says 172.21.15.55:35358) 2024-04-26T22:04:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.377+0000 7f2ba7fff640 1 -- 172.21.15.55:0/405519189 learned_addr learned my addr 172.21.15.55:0/405519189 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.378+0000 7f2ba5ffb640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2b900a3e40 0x7f2b900a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.378+0000 7f2ba5ffb640 1 -- 172.21.15.55:0/2974151091 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2b9c0f10b0 con 0x7f2ba810a010 2024-04-26T22:04:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.378+0000 7f2ba7fff640 1 -- 172.21.15.55:0/405519189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 msgr2=0x55fad9f2c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.379+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/405519189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f2c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.379+0000 7f2ba7fff640 1 -- 172.21.15.55:0/405519189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2aa00 msgr2=0x55fad9f2ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.379+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/405519189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2aa00 0x55fad9f2ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.379+0000 7f2ba7fff640 1 -- 172.21.15.55:0/405519189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad9e53c30 con 0x55fad9f2c590 2024-04-26T22:04:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.380+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/405519189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f32e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0bce90 tx=0x7f2b9c02a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b61af307ee2a204c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:53.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.386+0000 7f2bad277640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2b900a3e40 0x7f2b900a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.389+0000 7f2bad277640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2b900a3e40 0x7f2b900a6300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55fad9f2bad0 tx=0x7f2b98006c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:53.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.393+0000 7f2b8effd640 1 -- 172.21.15.55:0/405519189 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c049050 con 0x55fad9f2c590 2024-04-26T22:04:53.393 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.393+0000 7f2b8effd640 1 -- 172.21.15.55:0/405519189 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c0c1540 con 0x55fad9f2c590 2024-04-26T22:04:53.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.394+0000 7f2bb19fa740 1 -- 172.21.15.55:0/405519189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 msgr2=0x55fad9f32e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.394 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.394+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/405519189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f32e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0bce90 tx=0x7f2b9c02a440 comp rx=0 tx=0).stop 2024-04-26T22:04:53.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.395+0000 7f2bb19fa740 1 -- 172.21.15.55:0/405519189 shutdown_connections 2024-04-26T22:04:53.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.395+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/405519189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f32e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.395+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/405519189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f2c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.395+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/405519189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2aa00 0x55fad9f2ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.395+0000 7f2bb19fa740 1 -- 172.21.15.55:0/405519189 >> 172.21.15.55:0/405519189 conn(0x55fad9e6b910 msgr2=0x55fad9f262b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:53.398 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.398+0000 7f2bb19fa740 1 -- 172.21.15.55:0/405519189 shutdown_connections 2024-04-26T22:04:53.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.403+0000 7f2bb19fa740 1 -- 172.21.15.55:0/405519189 wait complete. 2024-04-26T22:04:53.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.403+0000 7f2bb19fa740 1 -- start start 2024-04-26T22:04:53.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.403+0000 7f2bb19fa740 1 Processor -- start 2024-04-26T22:04:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2aa00 0x55fad9f34990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f34ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55fad9e48a20 con 0x55fad9f2c590 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fad9e15dc0 con 0x55fad9f2aa00 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fad9f33360 con 0x55fad9f2bbd0 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bb19fa740 10 client.0 _ll_register_callbacks cb 0x55fad9e31600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bad277640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.404+0000 7f2bad277640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52158/0 (socket says 172.21.15.55:52158) 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.405+0000 7f2bad277640 1 -- 172.21.15.55:0/1990677918 learned_addr learned my addr 172.21.15.55:0/1990677918 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.406+0000 7f2bad277640 1 -- 172.21.15.55:0/1990677918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 msgr2=0x55fad9f34ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.406+0000 7f2bad277640 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f34ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.406+0000 7f2bad277640 1 -- 172.21.15.55:0/1990677918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2aa00 msgr2=0x55fad9f34990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.406+0000 7f2bad277640 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2aa00 0x55fad9f34990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.406+0000 7f2bad277640 1 -- 172.21.15.55:0/1990677918 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad9ba03f0 con 0x55fad9f2c590 2024-04-26T22:04:53.409 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.407+0000 7f2bad277640 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3bf10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2b98013f90 tx=0x7f2b980047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590fd251d73423a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.407+0000 7f2b87fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.407+0000 7f2b87fff640 1 -- 172.21.15.55:0/1990677918 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9801d5f0 con 0x55fad9f2c590 2024-04-26T22:04:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.408+0000 7f2bb19fa740 1 -- 172.21.15.55:0/1990677918 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9c0c1540 con 0x55fad9f2c590 2024-04-26T22:04:53.410 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.408+0000 7f2b87fff640 1 -- 172.21.15.55:0/1990677918 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2b9801d790 con 0x55fad9f2c590 2024-04-26T22:04:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.409+0000 7f2b87fff640 1 -- 172.21.15.55:0/1990677918 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2b9801d990 con 0x55fad9f2c590 2024-04-26T22:04:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.409+0000 7f2bb19fa740 10 client.17280 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.409+0000 7f2bb19fa740 1 -- 172.21.15.55:0/1990677918 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fad9c6a7d0 con 0x55fad9f2c590 2024-04-26T22:04:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.410+0000 7f2b87fff640 1 -- 172.21.15.55:0/1990677918 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c0c1540 con 0x55fad9f2c590 2024-04-26T22:04:53.411 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.410+0000 7f2b87fff640 1 -- 172.21.15.55:0/1990677918 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2b98012070 con 0x55fad9f2c590 2024-04-26T22:04:53.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.412+0000 7f2bb19fa740 10 client.17280 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:53.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.412+0000 7f2bb19fa740 10 client.17280 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:04:53.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.412+0000 7f2bb19fa740 20 client.17280 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:53.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.412+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fad9f3def0 0x55fad9f403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.412+0000 7f2bb19fa740 4 client.17280 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:53.412 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.412+0000 7f2bb19fa740 1 -- 172.21.15.55:0/1990677918 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad9e9c990 con 0x55fad9f3def0 2024-04-26T22:04:53.418 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.417+0000 7f2baca76640 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fad9f3def0 0x55fad9f403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.440+0000 7f2baca76640 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fad9f3def0 0x55fad9f403b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d50fbcc1a426ff78 server_cookie=9826229d7ebf01a3 in_seq=0 out_seq=0 2024-04-26T22:04:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.440+0000 7f2b87fff640 10 client.17280 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:04:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.448+0000 7f2b87fff640 1 -- 172.21.15.55:0/1990677918 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fad9e9c990 con 0x55fad9f3def0 2024-04-26T22:04:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.448+0000 7f2b87fff640 10 client.17280 handle_command_reply: tid=0 2024-04-26T22:04:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.448+0000 7f2bb19fa740 1 client.17280 shutdown 2024-04-26T22:04:53.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2bb19fa740 1 -- 172.21.15.55:0/1990677918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 msgr2=0x55fad9f3bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.452 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3bf10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2b98013f90 tx=0x7f2b980047d0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2bb19fa740 1 -- 172.21.15.55:0/1990677918 shutdown_connections 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fad9f3def0 0x55fad9f403b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3bf10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f34ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/1990677918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2aa00 0x55fad9f34990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2bb19fa740 1 -- 172.21.15.55:0/1990677918 >> 172.21.15.55:0/1990677918 conn(0x55fad9e6b910 msgr2=0x55fad9f25450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.452+0000 7f2b87fff640 0 client.17280 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.453+0000 7f2bb19fa740 1 -- 172.21.15.55:0/1990677918 shutdown_connections 2024-04-26T22:04:53.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.453+0000 7f2bb19fa740 1 -- 172.21.15.55:0/1990677918 wait complete. 2024-04-26T22:04:53.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.458+0000 7f2bb19fa740 20 client.17280 trim_cache size 0 max 16384 2024-04-26T22:04:53.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.514+0000 7f2bb19fa740 1 Processor -- start 2024-04-26T22:04:53.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.514+0000 7f2bb19fa740 1 -- start start 2024-04-26T22:04:53.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.515+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2aa00 0x55fad9f4c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.520+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f3bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.520+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f4c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.520+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2b94032fe0 con 0x55fad9f2aa00 2024-04-26T22:04:53.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.520+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b940308b0 con 0x55fad9f2c590 2024-04-26T22:04:53.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.520+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9402e700 con 0x55fad9f2bbd0 2024-04-26T22:04:53.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.521+0000 7f2ba7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f3bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.521+0000 7f2ba7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f3bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36226/0 (socket says 172.21.15.55:36226) 2024-04-26T22:04:53.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.521+0000 7f2ba7fff640 1 -- 172.21.15.55:0/751859740 learned_addr learned my addr 172.21.15.55:0/751859740 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.522+0000 7f2ba7fff640 1 -- 172.21.15.55:0/751859740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 msgr2=0x55fad9f4c7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.522+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/751859740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f4c7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.522+0000 7f2ba7fff640 1 -- 172.21.15.55:0/751859740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2aa00 msgr2=0x55fad9f4c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.522+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/751859740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2aa00 0x55fad9f4c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.522+0000 7f2ba7fff640 1 -- 172.21.15.55:0/751859740 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad9f2b650 con 0x55fad9f2bbd0 2024-04-26T22:04:53.523 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.523+0000 7f2ba7fff640 1 --2- 172.21.15.55:0/751859740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f3bed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0bd120 tx=0x7f2b9c0ee900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=622dbf0dc9fecf5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:53.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.523+0000 7f2b8e7fc640 1 -- 172.21.15.55:0/751859740 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0c5e00 con 0x55fad9f2bbd0 2024-04-26T22:04:53.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.524+0000 7f2b8e7fc640 1 -- 172.21.15.55:0/751859740 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c0c5b10 con 0x55fad9f2bbd0 2024-04-26T22:04:53.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.524+0000 7f2b8e7fc640 1 -- 172.21.15.55:0/751859740 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0c1110 con 0x55fad9f2bbd0 2024-04-26T22:04:53.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.526+0000 7f2bb19fa740 1 -- 172.21.15.55:0/751859740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 msgr2=0x55fad9f3bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.526+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/751859740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f3bed0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0bd120 tx=0x7f2b9c0ee900 comp rx=0 tx=0).stop 2024-04-26T22:04:53.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.529+0000 7f2bb19fa740 1 -- 172.21.15.55:0/751859740 shutdown_connections 2024-04-26T22:04:53.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.530+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/751859740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2c590 0x55fad9f4c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.530+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/751859740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2bbd0 0x55fad9f3bed0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.530+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/751859740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2aa00 0x55fad9f4c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.530+0000 7f2bb19fa740 1 -- 172.21.15.55:0/751859740 >> 172.21.15.55:0/751859740 conn(0x55fad9e6b910 msgr2=0x55fad9f25450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:53.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.531+0000 7f2bb19fa740 1 -- 172.21.15.55:0/751859740 shutdown_connections 2024-04-26T22:04:53.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.531+0000 7f2bb19fa740 1 -- 172.21.15.55:0/751859740 wait complete. 2024-04-26T22:04:53.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.532+0000 7f2bb19fa740 1 -- start start 2024-04-26T22:04:53.535 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.535+0000 7f2bb19fa740 1 Processor -- start 2024-04-26T22:04:53.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.536+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2aa00 0x55fad9f37bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.536+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2bbd0 0x55fad9f38110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.536 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.536+0000 7f2bb19fa740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.537+0000 7f2baca76640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2aa00 0x55fad9f37bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.537+0000 7f2baca76640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2aa00 0x55fad9f37bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36230/0 (socket says 172.21.15.55:36230) 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.538+0000 7f2ba7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2bbd0 0x55fad9f38110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.538+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9402ed70 con 0x55fad9f2c590 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.538+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2b940301e0 con 0x55fad9f2bbd0 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.538+0000 7f2bb19fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9402f440 con 0x55fad9f2aa00 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.538+0000 7f2bb19fa740 10 client.0 _ll_register_callbacks cb 0x7f2b98013c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2bad277640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2bad277640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52174/0 (socket says 172.21.15.55:52174) 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2bad277640 1 -- 172.21.15.55:0/3242054791 learned_addr learned my addr 172.21.15.55:0/3242054791 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:53.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2bad277640 1 -- 172.21.15.55:0/3242054791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2aa00 msgr2=0x55fad9f37bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:53.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2bad277640 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2aa00 0x55fad9f37bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2bad277640 1 -- 172.21.15.55:0/3242054791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2bbd0 msgr2=0x55fad9f38110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:53.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2bad277640 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2bbd0 0x55fad9f38110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2bad277640 1 -- 172.21.15.55:0/3242054791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad9f3b720 con 0x55fad9f2c590 2024-04-26T22:04:53.541 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.539+0000 7f2baca76640 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2aa00 0x55fad9f37bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.540+0000 7f2bad277640 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3b130 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009ef0 tx=0x7f2b98013530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef014988bf10316 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.540+0000 7f2b87fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.541+0000 7f2b87fff640 1 -- 172.21.15.55:0/3242054791 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98036060 con 0x55fad9f2c590 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.541+0000 7f2bb19fa740 1 -- 172.21.15.55:0/3242054791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9c0c5b10 con 0x55fad9f2c590 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.541+0000 7f2b87fff640 1 -- 172.21.15.55:0/3242054791 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2b9801d5f0 con 0x55fad9f2c590 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.541+0000 7f2b87fff640 1 -- 172.21.15.55:0/3242054791 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2b9801d7f0 con 0x55fad9f2c590 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.541+0000 7f2bb19fa740 10 client.17283 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.541+0000 7f2bb19fa740 1 -- 172.21.15.55:0/3242054791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55fad9fa6aa0 con 0x55fad9f2c590 2024-04-26T22:04:53.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.542+0000 7f2b87fff640 1 -- 172.21.15.55:0/3242054791 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c0c5b10 con 0x55fad9f2c590 2024-04-26T22:04:53.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.544+0000 7f2b87fff640 1 -- 172.21.15.55:0/3242054791 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f2b980066c0 con 0x55fad9f2c590 2024-04-26T22:04:53.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.544+0000 7f2bb19fa740 10 client.17283 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:53.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.544+0000 7f2bb19fa740 10 client.17283 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:04:53.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.544+0000 7f2bb19fa740 20 client.17283 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:53.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.544+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fad9fa74d0 0x55fad9fa98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:53.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.544+0000 7f2bb19fa740 4 client.17283 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:04:53.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.544+0000 7f2bb19fa740 1 -- 172.21.15.55:0/3242054791 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fad9e96800 con 0x55fad9fa74d0 2024-04-26T22:04:53.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.546+0000 7f2baca76640 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fad9fa74d0 0x55fad9fa98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:53.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.549+0000 7f2baca76640 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fad9fa74d0 0x55fad9fa98f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f716010daef3c4a server_cookie=81afeabb81fa1d45 in_seq=0 out_seq=0 2024-04-26T22:04:53.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.549+0000 7f2b87fff640 10 client.17283 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:04:53.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.550+0000 7f2b87fff640 1 -- 172.21.15.55:0/3242054791 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fad9e96800 con 0x55fad9fa74d0 2024-04-26T22:04:53.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.550+0000 7f2b87fff640 10 client.17283 handle_command_reply: tid=0 2024-04-26T22:04:53.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.550+0000 7f2bb19fa740 1 client.17283 shutdown 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.558+0000 7f2bb19fa740 1 -- 172.21.15.55:0/3242054791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 msgr2=0x55fad9f3b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.558+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3b130 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009ef0 tx=0x7f2b98013530 comp rx=0 tx=0).stop 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.558+0000 7f2bb19fa740 1 -- 172.21.15.55:0/3242054791 shutdown_connections 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.558+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55fad9fa74d0 0x55fad9fa98f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.558+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55fad9f2c590 0x55fad9f3b130 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.558+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fad9f2bbd0 0x55fad9f38110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.558+0000 7f2bb19fa740 1 --2- 172.21.15.55:0/3242054791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fad9f2aa00 0x55fad9f37bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.558+0000 7f2bb19fa740 1 -- 172.21.15.55:0/3242054791 >> 172.21.15.55:0/3242054791 conn(0x55fad9e6b910 msgr2=0x55fad9f291c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:53.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.559+0000 7f2b87fff640 0 client.17283 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:04:53.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.562+0000 7f2bb19fa740 1 -- 172.21.15.55:0/3242054791 shutdown_connections 2024-04-26T22:04:53.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.564+0000 7f2bb19fa740 1 -- 172.21.15.55:0/3242054791 wait complete. 2024-04-26T22:04:53.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.564+0000 7f2bb19fa740 20 client.17283 trim_cache size 0 max 16384 2024-04-26T22:04:53.571 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:04:53.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.574+0000 7f2bada78640 1 -- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2b900a3e40 msgr2=0x7f2b900a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.574+0000 7f2bada78640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2b900a3e40 0x7f2b900a6300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55fad9f2bad0 tx=0x7f2b98006c30 comp rx=0 tx=0).stop 2024-04-26T22:04:53.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.574+0000 7f2bada78640 1 -- 172.21.15.55:0/2974151091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 msgr2=0x7f2ba81a7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:53.575 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.574+0000 7f2bada78640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 0x7f2ba81a7e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c031470 tx=0x7f2b9c003cf0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.581+0000 7f2bada78640 1 -- 172.21.15.55:0/2974151091 shutdown_connections 2024-04-26T22:04:53.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.581+0000 7f2bada78640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f2b900a3e40 0x7f2b900a6300 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.581+0000 7f2bada78640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ba810a9b0 0x7f2ba807ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.581+0000 7f2bada78640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ba810a010 0x7f2ba81a7e40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.581+0000 7f2bada78640 1 --2- 172.21.15.55:0/2974151091 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8073d70 0x7f2ba81a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:53.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.581+0000 7f2bada78640 1 -- 172.21.15.55:0/2974151091 >> 172.21.15.55:0/2974151091 conn(0x7f2ba806f840 msgr2=0x7f2ba8071490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:53.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.586+0000 7f2bada78640 1 -- 172.21.15.55:0/2974151091 shutdown_connections 2024-04-26T22:04:53.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:53.586+0000 7f2bada78640 1 -- 172.21.15.55:0/2974151091 wait complete. 2024-04-26T22:04:53.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:04:53.940 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:53 smithi055 ceph-mon[29221]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 836 MiB/s wr, 1.33k op/s 2024-04-26T22:04:53.940 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:53 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/229076170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:04:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.011+0000 7f1246af9640 1 -- 172.21.15.55:0/1369947860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1240106340 msgr2=0x7f12401067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.012 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.011+0000 7f1246af9640 1 --2- 172.21.15.55:0/1369947860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1240106340 0x7f12401067c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1230009860 tx=0x7f123002a360 comp rx=0 tx=0).stop 2024-04-26T22:04:54.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.021+0000 7f1246af9640 1 -- 172.21.15.55:0/1369947860 shutdown_connections 2024-04-26T22:04:54.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.021+0000 7f1246af9640 1 --2- 172.21.15.55:0/1369947860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1240106340 0x7f12401067c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.021+0000 7f1246af9640 1 --2- 172.21.15.55:0/1369947860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1240103990 0x7f1240105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.021+0000 7f1246af9640 1 --2- 172.21.15.55:0/1369947860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1240101020 0x7f1240103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.021+0000 7f1246af9640 1 -- 172.21.15.55:0/1369947860 >> 172.21.15.55:0/1369947860 conn(0x7f12400fcc80 msgr2=0x7f12400ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.029 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.028+0000 7f1246af9640 1 -- 172.21.15.55:0/1369947860 shutdown_connections 2024-04-26T22:04:54.034 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.031+0000 7f1246af9640 1 -- 172.21.15.55:0/1369947860 wait complete. 2024-04-26T22:04:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.032+0000 7f1246af9640 1 Processor -- start 2024-04-26T22:04:54.035 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.032+0000 7f1246af9640 1 -- start start 2024-04-26T22:04:54.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.033+0000 7f1246af9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1240101020 0x7f12401968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.033+0000 7f1246af9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1240103990 0x7f1240196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.033+0000 7f1246af9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1240106340 0x7f124019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.033+0000 7f1246af9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f124010a120 con 0x7f1240103990 2024-04-26T22:04:54.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.033+0000 7f1246af9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1240109fa0 con 0x7f1240101020 2024-04-26T22:04:54.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.033+0000 7f1246af9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f124010a2a0 con 0x7f1240106340 2024-04-26T22:04:54.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.034+0000 7f123ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1240103990 0x7f1240196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.035+0000 7f123ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1240103990 0x7f1240196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52192/0 (socket says 172.21.15.55:52192) 2024-04-26T22:04:54.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.035+0000 7f123ffff640 1 -- 172.21.15.55:0/1839231736 learned_addr learned my addr 172.21.15.55:0/1839231736 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.038+0000 7f1245096640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1240106340 0x7f124019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.038+0000 7f1244895640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1240101020 0x7f12401968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.039+0000 7f123ffff640 1 -- 172.21.15.55:0/1839231736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1240106340 msgr2=0x7f124019de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.039+0000 7f123ffff640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1240106340 0x7f124019de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.039+0000 7f123ffff640 1 -- 172.21.15.55:0/1839231736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1240101020 msgr2=0x7f12401968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.039+0000 7f123ffff640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1240101020 0x7f12401968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.039+0000 7f123ffff640 1 -- 172.21.15.55:0/1839231736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124006c480 con 0x7f1240103990 2024-04-26T22:04:54.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.040+0000 7f123ffff640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1240103990 0x7f1240196df0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f123400b450 tx=0x7f123400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.042+0000 7f123dffb640 1 -- 172.21.15.55:0/1839231736 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1234015020 con 0x7f1240103990 2024-04-26T22:04:54.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.043+0000 7f123dffb640 1 -- 172.21.15.55:0/1839231736 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1234011d40 con 0x7f1240103990 2024-04-26T22:04:54.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.043+0000 7f123dffb640 1 -- 172.21.15.55:0/1839231736 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123400f340 con 0x7f1240103990 2024-04-26T22:04:54.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.043+0000 7f1246af9640 1 -- 172.21.15.55:0/1839231736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124006c770 con 0x7f1240103990 2024-04-26T22:04:54.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.043+0000 7f1246af9640 1 -- 172.21.15.55:0/1839231736 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124006cc70 con 0x7f1240103990 2024-04-26T22:04:54.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.055+0000 7f124980b740 1 Processor -- start 2024-04-26T22:04:54.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.056+0000 7f124980b740 1 -- start start 2024-04-26T22:04:54.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.056+0000 7f124980b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e58de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.056+0000 7f124980b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e58f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.056+0000 7f124980b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e595e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.056+0000 7f124980b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555e3e4f1d80 con 0x555e3e58ebd0 2024-04-26T22:04:54.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.056+0000 7f124980b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e3e4aba20 con 0x555e3e58da00 2024-04-26T22:04:54.060 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.056+0000 7f124980b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555e3e4edd40 con 0x555e3e58f590 2024-04-26T22:04:54.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.062+0000 7f123ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e595e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.064+0000 7f123dffb640 1 -- 172.21.15.55:0/1839231736 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1234018020 con 0x7f1240103990 2024-04-26T22:04:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.064+0000 7f1245096640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e58de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.065+0000 7f1244895640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e58f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.065+0000 7f1244895640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e58f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52198/0 (socket says 172.21.15.55:52198) 2024-04-26T22:04:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.065+0000 7f1244895640 1 -- 172.21.15.55:0/113670665 learned_addr learned my addr 172.21.15.55:0/113670665 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.065+0000 7f123dffb640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f12280a3d00 0x7f12280a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.066 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.066+0000 7f123dffb640 1 -- 172.21.15.55:0/1839231736 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f12340d3390 con 0x7f1240103990 2024-04-26T22:04:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.067+0000 7f1244895640 1 -- 172.21.15.55:0/113670665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 msgr2=0x555e3e595e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.067+0000 7f1244895640 1 --2- 172.21.15.55:0/113670665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e595e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.067+0000 7f1244895640 1 -- 172.21.15.55:0/113670665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 msgr2=0x555e3e58de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.067+0000 7f1244895640 1 --2- 172.21.15.55:0/113670665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e58de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.067+0000 7f1244895640 1 -- 172.21.15.55:0/113670665 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3e4b6c30 con 0x555e3e58ebd0 2024-04-26T22:04:54.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.067+0000 7f123ffff640 1 --2- 172.21.15.55:0/113670665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e595e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:54.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.068+0000 7f1245096640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f12280a3d00 0x7f12280a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.073+0000 7f1245096640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f12280a3d00 0x7f12280a61c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1230030380 tx=0x7f1230005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:53 smithi165 ceph-mon[33296]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 836 MiB/s wr, 1.33k op/s 2024-04-26T22:04:54.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:53 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/229076170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:04:54.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.074+0000 7f1244895640 1 --2- 172.21.15.55:0/113670665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e58f050 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f123801af90 tx=0x7f12380214e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8bf3e7b248104d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.079+0000 7f122effd640 1 -- 172.21.15.55:0/113670665 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123801d620 con 0x555e3e58ebd0 2024-04-26T22:04:54.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.079+0000 7f122effd640 1 -- 172.21.15.55:0/113670665 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123801dc70 con 0x555e3e58ebd0 2024-04-26T22:04:54.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.079+0000 7f122effd640 1 -- 172.21.15.55:0/113670665 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238029c80 con 0x555e3e58ebd0 2024-04-26T22:04:54.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.080+0000 7f124980b740 1 -- 172.21.15.55:0/113670665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 msgr2=0x555e3e58f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.080+0000 7f124980b740 1 --2- 172.21.15.55:0/113670665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e58f050 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f123801af90 tx=0x7f12380214e0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.081+0000 7f124980b740 1 -- 172.21.15.55:0/113670665 shutdown_connections 2024-04-26T22:04:54.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.081+0000 7f124980b740 1 --2- 172.21.15.55:0/113670665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e595e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.081+0000 7f124980b740 1 --2- 172.21.15.55:0/113670665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e58f050 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.081+0000 7f124980b740 1 --2- 172.21.15.55:0/113670665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e58de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.081+0000 7f124980b740 1 -- 172.21.15.55:0/113670665 >> 172.21.15.55:0/113670665 conn(0x555e3e4ce910 msgr2=0x555e3e5892b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.081+0000 7f124980b740 1 -- 172.21.15.55:0/113670665 shutdown_connections 2024-04-26T22:04:54.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.083+0000 7f124980b740 1 -- 172.21.15.55:0/113670665 wait complete. 2024-04-26T22:04:54.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.083+0000 7f124980b740 1 -- start start 2024-04-26T22:04:54.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.083+0000 7f124980b740 1 Processor -- start 2024-04-26T22:04:54.085 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f124980b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58da00 0x555e3e597ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f124980b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58ebd0 0x555e3e598220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f124980b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f124980b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x555e3e5965a0 con 0x555e3e58da00 2024-04-26T22:04:54.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f124980b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555e3e478dc0 con 0x555e3e58ebd0 2024-04-26T22:04:54.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f124980b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555e3e596720 con 0x555e3e58f590 2024-04-26T22:04:54.086 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f124980b740 10 client.0 _ll_register_callbacks cb 0x555e3e494600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:54.088 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f1244895640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58da00 0x555e3e597ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f1244895640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58da00 0x555e3e597ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52202/0 (socket says 172.21.15.55:52202) 2024-04-26T22:04:54.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.085+0000 7f1244895640 1 -- 172.21.15.55:0/1874697778 learned_addr learned my addr 172.21.15.55:0/1874697778 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.086+0000 7f1244895640 1 -- 172.21.15.55:0/1874697778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 msgr2=0x555e3e59f260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:54.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.086+0000 7f1244895640 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.087+0000 7f1244895640 1 -- 172.21.15.55:0/1874697778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58ebd0 msgr2=0x555e3e598220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:54.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.087+0000 7f1244895640 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58ebd0 0x555e3e598220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.087+0000 7f1244895640 1 -- 172.21.15.55:0/1874697778 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e3e2033f0 con 0x555e3e58da00 2024-04-26T22:04:54.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.088+0000 7f1244895640 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58da00 0x555e3e597ce0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1238000f80 tx=0x7f1238029680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b0614a12c5a54c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.090 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.089+0000 7f1223fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:54.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.089+0000 7f1223fff640 1 -- 172.21.15.55:0/1874697778 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238037020 con 0x555e3e58da00 2024-04-26T22:04:54.091 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.089+0000 7f1223fff640 1 -- 172.21.15.55:0/1874697778 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f123801aad0 con 0x555e3e58da00 2024-04-26T22:04:54.094 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.090+0000 7f124980b740 1 -- 172.21.15.55:0/1874697778 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f123801dc70 con 0x555e3e58da00 2024-04-26T22:04:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.094+0000 7f1223fff640 1 -- 172.21.15.55:0/1874697778 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1238042050 con 0x555e3e58da00 2024-04-26T22:04:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.097+0000 7f124980b740 10 client.17295 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:54.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.097+0000 7f124980b740 1 -- 172.21.15.55:0/1874697778 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555e3e2cd7d0 con 0x555e3e58da00 2024-04-26T22:04:54.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.098+0000 7f1223fff640 1 -- 172.21.15.55:0/1874697778 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123801dc70 con 0x555e3e58da00 2024-04-26T22:04:54.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.098+0000 7f1223fff640 1 -- 172.21.15.55:0/1874697778 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f1238032030 con 0x555e3e58da00 2024-04-26T22:04:54.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.098+0000 7f124980b740 10 client.17295 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:54.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.098+0000 7f124980b740 10 client.17295 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:04:54.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.098+0000 7f124980b740 20 client.17295 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:54.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.098+0000 7f124980b740 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x555e3e5a3890 0x555e3e5a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.098+0000 7f124980b740 4 client.17295 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:54.099 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.098+0000 7f124980b740 1 -- 172.21.15.55:0/1874697778 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e3e4ff990 con 0x555e3e5a3890 2024-04-26T22:04:54.102 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.102+0000 7f123ffff640 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x555e3e5a3890 0x555e3e5a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.112+0000 7f123ffff640 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x555e3e5a3890 0x555e3e5a5cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91d5010ca71e143e server_cookie=b4306e705e86440a in_seq=0 out_seq=0 2024-04-26T22:04:54.112 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.112+0000 7f1223fff640 10 client.17295 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:04:54.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.115+0000 7f1223fff640 1 -- 172.21.15.55:0/1874697778 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e3e4ff990 con 0x555e3e5a3890 2024-04-26T22:04:54.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.116+0000 7f1223fff640 10 client.17295 handle_command_reply: tid=0 2024-04-26T22:04:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.116+0000 7f124980b740 1 client.17295 shutdown 2024-04-26T22:04:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.116+0000 7f124980b740 1 -- 172.21.15.55:0/1874697778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58da00 msgr2=0x555e3e597ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.116+0000 7f124980b740 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58da00 0x555e3e597ce0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1238000f80 tx=0x7f1238029680 comp rx=0 tx=0).stop 2024-04-26T22:04:54.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f124980b740 1 -- 172.21.15.55:0/1874697778 shutdown_connections 2024-04-26T22:04:54.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f124980b740 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x555e3e5a3890 0x555e3e5a5cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f124980b740 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f124980b740 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58ebd0 0x555e3e598220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f124980b740 1 --2- 172.21.15.55:0/1874697778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58da00 0x555e3e597ce0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f124980b740 1 -- 172.21.15.55:0/1874697778 >> 172.21.15.55:0/1874697778 conn(0x555e3e4ce910 msgr2=0x555e3e588450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f1223fff640 0 client.17295 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:04:54.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f124980b740 1 -- 172.21.15.55:0/1874697778 shutdown_connections 2024-04-26T22:04:54.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.123+0000 7f124980b740 1 -- 172.21.15.55:0/1874697778 wait complete. 2024-04-26T22:04:54.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.124+0000 7f124980b740 20 client.17295 trim_cache size 0 max 16384 2024-04-26T22:04:54.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.169+0000 7f124980b740 1 Processor -- start 2024-04-26T22:04:54.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.169+0000 7f124980b740 1 -- start start 2024-04-26T22:04:54.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.172+0000 7f124980b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e5924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.172+0000 7f124980b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e592a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.172+0000 7f124980b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.172+0000 7f124980b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f1224010ef0 con 0x555e3e58ebd0 2024-04-26T22:04:54.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.172+0000 7f124980b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f122400e7c0 con 0x555e3e58da00 2024-04-26T22:04:54.172 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.172+0000 7f124980b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f122400c610 con 0x555e3e58f590 2024-04-26T22:04:54.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.175+0000 7f1244895640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e5924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.175+0000 7f1245096640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.175+0000 7f1245096640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36274/0 (socket says 172.21.15.55:36274) 2024-04-26T22:04:54.176 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.175+0000 7f1245096640 1 -- 172.21.15.55:0/2689388979 learned_addr learned my addr 172.21.15.55:0/2689388979 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.178+0000 7f123ffff640 1 --2- 172.21.15.55:0/2689388979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e592a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.181+0000 7f1244895640 1 -- 172.21.15.55:0/2689388979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 msgr2=0x555e3e59f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:54.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.181+0000 7f1244895640 1 --2- 172.21.15.55:0/2689388979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.181+0000 7f1244895640 1 -- 172.21.15.55:0/2689388979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 msgr2=0x555e3e592a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:54.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.181+0000 7f1244895640 1 --2- 172.21.15.55:0/2689388979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e592a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.181+0000 7f1244895640 1 -- 172.21.15.55:0/2689388979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3e598aa0 con 0x555e3e58da00 2024-04-26T22:04:54.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.193+0000 7f1244895640 1 --2- 172.21.15.55:0/2689388979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e5924d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f123801aee0 tx=0x7f1238004280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58385f2258fb8890 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.196+0000 7f122e7fc640 1 -- 172.21.15.55:0/2689388979 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238037040 con 0x555e3e58da00 2024-04-26T22:04:54.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.196+0000 7f122e7fc640 1 -- 172.21.15.55:0/2689388979 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123801a810 con 0x555e3e58da00 2024-04-26T22:04:54.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.196+0000 7f122e7fc640 1 -- 172.21.15.55:0/2689388979 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123802f040 con 0x555e3e58da00 2024-04-26T22:04:54.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.198+0000 7f124980b740 1 -- 172.21.15.55:0/2689388979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 msgr2=0x555e3e5924d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.198+0000 7f124980b740 1 --2- 172.21.15.55:0/2689388979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e5924d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f123801aee0 tx=0x7f1238004280 comp rx=0 tx=0).stop 2024-04-26T22:04:54.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.200+0000 7f124980b740 1 -- 172.21.15.55:0/2689388979 shutdown_connections 2024-04-26T22:04:54.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.200+0000 7f124980b740 1 --2- 172.21.15.55:0/2689388979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.200+0000 7f124980b740 1 --2- 172.21.15.55:0/2689388979 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e592a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.200+0000 7f124980b740 1 --2- 172.21.15.55:0/2689388979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e5924d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.200+0000 7f124980b740 1 -- 172.21.15.55:0/2689388979 >> 172.21.15.55:0/2689388979 conn(0x555e3e4ce910 msgr2=0x555e3e584240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.204+0000 7f124980b740 1 -- 172.21.15.55:0/2689388979 shutdown_connections 2024-04-26T22:04:54.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.204+0000 7f124980b740 1 -- 172.21.15.55:0/2689388979 wait complete. 2024-04-26T22:04:54.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.205+0000 7f124980b740 1 -- start start 2024-04-26T22:04:54.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.205+0000 7f124980b740 1 Processor -- start 2024-04-26T22:04:54.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.206+0000 7f124980b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e59edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.207+0000 7f124980b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e59ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.207+0000 7f124980b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.207+0000 7f124980b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f122400cc80 con 0x555e3e58ebd0 2024-04-26T22:04:54.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.207+0000 7f124980b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f122400e0f0 con 0x555e3e58da00 2024-04-26T22:04:54.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.208+0000 7f124980b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f122400d350 con 0x555e3e58f590 2024-04-26T22:04:54.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.208+0000 7f124980b740 10 client.0 _ll_register_callbacks cb 0x7f123801ae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:54.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.208+0000 7f123ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e59ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.208+0000 7f123ffff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e59ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52224/0 (socket says 172.21.15.55:52224) 2024-04-26T22:04:54.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.208+0000 7f123ffff640 1 -- 172.21.15.55:0/1446132467 learned_addr learned my addr 172.21.15.55:0/1446132467 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.209+0000 7f123ffff640 1 -- 172.21.15.55:0/1446132467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 msgr2=0x555e3e59bf80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:54.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.209+0000 7f123ffff640 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59bf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.209+0000 7f123ffff640 1 -- 172.21.15.55:0/1446132467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 msgr2=0x555e3e59edc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:54.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.209+0000 7f123ffff640 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e59edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.209+0000 7f123ffff640 1 -- 172.21.15.55:0/1446132467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e3e59f3d0 con 0x555e3e58ebd0 2024-04-26T22:04:54.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.210+0000 7f123ffff640 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e59ba40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f12340109e0 tx=0x7f123401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779edbf6de7cbd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.210+0000 7f1223fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:54.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.210+0000 7f1223fff640 1 -- 172.21.15.55:0/1446132467 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12340bc020 con 0x555e3e58ebd0 2024-04-26T22:04:54.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.210+0000 7f1223fff640 1 -- 172.21.15.55:0/1446132467 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f12340d0e40 con 0x555e3e58ebd0 2024-04-26T22:04:54.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.211+0000 7f124980b740 1 -- 172.21.15.55:0/1446132467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f123801a810 con 0x555e3e58ebd0 2024-04-26T22:04:54.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.211+0000 7f1223fff640 1 -- 172.21.15.55:0/1446132467 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f12340d1040 con 0x555e3e58ebd0 2024-04-26T22:04:54.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.211+0000 7f124980b740 10 client.17301 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:54.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.211+0000 7f124980b740 1 -- 172.21.15.55:0/1446132467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555e3e608e90 con 0x555e3e58ebd0 2024-04-26T22:04:54.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.216+0000 7f1223fff640 1 -- 172.21.15.55:0/1446132467 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12340b7070 con 0x555e3e58ebd0 2024-04-26T22:04:54.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.217+0000 7f1223fff640 1 -- 172.21.15.55:0/1446132467 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f12340be2c0 con 0x555e3e58ebd0 2024-04-26T22:04:54.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.217+0000 7f124980b740 10 client.17301 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:54.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.217+0000 7f124980b740 10 client.17301 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:04:54.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.217+0000 7f124980b740 20 client.17301 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:54.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.217+0000 7f124980b740 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x555e3e6097f0 0x555e3e60bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.217+0000 7f124980b740 4 client.17301 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:04:54.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.217+0000 7f124980b740 1 -- 172.21.15.55:0/1446132467 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f123801a810 con 0x555e3e6097f0 2024-04-26T22:04:54.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.233+0000 7f1244895640 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x555e3e6097f0 0x555e3e60bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.244+0000 7f1244895640 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x555e3e6097f0 0x555e3e60bc90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd758afeaa1e1e5d server_cookie=74b24a25f77ca0ee in_seq=0 out_seq=0 2024-04-26T22:04:54.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.244+0000 7f1223fff640 10 client.17301 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.246+0000 7f1223fff640 1 -- 172.21.15.55:0/1446132467 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f123801a810 con 0x555e3e6097f0 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.246+0000 7f1223fff640 10 client.17301 handle_command_reply: tid=0 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.246+0000 7f124980b740 1 client.17301 shutdown 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.246+0000 7f124980b740 1 -- 172.21.15.55:0/1446132467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 msgr2=0x555e3e59ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.246+0000 7f124980b740 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e59ba40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f12340109e0 tx=0x7f123401c000 comp rx=0 tx=0).stop 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f124980b740 1 -- 172.21.15.55:0/1446132467 shutdown_connections 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f124980b740 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x555e3e6097f0 0x555e3e60bc90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f124980b740 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555e3e58f590 0x555e3e59bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f124980b740 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x555e3e58ebd0 0x555e3e59ba40 secure :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f12340109e0 tx=0x7f123401c000 comp rx=0 tx=0).stop 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f124980b740 1 --2- 172.21.15.55:0/1446132467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555e3e58da00 0x555e3e59edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.248 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f124980b740 1 -- 172.21.15.55:0/1446132467 >> 172.21.15.55:0/1446132467 conn(0x555e3e4ce910 msgr2=0x555e3e5924d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f1223fff640 0 client.17301 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:04:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f124980b740 1 -- 172.21.15.55:0/1446132467 shutdown_connections 2024-04-26T22:04:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.247+0000 7f124980b740 1 -- 172.21.15.55:0/1446132467 wait complete. 2024-04-26T22:04:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.248+0000 7f124980b740 20 client.17301 trim_cache size 0 max 16384 2024-04-26T22:04:54.250 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:04:54.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:53 smithi064 ceph-mon[32765]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 836 MiB/s wr, 1.33k op/s 2024-04-26T22:04:54.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:53 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/229076170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:04:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.259+0000 7f1246af9640 1 -- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f12280a3d00 msgr2=0x7f12280a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.259+0000 7f1246af9640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f12280a3d00 0x7f12280a61c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1230030380 tx=0x7f1230005f90 comp rx=0 tx=0).stop 2024-04-26T22:04:54.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.259+0000 7f1246af9640 1 -- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1240103990 msgr2=0x7f1240196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.259+0000 7f1246af9640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1240103990 0x7f1240196df0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f123400b450 tx=0x7f123400b910 comp rx=0 tx=0).stop 2024-04-26T22:04:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.261+0000 7f1246af9640 1 -- 172.21.15.55:0/1839231736 shutdown_connections 2024-04-26T22:04:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.261+0000 7f1246af9640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f12280a3d00 0x7f12280a61c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.261+0000 7f1246af9640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1240106340 0x7f124019de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.261+0000 7f1246af9640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1240103990 0x7f1240196df0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.261+0000 7f1246af9640 1 --2- 172.21.15.55:0/1839231736 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1240101020 0x7f12401968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.261+0000 7f1246af9640 1 -- 172.21.15.55:0/1839231736 >> 172.21.15.55:0/1839231736 conn(0x7f12400fcc80 msgr2=0x7f12400ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.269+0000 7f1246af9640 1 -- 172.21.15.55:0/1839231736 shutdown_connections 2024-04-26T22:04:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.269+0000 7f1246af9640 1 -- 172.21.15.55:0/1839231736 wait complete. 2024-04-26T22:04:54.368 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:04:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.881+0000 7f8e96fbe640 1 -- 172.21.15.55:0/2525294968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90109f10 msgr2=0x7f8e9010a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.881+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/2525294968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90109f10 0x7f8e9010a390 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009900 tx=0x7f8e8002a440 comp rx=0 tx=0).stop 2024-04-26T22:04:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.883+0000 7f8e96fbe640 1 -- 172.21.15.55:0/2525294968 shutdown_connections 2024-04-26T22:04:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.883+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/2525294968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e9010a8d0 0x7f8e9010f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.883+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/2525294968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90109f10 0x7f8e9010a390 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.883+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/2525294968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e90073c10 0x7f8e90074030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.883+0000 7f8e96fbe640 1 -- 172.21.15.55:0/2525294968 >> 172.21.15.55:0/2525294968 conn(0x7f8e9006f780 msgr2=0x7f8e90071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.885+0000 7f8e96fbe640 1 -- 172.21.15.55:0/2525294968 shutdown_connections 2024-04-26T22:04:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.890+0000 7f8e96fbe640 1 -- 172.21.15.55:0/2525294968 wait complete. 2024-04-26T22:04:54.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.890+0000 7f8e96fbe640 1 Processor -- start 2024-04-26T22:04:54.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.892+0000 7f8e96fbe640 1 -- start start 2024-04-26T22:04:54.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.892+0000 7f8e96fbe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90073c10 0x7f8e901a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.892+0000 7f8e96fbe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e90109f10 0x7f8e901a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.892+0000 7f8e96fbe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e9010a8d0 0x7f8e901aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.892+0000 7f8e96fbe640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9011d230 con 0x7f8e90073c10 2024-04-26T22:04:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.892+0000 7f8e96fbe640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9011d0b0 con 0x7f8e9010a8d0 2024-04-26T22:04:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.892+0000 7f8e96fbe640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9011d3b0 con 0x7f8e90109f10 2024-04-26T22:04:54.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.893+0000 7f8e957bb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e90109f10 0x7f8e901a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.893+0000 7f8e95fbc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90073c10 0x7f8e901a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.894+0000 7f8e95fbc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90073c10 0x7f8e901a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52232/0 (socket says 172.21.15.55:52232) 2024-04-26T22:04:54.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.894+0000 7f8e95fbc640 1 -- 172.21.15.55:0/3578395294 learned_addr learned my addr 172.21.15.55:0/3578395294 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.895+0000 7f8e95fbc640 1 -- 172.21.15.55:0/3578395294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e90109f10 msgr2=0x7f8e901a7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.895+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e90109f10 0x7f8e901a7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.895+0000 7f8e95fbc640 1 -- 172.21.15.55:0/3578395294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e9010a8d0 msgr2=0x7f8e901aef60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.895+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e9010a8d0 0x7f8e901aef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.895+0000 7f8e95fbc640 1 -- 172.21.15.55:0/3578395294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e901af5d0 con 0x7f8e90073c10 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.896+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90073c10 0x7f8e901a79b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800ba50 tx=0x7f8e8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.896+0000 7f8e86ffd640 1 -- 172.21.15.55:0/3578395294 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88019070 con 0x7f8e90073c10 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.896+0000 7f8e86ffd640 1 -- 172.21.15.55:0/3578395294 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8800cc10 con 0x7f8e90073c10 2024-04-26T22:04:54.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.896+0000 7f8e86ffd640 1 -- 172.21.15.55:0/3578395294 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e880075d0 con 0x7f8e90073c10 2024-04-26T22:04:54.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.898+0000 7f8e96fbe640 1 -- 172.21.15.55:0/3578395294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e901af8c0 con 0x7f8e90073c10 2024-04-26T22:04:54.899 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.898+0000 7f8e96fbe640 1 -- 172.21.15.55:0/3578395294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e901afe10 con 0x7f8e90073c10 2024-04-26T22:04:54.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.904+0000 7f8e86ffd640 1 -- 172.21.15.55:0/3578395294 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8e88014980 con 0x7f8e90073c10 2024-04-26T22:04:54.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.905+0000 7f8e9b014740 1 Processor -- start 2024-04-26T22:04:54.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.906+0000 7f8e9b014740 1 -- start start 2024-04-26T22:04:54.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.906+0000 7f8e86ffd640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e7c0a3d80 0x7f8e7c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee53e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.907 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e86ffd640 1 -- 172.21.15.55:0/3578395294 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8e880cfa30 con 0x7f8e90073c10 2024-04-26T22:04:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 0x556b3ee55080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee555c0 0x556b3ee5be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e95fbc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee53e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e95fbc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee53e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52244/0 (socket says 172.21.15.55:52244) 2024-04-26T22:04:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e95fbc640 1 -- 172.21.15.55:0/505457135 learned_addr learned my addr 172.21.15.55:0/505457135 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e9b014740 1 -- 172.21.15.55:0/505457135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556b3edb7d80 con 0x556b3ee53a30 2024-04-26T22:04:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e9b014740 1 -- 172.21.15.55:0/505457135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556b3ed71a20 con 0x556b3ee54c00 2024-04-26T22:04:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.907+0000 7f8e9b014740 1 -- 172.21.15.55:0/505457135 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b3edb3d40 con 0x556b3ee555c0 2024-04-26T22:04:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.909+0000 7f8e95fbc640 1 -- 172.21.15.55:0/505457135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee555c0 msgr2=0x556b3ee5be50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:04:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.909+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/505457135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee555c0 0x556b3ee5be50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.909+0000 7f8e95fbc640 1 -- 172.21.15.55:0/505457135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 msgr2=0x556b3ee55080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.909+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/505457135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 0x556b3ee55080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.909+0000 7f8e95fbc640 1 -- 172.21.15.55:0/505457135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3ed7cc30 con 0x556b3ee53a30 2024-04-26T22:04:54.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.909+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/505457135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee53e50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8e880a4360 tx=0x7f8e8809be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2018a6ebe5fd72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.909+0000 7f8e7bfff640 1 -- 172.21.15.55:0/505457135 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88098340 con 0x556b3ee53a30 2024-04-26T22:04:54.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.909+0000 7f8e7bfff640 1 -- 172.21.15.55:0/505457135 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e880cd590 con 0x556b3ee53a30 2024-04-26T22:04:54.910 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.910+0000 7f8e7bfff640 1 -- 172.21.15.55:0/505457135 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e880a4660 con 0x556b3ee53a30 2024-04-26T22:04:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.911+0000 7f8e967bd640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e7c0a3d80 0x7f8e7c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.912+0000 7f8e9b014740 1 -- 172.21.15.55:0/505457135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 msgr2=0x556b3ee53e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.912+0000 7f8e9b014740 1 --2- 172.21.15.55:0/505457135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee53e50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8e880a4360 tx=0x7f8e8809be10 comp rx=0 tx=0).stop 2024-04-26T22:04:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.912+0000 7f8e9b014740 1 -- 172.21.15.55:0/505457135 shutdown_connections 2024-04-26T22:04:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.912+0000 7f8e9b014740 1 --2- 172.21.15.55:0/505457135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee555c0 0x556b3ee5be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.912+0000 7f8e9b014740 1 --2- 172.21.15.55:0/505457135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 0x556b3ee55080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.912+0000 7f8e9b014740 1 --2- 172.21.15.55:0/505457135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee53e50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.912+0000 7f8e9b014740 1 -- 172.21.15.55:0/505457135 >> 172.21.15.55:0/505457135 conn(0x556b3ed94910 msgr2=0x556b3ee4f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.913+0000 7f8e9b014740 1 -- 172.21.15.55:0/505457135 shutdown_connections 2024-04-26T22:04:54.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.913+0000 7f8e9b014740 1 -- 172.21.15.55:0/505457135 wait complete. 2024-04-26T22:04:54.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.913+0000 7f8e9b014740 1 -- start start 2024-04-26T22:04:54.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.913+0000 7f8e9b014740 1 Processor -- start 2024-04-26T22:04:54.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.914+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee5dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.914+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 0x556b3ee5e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.915 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.914+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 0x556b3ee65250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.914+0000 7f8e9b014740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x556b3ee5c5d0 con 0x556b3ee53a30 2024-04-26T22:04:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.914+0000 7f8e9b014740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556b3ed3edc0 con 0x556b3ee555c0 2024-04-26T22:04:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.914+0000 7f8e9b014740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b3ee5c750 con 0x556b3ee54c00 2024-04-26T22:04:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.914+0000 7f8e9b014740 10 client.0 _ll_register_callbacks cb 0x556b3ed5a600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e95fbc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee5dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e95fbc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee5dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52256/0 (socket says 172.21.15.55:52256) 2024-04-26T22:04:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e95fbc640 1 -- 172.21.15.55:0/2320651237 learned_addr learned my addr 172.21.15.55:0/2320651237 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e967bd640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e7c0a3d80 0x7f8e7c0a6240 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009530 tx=0x7f8e8c006b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e95fbc640 1 -- 172.21.15.55:0/2320651237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 msgr2=0x556b3ee5e210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 0x556b3ee5e210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e95fbc640 1 -- 172.21.15.55:0/2320651237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 msgr2=0x556b3ee65250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 0x556b3ee65250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.915+0000 7f8e95fbc640 1 -- 172.21.15.55:0/2320651237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3eac93f0 con 0x556b3ee53a30 2024-04-26T22:04:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.916+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee5dcd0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8e880986a0 tx=0x7f8e880bc130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4add4b82e19d3042 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.916+0000 7f8e797fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.916+0000 7f8e797fa640 1 -- 172.21.15.55:0/2320651237 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e880a4600 con 0x556b3ee53a30 2024-04-26T22:04:54.918 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.916+0000 7f8e797fa640 1 -- 172.21.15.55:0/2320651237 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8e8807ab40 con 0x556b3ee53a30 2024-04-26T22:04:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.917+0000 7f8e9b014740 1 -- 172.21.15.55:0/2320651237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e880cd590 con 0x556b3ee53a30 2024-04-26T22:04:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.917+0000 7f8e797fa640 1 -- 172.21.15.55:0/2320651237 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8e88080050 con 0x556b3ee53a30 2024-04-26T22:04:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.917+0000 7f8e9b014740 10 client.17313 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.917+0000 7f8e9b014740 1 -- 172.21.15.55:0/2320651237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556b3eb937d0 con 0x556b3ee53a30 2024-04-26T22:04:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.918+0000 7f8e797fa640 1 -- 172.21.15.55:0/2320651237 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e880cd590 con 0x556b3ee53a30 2024-04-26T22:04:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.918+0000 7f8e797fa640 1 -- 172.21.15.55:0/2320651237 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8e880a4ae0 con 0x556b3ee53a30 2024-04-26T22:04:54.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.918+0000 7f8e9b014740 10 client.17313 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:54.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.918+0000 7f8e9b014740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:54.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.918+0000 7f8e9b014740 20 client.17313 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:54.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.918+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556b3ee69740 0x556b3ee6bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.918+0000 7f8e9b014740 4 client.17313 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:54.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.918+0000 7f8e9b014740 1 -- 172.21.15.55:0/2320651237 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b3edc5990 con 0x556b3ee69740 2024-04-26T22:04:54.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.919+0000 7f8e957bb640 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556b3ee69740 0x556b3ee6bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.921+0000 7f8e797fa640 10 client.17313 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:54.922 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.921+0000 7f8e957bb640 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556b3ee69740 0x556b3ee6bb30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ee388f3fd0dec6 server_cookie=506ec5826f183853 in_seq=0 out_seq=0 2024-04-26T22:04:54.924 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.923+0000 7f8e797fa640 1 -- 172.21.15.55:0/2320651237 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b3edc5990 con 0x556b3ee69740 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.923+0000 7f8e797fa640 10 client.17313 handle_command_reply: tid=0 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 client.17313 shutdown 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 -- 172.21.15.55:0/2320651237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 msgr2=0x556b3ee5dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee5dcd0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8e880986a0 tx=0x7f8e880bc130 comp rx=0 tx=0).stop 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 -- 172.21.15.55:0/2320651237 shutdown_connections 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 0x556b3ee65250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 0x556b3ee5e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556b3ee69740 0x556b3ee6bb30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2320651237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee5dcd0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 -- 172.21.15.55:0/2320651237 >> 172.21.15.55:0/2320651237 conn(0x556b3ed94910 msgr2=0x556b3ee4e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e797fa640 0 client.17313 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 -- 172.21.15.55:0/2320651237 shutdown_connections 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 1 -- 172.21.15.55:0/2320651237 wait complete. 2024-04-26T22:04:54.925 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.924+0000 7f8e9b014740 20 client.17313 trim_cache size 0 max 16384 2024-04-26T22:04:54.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.989+0000 7f8e9b014740 1 Processor -- start 2024-04-26T22:04:54.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.994+0000 7f8e9b014740 1 -- start start 2024-04-26T22:04:54.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.994+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee53a30 0x556b3ee52b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.994+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 0x556b3ee65210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.994+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee555c0 0x556b3ee71220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:54.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.994+0000 7f8e9b014740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8e901bb700 con 0x556b3ee555c0 2024-04-26T22:04:54.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.994+0000 7f8e9b014740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e901b8fd0 con 0x556b3ee54c00 2024-04-26T22:04:54.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.995+0000 7f8e9b014740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8e901b6e20 con 0x556b3ee53a30 2024-04-26T22:04:54.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.996+0000 7f8e967bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee555c0 0x556b3ee71220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.996+0000 7f8e967bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee555c0 0x556b3ee71220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52270/0 (socket says 172.21.15.55:52270) 2024-04-26T22:04:54.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.996+0000 7f8e967bd640 1 -- 172.21.15.55:0/743996063 learned_addr learned my addr 172.21.15.55:0/743996063 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:54.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.996+0000 7f8e957bb640 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 0x556b3ee65210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:54.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:54.996+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee53a30 0x556b3ee52b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:55.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.007+0000 7f8e957bb640 1 -- 172.21.15.55:0/743996063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee53a30 msgr2=0x556b3ee52b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:55.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.007+0000 7f8e957bb640 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee53a30 0x556b3ee52b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.007+0000 7f8e957bb640 1 -- 172.21.15.55:0/743996063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee555c0 msgr2=0x556b3ee71220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:55.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.007+0000 7f8e957bb640 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee555c0 0x556b3ee71220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.007+0000 7f8e957bb640 1 -- 172.21.15.55:0/743996063 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3ee666d0 con 0x556b3ee54c00 2024-04-26T22:04:55.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.007+0000 7f8e967bd640 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee555c0 0x556b3ee71220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:55.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.016+0000 7f8e957bb640 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 0x556b3ee65210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009850 tx=0x7f8e80000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5470c8571f1fcb5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:55.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.022+0000 7f8e7b7fe640 1 -- 172.21.15.55:0/743996063 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80035040 con 0x556b3ee54c00 2024-04-26T22:04:55.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.022+0000 7f8e7b7fe640 1 -- 172.21.15.55:0/743996063 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e80049070 con 0x556b3ee54c00 2024-04-26T22:04:55.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.022+0000 7f8e7b7fe640 1 -- 172.21.15.55:0/743996063 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80031800 con 0x556b3ee54c00 2024-04-26T22:04:55.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.024+0000 7f8e9b014740 1 -- 172.21.15.55:0/743996063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 msgr2=0x556b3ee65210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:55.026 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.024+0000 7f8e9b014740 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 0x556b3ee65210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009850 tx=0x7f8e80000bc0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.032+0000 7f8e9b014740 1 -- 172.21.15.55:0/743996063 shutdown_connections 2024-04-26T22:04:55.032 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.032+0000 7f8e9b014740 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee555c0 0x556b3ee71220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.032+0000 7f8e9b014740 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee54c00 0x556b3ee65210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.032+0000 7f8e9b014740 1 --2- 172.21.15.55:0/743996063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee53a30 0x556b3ee52b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.033 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.032+0000 7f8e9b014740 1 -- 172.21.15.55:0/743996063 >> 172.21.15.55:0/743996063 conn(0x556b3ed94910 msgr2=0x556b3ee5df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:55.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.035+0000 7f8e9b014740 1 -- 172.21.15.55:0/743996063 shutdown_connections 2024-04-26T22:04:55.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.039+0000 7f8e9b014740 1 -- 172.21.15.55:0/743996063 wait complete. 2024-04-26T22:04:55.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.039+0000 7f8e9b014740 1 -- start start 2024-04-26T22:04:55.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.040+0000 7f8e9b014740 1 Processor -- start 2024-04-26T22:04:55.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.040+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee62860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:55.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.040+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 0x556b3ee62da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:55.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.040+0000 7f8e9b014740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 0x556b3ee63db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:55.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.040+0000 7f8e9b014740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f8e901b7490 con 0x556b3ee53a30 2024-04-26T22:04:55.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.040+0000 7f8e9b014740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e901b8900 con 0x556b3ee555c0 2024-04-26T22:04:55.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.040+0000 7f8e9b014740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8e901b7b60 con 0x556b3ee54c00 2024-04-26T22:04:55.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.040+0000 7f8e9b014740 10 client.0 _ll_register_callbacks cb 0x7f8e8809cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:55.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.042+0000 7f8e967bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 0x556b3ee63db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:55.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.043+0000 7f8e967bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 0x556b3ee63db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35494/0 (socket says 172.21.15.55:35494) 2024-04-26T22:04:55.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.043+0000 7f8e967bd640 1 -- 172.21.15.55:0/2302278686 learned_addr learned my addr 172.21.15.55:0/2302278686 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:55.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.043+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee62860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:55.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.047+0000 7f8e957bb640 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 0x556b3ee62da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:55.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.049+0000 7f8e95fbc640 1 -- 172.21.15.55:0/2302278686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 msgr2=0x556b3ee62da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:55.049 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.049+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 0x556b3ee62da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.049+0000 7f8e95fbc640 1 -- 172.21.15.55:0/2302278686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 msgr2=0x556b3ee63db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:55.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.049+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 0x556b3ee63db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.049+0000 7f8e95fbc640 1 -- 172.21.15.55:0/2302278686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3ee643a0 con 0x556b3ee53a30 2024-04-26T22:04:55.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.058+0000 7f8e95fbc640 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee62860 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8e8809a220 tx=0x7f8e88098550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f98da0559e8bca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:55.062 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.058+0000 7f8e797fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:55.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.065+0000 7f8e797fa640 1 -- 172.21.15.55:0/2302278686 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e880a8060 con 0x556b3ee53a30 2024-04-26T22:04:55.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.065+0000 7f8e9b014740 1 -- 172.21.15.55:0/2302278686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e80049070 con 0x556b3ee53a30 2024-04-26T22:04:55.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.066+0000 7f8e797fa640 1 -- 172.21.15.55:0/2302278686 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f8e880a4ae0 con 0x556b3ee53a30 2024-04-26T22:04:55.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.066+0000 7f8e797fa640 1 -- 172.21.15.55:0/2302278686 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8e88097f70 con 0x556b3ee53a30 2024-04-26T22:04:55.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.066+0000 7f8e9b014740 10 client.17319 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:55.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.066+0000 7f8e9b014740 1 -- 172.21.15.55:0/2302278686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556b3eecfa90 con 0x556b3ee53a30 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.068+0000 7f8e797fa640 1 -- 172.21.15.55:0/2302278686 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e80049070 con 0x556b3ee53a30 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.069+0000 7f8e797fa640 1 -- 172.21.15.55:0/2302278686 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f8e88097a20 con 0x556b3ee53a30 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.069+0000 7f8e9b014740 10 client.17319 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.069+0000 7f8e9b014740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.069+0000 7f8e9b014740 20 client.17319 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.069+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556b3eed0220 0x556b3eed2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.069+0000 7f8e9b014740 4 client.17319 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.069+0000 7f8e9b014740 1 -- 172.21.15.55:0/2302278686 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b3edbf800 con 0x556b3eed0220 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.072+0000 7f8e957bb640 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556b3eed0220 0x556b3eed2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:55.076 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.075+0000 7f8e957bb640 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556b3eed0220 0x556b3eed2640 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34b80f296d2b669 server_cookie=3918726e83a05d4e in_seq=0 out_seq=0 2024-04-26T22:04:55.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.076+0000 7f8e797fa640 10 client.17319 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.080+0000 7f8e797fa640 1 -- 172.21.15.55:0/2302278686 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b3edbf800 con 0x556b3eed0220 2024-04-26T22:04:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.080+0000 7f8e797fa640 10 client.17319 handle_command_reply: tid=0 2024-04-26T22:04:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.080+0000 7f8e9b014740 1 client.17319 shutdown 2024-04-26T22:04:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 -- 172.21.15.55:0/2302278686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 msgr2=0x556b3ee62860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee62860 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8e8809a220 tx=0x7f8e88098550 comp rx=0 tx=0).stop 2024-04-26T22:04:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 -- 172.21.15.55:0/2302278686 shutdown_connections 2024-04-26T22:04:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b3ee555c0 0x556b3ee63db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b3ee54c00 0x556b3ee62da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x556b3eed0220 0x556b3eed2640 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 --2- 172.21.15.55:0/2302278686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x556b3ee53a30 0x556b3ee62860 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 -- 172.21.15.55:0/2302278686 >> 172.21.15.55:0/2302278686 conn(0x556b3ed94910 msgr2=0x556b3ee52b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e797fa640 0 client.17319 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 -- 172.21.15.55:0/2302278686 shutdown_connections 2024-04-26T22:04:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 1 -- 172.21.15.55:0/2302278686 wait complete. 2024-04-26T22:04:55.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.082+0000 7f8e9b014740 20 client.17319 trim_cache size 0 max 16384 2024-04-26T22:04:55.089 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:04:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.097+0000 7f8e96fbe640 1 -- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e7c0a3d80 msgr2=0x7f8e7c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:55.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.097+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e7c0a3d80 0x7f8e7c0a6240 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009530 tx=0x7f8e8c006b70 comp rx=0 tx=0).stop 2024-04-26T22:04:55.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.097+0000 7f8e96fbe640 1 -- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90073c10 msgr2=0x7f8e901a79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:55.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.097+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90073c10 0x7f8e901a79b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800ba50 tx=0x7f8e8800bf10 comp rx=0 tx=0).stop 2024-04-26T22:04:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.102+0000 7f8e96fbe640 1 -- 172.21.15.55:0/3578395294 shutdown_connections 2024-04-26T22:04:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.102+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f8e7c0a3d80 0x7f8e7c0a6240 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.102+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e9010a8d0 0x7f8e901aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.102+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8e90109f10 0x7f8e901a7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.102+0000 7f8e96fbe640 1 --2- 172.21.15.55:0/3578395294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e90073c10 0x7f8e901a79b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.102+0000 7f8e96fbe640 1 -- 172.21.15.55:0/3578395294 >> 172.21.15.55:0/3578395294 conn(0x7f8e9006f780 msgr2=0x7f8e900701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:55.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.108+0000 7f8e96fbe640 1 -- 172.21.15.55:0/3578395294 shutdown_connections 2024-04-26T22:04:55.108 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:55.108+0000 7f8e96fbe640 1 -- 172.21.15.55:0/3578395294 wait complete. 2024-04-26T22:04:56.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:55 smithi165 ceph-mon[33296]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.41k op/s 2024-04-26T22:04:56.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:55 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:56.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:55 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:56.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:55 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:56.158 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:04:56.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:04:56.159 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:04:56.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:55 smithi055 ceph-mon[29221]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.41k op/s 2024-04-26T22:04:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:55 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:55 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:56.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:55 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:55 smithi064 ceph-mon[32765]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.41k op/s 2024-04-26T22:04:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:55 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:55 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:55 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:04:56.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.559+0000 7f508ec99640 1 -- 172.21.15.55:0/2032761072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f508810ec70 msgr2=0x7f508810f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:56.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.559+0000 7f508ec99640 1 --2- 172.21.15.55:0/2032761072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f508810ec70 0x7f508810f120 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5078009900 tx=0x7f507802a410 comp rx=0 tx=0).stop 2024-04-26T22:04:56.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.560+0000 7f508ec99640 1 -- 172.21.15.55:0/2032761072 shutdown_connections 2024-04-26T22:04:56.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.560+0000 7f508ec99640 1 --2- 172.21.15.55:0/2032761072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f508810ec70 0x7f508810f120 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.560+0000 7f508ec99640 1 --2- 172.21.15.55:0/2032761072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508810c2a0 0x7f508810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.560+0000 7f508ec99640 1 --2- 172.21.15.55:0/2032761072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5088109930 0x7f508810bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.561 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.560+0000 7f508ec99640 1 -- 172.21.15.55:0/2032761072 >> 172.21.15.55:0/2032761072 conn(0x7f508806f7c0 msgr2=0x7f5088071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:56.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.567+0000 7f508ec99640 1 -- 172.21.15.55:0/2032761072 shutdown_connections 2024-04-26T22:04:56.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.568+0000 7f508ec99640 1 -- 172.21.15.55:0/2032761072 wait complete. 2024-04-26T22:04:56.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.569+0000 7f508ec99640 1 Processor -- start 2024-04-26T22:04:56.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.570+0000 7f508ec99640 1 -- start start 2024-04-26T22:04:56.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.570+0000 7f508ec99640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5088109930 0x7f50881a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.570+0000 7f508ec99640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f508810c2a0 0x7f50881a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.571+0000 7f508ec99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508810ec70 0x7f50881aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.571+0000 7f508ec99640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f508811d350 con 0x7f5088109930 2024-04-26T22:04:56.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.571+0000 7f508ec99640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f508811d1d0 con 0x7f508810ec70 2024-04-26T22:04:56.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.571+0000 7f508ec99640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f508811d4d0 con 0x7f508810c2a0 2024-04-26T22:04:56.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.574+0000 7f5087fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f508810c2a0 0x7f50881a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.574+0000 7f508d236640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508810ec70 0x7f50881aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.575+0000 7f508d236640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508810ec70 0x7f50881aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35506/0 (socket says 172.21.15.55:35506) 2024-04-26T22:04:56.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.576+0000 7f508d236640 1 -- 172.21.15.55:0/3399262904 learned_addr learned my addr 172.21.15.55:0/3399262904 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:56.576 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.576+0000 7f508ca35640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5088109930 0x7f50881a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.591+0000 7f508ca35640 1 -- 172.21.15.55:0/3399262904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f508810c2a0 msgr2=0x7f50881a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:56.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.591+0000 7f508ca35640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f508810c2a0 0x7f50881a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.591+0000 7f508ca35640 1 -- 172.21.15.55:0/3399262904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508810ec70 msgr2=0x7f50881aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:56.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.591+0000 7f508ca35640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508810ec70 0x7f50881aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.591+0000 7f508ca35640 1 -- 172.21.15.55:0/3399262904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50881af5c0 con 0x7f5088109930 2024-04-26T22:04:56.597 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.597+0000 7f508ca35640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5088109930 0x7f50881a79f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f508000bd70 tx=0x7f508000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.598+0000 7f5085ffb640 1 -- 172.21.15.55:0/3399262904 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080019070 con 0x7f5088109930 2024-04-26T22:04:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.599+0000 7f508ec99640 1 -- 172.21.15.55:0/3399262904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50881af8b0 con 0x7f5088109930 2024-04-26T22:04:56.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.599+0000 7f508ec99640 1 -- 172.21.15.55:0/3399262904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50881afe00 con 0x7f5088109930 2024-04-26T22:04:56.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.599+0000 7f5085ffb640 1 -- 172.21.15.55:0/3399262904 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50800148f0 con 0x7f5088109930 2024-04-26T22:04:56.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.599+0000 7f5085ffb640 1 -- 172.21.15.55:0/3399262904 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080012860 con 0x7f5088109930 2024-04-26T22:04:56.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.607+0000 7f5085ffb640 1 -- 172.21.15.55:0/3399262904 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f508001c070 con 0x7f5088109930 2024-04-26T22:04:56.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.608+0000 7f5085ffb640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50700a3e40 0x7f50700a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.609+0000 7f5085ffb640 1 -- 172.21.15.55:0/3399262904 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f50800d0bf0 con 0x7f5088109930 2024-04-26T22:04:56.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.610+0000 7f50919bb740 1 Processor -- start 2024-04-26T22:04:56.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.610+0000 7f5087fff640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50700a3e40 0x7f50700a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.615+0000 7f50919bb740 1 -- start start 2024-04-26T22:04:56.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.615+0000 7f50919bb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7148b0 0x55d97e714cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.615+0000 7f50919bb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7152a0 0x55d97e719da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.615+0000 7f50919bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e71c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.615+0000 7f50919bb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d97e679b00 con 0x55d97e7152a0 2024-04-26T22:04:56.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.615+0000 7f50919bb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d97e615920 con 0x55d97e7148b0 2024-04-26T22:04:56.616 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.615+0000 7f50919bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d97e675ac0 con 0x55d97e71a410 2024-04-26T22:04:56.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.616+0000 7f508d236640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e71c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.616+0000 7f508d236640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e71c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36348/0 (socket says 172.21.15.55:36348) 2024-04-26T22:04:56.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.616+0000 7f508d236640 1 -- 172.21.15.55:0/725346733 learned_addr learned my addr 172.21.15.55:0/725346733 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:56.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.616+0000 7f508ca35640 1 --2- 172.21.15.55:0/725346733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7148b0 0x55d97e714cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.617 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.617+0000 7f5087fff640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50700a3e40 0x7f50700a6300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f50881a8eb0 tx=0x7f507c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:56.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.617+0000 7f508d236640 1 -- 172.21.15.55:0/725346733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7148b0 msgr2=0x55d97e714cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:56.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.617+0000 7f508d236640 1 --2- 172.21.15.55:0/725346733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7148b0 0x55d97e714cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.617+0000 7f508d236640 1 -- 172.21.15.55:0/725346733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7152a0 msgr2=0x55d97e719da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:56.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.617+0000 7f508d236640 1 --2- 172.21.15.55:0/725346733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7152a0 0x55d97e719da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.617+0000 7f508d236640 1 -- 172.21.15.55:0/725346733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97e4537e0 con 0x55d97e71a410 2024-04-26T22:04:56.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.618+0000 7f508d236640 1 --2- 172.21.15.55:0/725346733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e71c8a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5078031880 tx=0x7f5078009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bea979c7d6515c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:56.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.620+0000 7f506effd640 1 -- 172.21.15.55:0/725346733 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078035050 con 0x55d97e71a410 2024-04-26T22:04:56.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.620+0000 7f506effd640 1 -- 172.21.15.55:0/725346733 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5078031d30 con 0x55d97e71a410 2024-04-26T22:04:56.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.622+0000 7f50919bb740 1 -- 172.21.15.55:0/725346733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 msgr2=0x55d97e71c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:56.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.622+0000 7f50919bb740 1 --2- 172.21.15.55:0/725346733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e71c8a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5078031880 tx=0x7f5078009460 comp rx=0 tx=0).stop 2024-04-26T22:04:56.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.622+0000 7f50919bb740 1 -- 172.21.15.55:0/725346733 shutdown_connections 2024-04-26T22:04:56.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.622+0000 7f50919bb740 1 --2- 172.21.15.55:0/725346733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e71c8a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.622+0000 7f50919bb740 1 --2- 172.21.15.55:0/725346733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7152a0 0x55d97e719da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.622+0000 7f50919bb740 1 --2- 172.21.15.55:0/725346733 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7148b0 0x55d97e714cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.622+0000 7f50919bb740 1 -- 172.21.15.55:0/725346733 >> 172.21.15.55:0/725346733 conn(0x55d97e70ff20 msgr2=0x55d97e710320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:56.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.639+0000 7f50919bb740 1 -- 172.21.15.55:0/725346733 shutdown_connections 2024-04-26T22:04:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.639+0000 7f50919bb740 1 -- 172.21.15.55:0/725346733 wait complete. 2024-04-26T22:04:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.639+0000 7f50919bb740 1 -- start start 2024-04-26T22:04:56.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.639+0000 7f50919bb740 1 Processor -- start 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.640+0000 7f50919bb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.640+0000 7f50919bb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7152a0 0x55d97e71f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.640+0000 7f50919bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e7263e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.640+0000 7f50919bb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d97e71d3b0 con 0x55d97e7148b0 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.640+0000 7f50919bb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d97e6135d0 con 0x55d97e7152a0 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.640+0000 7f50919bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d97e71d530 con 0x55d97e71a410 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.640+0000 7f50919bb740 10 client.0 _ll_register_callbacks cb 0x55d97e601080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.640+0000 7f508ca35640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.641+0000 7f508ca35640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52308/0 (socket says 172.21.15.55:52308) 2024-04-26T22:04:56.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.641+0000 7f508ca35640 1 -- 172.21.15.55:0/816836423 learned_addr learned my addr 172.21.15.55:0/816836423 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.641+0000 7f508ca35640 1 -- 172.21.15.55:0/816836423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 msgr2=0x55d97e7263e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.641+0000 7f508ca35640 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e7263e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.641+0000 7f508ca35640 1 -- 172.21.15.55:0/816836423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7152a0 msgr2=0x55d97e71f3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.641+0000 7f508ca35640 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7152a0 0x55d97e71f3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.641+0000 7f508ca35640 1 -- 172.21.15.55:0/816836423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97e4b3f70 con 0x55d97e7148b0 2024-04-26T22:04:56.642 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.642+0000 7f508ca35640 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71ee60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f508000bd40 tx=0x7f508009cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e163775a0cd74758 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:56.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.643+0000 7f5067fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:04:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.644+0000 7f5067fff640 1 -- 172.21.15.55:0/816836423 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50800a5a50 con 0x55d97e7148b0 2024-04-26T22:04:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.644+0000 7f50919bb740 1 -- 172.21.15.55:0/816836423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d97e687710 con 0x55d97e7148b0 2024-04-26T22:04:56.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.644+0000 7f5067fff640 1 -- 172.21.15.55:0/816836423 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f50800a55c0 con 0x55d97e7148b0 2024-04-26T22:04:56.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.647+0000 7f5067fff640 1 -- 172.21.15.55:0/816836423 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5080099130 con 0x55d97e7148b0 2024-04-26T22:04:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.651+0000 7f50919bb740 10 client.17328 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:56.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.651+0000 7f50919bb740 1 -- 172.21.15.55:0/816836423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d97e34b170 con 0x55d97e7148b0 2024-04-26T22:04:56.657 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.656+0000 7f5067fff640 1 -- 172.21.15.55:0/816836423 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d97e687710 con 0x55d97e7148b0 2024-04-26T22:04:56.658 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.656+0000 7f5067fff640 1 -- 172.21.15.55:0/816836423 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f50800ceb10 con 0x55d97e7148b0 2024-04-26T22:04:56.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.656+0000 7f50919bb740 10 client.17328 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:56.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.656+0000 7f50919bb740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:56.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.656+0000 7f50919bb740 20 client.17328 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:56.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.656+0000 7f50919bb740 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d97e72af30 0x55d97e72d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.656+0000 7f50919bb740 4 client.17328 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:56.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.656+0000 7f50919bb740 1 -- 172.21.15.55:0/816836423 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d97e681580 con 0x55d97e72af30 2024-04-26T22:04:56.659 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.659+0000 7f5087fff640 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d97e72af30 0x55d97e72d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.661+0000 7f5087fff640 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d97e72af30 0x55d97e72d3d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3322a7fd83fbff87 server_cookie=db74ae5da4f7f2c5 in_seq=0 out_seq=0 2024-04-26T22:04:56.662 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.661+0000 7f5067fff640 10 client.17328 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:56.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.666+0000 7f5067fff640 1 -- 172.21.15.55:0/816836423 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d97e681580 con 0x55d97e72af30 2024-04-26T22:04:56.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.666+0000 7f5067fff640 10 client.17328 handle_command_reply: tid=0 2024-04-26T22:04:56.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.668+0000 7f50919bb740 1 client.17328 shutdown 2024-04-26T22:04:56.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.670+0000 7f50919bb740 1 -- 172.21.15.55:0/816836423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 msgr2=0x55d97e71ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:56.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.670+0000 7f50919bb740 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71ee60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f508000bd40 tx=0x7f508009cd90 comp rx=0 tx=0).stop 2024-04-26T22:04:56.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.676+0000 7f50919bb740 1 -- 172.21.15.55:0/816836423 shutdown_connections 2024-04-26T22:04:56.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.676+0000 7f50919bb740 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e71a410 0x55d97e7263e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.676+0000 7f50919bb740 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e7152a0 0x55d97e71f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.676+0000 7f50919bb740 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d97e72af30 0x55d97e72d3d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.676+0000 7f50919bb740 1 --2- 172.21.15.55:0/816836423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71ee60 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.676+0000 7f50919bb740 1 -- 172.21.15.55:0/816836423 >> 172.21.15.55:0/816836423 conn(0x55d97e70ff20 msgr2=0x55d97e70f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:56.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.676+0000 7f5067fff640 0 client.17328 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:56.677 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.677+0000 7f50919bb740 1 -- 172.21.15.55:0/816836423 shutdown_connections 2024-04-26T22:04:56.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.677+0000 7f50919bb740 1 -- 172.21.15.55:0/816836423 wait complete. 2024-04-26T22:04:56.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.677+0000 7f50919bb740 20 client.17328 trim_cache size 0 max 16384 2024-04-26T22:04:56.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.743+0000 7f50919bb740 1 Processor -- start 2024-04-26T22:04:56.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.743+0000 7f50919bb740 1 -- start start 2024-04-26T22:04:56.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.743+0000 7f50919bb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.743+0000 7f50919bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e726370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.744 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.744+0000 7f50919bb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 0x55d97e712e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.744+0000 7f50919bb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5074010f80 con 0x55d97e7148b0 2024-04-26T22:04:56.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.744+0000 7f50919bb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f507400e850 con 0x55d97e71a410 2024-04-26T22:04:56.745 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.744+0000 7f50919bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f507400c6a0 con 0x55d97e7152a0 2024-04-26T22:04:56.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.744+0000 7f508ca35640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.744+0000 7f508ca35640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52310/0 (socket says 172.21.15.55:52310) 2024-04-26T22:04:56.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.744+0000 7f508ca35640 1 -- 172.21.15.55:0/2202244971 learned_addr learned my addr 172.21.15.55:0/2202244971 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:56.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.744+0000 7f5087fff640 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e726370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.745+0000 7f508ca35640 1 -- 172.21.15.55:0/2202244971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 msgr2=0x55d97e726370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:56.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.745+0000 7f508ca35640 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e726370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.745+0000 7f508d236640 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 0x55d97e712e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.745+0000 7f508ca35640 1 -- 172.21.15.55:0/2202244971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 msgr2=0x55d97e712e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:56.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.745+0000 7f508ca35640 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 0x55d97e712e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.745+0000 7f508ca35640 1 -- 172.21.15.55:0/2202244971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d97e710bf0 con 0x55d97e7148b0 2024-04-26T22:04:56.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.745+0000 7f508d236640 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 0x55d97e712e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:04:56.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.746+0000 7f508ca35640 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71f180 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f508009cb90 tx=0x7f5080004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5510db18373e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:56.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.746+0000 7f506e7fc640 1 -- 172.21.15.55:0/2202244971 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50800a5a50 con 0x55d97e7148b0 2024-04-26T22:04:56.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.747+0000 7f506e7fc640 1 -- 172.21.15.55:0/2202244971 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50800b6070 con 0x55d97e7148b0 2024-04-26T22:04:56.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.747+0000 7f506e7fc640 1 -- 172.21.15.55:0/2202244971 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50800a5820 con 0x55d97e7148b0 2024-04-26T22:04:56.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.748+0000 7f50919bb740 1 -- 172.21.15.55:0/2202244971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 msgr2=0x55d97e71f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:56.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.748+0000 7f50919bb740 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71f180 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f508009cb90 tx=0x7f5080004a40 comp rx=0 tx=0).stop 2024-04-26T22:04:56.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.749+0000 7f50919bb740 1 -- 172.21.15.55:0/2202244971 shutdown_connections 2024-04-26T22:04:56.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.749+0000 7f50919bb740 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 0x55d97e712e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.749+0000 7f50919bb740 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e726370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.749+0000 7f50919bb740 1 --2- 172.21.15.55:0/2202244971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e71f180 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.749+0000 7f50919bb740 1 -- 172.21.15.55:0/2202244971 >> 172.21.15.55:0/2202244971 conn(0x55d97e70ff20 msgr2=0x55d97e71ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:56.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.752+0000 7f50919bb740 1 -- 172.21.15.55:0/2202244971 shutdown_connections 2024-04-26T22:04:56.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.752+0000 7f50919bb740 1 -- 172.21.15.55:0/2202244971 wait complete. 2024-04-26T22:04:56.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.753+0000 7f50919bb740 1 -- start start 2024-04-26T22:04:56.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.753+0000 7f50919bb740 1 Processor -- start 2024-04-26T22:04:56.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.754+0000 7f50919bb740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e7236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.754+0000 7f50919bb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e723c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.754+0000 7f50919bb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 0x55d97e724c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.755+0000 7f50919bb740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f507400cd10 con 0x55d97e7148b0 2024-04-26T22:04:56.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.755+0000 7f50919bb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f507400e180 con 0x55d97e71a410 2024-04-26T22:04:56.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.755+0000 7f50919bb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f507400d3e0 con 0x55d97e7152a0 2024-04-26T22:04:56.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.755+0000 7f5087fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e723c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.755+0000 7f50919bb740 10 client.0 _ll_register_callbacks cb 0x7f508009cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:56.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.755+0000 7f5087fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e723c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36360/0 (socket says 172.21.15.55:36360) 2024-04-26T22:04:56.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.755+0000 7f5087fff640 1 -- 172.21.15.55:0/2906911098 learned_addr learned my addr 172.21.15.55:0/2906911098 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:56.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.755+0000 7f508ca35640 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e7236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.756+0000 7f5087fff640 1 -- 172.21.15.55:0/2906911098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 msgr2=0x55d97e724c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.756+0000 7f5087fff640 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 0x55d97e724c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.756+0000 7f5087fff640 1 -- 172.21.15.55:0/2906911098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 msgr2=0x55d97e7236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.756+0000 7f5087fff640 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e7236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.756+0000 7f5087fff640 1 -- 172.21.15.55:0/2906911098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97e725220 con 0x55d97e7152a0 2024-04-26T22:04:56.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.758+0000 7f5087fff640 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e723c20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f507c013010 tx=0x7f507c013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9739a0a233ddfd23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:56.760 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.758+0000 7f5067fff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:04:56.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.758+0000 7f5067fff640 1 -- 172.21.15.55:0/2906911098 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c026920 con 0x55d97e7152a0 2024-04-26T22:04:56.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.759+0000 7f5067fff640 1 -- 172.21.15.55:0/2906911098 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f507c004020 con 0x55d97e7152a0 2024-04-26T22:04:56.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.759+0000 7f50919bb740 1 -- 172.21.15.55:0/2906911098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50800b6070 con 0x55d97e7152a0 2024-04-26T22:04:56.761 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.761+0000 7f5067fff640 1 -- 172.21.15.55:0/2906911098 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f507c037050 con 0x55d97e7152a0 2024-04-26T22:04:56.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.761+0000 7f50919bb740 10 client.26299 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:56.762 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.762+0000 7f50919bb740 1 -- 172.21.15.55:0/2906911098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d97e7265d0 con 0x55d97e7152a0 2024-04-26T22:04:56.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.762+0000 7f5067fff640 1 -- 172.21.15.55:0/2906911098 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f507c01dd00 con 0x55d97e7152a0 2024-04-26T22:04:56.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.763+0000 7f5067fff640 1 -- 172.21.15.55:0/2906911098 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f507c026070 con 0x55d97e7152a0 2024-04-26T22:04:56.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.763+0000 7f50919bb740 10 client.26299 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:56.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.763+0000 7f50919bb740 10 client.26299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:56.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.763+0000 7f50919bb740 20 client.26299 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:56.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.763+0000 7f50919bb740 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d97e791c90 0x55d97e7940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:56.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.763+0000 7f50919bb740 4 client.26299 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:04:56.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.763+0000 7f50919bb740 1 -- 172.21.15.55:0/2906911098 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f50800b6070 con 0x55d97e791c90 2024-04-26T22:04:56.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.819+0000 7f508ca35640 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d97e791c90 0x55d97e7940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:56.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.826+0000 7f5067fff640 10 client.26299 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:56.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:56.826+0000 7f508ca35640 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d97e791c90 0x55d97e7940b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca85b372f5a9340 server_cookie=99a9c1dbc94806c2 in_seq=0 out_seq=0 2024-04-26T22:04:57.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.541+0000 7f5067fff640 1 -- 172.21.15.55:0/2906911098 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f50800b6070 con 0x55d97e791c90 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.541+0000 7f5067fff640 10 client.26299 handle_command_reply: tid=0 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.541+0000 7f50919bb740 1 client.26299 shutdown 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.541+0000 7f50919bb740 1 -- 172.21.15.55:0/2906911098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 msgr2=0x55d97e723c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.541+0000 7f50919bb740 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e723c20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f507c013010 tx=0x7f507c013040 comp rx=0 tx=0).stop 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 1 -- 172.21.15.55:0/2906911098 shutdown_connections 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d97e71a410 0x55d97e724c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d97e7152a0 0x55d97e723c20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d97e791c90 0x55d97e7940b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 1 --2- 172.21.15.55:0/2906911098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d97e7148b0 0x55d97e7236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 1 -- 172.21.15.55:0/2906911098 >> 172.21.15.55:0/2906911098 conn(0x55d97e70ff20 msgr2=0x55d97e70e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:57.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f5067fff640 0 client.26299 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:57.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 1 -- 172.21.15.55:0/2906911098 shutdown_connections 2024-04-26T22:04:57.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 1 -- 172.21.15.55:0/2906911098 wait complete. 2024-04-26T22:04:57.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.542+0000 7f50919bb740 20 client.26299 trim_cache size 0 max 16384 2024-04-26T22:04:57.549 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:04:57.549 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:04:57.549 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "19edcd11-a03a-4ea7-83e0-cdd57ef13718", 2024-04-26T22:04:57.549 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:04:57.550 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:04:57.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.548+0000 7f508ec99640 1 -- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50700a3e40 msgr2=0x7f50700a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:57.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.549+0000 7f508ec99640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50700a3e40 0x7f50700a6300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f50881a8eb0 tx=0x7f507c005f90 comp rx=0 tx=0).stop 2024-04-26T22:04:57.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.549+0000 7f508ec99640 1 -- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5088109930 msgr2=0x7f50881a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:57.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.549+0000 7f508ec99640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5088109930 0x7f50881a79f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f508000bd70 tx=0x7f508000be60 comp rx=0 tx=0).stop 2024-04-26T22:04:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.553+0000 7f508ec99640 1 -- 172.21.15.55:0/3399262904 shutdown_connections 2024-04-26T22:04:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.553+0000 7f508ec99640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50700a3e40 0x7f50700a6300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.553+0000 7f508ec99640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f508810ec70 0x7f50881aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.553+0000 7f508ec99640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f508810c2a0 0x7f50881a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.553+0000 7f508ec99640 1 --2- 172.21.15.55:0/3399262904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5088109930 0x7f50881a79f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.553+0000 7f508ec99640 1 -- 172.21.15.55:0/3399262904 >> 172.21.15.55:0/3399262904 conn(0x7f508806f7c0 msgr2=0x7f508810e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:57.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.558+0000 7f508ec99640 1 -- 172.21.15.55:0/3399262904 shutdown_connections 2024-04-26T22:04:57.564 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.561+0000 7f508ec99640 1 -- 172.21.15.55:0/3399262904 wait complete. 2024-04-26T22:04:57.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:04:57.618 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:04:57.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:57 smithi055 ceph-mon[29221]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 796 MiB/s wr, 1.55k op/s 2024-04-26T22:04:57.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:57 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:04:57.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:57 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:57.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:57 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:04:57.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.983+0000 7f4e8b664640 1 -- 172.21.15.55:0/3199765431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e84102350 msgr2=0x7f4e84106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:57.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.983+0000 7f4e8b664640 1 --2- 172.21.15.55:0/3199765431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e84102350 0x7f4e84106bf0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009860 tx=0x7f4e8002a340 comp rx=0 tx=0).stop 2024-04-26T22:04:57.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.984+0000 7f4e8b664640 1 -- 172.21.15.55:0/3199765431 shutdown_connections 2024-04-26T22:04:57.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.984+0000 7f4e8b664640 1 --2- 172.21.15.55:0/3199765431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e84102350 0x7f4e84106bf0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.984+0000 7f4e8b664640 1 --2- 172.21.15.55:0/3199765431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e84101990 0x7f4e84101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.984+0000 7f4e8b664640 1 --2- 172.21.15.55:0/3199765431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e84100c30 0x7f4e84101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.984+0000 7f4e8b664640 1 -- 172.21.15.55:0/3199765431 >> 172.21.15.55:0/3199765431 conn(0x7f4e840fc560 msgr2=0x7f4e840fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.991+0000 7f4e8b664640 1 -- 172.21.15.55:0/3199765431 shutdown_connections 2024-04-26T22:04:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.991+0000 7f4e8b664640 1 -- 172.21.15.55:0/3199765431 wait complete. 2024-04-26T22:04:57.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.992+0000 7f4e8b664640 1 Processor -- start 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.994+0000 7f4e8b664640 1 -- start start 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e8b664640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e84100c30 0x7f4e8410d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e8b664640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e84101990 0x7f4e8410d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e8b664640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e84102350 0x7f4e84070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e8b664640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114b10 con 0x7f4e84102350 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e8b664640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114990 con 0x7f4e84101990 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e8b664640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84114c90 con 0x7f4e84100c30 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e89c01640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e84102350 0x7f4e84070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e89400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e84100c30 0x7f4e8410d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e89400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e84100c30 0x7f4e8410d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36374/0 (socket says 172.21.15.55:36374) 2024-04-26T22:04:57.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.995+0000 7f4e89400640 1 -- 172.21.15.55:0/847057285 learned_addr learned my addr 172.21.15.55:0/847057285 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:57.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.997+0000 7f4e88bff640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e84101990 0x7f4e8410d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.998+0000 7f4e89400640 1 -- 172.21.15.55:0/847057285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e84101990 msgr2=0x7f4e8410d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.998+0000 7f4e89400640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e84101990 0x7f4e8410d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.998+0000 7f4e89400640 1 -- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e84102350 msgr2=0x7f4e84070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.998+0000 7f4e89400640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e84102350 0x7f4e84070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:57.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:57.998+0000 7f4e89400640 1 -- 172.21.15.55:0/847057285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e840708b0 con 0x7f4e84100c30 2024-04-26T22:04:58.003 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.003+0000 7f4e89400640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e84100c30 0x7f4e8410d0a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400e9c0 tx=0x7f4e7400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:58.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.008+0000 7f4e7a7fc640 1 -- 172.21.15.55:0/847057285 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400ccc0 con 0x7f4e84100c30 2024-04-26T22:04:58.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.008+0000 7f4e8b664640 1 -- 172.21.15.55:0/847057285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e84070ba0 con 0x7f4e84100c30 2024-04-26T22:04:58.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.008+0000 7f4e8b664640 1 -- 172.21.15.55:0/847057285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e840710f0 con 0x7f4e84100c30 2024-04-26T22:04:58.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.008+0000 7f4e7a7fc640 1 -- 172.21.15.55:0/847057285 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e740054d0 con 0x7f4e84100c30 2024-04-26T22:04:58.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.010+0000 7f4e7a7fc640 1 -- 172.21.15.55:0/847057285 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74010430 con 0x7f4e84100c30 2024-04-26T22:04:58.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.015+0000 7f4e8e37e740 1 Processor -- start 2024-04-26T22:04:58.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.016+0000 7f4e8e37e740 1 -- start start 2024-04-26T22:04:58.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.016+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acb4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.017+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acb5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.017+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261acbc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.017+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56261ac18d80 con 0x56261acb50b0 2024-04-26T22:04:58.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.017+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56261abd28b0 con 0x56261acb5a70 2024-04-26T22:04:58.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.017+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56261ac14d40 con 0x56261acb3ee0 2024-04-26T22:04:58.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.019+0000 7f4e7a7fc640 1 -- 172.21.15.55:0/847057285 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4e74004640 con 0x7f4e84100c30 2024-04-26T22:04:58.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.020+0000 7f4e7a7fc640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e700a3d20 0x7f4e700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.020+0000 7f4e7a7fc640 1 -- 172.21.15.55:0/847057285 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4e740d0840 con 0x7f4e84100c30 2024-04-26T22:04:58.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.020+0000 7f4e88bff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261acbc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.020+0000 7f4e89c01640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acb4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.021+0000 7f4e89c01640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acb4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36378/0 (socket says 172.21.15.55:36378) 2024-04-26T22:04:58.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.021+0000 7f4e89c01640 1 -- 172.21.15.55:0/2151861692 learned_addr learned my addr 172.21.15.55:0/2151861692 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:58.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.021+0000 7f4e89c01640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e700a3d20 0x7f4e700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.023+0000 7f4e89400640 1 --2- 172.21.15.55:0/2151861692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acb5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.024+0000 7f4e89400640 1 -- 172.21.15.55:0/2151861692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 msgr2=0x56261acb4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:58.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.024+0000 7f4e89400640 1 --2- 172.21.15.55:0/2151861692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acb4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.024+0000 7f4e89400640 1 -- 172.21.15.55:0/2151861692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 msgr2=0x56261acbc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:58.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.024+0000 7f4e89400640 1 --2- 172.21.15.55:0/2151861692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261acbc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.024+0000 7f4e89400640 1 -- 172.21.15.55:0/2151861692 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261aba3a40 con 0x56261acb50b0 2024-04-26T22:04:58.028 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.025+0000 7f4e89400640 1 --2- 172.21.15.55:0/2151861692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acb5530 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4e740ce150 tx=0x7f4e7409c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4092ae0eb9790486 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:58.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.028+0000 7f4e6f7fe640 1 -- 172.21.15.55:0/2151861692 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7409c920 con 0x56261acb50b0 2024-04-26T22:04:58.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.028+0000 7f4e6f7fe640 1 -- 172.21.15.55:0/2151861692 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e740a0cd0 con 0x56261acb50b0 2024-04-26T22:04:58.030 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.028+0000 7f4e6f7fe640 1 -- 172.21.15.55:0/2151861692 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e740bbae0 con 0x56261acb50b0 2024-04-26T22:04:58.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.030+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2151861692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 msgr2=0x56261acb5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:58.031 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.030+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/2151861692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acb5530 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4e740ce150 tx=0x7f4e7409c340 comp rx=0 tx=0).stop 2024-04-26T22:04:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.037+0000 7f4e89c01640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e700a3d20 0x7f4e700a61e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56261acb4fb0 tx=0x7f4e8002a340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.039+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2151861692 shutdown_connections 2024-04-26T22:04:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.039+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/2151861692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261acbc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.039+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/2151861692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acb5530 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.039+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/2151861692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acb4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.039+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2151861692 >> 172.21.15.55:0/2151861692 conn(0x56261acaf3d0 msgr2=0x56261acaf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:58.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.045+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2151861692 shutdown_connections 2024-04-26T22:04:58.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.050+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2151861692 wait complete. 2024-04-26T22:04:58.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.050+0000 7f4e8e37e740 1 -- start start 2024-04-26T22:04:58.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.051+0000 7f4e8e37e740 1 Processor -- start 2024-04-26T22:04:58.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.052+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.052+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb50b0 0x56261acbe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.052 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.052+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb5a70 0x56261acc5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.052+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56261acbca20 con 0x56261acb3ee0 2024-04-26T22:04:58.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.052+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56261ab9fd10 con 0x56261acb50b0 2024-04-26T22:04:58.053 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.053+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56261acbcba0 con 0x56261acb5a70 2024-04-26T22:04:58.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.053+0000 7f4e89c01640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb5a70 0x56261acc5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.053+0000 7f4e89c01640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb5a70 0x56261acc5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36394/0 (socket says 172.21.15.55:36394) 2024-04-26T22:04:58.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.053+0000 7f4e89c01640 1 -- 172.21.15.55:0/3321633508 learned_addr learned my addr 172.21.15.55:0/3321633508 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:58.054 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.053+0000 7f4e8e37e740 10 client.0 _ll_register_callbacks cb 0x56261abcc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:58.055 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.054+0000 7f4e89400640 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.055+0000 7f4e89c01640 1 -- 172.21.15.55:0/3321633508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb50b0 msgr2=0x56261acbe910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:04:58.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.055+0000 7f4e89c01640 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb50b0 0x56261acbe910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.055+0000 7f4e89c01640 1 -- 172.21.15.55:0/3321633508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 msgr2=0x56261acbe3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:58.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.055+0000 7f4e89c01640 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.055+0000 7f4e89c01640 1 -- 172.21.15.55:0/3321633508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261a9f47d0 con 0x56261acb5a70 2024-04-26T22:04:58.056 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.055+0000 7f4e89400640 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:04:58.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.056+0000 7f4e89c01640 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb5a70 0x56261acc5950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e800058c0 tx=0x7f4e80005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac35d5095397776 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:58.057 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.057+0000 7f4e6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:04:58.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.058+0000 7f4e6cff9640 1 -- 172.21.15.55:0/3321633508 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80048040 con 0x56261acb5a70 2024-04-26T22:04:58.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.058+0000 7f4e6cff9640 1 -- 172.21.15.55:0/3321633508 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4e80003d70 con 0x56261acb5a70 2024-04-26T22:04:58.058 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.058+0000 7f4e8e37e740 1 -- 172.21.15.55:0/3321633508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e740a0cd0 con 0x56261acb5a70 2024-04-26T22:04:58.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.060+0000 7f4e6cff9640 1 -- 172.21.15.55:0/3321633508 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4e80037040 con 0x56261acb5a70 2024-04-26T22:04:58.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.061+0000 7f4e8e37e740 10 client.26311 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:58.061 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.061+0000 7f4e8e37e740 1 -- 172.21.15.55:0/3321633508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56261a92a3f0 con 0x56261acb5a70 2024-04-26T22:04:58.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.063+0000 7f4e6cff9640 1 -- 172.21.15.55:0/3321633508 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e740a0cd0 con 0x56261acb5a70 2024-04-26T22:04:58.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.063+0000 7f4e6cff9640 1 -- 172.21.15.55:0/3321633508 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4e8004b030 con 0x56261acb5a70 2024-04-26T22:04:58.064 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.064+0000 7f4e8e37e740 10 client.26311 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:58.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.064+0000 7f4e8e37e740 10 client.26311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:58.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.064+0000 7f4e8e37e740 20 client.26311 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:58.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.064+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56261acc9f90 0x56261accc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.064+0000 7f4e8e37e740 4 client.26311 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:04:58.065 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.064+0000 7f4e8e37e740 1 -- 172.21.15.55:0/3321633508 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56261ac26990 con 0x56261acc9f90 2024-04-26T22:04:58.068 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.068+0000 7f4e89400640 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56261acc9f90 0x56261accc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.074+0000 7f4e6cff9640 10 client.26311 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:58.075 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.074+0000 7f4e89400640 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56261acc9f90 0x56261accc3d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53fe142f4bf1d10 server_cookie=ed7096403f71f430 in_seq=0 out_seq=0 2024-04-26T22:04:58.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:57 smithi165 ceph-mon[33296]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 796 MiB/s wr, 1.55k op/s 2024-04-26T22:04:58.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:57 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:04:58.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:57 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:58.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:57 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:04:58.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.083+0000 7f4e6cff9640 1 -- 172.21.15.55:0/3321633508 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56261ac26990 con 0x56261acc9f90 2024-04-26T22:04:58.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.083+0000 7f4e6cff9640 10 client.26311 handle_command_reply: tid=0 2024-04-26T22:04:58.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.083+0000 7f4e8e37e740 1 client.26311 shutdown 2024-04-26T22:04:58.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.089+0000 7f4e8e37e740 1 -- 172.21.15.55:0/3321633508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb5a70 msgr2=0x56261acc5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:58.089 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.089+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb5a70 0x56261acc5950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e800058c0 tx=0x7f4e80005f90 comp rx=0 tx=0).stop 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 1 -- 172.21.15.55:0/3321633508 shutdown_connections 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb5a70 0x56261acc5950 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb50b0 0x56261acbe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56261acc9f90 0x56261accc3d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/3321633508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 1 -- 172.21.15.55:0/3321633508 >> 172.21.15.55:0/3321633508 conn(0x56261acaf3d0 msgr2=0x56261acae550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e6cff9640 0 client.26311 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 1 -- 172.21.15.55:0/3321633508 shutdown_connections 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 1 -- 172.21.15.55:0/3321633508 wait complete. 2024-04-26T22:04:58.098 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.097+0000 7f4e8e37e740 20 client.26311 trim_cache size 0 max 16384 2024-04-26T22:04:58.155 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.153+0000 7f4e8e37e740 1 Processor -- start 2024-04-26T22:04:58.156 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.156+0000 7f4e8e37e740 1 -- start start 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.156+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.156+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acbe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261acc5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e68010f90 con 0x56261acb50b0 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e89400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e88bff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acbe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6800e860 con 0x56261acb5a70 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6800c6b0 con 0x56261acb3ee0 2024-04-26T22:04:58.157 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e89400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:36398/0 (socket says 172.21.15.55:36398) 2024-04-26T22:04:58.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e89400640 1 -- 172.21.15.55:0/2107359856 learned_addr learned my addr 172.21.15.55:0/2107359856 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:58.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e89400640 1 -- 172.21.15.55:0/2107359856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 msgr2=0x56261acc5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:58.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e89400640 1 --2- 172.21.15.55:0/2107359856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261acc5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e89400640 1 -- 172.21.15.55:0/2107359856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 msgr2=0x56261acbe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:58.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e89400640 1 --2- 172.21.15.55:0/2107359856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acbe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.157+0000 7f4e89400640 1 -- 172.21.15.55:0/2107359856 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261a8ea0f0 con 0x56261acb3ee0 2024-04-26T22:04:58.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.163+0000 7f4e89400640 1 --2- 172.21.15.55:0/2107359856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400c6a0 tx=0x7f4e74009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef97482a2bac2209 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:58.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.164+0000 7f4e6effd640 1 -- 172.21.15.55:0/2107359856 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e740a0cd0 con 0x56261acb3ee0 2024-04-26T22:04:58.164 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.164+0000 7f4e6effd640 1 -- 172.21.15.55:0/2107359856 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e7409c920 con 0x56261acb3ee0 2024-04-26T22:04:58.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.165+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2107359856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 msgr2=0x56261acbe3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:58.167 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.165+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/2107359856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400c6a0 tx=0x7f4e74009f90 comp rx=0 tx=0).stop 2024-04-26T22:04:58.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.168+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2107359856 shutdown_connections 2024-04-26T22:04:58.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.168+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/2107359856 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261acc5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.168+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/2107359856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb50b0 0x56261acbe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.168+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/2107359856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb3ee0 0x56261acbe3d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.168+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2107359856 >> 172.21.15.55:0/2107359856 conn(0x56261acaf3d0 msgr2=0x56261acb80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:58.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.174+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2107359856 shutdown_connections 2024-04-26T22:04:58.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.179+0000 7f4e8e37e740 1 -- 172.21.15.55:0/2107359856 wait complete. 2024-04-26T22:04:58.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.179+0000 7f4e8e37e740 1 -- start start 2024-04-26T22:04:58.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.179+0000 7f4e8e37e740 1 Processor -- start 2024-04-26T22:04:58.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.180+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 0x56261acc0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.180 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.180+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb50b0 0x56261acc0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.180+0000 7f4e8e37e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261ad2e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.180+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6800cd20 con 0x56261acb3ee0 2024-04-26T22:04:58.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.180+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6800e190 con 0x56261acb5a70 2024-04-26T22:04:58.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.180+0000 7f4e8e37e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6800d3f0 con 0x56261acb50b0 2024-04-26T22:04:58.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.180+0000 7f4e8e37e740 10 client.0 _ll_register_callbacks cb 0x7f4e800096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:04:58.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.183+0000 7f4e89c01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261ad2e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.183+0000 7f4e89c01640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261ad2e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:35572/0 (socket says 172.21.15.55:35572) 2024-04-26T22:04:58.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.183+0000 7f4e89c01640 1 -- 172.21.15.55:0/945165314 learned_addr learned my addr 172.21.15.55:0/945165314 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:04:58.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.184+0000 7f4e88bff640 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb50b0 0x56261acc0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.185+0000 7f4e89c01640 1 -- 172.21.15.55:0/945165314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb50b0 msgr2=0x56261acc0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:58.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.185+0000 7f4e89c01640 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb50b0 0x56261acc0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.185+0000 7f4e89c01640 1 -- 172.21.15.55:0/945165314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 msgr2=0x56261acc0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:04:58.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.185+0000 7f4e89c01640 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 0x56261acc0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.185+0000 7f4e89c01640 1 -- 172.21.15.55:0/945165314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261abd94b0 con 0x56261acb5a70 2024-04-26T22:04:58.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.193+0000 7f4e89c01640 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261ad2e540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e800095f0 tx=0x7f4e80030c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e1f465f5642fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:04:58.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.193+0000 7f4e6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:04:58.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.195+0000 7f4e6cff9640 1 -- 172.21.15.55:0/945165314 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80003cf0 con 0x56261acb5a70 2024-04-26T22:04:58.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.195+0000 7f4e6cff9640 1 -- 172.21.15.55:0/945165314 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4e800324e0 con 0x56261acb5a70 2024-04-26T22:04:58.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.196+0000 7f4e8e37e740 1 -- 172.21.15.55:0/945165314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56261ac20800 con 0x56261acb5a70 2024-04-26T22:04:58.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.203+0000 7f4e6cff9640 1 -- 172.21.15.55:0/945165314 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4e80003e90 con 0x56261acb5a70 2024-04-26T22:04:58.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.206+0000 7f4e8e37e740 10 client.26288 fetch_fsmap learned FSMap version 23 2024-04-26T22:04:58.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.207+0000 7f4e8e37e740 1 -- 172.21.15.55:0/945165314 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56261ad2fab0 con 0x56261acb5a70 2024-04-26T22:04:58.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.210+0000 7f4e6cff9640 1 -- 172.21.15.55:0/945165314 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56261ac20800 con 0x56261acb5a70 2024-04-26T22:04:58.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.211+0000 7f4e6cff9640 1 -- 172.21.15.55:0/945165314 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4e8004b040 con 0x56261acb5a70 2024-04-26T22:04:58.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.217+0000 7f4e8e37e740 10 client.26288 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:04:58.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.217+0000 7f4e8e37e740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:04:58.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.217+0000 7f4e8e37e740 20 client.26288 populate_metadata read hostname 'smithi055' 2024-04-26T22:04:58.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.217+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56261ad303d0 0x56261ad327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:04:58.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.217+0000 7f4e8e37e740 4 client.26288 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:04:58.217 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.217+0000 7f4e8e37e740 1 -- 172.21.15.55:0/945165314 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56261ab9b4a0 con 0x56261ad303d0 2024-04-26T22:04:58.222 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.221+0000 7f4e89400640 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56261ad303d0 0x56261ad327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:04:58.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.231+0000 7f4e89400640 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56261ad303d0 0x56261ad327f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab9de3a5672897a server_cookie=987dbb32eb259db4 in_seq=0 out_seq=0 2024-04-26T22:04:58.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.231+0000 7f4e6cff9640 10 client.26288 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:58.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.239+0000 7f4e6cff9640 1 -- 172.21.15.55:0/945165314 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x56261ab9b4a0 con 0x56261ad303d0 2024-04-26T22:04:58.240 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.239+0000 7f4e6cff9640 10 client.26288 handle_command_reply: tid=0 2024-04-26T22:04:58.241 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.241+0000 7f4e8e37e740 1 client.26288 shutdown 2024-04-26T22:04:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.246+0000 7f4e8e37e740 1 -- 172.21.15.55:0/945165314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 msgr2=0x56261ad2e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.246+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261ad2e540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e800095f0 tx=0x7f4e80030c50 comp rx=0 tx=0).stop 2024-04-26T22:04:58.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.249+0000 7f4e8e37e740 1 -- 172.21.15.55:0/945165314 shutdown_connections 2024-04-26T22:04:58.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.249+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56261acb5a70 0x56261ad2e540 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.249+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56261acb50b0 0x56261acc0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.249+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x56261ad303d0 0x56261ad327f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.249 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.249+0000 7f4e8e37e740 1 --2- 172.21.15.55:0/945165314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56261acb3ee0 0x56261acc0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.250 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.249+0000 7f4e8e37e740 1 -- 172.21.15.55:0/945165314 >> 172.21.15.55:0/945165314 conn(0x56261acaf3d0 msgr2=0x56261acb8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:58.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.251+0000 7f4e6cff9640 0 client.26288 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:04:58.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.252+0000 7f4e8e37e740 1 -- 172.21.15.55:0/945165314 shutdown_connections 2024-04-26T22:04:58.253 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.252+0000 7f4e8e37e740 1 -- 172.21.15.55:0/945165314 wait complete. 2024-04-26T22:04:58.255 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.255+0000 7f4e8e37e740 20 client.26288 trim_cache size 0 max 16384 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (15 inodes in the stack)", 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout: "19edcd11-a03a-4ea7-83e0-cdd57ef13718": { 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "19edcd11-a03a-4ea7-83e0-cdd57ef13718", 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:04:58.257 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:04:58.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:57 smithi064 ceph-mon[32765]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 796 MiB/s wr, 1.55k op/s 2024-04-26T22:04:58.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:57 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:04:58.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:57 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:58.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:57 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:04:58.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.264+0000 7f4e8b664640 1 -- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e700a3d20 msgr2=0x7f4e700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:58.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.264+0000 7f4e8b664640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e700a3d20 0x7f4e700a61e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56261acb4fb0 tx=0x7f4e8002a340 comp rx=0 tx=0).stop 2024-04-26T22:04:58.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.264+0000 7f4e8b664640 1 -- 172.21.15.55:0/847057285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e84100c30 msgr2=0x7f4e8410d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:04:58.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.264+0000 7f4e8b664640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e84100c30 0x7f4e8410d0a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400e9c0 tx=0x7f4e7400ee80 comp rx=0 tx=0).stop 2024-04-26T22:04:58.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.265+0000 7f4e8b664640 1 -- 172.21.15.55:0/847057285 shutdown_connections 2024-04-26T22:04:58.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.265+0000 7f4e8b664640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f4e700a3d20 0x7f4e700a61e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.266 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.265+0000 7f4e8b664640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e84102350 0x7f4e84070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.265+0000 7f4e8b664640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e84101990 0x7f4e8410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.265+0000 7f4e8b664640 1 --2- 172.21.15.55:0/847057285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e84100c30 0x7f4e8410d0a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:04:58.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.265+0000 7f4e8b664640 1 -- 172.21.15.55:0/847057285 >> 172.21.15.55:0/847057285 conn(0x7f4e840fc560 msgr2=0x7f4e84102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:04:58.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.268+0000 7f4e8b664640 1 -- 172.21.15.55:0/847057285 shutdown_connections 2024-04-26T22:04:58.271 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:04:58.271+0000 7f4e8b664640 1 -- 172.21.15.55:0/847057285 wait complete. 2024-04-26T22:04:58.324 INFO:tasks.cephfs.filesystem:scrub status for tag:19edcd11-a03a-4ea7-83e0-cdd57ef13718 - {'path': '/', 'tag': '19edcd11-a03a-4ea7-83e0-cdd57ef13718', 'options': 'recursive,force'} 2024-04-26T22:04:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:58 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:04:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:58 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:58 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:04:59.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:58 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:04:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:58 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:04:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:58 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:00.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:59 smithi165 ceph-mon[33296]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.98k op/s 2024-04-26T22:05:00.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:04:59 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:00.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:59 smithi055 ceph-mon[29221]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.98k op/s 2024-04-26T22:05:00.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:04:59 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:59 smithi064 ceph-mon[32765]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.98k op/s 2024-04-26T22:05:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:04:59 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:02.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:01 smithi165 ceph-mon[33296]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 816 MiB/s wr, 2.09k op/s 2024-04-26T22:05:02.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:01 smithi055 ceph-mon[29221]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 816 MiB/s wr, 2.09k op/s 2024-04-26T22:05:02.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:01 smithi064 ceph-mon[32765]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 816 MiB/s wr, 2.09k op/s 2024-04-26T22:05:04.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:03 smithi165 ceph-mon[33296]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 2.38k op/s 2024-04-26T22:05:04.192 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:03 smithi055 ceph-mon[29221]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 2.38k op/s 2024-04-26T22:05:04.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:03 smithi064 ceph-mon[32765]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 2.38k op/s 2024-04-26T22:05:06.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:05 smithi165 ceph-mon[33296]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 818 MiB/s wr, 2.24k op/s 2024-04-26T22:05:06.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:05 smithi055 ceph-mon[29221]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 818 MiB/s wr, 2.24k op/s 2024-04-26T22:05:06.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:05 smithi064 ceph-mon[32765]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 818 MiB/s wr, 2.24k op/s 2024-04-26T22:05:07.157 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:06 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:05:07.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:06 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:05:07.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:06 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:05:08.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:07 smithi055 ceph-mon[29221]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 786 MiB/s wr, 2.07k op/s 2024-04-26T22:05:08.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:07 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:05:08.227 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:07 smithi165 ceph-mon[33296]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 786 MiB/s wr, 2.07k op/s 2024-04-26T22:05:08.227 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:07 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:05:08.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:07 smithi064 ceph-mon[32765]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 786 MiB/s wr, 2.07k op/s 2024-04-26T22:05:08.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:07 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:05:09.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:08 smithi055 ceph-mon[29221]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 882 MiB/s wr, 2.33k op/s 2024-04-26T22:05:09.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:08 smithi064 ceph-mon[32765]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 882 MiB/s wr, 2.33k op/s 2024-04-26T22:05:09.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:08 smithi165 ceph-mon[33296]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 882 MiB/s wr, 2.33k op/s 2024-04-26T22:05:11.329 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:11 smithi165 ceph-mon[33296]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 2.32k op/s 2024-04-26T22:05:11.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:11 smithi055 ceph-mon[29221]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 2.32k op/s 2024-04-26T22:05:11.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:11 smithi064 ceph-mon[32765]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 836 MiB/s wr, 2.32k op/s 2024-04-26T22:05:13.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:13 smithi165 ceph-mon[33296]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 859 MiB/s wr, 2.63k op/s 2024-04-26T22:05:13.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:13 smithi055 ceph-mon[29221]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 859 MiB/s wr, 2.63k op/s 2024-04-26T22:05:13.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:13 smithi064 ceph-mon[32765]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 859 MiB/s wr, 2.63k op/s 2024-04-26T22:05:15.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:15 smithi165 ceph-mon[33296]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 793 MiB/s wr, 2.60k op/s 2024-04-26T22:05:15.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:15 smithi055 ceph-mon[29221]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 793 MiB/s wr, 2.60k op/s 2024-04-26T22:05:15.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:15 smithi064 ceph-mon[32765]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 793 MiB/s wr, 2.60k op/s 2024-04-26T22:05:17.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:17 smithi055 ceph-mon[29221]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 MiB/s wr, 2.60k op/s 2024-04-26T22:05:17.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:17 smithi064 ceph-mon[32765]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 MiB/s wr, 2.60k op/s 2024-04-26T22:05:17.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:17 smithi165 ceph-mon[33296]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 MiB/s wr, 2.60k op/s 2024-04-26T22:05:19.444 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:19 smithi055 ceph-mon[29221]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 MiB/s wr, 2.81k op/s 2024-04-26T22:05:19.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:19 smithi064 ceph-mon[32765]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 MiB/s wr, 2.81k op/s 2024-04-26T22:05:19.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:19 smithi165 ceph-mon[33296]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 877 MiB/s wr, 2.81k op/s 2024-04-26T22:05:21.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:21 smithi055 ceph-mon[29221]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 830 MiB/s wr, 2.57k op/s 2024-04-26T22:05:21.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:21 smithi064 ceph-mon[32765]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 830 MiB/s wr, 2.57k op/s 2024-04-26T22:05:21.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:21 smithi165 ceph-mon[33296]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 830 MiB/s wr, 2.57k op/s 2024-04-26T22:05:23.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:23 smithi055 ceph-mon[29221]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 2.20k op/s 2024-04-26T22:05:23.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:23 smithi064 ceph-mon[32765]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 2.20k op/s 2024-04-26T22:05:23.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:23 smithi165 ceph-mon[33296]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 2.20k op/s 2024-04-26T22:05:25.436 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:25 smithi055 ceph-mon[29221]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 859 MiB/s wr, 1.70k op/s 2024-04-26T22:05:25.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:25 smithi064 ceph-mon[32765]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 859 MiB/s wr, 1.70k op/s 2024-04-26T22:05:25.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:25 smithi165 ceph-mon[33296]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 859 MiB/s wr, 1.70k op/s 2024-04-26T22:05:27.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:27 smithi055 ceph-mon[29221]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 773 MiB/s wr, 1.31k op/s 2024-04-26T22:05:27.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:27 smithi064 ceph-mon[32765]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 773 MiB/s wr, 1.31k op/s 2024-04-26T22:05:27.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:27 smithi165 ceph-mon[33296]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 773 MiB/s wr, 1.31k op/s 2024-04-26T22:05:28.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:05:28.325 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:05:28.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.763+0000 7f4765712640 1 -- 172.21.15.55:0/3302300421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f476010bf10 msgr2=0x7f476010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:28.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.763+0000 7f4765712640 1 --2- 172.21.15.55:0/3302300421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f476010bf10 0x7f476010e3a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f475402a440 comp rx=0 tx=0).stop 2024-04-26T22:05:28.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.768+0000 7f4765712640 1 -- 172.21.15.55:0/3302300421 shutdown_connections 2024-04-26T22:05:28.770 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.768+0000 7f4765712640 1 --2- 172.21.15.55:0/3302300421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476010ea10 0x7f4760119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.768+0000 7f4765712640 1 --2- 172.21.15.55:0/3302300421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f476010bf10 0x7f476010e3a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.768+0000 7f4765712640 1 --2- 172.21.15.55:0/3302300421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4760074760 0x7f476010b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.771 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.768+0000 7f4765712640 1 -- 172.21.15.55:0/3302300421 >> 172.21.15.55:0/3302300421 conn(0x7f476006f7d0 msgr2=0x7f4760071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:28.779 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.779+0000 7f4765712640 1 -- 172.21.15.55:0/3302300421 shutdown_connections 2024-04-26T22:05:28.783 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.783+0000 7f4765712640 1 -- 172.21.15.55:0/3302300421 wait complete. 2024-04-26T22:05:28.784 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.784+0000 7f4765712640 1 Processor -- start 2024-04-26T22:05:28.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.787+0000 7f4765712640 1 -- start start 2024-04-26T22:05:28.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.788+0000 7f4765712640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4760074760 0x7f47601a7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.788+0000 7f4765712640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476010bf10 0x7f47601a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.788+0000 7f4765712640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476010ea10 0x7f47601aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.788+0000 7f4765712640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f476011d180 con 0x7f4760074760 2024-04-26T22:05:28.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.788+0000 7f4765712640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f476011d000 con 0x7f476010ea10 2024-04-26T22:05:28.790 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.788+0000 7f4765712640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f476011d300 con 0x7f476010bf10 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.788+0000 7f475ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476010ea10 0x7f47601aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.789+0000 7f475effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4760074760 0x7f47601a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.789+0000 7f475ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476010ea10 0x7f47601aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:32788/0 (socket says 172.21.15.55:32788) 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.789+0000 7f475ffff640 1 -- 172.21.15.55:0/2754878624 learned_addr learned my addr 172.21.15.55:0/2754878624 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.790+0000 7f475ffff640 1 -- 172.21.15.55:0/2754878624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476010bf10 msgr2=0x7f47601a7d90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.790+0000 7f475ffff640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476010bf10 0x7f47601a7d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.790+0000 7f475ffff640 1 -- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4760074760 msgr2=0x7f47601a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.790+0000 7f475ffff640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4760074760 0x7f47601a7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.791 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.790+0000 7f475ffff640 1 -- 172.21.15.55:0/2754878624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47601af470 con 0x7f476010ea10 2024-04-26T22:05:28.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.794+0000 7f475ffff640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476010ea10 0x7f47601aee00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f475000b3f0 tx=0x7f475000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:28.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.797+0000 7f4747fff640 1 -- 172.21.15.55:0/2754878624 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750015020 con 0x7f476010ea10 2024-04-26T22:05:28.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.797+0000 7f4747fff640 1 -- 172.21.15.55:0/2754878624 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4750011ce0 con 0x7f476010ea10 2024-04-26T22:05:28.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.797+0000 7f4747fff640 1 -- 172.21.15.55:0/2754878624 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750010da0 con 0x7f476010ea10 2024-04-26T22:05:28.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.799+0000 7f4765712640 1 -- 172.21.15.55:0/2754878624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47601af760 con 0x7f476010ea10 2024-04-26T22:05:28.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.799+0000 7f4765712640 1 -- 172.21.15.55:0/2754878624 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47601afce0 con 0x7f476010ea10 2024-04-26T22:05:28.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.805+0000 7f476841f740 1 Processor -- start 2024-04-26T22:05:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.807+0000 7f476841f740 1 -- start start 2024-04-26T22:05:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.807+0000 7f4747fff640 1 -- 172.21.15.55:0/2754878624 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4750018020 con 0x7f476010ea10 2024-04-26T22:05:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.808+0000 7f476841f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f5d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.808+0000 7f476841f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 0x55e0c9f5e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.808+0000 7f476841f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.808+0000 7f476841f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e0c9ec1d80 con 0x55e0c9f5ec40 2024-04-26T22:05:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.808+0000 7f476841f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0c9e7b8b0 con 0x55e0c9f5e280 2024-04-26T22:05:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.808+0000 7f476841f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0c9ebdd40 con 0x55e0c9f5d0b0 2024-04-26T22:05:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.809+0000 7f475e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.809+0000 7f475e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f654d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35316/0 (socket says 172.21.15.55:35316) 2024-04-26T22:05:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.809+0000 7f475e7fc640 1 -- 172.21.15.55:0/698689904 learned_addr learned my addr 172.21.15.55:0/698689904 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.809+0000 7f4747fff640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f47480a3e40 0x7f47480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.810+0000 7f4747fff640 1 -- 172.21.15.55:0/2754878624 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f47500cfd10 con 0x7f476010ea10 2024-04-26T22:05:28.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.813+0000 7f475ffff640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f47480a3e40 0x7f47480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.814+0000 7f475effd640 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 0x55e0c9f5e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.815 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.814+0000 7f475e7fc640 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f5d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.815+0000 7f475e7fc640 1 -- 172.21.15.55:0/698689904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 msgr2=0x55e0c9f5d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.815+0000 7f475e7fc640 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f5d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.815+0000 7f475e7fc640 1 -- 172.21.15.55:0/698689904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 msgr2=0x55e0c9f5e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.815+0000 7f475e7fc640 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 0x55e0c9f5e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.815+0000 7f475e7fc640 1 -- 172.21.15.55:0/698689904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c9e4ca40 con 0x55e0c9f5ec40 2024-04-26T22:05:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.816+0000 7f475e7fc640 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f654d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f47540096c0 tx=0x7f47540314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba2f4304012873e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.816+0000 7f47457fa640 1 -- 172.21.15.55:0/698689904 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754041020 con 0x55e0c9f5ec40 2024-04-26T22:05:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.817+0000 7f47457fa640 1 -- 172.21.15.55:0/698689904 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4754031cf0 con 0x55e0c9f5ec40 2024-04-26T22:05:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.818+0000 7f476841f740 1 -- 172.21.15.55:0/698689904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 msgr2=0x55e0c9f654d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.818+0000 7f476841f740 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f654d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f47540096c0 tx=0x7f47540314a0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.818+0000 7f475ffff640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f47480a3e40 0x7f47480a6300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f47500a4640 tx=0x7f4750010fe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.819+0000 7f476841f740 1 -- 172.21.15.55:0/698689904 shutdown_connections 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.819+0000 7f476841f740 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f654d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.819+0000 7f476841f740 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 0x55e0c9f5e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.819+0000 7f476841f740 1 --2- 172.21.15.55:0/698689904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f5d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.819+0000 7f476841f740 1 -- 172.21.15.55:0/698689904 >> 172.21.15.55:0/698689904 conn(0x55e0c9f585a0 msgr2=0x55e0c9f589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.823+0000 7f476841f740 1 -- 172.21.15.55:0/698689904 shutdown_connections 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.823+0000 7f476841f740 1 -- 172.21.15.55:0/698689904 wait complete. 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.824+0000 7f476841f740 1 -- start start 2024-04-26T22:05:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.824+0000 7f476841f740 1 Processor -- start 2024-04-26T22:05:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.824+0000 7f476841f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.824+0000 7f476841f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5e280 0x55e0c9f67af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.824+0000 7f476841f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.824+0000 7f476841f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e0c9f65c40 con 0x55e0c9f5ec40 2024-04-26T22:05:28.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.824+0000 7f476841f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0c9e48d10 con 0x55e0c9f5d0b0 2024-04-26T22:05:28.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.824+0000 7f476841f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e0c9f65dc0 con 0x55e0c9f5e280 2024-04-26T22:05:28.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.825+0000 7f476841f740 10 client.0 _ll_register_callbacks cb 0x55e0c9e75210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.831+0000 7f475e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.831+0000 7f475e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.831+0000 7f475e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35330/0 (socket says 172.21.15.55:35330) 2024-04-26T22:05:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.831+0000 7f475e7fc640 1 -- 172.21.15.55:0/436943452 learned_addr learned my addr 172.21.15.55:0/436943452 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.832+0000 7f475e7fc640 1 -- 172.21.15.55:0/436943452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5e280 msgr2=0x55e0c9f67af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.832+0000 7f475e7fc640 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5e280 0x55e0c9f67af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.832+0000 7f475e7fc640 1 -- 172.21.15.55:0/436943452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5d0b0 msgr2=0x55e0c9f675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.832+0000 7f475e7fc640 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.832+0000 7f475e7fc640 1 -- 172.21.15.55:0/436943452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c9c9d7d0 con 0x55e0c9f5ec40 2024-04-26T22:05:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.832+0000 7f475e7fc640 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6eac0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4754031f50 tx=0x7f475402a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1878824ebff97f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:28.842 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.842+0000 7f4742ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:05:28.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.842+0000 7f4742ff5640 1 -- 172.21.15.55:0/436943452 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475404b020 con 0x55e0c9f5ec40 2024-04-26T22:05:28.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.843+0000 7f476841f740 1 -- 172.21.15.55:0/436943452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4754031cf0 con 0x55e0c9f5ec40 2024-04-26T22:05:28.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.843+0000 7f4742ff5640 1 -- 172.21.15.55:0/436943452 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4754031a50 con 0x55e0c9f5ec40 2024-04-26T22:05:28.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.845+0000 7f4742ff5640 1 -- 172.21.15.55:0/436943452 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4754041070 con 0x55e0c9f5ec40 2024-04-26T22:05:28.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.846+0000 7f476841f740 10 client.17358 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:28.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.846+0000 7f476841f740 1 -- 172.21.15.55:0/436943452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e0c9bd33f0 con 0x55e0c9f5ec40 2024-04-26T22:05:28.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.848+0000 7f4742ff5640 1 -- 172.21.15.55:0/436943452 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4754031cf0 con 0x55e0c9f5ec40 2024-04-26T22:05:28.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.850+0000 7f4742ff5640 1 -- 172.21.15.55:0/436943452 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f475405a020 con 0x55e0c9f5ec40 2024-04-26T22:05:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.850+0000 7f476841f740 10 client.17358 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.850+0000 7f476841f740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.850+0000 7f476841f740 20 client.17358 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.850+0000 7f476841f740 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e0c9f83c30 0x55e0c9f86020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.850+0000 7f476841f740 4 client.17358 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.850+0000 7f476841f740 1 -- 172.21.15.55:0/436943452 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0c9ecf990 con 0x55e0c9f83c30 2024-04-26T22:05:28.852 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.851+0000 7f475effd640 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e0c9f83c30 0x55e0c9f86020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.866+0000 7f475effd640 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e0c9f83c30 0x55e0c9f86020 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8a22399973add9 server_cookie=338aed47fd9526fb in_seq=0 out_seq=0 2024-04-26T22:05:28.867 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.866+0000 7f4742ff5640 10 client.17358 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.873+0000 7f4742ff5640 1 -- 172.21.15.55:0/436943452 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0c9ecf990 con 0x55e0c9f83c30 2024-04-26T22:05:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.873+0000 7f4742ff5640 10 client.17358 handle_command_reply: tid=0 2024-04-26T22:05:28.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.876+0000 7f476841f740 1 client.17358 shutdown 2024-04-26T22:05:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.877+0000 7f476841f740 1 -- 172.21.15.55:0/436943452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 msgr2=0x55e0c9f6eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.877+0000 7f476841f740 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6eac0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4754031f50 tx=0x7f475402a440 comp rx=0 tx=0).stop 2024-04-26T22:05:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.878+0000 7f476841f740 1 -- 172.21.15.55:0/436943452 shutdown_connections 2024-04-26T22:05:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.878+0000 7f476841f740 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e0c9f83c30 0x55e0c9f86020 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.878+0000 7f476841f740 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6eac0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.878+0000 7f476841f740 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5e280 0x55e0c9f67af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.878+0000 7f476841f740 1 --2- 172.21.15.55:0/436943452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.878+0000 7f476841f740 1 -- 172.21.15.55:0/436943452 >> 172.21.15.55:0/436943452 conn(0x55e0c9f585a0 msgr2=0x55e0c9f576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.879+0000 7f4742ff5640 0 client.17358 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:28.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.879+0000 7f476841f740 1 -- 172.21.15.55:0/436943452 shutdown_connections 2024-04-26T22:05:28.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.880+0000 7f476841f740 1 -- 172.21.15.55:0/436943452 wait complete. 2024-04-26T22:05:28.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.882+0000 7f476841f740 20 client.17358 trim_cache size 0 max 16384 2024-04-26T22:05:28.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.958+0000 7f476841f740 1 Processor -- start 2024-04-26T22:05:28.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.958+0000 7f476841f740 1 -- start start 2024-04-26T22:05:28.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.958+0000 7f476841f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.958+0000 7f476841f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.958+0000 7f476841f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f57530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.958+0000 7f476841f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f47601b2550 con 0x55e0c9f5d0b0 2024-04-26T22:05:28.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.958+0000 7f476841f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47601b2c20 con 0x55e0c9f5e280 2024-04-26T22:05:28.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.958+0000 7f476841f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f47601b04f0 con 0x55e0c9f5ec40 2024-04-26T22:05:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.962+0000 7f475effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.963+0000 7f475e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.963+0000 7f475e7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35344/0 (socket says 172.21.15.55:35344) 2024-04-26T22:05:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.963+0000 7f475e7fc640 1 -- 172.21.15.55:0/2209808216 learned_addr learned my addr 172.21.15.55:0/2209808216 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:28.965 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.963+0000 7f475e7fc640 1 --2- 172.21.15.55:0/2209808216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f57530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.965+0000 7f475e7fc640 1 -- 172.21.15.55:0/2209808216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5ec40 msgr2=0x55e0c9f57530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.965+0000 7f475e7fc640 1 --2- 172.21.15.55:0/2209808216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f57530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.965+0000 7f475e7fc640 1 -- 172.21.15.55:0/2209808216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 msgr2=0x55e0c9f6ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.965+0000 7f475e7fc640 1 --2- 172.21.15.55:0/2209808216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.966+0000 7f475e7fc640 1 -- 172.21.15.55:0/2209808216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c9b930f0 con 0x55e0c9f5d0b0 2024-04-26T22:05:28.969 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.967+0000 7f475e7fc640 1 --2- 172.21.15.55:0/2209808216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4754031500 tx=0x7f4754000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc92d5a1a5de0d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.968+0000 7f4744ff9640 1 -- 172.21.15.55:0/2209808216 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475404b070 con 0x55e0c9f5d0b0 2024-04-26T22:05:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.969+0000 7f4744ff9640 1 -- 172.21.15.55:0/2209808216 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4754062070 con 0x55e0c9f5d0b0 2024-04-26T22:05:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.969+0000 7f4744ff9640 1 -- 172.21.15.55:0/2209808216 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754031850 con 0x55e0c9f5d0b0 2024-04-26T22:05:28.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.970+0000 7f476841f740 1 -- 172.21.15.55:0/2209808216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5d0b0 msgr2=0x55e0c9f675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:28.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.970+0000 7f476841f740 1 --2- 172.21.15.55:0/2209808216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4754031500 tx=0x7f4754000bc0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.975+0000 7f476841f740 1 -- 172.21.15.55:0/2209808216 shutdown_connections 2024-04-26T22:05:28.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.975+0000 7f476841f740 1 --2- 172.21.15.55:0/2209808216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f57530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.975+0000 7f476841f740 1 --2- 172.21.15.55:0/2209808216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.975+0000 7f476841f740 1 --2- 172.21.15.55:0/2209808216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f675b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.976+0000 7f476841f740 1 -- 172.21.15.55:0/2209808216 >> 172.21.15.55:0/2209808216 conn(0x55e0c9f585a0 msgr2=0x55e0c9f56e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:28.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.977+0000 7f476841f740 1 -- 172.21.15.55:0/2209808216 shutdown_connections 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.978+0000 7f476841f740 1 -- 172.21.15.55:0/2209808216 wait complete. 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 1 -- start start 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 1 Processor -- start 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f4760118270 con 0x55e0c9f5e280 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4760117cb0 con 0x55e0c9f5ec40 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4760118940 con 0x55e0c9f5d0b0 2024-04-26T22:05:28.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.979+0000 7f476841f740 10 client.0 _ll_register_callbacks cb 0x7f4754004980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:28.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.980+0000 7f475effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.980+0000 7f475effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35348/0 (socket says 172.21.15.55:35348) 2024-04-26T22:05:28.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.980+0000 7f475effd640 1 -- 172.21.15.55:0/3692949170 learned_addr learned my addr 172.21.15.55:0/3692949170 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:28.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.980+0000 7f475e7fc640 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.981+0000 7f475e7fc640 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:28.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.981+0000 7f475effd640 1 -- 172.21.15.55:0/3692949170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 msgr2=0x55e0c9f6c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:28.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.981+0000 7f475effd640 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f6c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.981+0000 7f475effd640 1 -- 172.21.15.55:0/3692949170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5ec40 msgr2=0x55e0c9f6db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:28.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.981+0000 7f475effd640 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:28.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.981+0000 7f475effd640 1 -- 172.21.15.55:0/3692949170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c9e824b0 con 0x55e0c9f5e280 2024-04-26T22:05:28.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.982+0000 7f475effd640 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6cb50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4758001290 tx=0x7f47580137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7e7d8520cebab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:28.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.982+0000 7f4742ff5640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:05:28.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.983+0000 7f4742ff5640 1 -- 172.21.15.55:0/3692949170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47580127a0 con 0x55e0c9f5e280 2024-04-26T22:05:28.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.983+0000 7f4742ff5640 1 -- 172.21.15.55:0/3692949170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f4758012940 con 0x55e0c9f5e280 2024-04-26T22:05:28.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.983+0000 7f476841f740 1 -- 172.21.15.55:0/3692949170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4754062070 con 0x55e0c9f5e280 2024-04-26T22:05:28.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.983+0000 7f4742ff5640 1 -- 172.21.15.55:0/3692949170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4758012b40 con 0x55e0c9f5e280 2024-04-26T22:05:28.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.983+0000 7f476841f740 10 client.17364 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:28.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.983+0000 7f476841f740 1 -- 172.21.15.55:0/3692949170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e0c9fea400 con 0x55e0c9f5e280 2024-04-26T22:05:28.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.984+0000 7f4742ff5640 1 -- 172.21.15.55:0/3692949170 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4754062070 con 0x55e0c9f5e280 2024-04-26T22:05:28.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.986+0000 7f4742ff5640 1 -- 172.21.15.55:0/3692949170 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f475802c3f0 con 0x55e0c9f5e280 2024-04-26T22:05:28.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.986+0000 7f476841f740 10 client.17364 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:28.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.986+0000 7f476841f740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:28.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.986+0000 7f476841f740 20 client.17364 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:28.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.986+0000 7f476841f740 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e0c9fead60 0x55e0c9fed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:28.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.986+0000 7f476841f740 4 client.17364 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:05:28.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.986+0000 7f476841f740 1 -- 172.21.15.55:0/3692949170 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e0c9ec9800 con 0x55e0c9fead60 2024-04-26T22:05:28.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:28.990+0000 7f475e7fc640 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e0c9fead60 0x55e0c9fed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:29.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.001+0000 7f475e7fc640 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e0c9fead60 0x55e0c9fed180 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7692a696d1a52e85 server_cookie=ee51570a03110d7b in_seq=0 out_seq=0 2024-04-26T22:05:29.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.001+0000 7f4742ff5640 10 client.17364 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:29.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.004+0000 7f4742ff5640 1 -- 172.21.15.55:0/3692949170 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e0c9ec9800 con 0x55e0c9fead60 2024-04-26T22:05:29.005 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.004+0000 7f4742ff5640 10 client.17364 handle_command_reply: tid=0 2024-04-26T22:05:29.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.004+0000 7f476841f740 1 client.17364 shutdown 2024-04-26T22:05:29.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.004+0000 7f476841f740 1 -- 172.21.15.55:0/3692949170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5e280 msgr2=0x55e0c9f6cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:29.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.004+0000 7f476841f740 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6cb50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4758001290 tx=0x7f47580137d0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 1 -- 172.21.15.55:0/3692949170 shutdown_connections 2024-04-26T22:05:29.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0c9f5ec40 0x55e0c9f6db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e0c9fead60 0x55e0c9fed180 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e0c9f5e280 0x55e0c9f6cb50 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 1 --2- 172.21.15.55:0/3692949170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e0c9f5d0b0 0x55e0c9f6c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f4742ff5640 0 client.17364 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 1 -- 172.21.15.55:0/3692949170 >> 172.21.15.55:0/3692949170 conn(0x55e0c9f585a0 msgr2=0x55e0c9f576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 1 -- 172.21.15.55:0/3692949170 shutdown_connections 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 1 -- 172.21.15.55:0/3692949170 wait complete. 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.005+0000 7f476841f740 20 client.17364 trim_cache size 0 max 16384 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:05:29.008 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:05:29.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 -- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f47480a3e40 msgr2=0x7f47480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:29.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f47480a3e40 0x7f47480a6300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f47500a4640 tx=0x7f4750010fe0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 -- 172.21.15.55:0/2754878624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476010ea10 msgr2=0x7f47601aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:29.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476010ea10 0x7f47601aee00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f475000b3f0 tx=0x7f475000b8b0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 -- 172.21.15.55:0/2754878624 shutdown_connections 2024-04-26T22:05:29.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f47480a3e40 0x7f47480a6300 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f476010ea10 0x7f47601aee00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476010bf10 0x7f47601a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 --2- 172.21.15.55:0/2754878624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4760074760 0x7f47601a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.006+0000 7f4765712640 1 -- 172.21.15.55:0/2754878624 >> 172.21.15.55:0/2754878624 conn(0x7f476006f7d0 msgr2=0x7f4760109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:29.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.007+0000 7f4765712640 1 -- 172.21.15.55:0/2754878624 shutdown_connections 2024-04-26T22:05:29.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.007+0000 7f4765712640 1 -- 172.21.15.55:0/2754878624 wait complete. 2024-04-26T22:05:29.036 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:05:29.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:05:29.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.291+0000 7f5f4bd33640 1 -- 172.21.15.55:0/70468198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4410a290 msgr2=0x7f5f44110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:29.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.291+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/70468198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4410a290 0x7f5f44110c70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009ca0 tx=0x7f5f3c02a8f0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.291+0000 7f5f4bd33640 1 -- 172.21.15.55:0/70468198 shutdown_connections 2024-04-26T22:05:29.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.291+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/70468198 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f441111b0 0x7f5f441135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.291+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/70468198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f4410a290 0x7f5f44110c70 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.291+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/70468198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f44109930 0x7f5f44109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.291+0000 7f5f4bd33640 1 -- 172.21.15.55:0/70468198 >> 172.21.15.55:0/70468198 conn(0x7f5f4406f7d0 msgr2=0x7f5f44071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.292+0000 7f5f4bd33640 1 -- 172.21.15.55:0/70468198 shutdown_connections 2024-04-26T22:05:29.293 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.292+0000 7f5f4bd33640 1 -- 172.21.15.55:0/70468198 wait complete. 2024-04-26T22:05:29.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.293+0000 7f5f4bd33640 1 Processor -- start 2024-04-26T22:05:29.294 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.293+0000 7f5f4bd33640 1 -- start start 2024-04-26T22:05:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.294+0000 7f5f4bd33640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f44109930 0x7f5f441a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.294+0000 7f5f4bd33640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4410a290 0x7f5f441a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.294+0000 7f5f4bd33640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f441111b0 0x7f5f441aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.294+0000 7f5f4bd33640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44116e20 con 0x7f5f441111b0 2024-04-26T22:05:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.294+0000 7f5f4bd33640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44116ca0 con 0x7f5f4410a290 2024-04-26T22:05:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.294+0000 7f5f4bd33640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44116fa0 con 0x7f5f44109930 2024-04-26T22:05:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.295+0000 7f5f4a2d0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f441111b0 0x7f5f441aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.295+0000 7f5f4a2d0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f441111b0 0x7f5f441aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35362/0 (socket says 172.21.15.55:35362) 2024-04-26T22:05:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.295+0000 7f5f4a2d0640 1 -- 172.21.15.55:0/4076439748 learned_addr learned my addr 172.21.15.55:0/4076439748 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:29.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.295+0000 7f5f492ce640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4410a290 0x7f5f441a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.297+0000 7f5f492ce640 1 -- 172.21.15.55:0/4076439748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f44109930 msgr2=0x7f5f441a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:29.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.297+0000 7f5f492ce640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f44109930 0x7f5f441a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.297+0000 7f5f492ce640 1 -- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f441111b0 msgr2=0x7f5f441aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:29.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.297+0000 7f5f492ce640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f441111b0 0x7f5f441aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.297+0000 7f5f492ce640 1 -- 172.21.15.55:0/4076439748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f441ab3e0 con 0x7f5f4410a290 2024-04-26T22:05:29.298 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.298+0000 7f5f492ce640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4410a290 0x7f5f441a3d00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c02aee0 tx=0x7f5f3c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:29.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.299+0000 7f5f3affd640 1 -- 172.21.15.55:0/4076439748 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c004040 con 0x7f5f4410a290 2024-04-26T22:05:29.299 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.299+0000 7f5f3affd640 1 -- 172.21.15.55:0/4076439748 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f3c02d070 con 0x7f5f4410a290 2024-04-26T22:05:29.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.299+0000 7f5f3affd640 1 -- 172.21.15.55:0/4076439748 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c02fbf0 con 0x7f5f4410a290 2024-04-26T22:05:29.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.299+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f441ab670 con 0x7f5f4410a290 2024-04-26T22:05:29.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.300+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f441abbf0 con 0x7f5f4410a290 2024-04-26T22:05:29.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.302+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f441abe50 con 0x7f5f4410a290 2024-04-26T22:05:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.314+0000 7f5f3affd640 1 -- 172.21.15.55:0/4076439748 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5f3c0305f0 con 0x7f5f4410a290 2024-04-26T22:05:29.316 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.316+0000 7f5f3affd640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5f300a3e40 0x7f5f300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:29.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.316+0000 7f5f49acf640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5f300a3e40 0x7f5f300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:29.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.317+0000 7f5f3affd640 1 -- 172.21.15.55:0/4076439748 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5f3c0f3400 con 0x7f5f4410a290 2024-04-26T22:05:29.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.318+0000 7f5f3affd640 1 -- 172.21.15.55:0/4076439748 <== mon.2 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5f3c0bfd70 con 0x7f5f4410a290 2024-04-26T22:05:29.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.319+0000 7f5f49acf640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5f300a3e40 0x7f5f300a6300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5f340096f0 tx=0x7f5f34006c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:29.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:29 smithi055 ceph-mon[29221]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 912 MiB/s wr, 1.54k op/s 2024-04-26T22:05:29.506 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:29 smithi064 ceph-mon[32765]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 912 MiB/s wr, 1.54k op/s 2024-04-26T22:05:29.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:29 smithi165 ceph-mon[33296]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 912 MiB/s wr, 1.54k op/s 2024-04-26T22:05:29.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.967+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f441ac020 con 0x7f5f4410a290 2024-04-26T22:05:29.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.972+0000 7f5f3affd640 1 -- 172.21.15.55:0/4076439748 <== mon.2 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f5f3c0bf670 con 0x7f5f4410a290 2024-04-26T22:05:29.976 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:05:29.976 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:05:29.977 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:05:29.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.996+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5f300a3e40 msgr2=0x7f5f300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:29.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.996+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5f300a3e40 0x7f5f300a6300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5f340096f0 tx=0x7f5f34006c40 comp rx=0 tx=0).stop 2024-04-26T22:05:29.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.996+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4410a290 msgr2=0x7f5f441a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:29.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.996+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4410a290 0x7f5f441a3d00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c02aee0 tx=0x7f5f3c003ba0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.998+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 shutdown_connections 2024-04-26T22:05:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.998+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5f300a3e40 0x7f5f300a6300 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.998+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f441111b0 0x7f5f441aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.998+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f4410a290 0x7f5f441a3d00 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c02aee0 tx=0x7f5f3c003ba0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.998+0000 7f5f4bd33640 1 --2- 172.21.15.55:0/4076439748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f44109930 0x7f5f441a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:29.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:29.998+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 >> 172.21.15.55:0/4076439748 conn(0x7f5f4406f7d0 msgr2=0x7f5f4410aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.005+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 shutdown_connections 2024-04-26T22:05:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.009+0000 7f5f4bd33640 1 -- 172.21.15.55:0/4076439748 wait complete. 2024-04-26T22:05:30.063 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:05:30.437 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:30 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4076439748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:05:30.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.489+0000 7f39acac9640 1 -- 172.21.15.55:0/1519183302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 msgr2=0x7f39a8074190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.489+0000 7f39acac9640 1 --2- 172.21.15.55:0/1519183302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 0x7f39a8074190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f399c009860 tx=0x7f399c02a360 comp rx=0 tx=0).stop 2024-04-26T22:05:30.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.495+0000 7f39acac9640 1 -- 172.21.15.55:0/1519183302 shutdown_connections 2024-04-26T22:05:30.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.495+0000 7f39acac9640 1 --2- 172.21.15.55:0/1519183302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39a810a9b0 0x7f39a810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.495+0000 7f39acac9640 1 --2- 172.21.15.55:0/1519183302 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39a810a010 0x7f39a810a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.495+0000 7f39acac9640 1 --2- 172.21.15.55:0/1519183302 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 0x7f39a8074190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.495+0000 7f39acac9640 1 -- 172.21.15.55:0/1519183302 >> 172.21.15.55:0/1519183302 conn(0x7f39a806f840 msgr2=0x7f39a8071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.500+0000 7f39acac9640 1 -- 172.21.15.55:0/1519183302 shutdown_connections 2024-04-26T22:05:30.505 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.503+0000 7f39acac9640 1 -- 172.21.15.55:0/1519183302 wait complete. 2024-04-26T22:05:30.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.505+0000 7f39acac9640 1 Processor -- start 2024-04-26T22:05:30.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:30 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4076439748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:05:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.506+0000 7f39acac9640 1 -- start start 2024-04-26T22:05:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.507+0000 7f39acac9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 0x7f39a81a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.507+0000 7f39acac9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39a810a010 0x7f39a81a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.507+0000 7f39acac9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39a810a9b0 0x7f39a81aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.507+0000 7f39acac9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f39a811d1c0 con 0x7f39a810a9b0 2024-04-26T22:05:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.507+0000 7f39acac9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39a811d040 con 0x7f39a8073d70 2024-04-26T22:05:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.507+0000 7f39acac9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39a811d340 con 0x7f39a810a010 2024-04-26T22:05:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.509+0000 7f39a7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39a810a9b0 0x7f39a81aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.509+0000 7f39a7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39a810a9b0 0x7f39a81aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35378/0 (socket says 172.21.15.55:35378) 2024-04-26T22:05:30.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.509+0000 7f39a7fff640 1 -- 172.21.15.55:0/3919095918 learned_addr learned my addr 172.21.15.55:0/3919095918 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:30.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.515+0000 7f39a77fe640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 0x7f39a81a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.517+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39a810a010 0x7f39a81a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.519+0000 7f39a77fe640 1 -- 172.21.15.55:0/3919095918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39a810a010 msgr2=0x7f39a81a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.519+0000 7f39a77fe640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39a810a010 0x7f39a81a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.519+0000 7f39a77fe640 1 -- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39a810a9b0 msgr2=0x7f39a81aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.519+0000 7f39a77fe640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39a810a9b0 0x7f39a81aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.519+0000 7f39a77fe640 1 -- 172.21.15.55:0/3919095918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a8111190 con 0x7f39a8073d70 2024-04-26T22:05:30.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.526+0000 7f39a77fe640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 0x7f39a81a78b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f399c009780 tx=0x7f399c030d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.530 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.529+0000 7f39a4ff9640 1 -- 172.21.15.55:0/3919095918 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c037040 con 0x7f39a8073d70 2024-04-26T22:05:30.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.529+0000 7f39a4ff9640 1 -- 172.21.15.55:0/3919095918 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f399c02adc0 con 0x7f39a8073d70 2024-04-26T22:05:30.531 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.529+0000 7f39a4ff9640 1 -- 172.21.15.55:0/3919095918 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c005340 con 0x7f39a8073d70 2024-04-26T22:05:30.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.531+0000 7f39acac9640 1 -- 172.21.15.55:0/3919095918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a8111420 con 0x7f39a8073d70 2024-04-26T22:05:30.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.532+0000 7f39acac9640 1 -- 172.21.15.55:0/3919095918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a8111860 con 0x7f39a8073d70 2024-04-26T22:05:30.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.539+0000 7f39b0a46740 1 Processor -- start 2024-04-26T22:05:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.546+0000 7f39a4ff9640 1 -- 172.21.15.55:0/3919095918 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f399c043070 con 0x7f39a8073d70 2024-04-26T22:05:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.548+0000 7f39a4ff9640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39900a3e40 0x7f39900a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.549+0000 7f39a4ff9640 1 -- 172.21.15.55:0/3919095918 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f399c0ef7f0 con 0x7f39a8073d70 2024-04-26T22:05:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.549+0000 7f39b0a46740 1 -- start start 2024-04-26T22:05:30.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.550+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e157a00 0x56144e157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.550+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e159050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.552 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.550+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 0x56144e15fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.551+0000 7f39b0a46740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56144e0bbd80 con 0x56144e157a00 2024-04-26T22:05:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.551+0000 7f39b0a46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56144e075a20 con 0x56144e158bd0 2024-04-26T22:05:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.551+0000 7f39b0a46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56144e0b7d40 con 0x56144e159590 2024-04-26T22:05:30.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.553+0000 7f39a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.553+0000 7f39a6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:32842/0 (socket says 172.21.15.55:32842) 2024-04-26T22:05:30.555 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.553+0000 7f39a6ffd640 1 -- 172.21.15.55:0/1985632975 learned_addr learned my addr 172.21.15.55:0/1985632975 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:30.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.555+0000 7f39a7fff640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39900a3e40 0x7f39900a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.555+0000 7f39a7fff640 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 0x56144e15fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.559+0000 7f39a77fe640 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e157a00 0x56144e157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.559+0000 7f39a6ffd640 1 -- 172.21.15.55:0/1985632975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 msgr2=0x56144e15fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.559+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 0x56144e15fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.559+0000 7f39a6ffd640 1 -- 172.21.15.55:0/1985632975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e157a00 msgr2=0x56144e157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.559+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e157a00 0x56144e157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.559 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.559+0000 7f39a6ffd640 1 -- 172.21.15.55:0/1985632975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56144e080c30 con 0x56144e158bd0 2024-04-26T22:05:30.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.560+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e159050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39a000f430 tx=0x7f39a000f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=657aa57b9823b889 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.561+0000 7f398dffb640 1 -- 172.21.15.55:0/1985632975 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a001d070 con 0x56144e158bd0 2024-04-26T22:05:30.562 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.561+0000 7f398dffb640 1 -- 172.21.15.55:0/1985632975 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a000cb50 con 0x56144e158bd0 2024-04-26T22:05:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.563+0000 7f39b0a46740 1 -- 172.21.15.55:0/1985632975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 msgr2=0x56144e159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.563+0000 7f39b0a46740 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e159050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f39a000f430 tx=0x7f39a000f8f0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.567+0000 7f39b0a46740 1 -- 172.21.15.55:0/1985632975 shutdown_connections 2024-04-26T22:05:30.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.567+0000 7f39b0a46740 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 0x56144e15fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.567+0000 7f39b0a46740 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e159050 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.567+0000 7f39b0a46740 1 --2- 172.21.15.55:0/1985632975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e157a00 0x56144e157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.568 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.567+0000 7f39b0a46740 1 -- 172.21.15.55:0/1985632975 >> 172.21.15.55:0/1985632975 conn(0x56144e098910 msgr2=0x56144e1532b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.567+0000 7f39b0a46740 1 -- 172.21.15.55:0/1985632975 shutdown_connections 2024-04-26T22:05:30.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.567+0000 7f39b0a46740 1 -- 172.21.15.55:0/1985632975 wait complete. 2024-04-26T22:05:30.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.567+0000 7f39b0a46740 1 -- start start 2024-04-26T22:05:30.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.568+0000 7f39b0a46740 1 Processor -- start 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.568+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e157a00 0x56144e161c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.569+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 0x56144e162190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.569+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 0x56144e1691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.569+0000 7f39a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e157a00 0x56144e161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.569+0000 7f39a77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e157a00 0x56144e161c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:32852/0 (socket says 172.21.15.55:32852) 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.569+0000 7f39a77fe640 1 -- 172.21.15.55:0/3321556139 learned_addr learned my addr 172.21.15.55:0/3321556139 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.570+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x56144e1605f0 con 0x56144e158bd0 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.570+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56144e042dc0 con 0x56144e157a00 2024-04-26T22:05:30.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.570+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56144e160770 con 0x56144e159590 2024-04-26T22:05:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.570+0000 7f39b0a46740 10 client.0 _ll_register_callbacks cb 0x56144e05e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.571+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 0x56144e162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.571+0000 7f39a77fe640 1 -- 172.21.15.55:0/3321556139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 msgr2=0x56144e1691d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:05:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.571+0000 7f39a77fe640 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 0x56144e1691d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.571+0000 7f39a77fe640 1 -- 172.21.15.55:0/3321556139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 msgr2=0x56144e162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.571+0000 7f39a77fe640 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 0x56144e162190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.571+0000 7f39a77fe640 1 -- 172.21.15.55:0/3321556139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56144ddcd3f0 con 0x56144e157a00 2024-04-26T22:05:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.572+0000 7f39a77fe640 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e157a00 0x56144e161c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f399c009830 tx=0x7f399c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3fc7c5ebaa35646 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.572+0000 7f39a7fff640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39900a3e40 0x7f39900a6300 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3998005f80 tx=0x7f399800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:30 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4076439748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:05:30.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.595+0000 7f3986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:05:30.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.595+0000 7f3986ffd640 1 -- 172.21.15.55:0/3321556139 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c0bbe40 con 0x56144e157a00 2024-04-26T22:05:30.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.595+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a000cb50 con 0x56144e157a00 2024-04-26T22:05:30.596 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.595+0000 7f3986ffd640 1 -- 172.21.15.55:0/3321556139 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f399c0c41c0 con 0x56144e157a00 2024-04-26T22:05:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.597+0000 7f3986ffd640 1 -- 172.21.15.55:0/3321556139 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f399c0bb9e0 con 0x56144e157a00 2024-04-26T22:05:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.597+0000 7f39b0a46740 10 client.26312 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:30.599 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.597+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56144de977d0 con 0x56144e157a00 2024-04-26T22:05:30.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.601+0000 7f3986ffd640 1 -- 172.21.15.55:0/3321556139 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a000cb50 con 0x56144e157a00 2024-04-26T22:05:30.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.601+0000 7f3986ffd640 1 -- 172.21.15.55:0/3321556139 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f399c0daba0 con 0x56144e157a00 2024-04-26T22:05:30.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.601+0000 7f39b0a46740 10 client.26312 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:30.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.601+0000 7f39b0a46740 10 client.26312 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:05:30.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.601+0000 7f39b0a46740 20 client.26312 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:30.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.601+0000 7f39b0a46740 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x56144e16dc30 0x56144e1700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.601+0000 7f39b0a46740 4 client.26312 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:30.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.601+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56144e0c9990 con 0x56144e16dc30 2024-04-26T22:05:30.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.602+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x56144e16dc30 0x56144e1700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.604+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x56144e16dc30 0x56144e1700d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45e68a4c2d6a2c44 server_cookie=f05d58913f51d740 in_seq=0 out_seq=0 2024-04-26T22:05:30.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.609+0000 7f3986ffd640 10 client.26312 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:05:30.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.609+0000 7f3986ffd640 1 -- 172.21.15.55:0/3321556139 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56144e0c9990 con 0x56144e16dc30 2024-04-26T22:05:30.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.609+0000 7f3986ffd640 10 client.26312 handle_command_reply: tid=0 2024-04-26T22:05:30.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.609+0000 7f39b0a46740 1 client.26312 shutdown 2024-04-26T22:05:30.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.618+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e157a00 msgr2=0x56144e161c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.619 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.618+0000 7f39b0a46740 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e157a00 0x56144e161c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f399c009830 tx=0x7f399c009510 comp rx=0 tx=0).stop 2024-04-26T22:05:30.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.618+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 shutdown_connections 2024-04-26T22:05:30.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.618+0000 7f39b0a46740 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x56144e16dc30 0x56144e1700d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.618+0000 7f39b0a46740 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e159590 0x56144e1691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.618+0000 7f39b0a46740 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 0x56144e162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.618+0000 7f39b0a46740 1 --2- 172.21.15.55:0/3321556139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e157a00 0x56144e161c50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.618+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 >> 172.21.15.55:0/3321556139 conn(0x56144e098910 msgr2=0x56144e152450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.619+0000 7f3986ffd640 0 client.26312 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:05:30.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.621+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 shutdown_connections 2024-04-26T22:05:30.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.622+0000 7f39b0a46740 1 -- 172.21.15.55:0/3321556139 wait complete. 2024-04-26T22:05:30.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.622+0000 7f39b0a46740 20 client.26312 trim_cache size 0 max 16384 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.687+0000 7f39b0a46740 1 Processor -- start 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.687+0000 7f39b0a46740 1 -- start start 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.687+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e16fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.688+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e169190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.688+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e159590 0x56144e161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.688+0000 7f39b0a46740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81c3ba0 con 0x56144e159590 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.688+0000 7f39a7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e159590 0x56144e161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.688+0000 7f39b0a46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81c1470 con 0x56144e158bd0 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.688+0000 7f39a7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e159590 0x56144e161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54600/0 (socket says 172.21.15.55:54600) 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.688+0000 7f39a7fff640 1 -- 172.21.15.55:0/2189383341 learned_addr learned my addr 172.21.15.55:0/2189383341 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:30.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.688+0000 7f39b0a46740 1 -- 172.21.15.55:0/2189383341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8118780 con 0x56144e157a00 2024-04-26T22:05:30.693 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.691+0000 7f39a77fe640 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e16fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.691+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e169190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.693+0000 7f39a7fff640 1 -- 172.21.15.55:0/2189383341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 msgr2=0x56144e16fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.693+0000 7f39a7fff640 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e16fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.693+0000 7f39a7fff640 1 -- 172.21.15.55:0/2189383341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 msgr2=0x56144e169190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.693+0000 7f39a7fff640 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e169190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.693+0000 7f39a7fff640 1 -- 172.21.15.55:0/2189383341 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56144e153fb0 con 0x56144e159590 2024-04-26T22:05:30.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.694+0000 7f39a7fff640 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e159590 0x56144e161f70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3998010430 tx=0x7f3998004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccce7bb8dd14413 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.697+0000 7f398d7fa640 1 -- 172.21.15.55:0/2189383341 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801f650 con 0x56144e159590 2024-04-26T22:05:30.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.697+0000 7f398d7fa640 1 -- 172.21.15.55:0/2189383341 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f399801fca0 con 0x56144e159590 2024-04-26T22:05:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.698+0000 7f39b0a46740 1 -- 172.21.15.55:0/2189383341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e159590 msgr2=0x56144e161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.698+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e159590 0x56144e161f70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3998010430 tx=0x7f3998004730 comp rx=0 tx=0).stop 2024-04-26T22:05:30.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.699+0000 7f39b0a46740 1 -- 172.21.15.55:0/2189383341 shutdown_connections 2024-04-26T22:05:30.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.699+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e159590 0x56144e161f70 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.699+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e158bd0 0x56144e169190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.699+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2189383341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e16fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.699+0000 7f39b0a46740 1 -- 172.21.15.55:0/2189383341 >> 172.21.15.55:0/2189383341 conn(0x56144e098910 msgr2=0x56144e161860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.705+0000 7f39b0a46740 1 -- 172.21.15.55:0/2189383341 shutdown_connections 2024-04-26T22:05:30.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.706+0000 7f39b0a46740 1 -- 172.21.15.55:0/2189383341 wait complete. 2024-04-26T22:05:30.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.706+0000 7f39b0a46740 1 -- start start 2024-04-26T22:05:30.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.707+0000 7f39b0a46740 1 Processor -- start 2024-04-26T22:05:30.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.707+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.707+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 0x56144e167070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.707+0000 7f39b0a46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e159590 0x56144e168030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.707+0000 7f39b0a46740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8118df0 con 0x56144e158bd0 2024-04-26T22:05:30.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.707+0000 7f39b0a46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81c0da0 con 0x56144e159590 2024-04-26T22:05:30.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.707+0000 7f39b0a46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81194c0 con 0x56144e157a00 2024-04-26T22:05:30.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.708+0000 7f39b0a46740 10 client.0 _ll_register_callbacks cb 0x7f399c0b7bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:30.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.712+0000 7f39a77fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.712+0000 7f39a77fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e166b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:34918/0 (socket says 172.21.15.55:34918) 2024-04-26T22:05:30.712 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.712+0000 7f39a77fe640 1 -- 172.21.15.55:0/2921341474 learned_addr learned my addr 172.21.15.55:0/2921341474 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:30.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.713+0000 7f39a7fff640 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e159590 0x56144e168030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.718+0000 7f39a6ffd640 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 0x56144e167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.720+0000 7f39a7fff640 1 -- 172.21.15.55:0/2921341474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 msgr2=0x56144e166b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.720+0000 7f39a7fff640 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e166b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.720+0000 7f39a7fff640 1 -- 172.21.15.55:0/2921341474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 msgr2=0x56144e167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:30.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.720+0000 7f39a7fff640 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 0x56144e167070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.720+0000 7f39a7fff640 1 -- 172.21.15.55:0/2921341474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56144e168620 con 0x56144e159590 2024-04-26T22:05:30.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.728+0000 7f39a7fff640 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e159590 0x56144e168030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f399800ebb0 tx=0x7f3998027630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ae5a9a06c4c07f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:30.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.729+0000 7f3986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:05:30.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.733+0000 7f3986ffd640 1 -- 172.21.15.55:0/2921341474 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801f450 con 0x56144e159590 2024-04-26T22:05:30.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.734+0000 7f39b0a46740 1 -- 172.21.15.55:0/2921341474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56144e0c3800 con 0x56144e159590 2024-04-26T22:05:30.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.734+0000 7f3986ffd640 1 -- 172.21.15.55:0/2921341474 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f399801f5f0 con 0x56144e159590 2024-04-26T22:05:30.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.737+0000 7f3986ffd640 1 -- 172.21.15.55:0/2921341474 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f399801f830 con 0x56144e159590 2024-04-26T22:05:30.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.737+0000 7f39b0a46740 10 client.26318 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:30.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.737+0000 7f39b0a46740 1 -- 172.21.15.55:0/2921341474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56144e1d4540 con 0x56144e159590 2024-04-26T22:05:30.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.745+0000 7f3986ffd640 1 -- 172.21.15.55:0/2921341474 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56144e0c3800 con 0x56144e159590 2024-04-26T22:05:30.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.745+0000 7f3986ffd640 1 -- 172.21.15.55:0/2921341474 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f399801c040 con 0x56144e159590 2024-04-26T22:05:30.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.745+0000 7f39b0a46740 10 client.26318 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:30.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.745+0000 7f39b0a46740 10 client.26318 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:05:30.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.745+0000 7f39b0a46740 20 client.26318 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:30.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.745+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x56144e1d4d20 0x56144e1d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:30.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.745+0000 7f39b0a46740 4 client.26318 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:05:30.746 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.745+0000 7f39b0a46740 1 -- 172.21.15.55:0/2921341474 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56144e03e610 con 0x56144e1d4d20 2024-04-26T22:05:30.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.746+0000 7f39a77fe640 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x56144e1d4d20 0x56144e1d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:30.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.748+0000 7f39a77fe640 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x56144e1d4d20 0x56144e1d7140 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84e85a02d13ba018 server_cookie=2035787a2c1be59a in_seq=0 out_seq=0 2024-04-26T22:05:30.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.748+0000 7f3986ffd640 10 client.26318 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:05:30.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.748+0000 7f3986ffd640 1 -- 172.21.15.55:0/2921341474 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56144e03e610 con 0x56144e1d4d20 2024-04-26T22:05:30.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.748+0000 7f3986ffd640 10 client.26318 handle_command_reply: tid=0 2024-04-26T22:05:30.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 client.26318 shutdown 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 -- 172.21.15.55:0/2921341474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e159590 msgr2=0x56144e168030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e159590 0x56144e168030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f399800ebb0 tx=0x7f3998027630 comp rx=0 tx=0).stop 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 -- 172.21.15.55:0/2921341474 shutdown_connections 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x56144e1d4d20 0x56144e1d7140 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56144e159590 0x56144e168030 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x56144e158bd0 0x56144e167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 --2- 172.21.15.55:0/2921341474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56144e157a00 0x56144e166b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 -- 172.21.15.55:0/2921341474 >> 172.21.15.55:0/2921341474 conn(0x56144e098910 msgr2=0x56144e1561f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f3986ffd640 0 client.26318 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 -- 172.21.15.55:0/2921341474 shutdown_connections 2024-04-26T22:05:30.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 1 -- 172.21.15.55:0/2921341474 wait complete. 2024-04-26T22:05:30.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.749+0000 7f39b0a46740 20 client.26318 trim_cache size 0 max 16384 2024-04-26T22:05:30.751 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 -- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39900a3e40 msgr2=0x7f39900a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39900a3e40 0x7f39900a6300 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3998005f80 tx=0x7f399800b7b0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 -- 172.21.15.55:0/3919095918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 msgr2=0x7f39a81a78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 0x7f39a81a78b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f399c009780 tx=0x7f399c030d80 comp rx=0 tx=0).stop 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 -- 172.21.15.55:0/3919095918 shutdown_connections 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f39900a3e40 0x7f39900a6300 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f39a810a9b0 0x7f39a81aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39a810a010 0x7f39a81a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 --2- 172.21.15.55:0/3919095918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39a8073d70 0x7f39a81a78b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:30.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 -- 172.21.15.55:0/3919095918 >> 172.21.15.55:0/3919095918 conn(0x7f39a806f840 msgr2=0x7f39a80715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 -- 172.21.15.55:0/3919095918 shutdown_connections 2024-04-26T22:05:30.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:30.750+0000 7f39acac9640 1 -- 172.21.15.55:0/3919095918 wait complete. 2024-04-26T22:05:30.769 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:05:31.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.109+0000 7f6359bf0640 1 -- 172.21.15.55:0/3657453890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f635410c2a0 msgr2=0x7f635410e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.110 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.109+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3657453890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f635410c2a0 0x7f635410e730 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6344009900 tx=0x7f634402a440 comp rx=0 tx=0).stop 2024-04-26T22:05:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.115+0000 7f6359bf0640 1 -- 172.21.15.55:0/3657453890 shutdown_connections 2024-04-26T22:05:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.115+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3657453890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635410ec70 0x7f635410f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.115+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3657453890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f635410c2a0 0x7f635410e730 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.115+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3657453890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6354109930 0x7f635410bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.115+0000 7f6359bf0640 1 -- 172.21.15.55:0/3657453890 >> 172.21.15.55:0/3657453890 conn(0x7f635406f7c0 msgr2=0x7f6354071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.119+0000 7f6359bf0640 1 -- 172.21.15.55:0/3657453890 shutdown_connections 2024-04-26T22:05:31.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.120+0000 7f6359bf0640 1 -- 172.21.15.55:0/3657453890 wait complete. 2024-04-26T22:05:31.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.122+0000 7f6359bf0640 1 Processor -- start 2024-04-26T22:05:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.123+0000 7f6359bf0640 1 -- start start 2024-04-26T22:05:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.123+0000 7f6359bf0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6354109930 0x7f63541a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.123+0000 7f6359bf0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f635410c2a0 0x7f63541a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.123+0000 7f6359bf0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635410ec70 0x7f63541aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.124+0000 7f6359bf0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f635411d2d0 con 0x7f6354109930 2024-04-26T22:05:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.124+0000 7f6359bf0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f635411d150 con 0x7f635410ec70 2024-04-26T22:05:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.124+0000 7f6359bf0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f635411d450 con 0x7f635410c2a0 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.124+0000 7f6353fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635410ec70 0x7f63541aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.124+0000 7f6353fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635410ec70 0x7f63541aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45754/0 (socket says 172.21.15.55:45754) 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.124+0000 7f6353fff640 1 -- 172.21.15.55:0/3506513926 learned_addr learned my addr 172.21.15.55:0/3506513926 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.124+0000 7f6352ffd640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f635410c2a0 0x7f63541a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.125+0000 7f6353fff640 1 -- 172.21.15.55:0/3506513926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f635410c2a0 msgr2=0x7f63541a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.125+0000 7f6353fff640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f635410c2a0 0x7f63541a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.125+0000 7f6353fff640 1 -- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6354109930 msgr2=0x7f63541a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.125+0000 7f6353fff640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6354109930 0x7f63541a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.125+0000 7f6353fff640 1 -- 172.21.15.55:0/3506513926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6354111190 con 0x7f635410ec70 2024-04-26T22:05:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.126+0000 7f6353fff640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635410ec70 0x7f63541aefd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f634c00e9d0 tx=0x7f634c00ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.126+0000 7f6350ff9640 1 -- 172.21.15.55:0/3506513926 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c00cd80 con 0x7f635410ec70 2024-04-26T22:05:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.126+0000 7f6359bf0640 1 -- 172.21.15.55:0/3506513926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6354111480 con 0x7f635410ec70 2024-04-26T22:05:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.126+0000 7f6359bf0640 1 -- 172.21.15.55:0/3506513926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63541119a0 con 0x7f635410ec70 2024-04-26T22:05:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.127+0000 7f6350ff9640 1 -- 172.21.15.55:0/3506513926 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f634c005020 con 0x7f635410ec70 2024-04-26T22:05:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.127+0000 7f6350ff9640 1 -- 172.21.15.55:0/3506513926 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c0336e0 con 0x7f635410ec70 2024-04-26T22:05:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.131+0000 7f6350ff9640 1 -- 172.21.15.55:0/3506513926 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f634c0051d0 con 0x7f635410ec70 2024-04-26T22:05:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.132+0000 7f6350ff9640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f633c0a3d80 0x7f633c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.132+0000 7f6350ff9640 1 -- 172.21.15.55:0/3506513926 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f634c0f2770 con 0x7f635410ec70 2024-04-26T22:05:31.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.140+0000 7f63537fe640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f633c0a3d80 0x7f633c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.151 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.147+0000 7f63537fe640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f633c0a3d80 0x7f633c0a6240 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f634800a8f0 tx=0x7f6348005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.151+0000 7f635c9e2740 1 Processor -- start 2024-04-26T22:05:31.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.153+0000 7f635c9e2740 1 -- start start 2024-04-26T22:05:31.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.153+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88e820 0x5597db88ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.153+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88f9f0 0x5597db88fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.153+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db896c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.158 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.153+0000 7f635c9e2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5597db7f2d80 con 0x5597db88f9f0 2024-04-26T22:05:31.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.153+0000 7f635c9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597db7aca20 con 0x5597db8903b0 2024-04-26T22:05:31.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.153+0000 7f635c9e2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597db7eed40 con 0x5597db88e820 2024-04-26T22:05:31.159 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.156+0000 7f63537fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88e820 0x5597db88ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.158+0000 7f63537fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88e820 0x5597db88ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:34950/0 (socket says 172.21.15.55:34950) 2024-04-26T22:05:31.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.158+0000 7f63537fe640 1 -- 172.21.15.55:0/583222690 learned_addr learned my addr 172.21.15.55:0/583222690 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.162 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.160+0000 7f6353fff640 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db896c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.164+0000 7f6352ffd640 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88f9f0 0x5597db88fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.169 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.165+0000 7f63537fe640 1 -- 172.21.15.55:0/583222690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 msgr2=0x5597db896c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.165+0000 7f63537fe640 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db896c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.165+0000 7f63537fe640 1 -- 172.21.15.55:0/583222690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88f9f0 msgr2=0x5597db88fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.165+0000 7f63537fe640 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88f9f0 0x5597db88fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.165+0000 7f63537fe640 1 -- 172.21.15.55:0/583222690 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597db7b7c30 con 0x5597db88e820 2024-04-26T22:05:31.170 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.169+0000 7f6352ffd640 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88f9f0 0x5597db88fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:05:31.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.170+0000 7f63537fe640 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88e820 0x5597db88ec40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f634800cc90 tx=0x7f634800f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=460aabdfb50a5b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.177+0000 7f6339ffb640 1 -- 172.21.15.55:0/583222690 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348020070 con 0x5597db88e820 2024-04-26T22:05:31.178 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.177+0000 7f6339ffb640 1 -- 172.21.15.55:0/583222690 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6348016070 con 0x5597db88e820 2024-04-26T22:05:31.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.178+0000 7f635c9e2740 1 -- 172.21.15.55:0/583222690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88e820 msgr2=0x5597db88ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.179 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.178+0000 7f635c9e2740 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88e820 0x5597db88ec40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f634800cc90 tx=0x7f634800f000 comp rx=0 tx=0).stop 2024-04-26T22:05:31.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.181+0000 7f635c9e2740 1 -- 172.21.15.55:0/583222690 shutdown_connections 2024-04-26T22:05:31.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.181+0000 7f635c9e2740 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db896c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.181+0000 7f635c9e2740 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88f9f0 0x5597db88fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.181+0000 7f635c9e2740 1 --2- 172.21.15.55:0/583222690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88e820 0x5597db88ec40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.181 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.181+0000 7f635c9e2740 1 -- 172.21.15.55:0/583222690 >> 172.21.15.55:0/583222690 conn(0x5597db7cf910 msgr2=0x5597db88a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.189+0000 7f635c9e2740 1 -- 172.21.15.55:0/583222690 shutdown_connections 2024-04-26T22:05:31.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.191+0000 7f635c9e2740 1 -- 172.21.15.55:0/583222690 wait complete. 2024-04-26T22:05:31.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.192+0000 7f635c9e2740 1 -- start start 2024-04-26T22:05:31.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.192+0000 7f635c9e2740 1 Processor -- start 2024-04-26T22:05:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.195+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88e820 0x5597db898ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.195+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 0x5597db899010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.195+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db8903b0 0x5597db8a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.195+0000 7f635c9e2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5597db897410 con 0x5597db8903b0 2024-04-26T22:05:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.195+0000 7f635c9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5597db779dc0 con 0x5597db88e820 2024-04-26T22:05:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.195+0000 7f635c9e2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597db897590 con 0x5597db88f9f0 2024-04-26T22:05:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.195+0000 7f635c9e2740 10 client.0 _ll_register_callbacks cb 0x5597db795600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.196+0000 7f6353fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db8903b0 0x5597db8a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.196+0000 7f6353fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db8903b0 0x5597db8a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54646/0 (socket says 172.21.15.55:54646) 2024-04-26T22:05:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.196+0000 7f6353fff640 1 -- 172.21.15.55:0/3792530854 learned_addr learned my addr 172.21.15.55:0/3792530854 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.198+0000 7f6352ffd640 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 0x5597db899010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.198+0000 7f63537fe640 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88e820 0x5597db898ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.198+0000 7f6353fff640 1 -- 172.21.15.55:0/3792530854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 msgr2=0x5597db899010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.198+0000 7f6353fff640 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 0x5597db899010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.198+0000 7f6353fff640 1 -- 172.21.15.55:0/3792530854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88e820 msgr2=0x5597db898ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.198+0000 7f6353fff640 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88e820 0x5597db898ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.198+0000 7f6353fff640 1 -- 172.21.15.55:0/3792530854 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597db5043f0 con 0x5597db8903b0 2024-04-26T22:05:31.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.199+0000 7f6353fff640 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db8903b0 0x5597db8a0050 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f634c0bb220 tx=0x7f634c0bf090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05066ec1bec4cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.199+0000 7f6332ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:05:31.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.199+0000 7f6332ffd640 1 -- 172.21.15.55:0/3792530854 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c041040 con 0x5597db8903b0 2024-04-26T22:05:31.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.200+0000 7f6332ffd640 1 -- 172.21.15.55:0/3792530854 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f634c0c7030 con 0x5597db8903b0 2024-04-26T22:05:31.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.200+0000 7f635c9e2740 1 -- 172.21.15.55:0/3792530854 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6348016070 con 0x5597db8903b0 2024-04-26T22:05:31.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.200+0000 7f6332ffd640 1 -- 172.21.15.55:0/3792530854 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f634c0c7230 con 0x5597db8903b0 2024-04-26T22:05:31.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.201+0000 7f635c9e2740 10 client.17394 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:31.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.201+0000 7f635c9e2740 1 -- 172.21.15.55:0/3792530854 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5597db5ce7d0 con 0x5597db8903b0 2024-04-26T22:05:31.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.211+0000 7f6332ffd640 1 -- 172.21.15.55:0/3792530854 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6348016070 con 0x5597db8903b0 2024-04-26T22:05:31.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.211+0000 7f6332ffd640 1 -- 172.21.15.55:0/3792530854 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f634c0d4070 con 0x5597db8903b0 2024-04-26T22:05:31.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.211+0000 7f635c9e2740 10 client.17394 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:31.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.211+0000 7f635c9e2740 10 client.17394 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:05:31.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.211+0000 7f635c9e2740 20 client.17394 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:31.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.211+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5597db8a4840 0x5597db8a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.211+0000 7f635c9e2740 4 client.17394 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:31.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.211+0000 7f635c9e2740 1 -- 172.21.15.55:0/3792530854 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597db800990 con 0x5597db8a4840 2024-04-26T22:05:31.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.212+0000 7f63537fe640 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5597db8a4840 0x5597db8a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.215+0000 7f6332ffd640 10 client.17394 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:05:31.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.215+0000 7f63537fe640 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5597db8a4840 0x5597db8a6d00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eda636dec0237db8 server_cookie=4014be6ca371d931 in_seq=0 out_seq=0 2024-04-26T22:05:31.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.218+0000 7f6332ffd640 1 -- 172.21.15.55:0/3792530854 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597db800990 con 0x5597db8a4840 2024-04-26T22:05:31.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.218+0000 7f6332ffd640 10 client.17394 handle_command_reply: tid=0 2024-04-26T22:05:31.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.218+0000 7f635c9e2740 1 client.17394 shutdown 2024-04-26T22:05:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 -- 172.21.15.55:0/3792530854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db8903b0 msgr2=0x5597db8a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db8903b0 0x5597db8a0050 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f634c0bb220 tx=0x7f634c0bf090 comp rx=0 tx=0).stop 2024-04-26T22:05:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 -- 172.21.15.55:0/3792530854 shutdown_connections 2024-04-26T22:05:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5597db8a4840 0x5597db8a6d00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db8903b0 0x5597db8a0050 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 0x5597db899010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3792530854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88e820 0x5597db898ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 -- 172.21.15.55:0/3792530854 >> 172.21.15.55:0/3792530854 conn(0x5597db7cf910 msgr2=0x5597db889280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f6332ffd640 0 client.17394 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:05:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 -- 172.21.15.55:0/3792530854 shutdown_connections 2024-04-26T22:05:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.231+0000 7f635c9e2740 1 -- 172.21.15.55:0/3792530854 wait complete. 2024-04-26T22:05:31.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.232+0000 7f635c9e2740 20 client.17394 trim_cache size 0 max 16384 2024-04-26T22:05:31.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.273+0000 7f635c9e2740 1 Processor -- start 2024-04-26T22:05:31.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.273+0000 7f635c9e2740 1 -- start start 2024-04-26T22:05:31.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.273+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 0x5597db88a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.273+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88f9f0 0x5597db8a0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.273+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db8903b0 0x5597db8a6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.273+0000 7f635c9e2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f6340010eb0 con 0x5597db88e820 2024-04-26T22:05:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.273+0000 7f635c9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f634000e780 con 0x5597db88f9f0 2024-04-26T22:05:31.274 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.273+0000 7f635c9e2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f634000c5d0 con 0x5597db8903b0 2024-04-26T22:05:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.275+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88f9f0 0x5597db8a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.275+0000 7f6352ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88f9f0 0x5597db8a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45778/0 (socket says 172.21.15.55:45778) 2024-04-26T22:05:31.275 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.275+0000 7f6352ffd640 1 -- 172.21.15.55:0/3874083385 learned_addr learned my addr 172.21.15.55:0/3874083385 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.276+0000 7f6352ffd640 1 -- 172.21.15.55:0/3874083385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db8903b0 msgr2=0x5597db8a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.276+0000 7f6352ffd640 1 --2- 172.21.15.55:0/3874083385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db8903b0 0x5597db8a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.276 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.276+0000 7f6352ffd640 1 -- 172.21.15.55:0/3874083385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 msgr2=0x5597db88a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.276+0000 7f6352ffd640 1 --2- 172.21.15.55:0/3874083385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 0x5597db88a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.276+0000 7f6352ffd640 1 -- 172.21.15.55:0/3874083385 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597db896fb0 con 0x5597db88f9f0 2024-04-26T22:05:31.277 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.277+0000 7f6352ffd640 1 --2- 172.21.15.55:0/3874083385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88f9f0 0x5597db8a0010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6344006010 tx=0x7f63440041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de56c48fabadf62e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.280+0000 7f63397fa640 1 -- 172.21.15.55:0/3874083385 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344035030 con 0x5597db88f9f0 2024-04-26T22:05:31.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.281+0000 7f63397fa640 1 -- 172.21.15.55:0/3874083385 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6344041070 con 0x5597db88f9f0 2024-04-26T22:05:31.281 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.281+0000 7f63397fa640 1 -- 172.21.15.55:0/3874083385 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63440035f0 con 0x5597db88f9f0 2024-04-26T22:05:31.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.282+0000 7f635c9e2740 1 -- 172.21.15.55:0/3874083385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88f9f0 msgr2=0x5597db8a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.282+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3874083385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88f9f0 0x5597db8a0010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6344006010 tx=0x7f63440041a0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.283+0000 7f635c9e2740 1 -- 172.21.15.55:0/3874083385 shutdown_connections 2024-04-26T22:05:31.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.283+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3874083385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db8903b0 0x5597db8a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.283+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3874083385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db88f9f0 0x5597db8a0010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.283+0000 7f635c9e2740 1 --2- 172.21.15.55:0/3874083385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 0x5597db88a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.283+0000 7f635c9e2740 1 -- 172.21.15.55:0/3874083385 >> 172.21.15.55:0/3874083385 conn(0x5597db7cf910 msgr2=0x5597db889c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.283 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.283+0000 7f635c9e2740 1 -- 172.21.15.55:0/3874083385 shutdown_connections 2024-04-26T22:05:31.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.283+0000 7f635c9e2740 1 -- 172.21.15.55:0/3874083385 wait complete. 2024-04-26T22:05:31.284 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.284+0000 7f635c9e2740 1 -- start start 2024-04-26T22:05:31.285 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.285+0000 7f635c9e2740 1 Processor -- start 2024-04-26T22:05:31.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.285+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 0x5597db89d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.285+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 0x5597db89dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.285+0000 7f635c9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db89eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.285+0000 7f635c9e2740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f634000cc40 con 0x5597db88e820 2024-04-26T22:05:31.288 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.285+0000 7f635c9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f634000e0b0 con 0x5597db8903b0 2024-04-26T22:05:31.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.285+0000 7f635c9e2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f634000d310 con 0x5597db88f9f0 2024-04-26T22:05:31.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.285+0000 7f635c9e2740 10 client.0 _ll_register_callbacks cb 0x7f634c0ba430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:31.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.287+0000 7f6353fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db89eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.288+0000 7f6353fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db89eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45790/0 (socket says 172.21.15.55:45790) 2024-04-26T22:05:31.289 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.288+0000 7f6353fff640 1 -- 172.21.15.55:0/2767048525 learned_addr learned my addr 172.21.15.55:0/2767048525 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.289+0000 7f63537fe640 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 0x5597db89d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.289+0000 7f6352ffd640 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 0x5597db89dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.290+0000 7f63537fe640 1 -- 172.21.15.55:0/2767048525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 msgr2=0x5597db89dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.290+0000 7f63537fe640 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 0x5597db89dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.290+0000 7f63537fe640 1 -- 172.21.15.55:0/2767048525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 msgr2=0x5597db89eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.290+0000 7f63537fe640 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db89eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.290 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.290+0000 7f63537fe640 1 -- 172.21.15.55:0/2767048525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597db8858b0 con 0x5597db88e820 2024-04-26T22:05:31.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.290+0000 7f63537fe640 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 0x5597db89d660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6348011040 tx=0x7f634800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fa201c50bd48d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.290+0000 7f6332ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:05:31.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.291+0000 7f6332ffd640 1 -- 172.21.15.55:0/2767048525 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348020040 con 0x5597db88e820 2024-04-26T22:05:31.291 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.291+0000 7f635c9e2740 1 -- 172.21.15.55:0/2767048525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6344041070 con 0x5597db88e820 2024-04-26T22:05:31.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.291+0000 7f6332ffd640 1 -- 172.21.15.55:0/2767048525 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f63480040e0 con 0x5597db88e820 2024-04-26T22:05:31.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.292+0000 7f6332ffd640 1 -- 172.21.15.55:0/2767048525 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f63480035f0 con 0x5597db88e820 2024-04-26T22:05:31.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.292+0000 7f635c9e2740 10 client.17397 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:31.292 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.292+0000 7f635c9e2740 1 -- 172.21.15.55:0/2767048525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5597db8a0280 con 0x5597db88e820 2024-04-26T22:05:31.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.295+0000 7f6332ffd640 1 -- 172.21.15.55:0/2767048525 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6344041070 con 0x5597db88e820 2024-04-26T22:05:31.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.295+0000 7f6332ffd640 1 -- 172.21.15.55:0/2767048525 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f634801b070 con 0x5597db88e820 2024-04-26T22:05:31.295 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.295+0000 7f635c9e2740 10 client.17397 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:31.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.295+0000 7f635c9e2740 10 client.17397 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:05:31.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.295+0000 7f635c9e2740 20 client.17397 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:31.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.295+0000 7f635c9e2740 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5597db90b510 0x5597db90d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.295+0000 7f635c9e2740 4 client.17397 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:05:31.296 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.295+0000 7f635c9e2740 1 -- 172.21.15.55:0/2767048525 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597db7fa800 con 0x5597db90b510 2024-04-26T22:05:31.297 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.297+0000 7f6352ffd640 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5597db90b510 0x5597db90d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.299+0000 7f6352ffd640 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5597db90b510 0x5597db90d930 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d98fcb0c787b86c4 server_cookie=c2f0aca801ef3903 in_seq=0 out_seq=0 2024-04-26T22:05:31.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.299+0000 7f6332ffd640 10 client.17397 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:05:31.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.299+0000 7f6332ffd640 1 -- 172.21.15.55:0/2767048525 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597db7fa800 con 0x5597db90b510 2024-04-26T22:05:31.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.299+0000 7f6332ffd640 10 client.17397 handle_command_reply: tid=0 2024-04-26T22:05:31.300 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.299+0000 7f635c9e2740 1 client.17397 shutdown 2024-04-26T22:05:31.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.301+0000 7f635c9e2740 1 -- 172.21.15.55:0/2767048525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 msgr2=0x5597db89d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.301 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.301+0000 7f635c9e2740 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 0x5597db89d660 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6348011040 tx=0x7f634800c950 comp rx=0 tx=0).stop 2024-04-26T22:05:31.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.302+0000 7f635c9e2740 1 -- 172.21.15.55:0/2767048525 shutdown_connections 2024-04-26T22:05:31.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.302+0000 7f635c9e2740 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x5597db90b510 0x5597db90d930 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.302+0000 7f635c9e2740 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5597db8903b0 0x5597db89eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.302+0000 7f635c9e2740 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597db88f9f0 0x5597db89dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.302+0000 7f635c9e2740 1 --2- 172.21.15.55:0/2767048525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5597db88e820 0x5597db89d660 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.303 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.302+0000 7f635c9e2740 1 -- 172.21.15.55:0/2767048525 >> 172.21.15.55:0/2767048525 conn(0x5597db7cf910 msgr2=0x5597db8889f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.303+0000 7f6332ffd640 0 client.17397 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:05:31.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.303+0000 7f635c9e2740 1 -- 172.21.15.55:0/2767048525 shutdown_connections 2024-04-26T22:05:31.304 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.303+0000 7f635c9e2740 1 -- 172.21.15.55:0/2767048525 wait complete. 2024-04-26T22:05:31.308 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.308+0000 7f635c9e2740 20 client.17397 trim_cache size 0 max 16384 2024-04-26T22:05:31.309 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:05:31.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.312+0000 7f6359bf0640 1 -- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f633c0a3d80 msgr2=0x7f633c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.312+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f633c0a3d80 0x7f633c0a6240 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f634800a8f0 tx=0x7f6348005bf0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.312+0000 7f6359bf0640 1 -- 172.21.15.55:0/3506513926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635410ec70 msgr2=0x7f63541aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.312+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635410ec70 0x7f63541aefd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f634c00e9d0 tx=0x7f634c00ee90 comp rx=0 tx=0).stop 2024-04-26T22:05:31.313 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.312+0000 7f6359bf0640 1 -- 172.21.15.55:0/3506513926 shutdown_connections 2024-04-26T22:05:31.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.312+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f633c0a3d80 0x7f633c0a6240 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.312+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f635410ec70 0x7f63541aefd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.313+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f635410c2a0 0x7f63541a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.313+0000 7f6359bf0640 1 --2- 172.21.15.55:0/3506513926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6354109930 0x7f63541a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.313+0000 7f6359bf0640 1 -- 172.21.15.55:0/3506513926 >> 172.21.15.55:0/3506513926 conn(0x7f635406f7c0 msgr2=0x7f635410afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.313+0000 7f6359bf0640 1 -- 172.21.15.55:0/3506513926 shutdown_connections 2024-04-26T22:05:31.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.315+0000 7f6359bf0640 1 -- 172.21.15.55:0/3506513926 wait complete. 2024-04-26T22:05:31.351 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:05:31.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:31 smithi055 ceph-mon[29221]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 866 MiB/s wr, 1.38k op/s 2024-04-26T22:05:31.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:31 smithi064 ceph-mon[32765]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 866 MiB/s wr, 1.38k op/s 2024-04-26T22:05:31.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:31 smithi165 ceph-mon[33296]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 866 MiB/s wr, 1.38k op/s 2024-04-26T22:05:31.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.703+0000 7fe4927fc640 1 -- 172.21.15.55:0/2303839142 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484035020 con 0x7fe494073d70 2024-04-26T22:05:31.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.704+0000 7fe498ff9640 1 -- 172.21.15.55:0/2303839142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe494073d70 msgr2=0x7fe494074190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.704+0000 7fe498ff9640 1 --2- 172.21.15.55:0/2303839142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe494073d70 0x7fe494074190 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe484009900 tx=0x7fe48402a440 comp rx=0 tx=0).stop 2024-04-26T22:05:31.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.705+0000 7fe498ff9640 1 -- 172.21.15.55:0/2303839142 shutdown_connections 2024-04-26T22:05:31.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.705+0000 7fe498ff9640 1 --2- 172.21.15.55:0/2303839142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe49410a9b0 0x7fe49410f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.705+0000 7fe498ff9640 1 --2- 172.21.15.55:0/2303839142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe49410a010 0x7fe49410a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.706+0000 7fe498ff9640 1 --2- 172.21.15.55:0/2303839142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe494073d70 0x7fe494074190 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.707 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.706+0000 7fe498ff9640 1 -- 172.21.15.55:0/2303839142 >> 172.21.15.55:0/2303839142 conn(0x7fe49406f840 msgr2=0x7fe494071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.709+0000 7fe498ff9640 1 -- 172.21.15.55:0/2303839142 shutdown_connections 2024-04-26T22:05:31.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.711+0000 7fe498ff9640 1 -- 172.21.15.55:0/2303839142 wait complete. 2024-04-26T22:05:31.714 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.712+0000 7fe498ff9640 1 Processor -- start 2024-04-26T22:05:31.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.719+0000 7fe498ff9640 1 -- start start 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.719+0000 7fe498ff9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe494073d70 0x7fe4941a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.719+0000 7fe498ff9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe49410a010 0x7fe4941a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.719+0000 7fe498ff9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe49410a9b0 0x7fe4941aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.719+0000 7fe498ff9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe49411d220 con 0x7fe49410a9b0 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.719+0000 7fe498ff9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe49411d0a0 con 0x7fe494073d70 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.719+0000 7fe498ff9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe49411d3a0 con 0x7fe49410a010 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.721+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe49410a010 0x7fe4941a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.723+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe49410a010 0x7fe4941a7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:34988/0 (socket says 172.21.15.55:34988) 2024-04-26T22:05:31.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.723+0000 7fe492ffd640 1 -- 172.21.15.55:0/4136029229 learned_addr learned my addr 172.21.15.55:0/4136029229 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.727 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.726+0000 7fe493fff640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe49410a9b0 0x7fe4941aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.729 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.729+0000 7fe4937fe640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe494073d70 0x7fe4941a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.730+0000 7fe493fff640 1 -- 172.21.15.55:0/4136029229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe49410a010 msgr2=0x7fe4941a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.730+0000 7fe493fff640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe49410a010 0x7fe4941a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.730+0000 7fe493fff640 1 -- 172.21.15.55:0/4136029229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe494073d70 msgr2=0x7fe4941a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.730+0000 7fe493fff640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe494073d70 0x7fe4941a78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.730 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.730+0000 7fe493fff640 1 -- 172.21.15.55:0/4136029229 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4941af4f0 con 0x7fe49410a9b0 2024-04-26T22:05:31.731 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.731+0000 7fe493fff640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe49410a9b0 0x7fe4941aee80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c710 tx=0x7fe48800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.732+0000 7fe490ff9640 1 -- 172.21.15.55:0/4136029229 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488004220 con 0x7fe49410a9b0 2024-04-26T22:05:31.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.732+0000 7fe498ff9640 1 -- 172.21.15.55:0/4136029229 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4941af7e0 con 0x7fe49410a9b0 2024-04-26T22:05:31.733 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.732+0000 7fe498ff9640 1 -- 172.21.15.55:0/4136029229 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4941afd30 con 0x7fe49410a9b0 2024-04-26T22:05:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.737+0000 7fe49cf70740 1 Processor -- start 2024-04-26T22:05:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.737+0000 7fe490ff9640 1 -- 172.21.15.55:0/4136029229 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe48800ce20 con 0x7fe49410a9b0 2024-04-26T22:05:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.737+0000 7fe490ff9640 1 -- 172.21.15.55:0/4136029229 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488003db0 con 0x7fe49410a9b0 2024-04-26T22:05:31.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.741+0000 7fe490ff9640 1 -- 172.21.15.55:0/4136029229 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe48800f450 con 0x7fe49410a9b0 2024-04-26T22:05:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.743+0000 7fe490ff9640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe47c0a3e10 0x7fe47c0a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.747 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.746+0000 7fe490ff9640 1 -- 172.21.15.55:0/4136029229 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe4880d0080 con 0x7fe49410a9b0 2024-04-26T22:05:31.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.747+0000 7fe49cf70740 1 -- start start 2024-04-26T22:05:31.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.747+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 0x560b0e507de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.748+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e509040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.748 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.748+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e509580 0x560b0e50fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.748+0000 7fe49cf70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560b0e46bd80 con 0x560b0e508be0 2024-04-26T22:05:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.748+0000 7fe49cf70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560b0e425a20 con 0x560b0e5079c0 2024-04-26T22:05:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.748+0000 7fe49cf70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560b0e467d40 con 0x560b0e509580 2024-04-26T22:05:31.749 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.749+0000 7fe492ffd640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe47c0a3e10 0x7fe47c0a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.751+0000 7fe493fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e509580 0x560b0e50fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.753+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e509040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.753+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e509040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54684/0 (socket says 172.21.15.55:54684) 2024-04-26T22:05:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.753+0000 7fe492ffd640 1 -- 172.21.15.55:0/710884814 learned_addr learned my addr 172.21.15.55:0/710884814 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.755+0000 7fe4937fe640 1 --2- 172.21.15.55:0/710884814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 0x560b0e507de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.755+0000 7fe493fff640 1 -- 172.21.15.55:0/710884814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 msgr2=0x560b0e507de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.755+0000 7fe493fff640 1 --2- 172.21.15.55:0/710884814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 0x560b0e507de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.756+0000 7fe493fff640 1 -- 172.21.15.55:0/710884814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 msgr2=0x560b0e509040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.756+0000 7fe493fff640 1 --2- 172.21.15.55:0/710884814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e509040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.756+0000 7fe493fff640 1 -- 172.21.15.55:0/710884814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0e430c30 con 0x560b0e509580 2024-04-26T22:05:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.762+0000 7fe493fff640 1 --2- 172.21.15.55:0/710884814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e509580 0x560b0e50fe30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe4880cd960 tx=0x7fe48809bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d18fc86f273dc96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.763+0000 7fe492ffd640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe47c0a3e10 0x7fe47c0a62d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0138d0 tx=0x7fe48c013cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.766+0000 7fe498ff9640 1 -- 172.21.15.55:0/710884814 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4880b5070 con 0x560b0e509580 2024-04-26T22:05:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.766+0000 7fe498ff9640 1 -- 172.21.15.55:0/710884814 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4880a4dd0 con 0x560b0e509580 2024-04-26T22:05:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.766+0000 7fe498ff9640 1 -- 172.21.15.55:0/710884814 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48801d040 con 0x560b0e509580 2024-04-26T22:05:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.773+0000 7fe49cf70740 1 -- 172.21.15.55:0/710884814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e509580 msgr2=0x560b0e50fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.773+0000 7fe49cf70740 1 --2- 172.21.15.55:0/710884814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e509580 0x560b0e50fe30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe4880cd960 tx=0x7fe48809bbf0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.776+0000 7fe49cf70740 1 -- 172.21.15.55:0/710884814 shutdown_connections 2024-04-26T22:05:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.776+0000 7fe49cf70740 1 --2- 172.21.15.55:0/710884814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e509580 0x560b0e50fe30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.776+0000 7fe49cf70740 1 --2- 172.21.15.55:0/710884814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e509040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.776+0000 7fe49cf70740 1 --2- 172.21.15.55:0/710884814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 0x560b0e507de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.776+0000 7fe49cf70740 1 -- 172.21.15.55:0/710884814 >> 172.21.15.55:0/710884814 conn(0x560b0e448910 msgr2=0x560b0e5035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.782+0000 7fe49cf70740 1 -- 172.21.15.55:0/710884814 shutdown_connections 2024-04-26T22:05:31.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.789+0000 7fe49cf70740 1 -- 172.21.15.55:0/710884814 wait complete. 2024-04-26T22:05:31.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.789+0000 7fe49cf70740 1 -- start start 2024-04-26T22:05:31.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.790+0000 7fe49cf70740 1 Processor -- start 2024-04-26T22:05:31.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.790+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e5079c0 0x560b0e511cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.790+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e508be0 0x560b0e5121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.792 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.790+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e509580 0x560b0e519230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.790+0000 7fe49cf70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560b0e510550 con 0x560b0e509580 2024-04-26T22:05:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.790+0000 7fe49cf70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560b0e3f2dc0 con 0x560b0e508be0 2024-04-26T22:05:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.790+0000 7fe49cf70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560b0e5106d0 con 0x560b0e5079c0 2024-04-26T22:05:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.790+0000 7fe49cf70740 10 client.0 _ll_register_callbacks cb 0x560b0e40e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.793+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e508be0 0x560b0e5121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.793+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e508be0 0x560b0e5121f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45840/0 (socket says 172.21.15.55:45840) 2024-04-26T22:05:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.793+0000 7fe492ffd640 1 -- 172.21.15.55:0/890809780 learned_addr learned my addr 172.21.15.55:0/890809780 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.794+0000 7fe492ffd640 1 -- 172.21.15.55:0/890809780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e5079c0 msgr2=0x560b0e511cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:05:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.794+0000 7fe492ffd640 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e5079c0 0x560b0e511cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.794+0000 7fe492ffd640 1 -- 172.21.15.55:0/890809780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e509580 msgr2=0x560b0e519230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.794+0000 7fe492ffd640 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e509580 0x560b0e519230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.794+0000 7fe492ffd640 1 -- 172.21.15.55:0/890809780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b0e17d3f0 con 0x560b0e508be0 2024-04-26T22:05:31.798 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.796+0000 7fe492ffd640 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e508be0 0x560b0e5121f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe48c015da0 tx=0x7fe48c004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a7432e03bc4af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.799+0000 7fe472ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:05:31.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.799+0000 7fe472ffd640 1 -- 172.21.15.55:0/890809780 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c029060 con 0x560b0e508be0 2024-04-26T22:05:31.799 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.799+0000 7fe472ffd640 1 -- 172.21.15.55:0/890809780 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe48c018e00 con 0x560b0e508be0 2024-04-26T22:05:31.800 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.799+0000 7fe49cf70740 1 -- 172.21.15.55:0/890809780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4880a4dd0 con 0x560b0e508be0 2024-04-26T22:05:31.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.803+0000 7fe472ffd640 1 -- 172.21.15.55:0/890809780 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe48c024400 con 0x560b0e508be0 2024-04-26T22:05:31.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.803+0000 7fe49cf70740 10 client.26333 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:31.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.803+0000 7fe49cf70740 1 -- 172.21.15.55:0/890809780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560b0e2477d0 con 0x560b0e508be0 2024-04-26T22:05:31.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.805+0000 7fe472ffd640 1 -- 172.21.15.55:0/890809780 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4880a4dd0 con 0x560b0e508be0 2024-04-26T22:05:31.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.805+0000 7fe472ffd640 1 -- 172.21.15.55:0/890809780 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe48c02d5c0 con 0x560b0e508be0 2024-04-26T22:05:31.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.806+0000 7fe49cf70740 10 client.26333 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:31.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.806+0000 7fe49cf70740 10 client.26333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:31.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.806+0000 7fe49cf70740 20 client.26333 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:31.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.806+0000 7fe49cf70740 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560b0e53fab0 0x560b0e541f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.806+0000 7fe49cf70740 4 client.26333 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:31.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.806+0000 7fe49cf70740 1 -- 172.21.15.55:0/890809780 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b0e479990 con 0x560b0e53fab0 2024-04-26T22:05:31.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.818+0000 7fe4937fe640 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560b0e53fab0 0x560b0e541f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.825+0000 7fe4937fe640 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560b0e53fab0 0x560b0e541f30 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d79526a3a62262 server_cookie=e998b18528fe2131 in_seq=0 out_seq=0 2024-04-26T22:05:31.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.826+0000 7fe472ffd640 10 client.26333 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:31.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.833+0000 7fe472ffd640 1 -- 172.21.15.55:0/890809780 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b0e479990 con 0x560b0e53fab0 2024-04-26T22:05:31.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.833+0000 7fe472ffd640 10 client.26333 handle_command_reply: tid=0 2024-04-26T22:05:31.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.833+0000 7fe49cf70740 1 client.26333 shutdown 2024-04-26T22:05:31.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.839+0000 7fe49cf70740 1 -- 172.21.15.55:0/890809780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e508be0 msgr2=0x560b0e5121f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.840 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.839+0000 7fe49cf70740 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e508be0 0x560b0e5121f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe48c015da0 tx=0x7fe48c004a60 comp rx=0 tx=0).stop 2024-04-26T22:05:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.844+0000 7fe49cf70740 1 -- 172.21.15.55:0/890809780 shutdown_connections 2024-04-26T22:05:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.844+0000 7fe49cf70740 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560b0e53fab0 0x560b0e541f30 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.844+0000 7fe49cf70740 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e509580 0x560b0e519230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.844+0000 7fe49cf70740 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e508be0 0x560b0e5121f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.844+0000 7fe49cf70740 1 --2- 172.21.15.55:0/890809780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e5079c0 0x560b0e511cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.844+0000 7fe49cf70740 1 -- 172.21.15.55:0/890809780 >> 172.21.15.55:0/890809780 conn(0x560b0e448910 msgr2=0x560b0e502730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.844+0000 7fe472ffd640 0 client.26333 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:31.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.845+0000 7fe49cf70740 1 -- 172.21.15.55:0/890809780 shutdown_connections 2024-04-26T22:05:31.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.845+0000 7fe49cf70740 1 -- 172.21.15.55:0/890809780 wait complete. 2024-04-26T22:05:31.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.847+0000 7fe49cf70740 20 client.26333 trim_cache size 0 max 16384 2024-04-26T22:05:31.900 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.900+0000 7fe49cf70740 1 Processor -- start 2024-04-26T22:05:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.901+0000 7fe49cf70740 1 -- start start 2024-04-26T22:05:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.901+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e448910 0x560b0e4fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.901+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e5079c0 0x560b0e5191f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.901+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e508be0 0x560b0e506cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.901+0000 7fe49cf70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe480010f90 con 0x560b0e5079c0 2024-04-26T22:05:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.901+0000 7fe49cf70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe48000e860 con 0x560b0e448910 2024-04-26T22:05:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.901+0000 7fe49cf70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe48000c6b0 con 0x560b0e508be0 2024-04-26T22:05:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.902+0000 7fe493fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e508be0 0x560b0e506cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.902+0000 7fe493fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e508be0 0x560b0e506cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:34996/0 (socket says 172.21.15.55:34996) 2024-04-26T22:05:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.902+0000 7fe493fff640 1 -- 172.21.15.55:0/1340820394 learned_addr learned my addr 172.21.15.55:0/1340820394 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.902+0000 7fe4937fe640 1 --2- 172.21.15.55:0/1340820394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e448910 0x560b0e4fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.903+0000 7fe493fff640 1 -- 172.21.15.55:0/1340820394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e448910 msgr2=0x560b0e4fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.903+0000 7fe493fff640 1 --2- 172.21.15.55:0/1340820394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e448910 0x560b0e4fe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.903+0000 7fe493fff640 1 -- 172.21.15.55:0/1340820394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e5079c0 msgr2=0x560b0e5191f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.903+0000 7fe493fff640 1 --2- 172.21.15.55:0/1340820394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e5079c0 0x560b0e5191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.903+0000 7fe493fff640 1 -- 172.21.15.55:0/1340820394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b0e4ff010 con 0x560b0e508be0 2024-04-26T22:05:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.903+0000 7fe493fff640 1 --2- 172.21.15.55:0/1340820394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e508be0 0x560b0e506cd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe488003920 tx=0x7fe4880a0690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132f8ca760b8b440 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.905+0000 7fe4797fa640 1 -- 172.21.15.55:0/1340820394 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4880ba070 con 0x560b0e508be0 2024-04-26T22:05:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.905+0000 7fe4797fa640 1 -- 172.21.15.55:0/1340820394 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4880035f0 con 0x560b0e508be0 2024-04-26T22:05:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.906+0000 7fe49cf70740 1 -- 172.21.15.55:0/1340820394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e508be0 msgr2=0x560b0e506cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.906+0000 7fe49cf70740 1 --2- 172.21.15.55:0/1340820394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e508be0 0x560b0e506cd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe488003920 tx=0x7fe4880a0690 comp rx=0 tx=0).stop 2024-04-26T22:05:31.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.908+0000 7fe49cf70740 1 -- 172.21.15.55:0/1340820394 shutdown_connections 2024-04-26T22:05:31.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.908+0000 7fe49cf70740 1 --2- 172.21.15.55:0/1340820394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e508be0 0x560b0e506cd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.908+0000 7fe49cf70740 1 --2- 172.21.15.55:0/1340820394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e5079c0 0x560b0e5191f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.908+0000 7fe49cf70740 1 --2- 172.21.15.55:0/1340820394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e448910 0x560b0e4fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.908 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.908+0000 7fe49cf70740 1 -- 172.21.15.55:0/1340820394 >> 172.21.15.55:0/1340820394 conn(0x560b0e4feb70 msgr2=0x560b0e5065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.912 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.911+0000 7fe49cf70740 1 -- 172.21.15.55:0/1340820394 shutdown_connections 2024-04-26T22:05:31.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.911+0000 7fe49cf70740 1 -- 172.21.15.55:0/1340820394 wait complete. 2024-04-26T22:05:31.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.912+0000 7fe49cf70740 1 -- start start 2024-04-26T22:05:31.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.912+0000 7fe49cf70740 1 Processor -- start 2024-04-26T22:05:31.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.913+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e448910 0x560b0e514400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.913+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 0x560b0e514940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.913 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.913+0000 7fe49cf70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e5178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.913+0000 7fe49cf70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe48000cd20 con 0x560b0e508be0 2024-04-26T22:05:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.913+0000 7fe49cf70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe48000e190 con 0x560b0e5079c0 2024-04-26T22:05:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.913+0000 7fe49cf70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe48000d3f0 con 0x560b0e448910 2024-04-26T22:05:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.913+0000 7fe49cf70740 10 client.0 _ll_register_callbacks cb 0x7fe48c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.914+0000 7fe4937fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e448910 0x560b0e514400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.917+0000 7fe492ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 0x560b0e514940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.917+0000 7fe4937fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e448910 0x560b0e514400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:35010/0 (socket says 172.21.15.55:35010) 2024-04-26T22:05:31.917 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.917+0000 7fe4937fe640 1 -- 172.21.15.55:0/554025389 learned_addr learned my addr 172.21.15.55:0/554025389 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:31.919 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.919+0000 7fe493fff640 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e5178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.919+0000 7fe4937fe640 1 -- 172.21.15.55:0/554025389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 msgr2=0x560b0e514940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.919+0000 7fe4937fe640 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 0x560b0e514940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.919+0000 7fe4937fe640 1 -- 172.21.15.55:0/554025389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 msgr2=0x560b0e5178c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:31.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.919+0000 7fe4937fe640 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e5178c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.920 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.919+0000 7fe4937fe640 1 -- 172.21.15.55:0/554025389 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b0e517eb0 con 0x560b0e448910 2024-04-26T22:05:31.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.921+0000 7fe4937fe640 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e448910 0x560b0e514400 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe48402a960 tx=0x7fe484005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=114772fc8e804fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:31.921 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.921+0000 7fe472ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:05:31.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.926+0000 7fe472ffd640 1 -- 172.21.15.55:0/554025389 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484049070 con 0x560b0e448910 2024-04-26T22:05:31.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.926+0000 7fe49cf70740 1 -- 172.21.15.55:0/554025389 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4880035f0 con 0x560b0e448910 2024-04-26T22:05:31.926 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.926+0000 7fe472ffd640 1 -- 172.21.15.55:0/554025389 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe484045dd0 con 0x560b0e448910 2024-04-26T22:05:31.937 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.937+0000 7fe472ffd640 1 -- 172.21.15.55:0/554025389 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe484053050 con 0x560b0e448910 2024-04-26T22:05:31.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.938+0000 7fe49cf70740 10 client.26356 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:31.938 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.938+0000 7fe49cf70740 1 -- 172.21.15.55:0/554025389 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560b0e519220 con 0x560b0e448910 2024-04-26T22:05:31.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.941+0000 7fe472ffd640 1 -- 172.21.15.55:0/554025389 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4880035f0 con 0x560b0e448910 2024-04-26T22:05:31.941 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.941+0000 7fe472ffd640 1 -- 172.21.15.55:0/554025389 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe48403a040 con 0x560b0e448910 2024-04-26T22:05:31.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.942+0000 7fe49cf70740 10 client.26356 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:31.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.942+0000 7fe49cf70740 10 client.26356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:31.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.942+0000 7fe49cf70740 20 client.26356 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:31.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.942+0000 7fe49cf70740 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560b0e509580 0x560b0e5b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:31.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.942+0000 7fe49cf70740 4 client.26356 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:05:31.943 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.942+0000 7fe49cf70740 1 -- 172.21.15.55:0/554025389 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b0e473800 con 0x560b0e509580 2024-04-26T22:05:31.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.944+0000 7fe492ffd640 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560b0e509580 0x560b0e5b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:31.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.959+0000 7fe472ffd640 10 client.26356 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:31.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.959+0000 7fe492ffd640 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560b0e509580 0x560b0e5b9630 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93023b7fa319d3b1 server_cookie=e43ac5478bf45969 in_seq=0 out_seq=0 2024-04-26T22:05:31.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.964+0000 7fe472ffd640 1 -- 172.21.15.55:0/554025389 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b0e473800 con 0x560b0e509580 2024-04-26T22:05:31.964 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.964+0000 7fe472ffd640 10 client.26356 handle_command_reply: tid=0 2024-04-26T22:05:31.966 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.965+0000 7fe49cf70740 1 client.26356 shutdown 2024-04-26T22:05:31.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.968+0000 7fe49cf70740 1 -- 172.21.15.55:0/554025389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e448910 msgr2=0x560b0e514400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.968+0000 7fe49cf70740 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e448910 0x560b0e514400 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe48402a960 tx=0x7fe484005450 comp rx=0 tx=0).stop 2024-04-26T22:05:31.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.972+0000 7fe49cf70740 1 -- 172.21.15.55:0/554025389 shutdown_connections 2024-04-26T22:05:31.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.972+0000 7fe49cf70740 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560b0e509580 0x560b0e5b9630 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.972+0000 7fe49cf70740 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560b0e508be0 0x560b0e5178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.972+0000 7fe49cf70740 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560b0e5079c0 0x560b0e514940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.972+0000 7fe49cf70740 1 --2- 172.21.15.55:0/554025389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b0e448910 0x560b0e514400 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.972+0000 7fe472ffd640 0 client.26356 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:31.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.972+0000 7fe49cf70740 1 -- 172.21.15.55:0/554025389 >> 172.21.15.55:0/554025389 conn(0x560b0e4feb70 msgr2=0x560b0e506e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.975+0000 7fe49cf70740 1 -- 172.21.15.55:0/554025389 shutdown_connections 2024-04-26T22:05:31.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.980+0000 7fe49cf70740 1 -- 172.21.15.55:0/554025389 wait complete. 2024-04-26T22:05:31.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.980+0000 7fe49cf70740 20 client.26356 trim_cache size 0 max 16384 2024-04-26T22:05:31.981 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:05:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.983+0000 7fe47a7fc640 1 -- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe47c0a3e10 msgr2=0x7fe47c0a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.983+0000 7fe47a7fc640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe47c0a3e10 0x7fe47c0a62d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0138d0 tx=0x7fe48c013cd0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.983+0000 7fe47a7fc640 1 -- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe49410a9b0 msgr2=0x7fe4941aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.983+0000 7fe47a7fc640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe49410a9b0 0x7fe4941aee80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c710 tx=0x7fe48800cbd0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.990+0000 7fe47a7fc640 1 -- 172.21.15.55:0/4136029229 shutdown_connections 2024-04-26T22:05:31.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.990+0000 7fe47a7fc640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe47c0a3e10 0x7fe47c0a62d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.990+0000 7fe47a7fc640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe49410a9b0 0x7fe4941aee80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.990+0000 7fe47a7fc640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe49410a010 0x7fe4941a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.990+0000 7fe47a7fc640 1 --2- 172.21.15.55:0/4136029229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe494073d70 0x7fe4941a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:31.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.990+0000 7fe47a7fc640 1 -- 172.21.15.55:0/4136029229 >> 172.21.15.55:0/4136029229 conn(0x7fe49406f840 msgr2=0x7fe4940714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:31.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.995+0000 7fe47a7fc640 1 -- 172.21.15.55:0/4136029229 shutdown_connections 2024-04-26T22:05:31.999 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:31.998+0000 7fe47a7fc640 1 -- 172.21.15.55:0/4136029229 wait complete. 2024-04-26T22:05:33.036 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:05:33.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:05:33.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:05:33.438 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:33 smithi055 ceph-mon[29221]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 850 MiB/s wr, 1.24k op/s 2024-04-26T22:05:33.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.503+0000 7f0182fe9640 1 -- 172.21.15.55:0/383442658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f017c105d80 msgr2=0x7f017c106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:33.506 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.503+0000 7f0182fe9640 1 --2- 172.21.15.55:0/383442658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f017c105d80 0x7f017c106200 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0170009860 tx=0x7f017002a360 comp rx=0 tx=0).stop 2024-04-26T22:05:33.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:33 smithi064 ceph-mon[32765]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 850 MiB/s wr, 1.24k op/s 2024-04-26T22:05:33.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.511+0000 7f0182fe9640 1 -- 172.21.15.55:0/383442658 shutdown_connections 2024-04-26T22:05:33.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.511+0000 7f0182fe9640 1 --2- 172.21.15.55:0/383442658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c106740 0x7f017c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.511+0000 7f0182fe9640 1 --2- 172.21.15.55:0/383442658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f017c105d80 0x7f017c106200 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.511+0000 7f0182fe9640 1 --2- 172.21.15.55:0/383442658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f017c104b90 0x7f017c104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.511+0000 7f0182fe9640 1 -- 172.21.15.55:0/383442658 >> 172.21.15.55:0/383442658 conn(0x7f017c100120 msgr2=0x7f017c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:33.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.514+0000 7f0182fe9640 1 -- 172.21.15.55:0/383442658 shutdown_connections 2024-04-26T22:05:33.515 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.515+0000 7f0182fe9640 1 -- 172.21.15.55:0/383442658 wait complete. 2024-04-26T22:05:33.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.516+0000 7f0182fe9640 1 Processor -- start 2024-04-26T22:05:33.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.516+0000 7f0182fe9640 1 -- start start 2024-04-26T22:05:33.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.516+0000 7f0182fe9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f017c104b90 0x7f017c07a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.516+0000 7f0182fe9640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f017c105d80 0x7f017c07ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.516+0000 7f0182fe9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c106740 0x7f017c077250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.516+0000 7f0182fe9640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1106f0 con 0x7f017c105d80 2024-04-26T22:05:33.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.516+0000 7f0182fe9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f017c110570 con 0x7f017c106740 2024-04-26T22:05:33.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.516+0000 7f0182fe9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f017c110870 con 0x7f017c104b90 2024-04-26T22:05:33.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.518+0000 7f017bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f017c105d80 0x7f017c07ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.518+0000 7f017bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f017c105d80 0x7f017c07ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54724/0 (socket says 172.21.15.55:54724) 2024-04-26T22:05:33.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.518+0000 7f017bfff640 1 -- 172.21.15.55:0/321903268 learned_addr learned my addr 172.21.15.55:0/321903268 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:33.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.520+0000 7f0181586640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c106740 0x7f017c077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.520 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.520+0000 7f0180d85640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f017c104b90 0x7f017c07a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.521+0000 7f017bfff640 1 -- 172.21.15.55:0/321903268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f017c104b90 msgr2=0x7f017c07a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:33.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.521+0000 7f017bfff640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f017c104b90 0x7f017c07a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.521+0000 7f017bfff640 1 -- 172.21.15.55:0/321903268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c106740 msgr2=0x7f017c077250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:33.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.521+0000 7f017bfff640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c106740 0x7f017c077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.521+0000 7f017bfff640 1 -- 172.21.15.55:0/321903268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c077b10 con 0x7f017c105d80 2024-04-26T22:05:33.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.523+0000 7f017bfff640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f017c105d80 0x7f017c07ac20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f017002a950 tx=0x7f0170032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:33.524 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.524+0000 7f0179ffb640 1 -- 172.21.15.55:0/321903268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170037050 con 0x7f017c105d80 2024-04-26T22:05:33.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.525+0000 7f0182fe9640 1 -- 172.21.15.55:0/321903268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c1a72f0 con 0x7f017c105d80 2024-04-26T22:05:33.526 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.525+0000 7f0182fe9640 1 -- 172.21.15.55:0/321903268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c1a76a0 con 0x7f017c105d80 2024-04-26T22:05:33.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.528+0000 7f0179ffb640 1 -- 172.21.15.55:0/321903268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0170038040 con 0x7f017c105d80 2024-04-26T22:05:33.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.529+0000 7f0179ffb640 1 -- 172.21.15.55:0/321903268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170004920 con 0x7f017c105d80 2024-04-26T22:05:33.540 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.539+0000 7f0179ffb640 1 -- 172.21.15.55:0/321903268 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0170004ac0 con 0x7f017c105d80 2024-04-26T22:05:33.542 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.542+0000 7f0185dde740 1 Processor -- start 2024-04-26T22:05:33.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.542+0000 7f0185dde740 1 -- start start 2024-04-26T22:05:33.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.542+0000 7f0185dde740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.542+0000 7f0185dde740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.542+0000 7f0185dde740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.542+0000 7f0185dde740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bdc2815b00 con 0x55bdc28b12a0 2024-04-26T22:05:33.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.542+0000 7f0185dde740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bdc27b1920 con 0x55bdc28b6410 2024-04-26T22:05:33.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.542+0000 7f0185dde740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bdc2811ac0 con 0x55bdc28b08b0 2024-04-26T22:05:33.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.543+0000 7f0180d85640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.543+0000 7f0180d85640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54732/0 (socket says 172.21.15.55:54732) 2024-04-26T22:05:33.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.543+0000 7f0180d85640 1 -- 172.21.15.55:0/317553487 learned_addr learned my addr 172.21.15.55:0/317553487 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:33.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.544+0000 7f0179ffb640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01640a3e40 0x7f01640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.544 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.544+0000 7f017bfff640 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.544+0000 7f0179ffb640 1 -- 172.21.15.55:0/321903268 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f01700f1ba0 con 0x7f017c105d80 2024-04-26T22:05:33.546 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.545+0000 7f0181586640 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.548 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.547+0000 7f0180d85640 1 -- 172.21.15.55:0/317553487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 msgr2=0x55bdc28b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.547+0000 7f0180d85640 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.547+0000 7f0180d85640 1 -- 172.21.15.55:0/317553487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 msgr2=0x55bdc28b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.547+0000 7f0180d85640 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.547+0000 7f0180d85640 1 -- 172.21.15.55:0/317553487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc25ef7e0 con 0x55bdc28b12a0 2024-04-26T22:05:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.548+0000 7f017bfff640 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.548+0000 7f0181586640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01640a3e40 0x7f01640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.549+0000 7f0180d85640 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28b5da0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f017400f430 tx=0x7f017400f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ac213b78c6cb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:33.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.550+0000 7f016affd640 1 -- 172.21.15.55:0/317553487 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017401d070 con 0x55bdc28b12a0 2024-04-26T22:05:33.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.550+0000 7f016affd640 1 -- 172.21.15.55:0/317553487 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f017400cb50 con 0x55bdc28b12a0 2024-04-26T22:05:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.552+0000 7f0185dde740 1 -- 172.21.15.55:0/317553487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 msgr2=0x55bdc28b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.552+0000 7f0185dde740 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28b5da0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f017400f430 tx=0x7f017400f8f0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.554+0000 7f0185dde740 1 -- 172.21.15.55:0/317553487 shutdown_connections 2024-04-26T22:05:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.554+0000 7f0185dde740 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.554+0000 7f0185dde740 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28b5da0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.554+0000 7f0185dde740 1 --2- 172.21.15.55:0/317553487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.554+0000 7f0185dde740 1 -- 172.21.15.55:0/317553487 >> 172.21.15.55:0/317553487 conn(0x55bdc28abf20 msgr2=0x55bdc28ac320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:33.558 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.556+0000 7f0181586640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01640a3e40 0x7f01640a6300 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f016c005fd0 tx=0x7f016c005a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:33.560 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.558+0000 7f0185dde740 1 -- 172.21.15.55:0/317553487 shutdown_connections 2024-04-26T22:05:33.563 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.560+0000 7f0185dde740 1 -- 172.21.15.55:0/317553487 wait complete. 2024-04-26T22:05:33.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.561+0000 7f0185dde740 1 -- start start 2024-04-26T22:05:33.569 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.561+0000 7f0185dde740 1 Processor -- start 2024-04-26T22:05:33.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.568+0000 7f0185dde740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.568+0000 7f0185dde740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b12a0 0x55bdc28cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.568+0000 7f0185dde740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.569+0000 7f0185dde740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55bdc28ca420 con 0x55bdc28b6410 2024-04-26T22:05:33.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.569+0000 7f0185dde740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bdc27af5d0 con 0x55bdc28b12a0 2024-04-26T22:05:33.570 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.569+0000 7f0185dde740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bdc28ca5a0 con 0x55bdc28b08b0 2024-04-26T22:05:33.571 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.569+0000 7f0185dde740 10 client.0 _ll_register_callbacks cb 0x55bdc279d080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:33.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.570+0000 7f0181586640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.570+0000 7f0181586640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54746/0 (socket says 172.21.15.55:54746) 2024-04-26T22:05:33.572 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.570+0000 7f0181586640 1 -- 172.21.15.55:0/2845758076 learned_addr learned my addr 172.21.15.55:0/2845758076 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:33.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.571+0000 7f0180d85640 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.573 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.571+0000 7f017bfff640 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b12a0 0x55bdc28cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.572+0000 7f0180d85640 1 -- 172.21.15.55:0/2845758076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b12a0 msgr2=0x55bdc28cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:33.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.572+0000 7f0180d85640 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b12a0 0x55bdc28cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.572+0000 7f0180d85640 1 -- 172.21.15.55:0/2845758076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 msgr2=0x55bdc28d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:33.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.572+0000 7f0180d85640 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.572+0000 7f0180d85640 1 -- 172.21.15.55:0/2845758076 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc264ff70 con 0x55bdc28b08b0 2024-04-26T22:05:33.574 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.573+0000 7f0180d85640 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cbd00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55bdc28acdf0 tx=0x7f017400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=189b48af93152c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:33.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:33 smithi165 ceph-mon[33296]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 850 MiB/s wr, 1.24k op/s 2024-04-26T22:05:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.577+0000 7f015ffff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:05:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.577+0000 7f015ffff640 1 -- 172.21.15.55:0/2845758076 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174029060 con 0x55bdc28b08b0 2024-04-26T22:05:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.578+0000 7f0185dde740 1 -- 172.21.15.55:0/2845758076 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f017400cb50 con 0x55bdc28b08b0 2024-04-26T22:05:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.578+0000 7f015ffff640 1 -- 172.21.15.55:0/2845758076 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f0174003970 con 0x55bdc28b08b0 2024-04-26T22:05:33.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.578+0000 7f015ffff640 1 -- 172.21.15.55:0/2845758076 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f017401d040 con 0x55bdc28b08b0 2024-04-26T22:05:33.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.581+0000 7f0185dde740 10 client.26368 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:33.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.581+0000 7f0185dde740 1 -- 172.21.15.55:0/2845758076 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55bdc24e7170 con 0x55bdc28b08b0 2024-04-26T22:05:33.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.588+0000 7f015ffff640 1 -- 172.21.15.55:0/2845758076 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f017400cb50 con 0x55bdc28b08b0 2024-04-26T22:05:33.589 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.588+0000 7f015ffff640 1 -- 172.21.15.55:0/2845758076 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f017400b040 con 0x55bdc28b08b0 2024-04-26T22:05:33.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.589+0000 7f0185dde740 10 client.26368 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:33.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.589+0000 7f0185dde740 10 client.26368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:33.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.589+0000 7f0185dde740 20 client.26368 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:33.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.589+0000 7f0185dde740 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bdc28d7490 0x55bdc28d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.589+0000 7f0185dde740 4 client.26368 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:33.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.589+0000 7f0185dde740 1 -- 172.21.15.55:0/2845758076 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdc2823710 con 0x55bdc28d7490 2024-04-26T22:05:33.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.593+0000 7f017bfff640 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bdc28d7490 0x55bdc28d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.605+0000 7f017bfff640 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bdc28d7490 0x55bdc28d9880 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e412e5b39904fc78 server_cookie=8df8e7776869fcc in_seq=0 out_seq=0 2024-04-26T22:05:33.610 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.607+0000 7f015ffff640 10 client.26368 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:33.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.610+0000 7f015ffff640 1 -- 172.21.15.55:0/2845758076 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdc2823710 con 0x55bdc28d7490 2024-04-26T22:05:33.612 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.610+0000 7f015ffff640 10 client.26368 handle_command_reply: tid=0 2024-04-26T22:05:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.615+0000 7f0185dde740 1 client.26368 shutdown 2024-04-26T22:05:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f0185dde740 1 -- 172.21.15.55:0/2845758076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 msgr2=0x55bdc28cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f0185dde740 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cbd00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55bdc28acdf0 tx=0x7f017400c8d0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f0185dde740 1 -- 172.21.15.55:0/2845758076 shutdown_connections 2024-04-26T22:05:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f0185dde740 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bdc28d7490 0x55bdc28d9880 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f0185dde740 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f0185dde740 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b12a0 0x55bdc28cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f0185dde740 1 --2- 172.21.15.55:0/2845758076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cbd00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f0185dde740 1 -- 172.21.15.55:0/2845758076 >> 172.21.15.55:0/2845758076 conn(0x55bdc28abf20 msgr2=0x55bdc28ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.618+0000 7f015ffff640 0 client.26368 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.621+0000 7f0185dde740 1 -- 172.21.15.55:0/2845758076 shutdown_connections 2024-04-26T22:05:33.625 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.624+0000 7f0185dde740 1 -- 172.21.15.55:0/2845758076 wait complete. 2024-04-26T22:05:33.626 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.624+0000 7f0185dde740 20 client.26368 trim_cache size 0 max 16384 2024-04-26T22:05:33.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.687+0000 7f0185dde740 1 Processor -- start 2024-04-26T22:05:33.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.688+0000 7f0185dde740 1 -- start start 2024-04-26T22:05:33.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.688+0000 7f0185dde740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b08b0 0x55bdc28aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.688+0000 7f0185dde740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b12a0 0x55bdc28caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.688+0000 7f0185dde740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.688+0000 7f0185dde740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0160010fb0 con 0x55bdc28b6410 2024-04-26T22:05:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.688+0000 7f0185dde740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f016000e880 con 0x55bdc28b08b0 2024-04-26T22:05:33.689 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.688+0000 7f0185dde740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f016000c6d0 con 0x55bdc28b12a0 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0181586640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0181586640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54750/0 (socket says 172.21.15.55:54750) 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0181586640 1 -- 172.21.15.55:0/1751085446 learned_addr learned my addr 172.21.15.55:0/1751085446 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0180d85640 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b08b0 0x55bdc28aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f017bfff640 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b12a0 0x55bdc28caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0180d85640 1 -- 172.21.15.55:0/1751085446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b12a0 msgr2=0x55bdc28caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0180d85640 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b12a0 0x55bdc28caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0180d85640 1 -- 172.21.15.55:0/1751085446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 msgr2=0x55bdc28cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0180d85640 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.690 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.689+0000 7f0180d85640 1 -- 172.21.15.55:0/1751085446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc28ccac0 con 0x55bdc28b08b0 2024-04-26T22:05:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.690+0000 7f0180d85640 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b08b0 0x55bdc28aaf60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0174005e30 tx=0x7f01740095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c4ab2b6e7428d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.691+0000 7f016a7fc640 1 -- 172.21.15.55:0/1751085446 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174033020 con 0x55bdc28b08b0 2024-04-26T22:05:33.691 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.691+0000 7f016a7fc640 1 -- 172.21.15.55:0/1751085446 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01740055c0 con 0x55bdc28b08b0 2024-04-26T22:05:33.692 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.691+0000 7f016a7fc640 1 -- 172.21.15.55:0/1751085446 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174029070 con 0x55bdc28b08b0 2024-04-26T22:05:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.693+0000 7f0185dde740 1 -- 172.21.15.55:0/1751085446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b08b0 msgr2=0x55bdc28aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:33.694 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.693+0000 7f0185dde740 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b08b0 0x55bdc28aaf60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0174005e30 tx=0x7f01740095e0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.696+0000 7f0185dde740 1 -- 172.21.15.55:0/1751085446 shutdown_connections 2024-04-26T22:05:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.696+0000 7f0185dde740 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b6410 0x55bdc28cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.696+0000 7f0185dde740 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b12a0 0x55bdc28caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.696+0000 7f0185dde740 1 --2- 172.21.15.55:0/1751085446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b08b0 0x55bdc28aaf60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.696+0000 7f0185dde740 1 -- 172.21.15.55:0/1751085446 >> 172.21.15.55:0/1751085446 conn(0x55bdc28abf20 msgr2=0x55bdc28aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:33.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.696+0000 7f0185dde740 1 -- 172.21.15.55:0/1751085446 shutdown_connections 2024-04-26T22:05:33.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.696+0000 7f0185dde740 1 -- 172.21.15.55:0/1751085446 wait complete. 2024-04-26T22:05:33.698 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.697+0000 7f0185dde740 1 -- start start 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.697+0000 7f0185dde740 1 Processor -- start 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.698+0000 7f0185dde740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.698+0000 7f0185dde740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.698+0000 7f0185dde740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.698+0000 7f0185dde740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f016000cd40 con 0x55bdc28b12a0 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.698+0000 7f0185dde740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f016000e1b0 con 0x55bdc28b6410 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.698+0000 7f0185dde740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f016000d410 con 0x55bdc28b08b0 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.698+0000 7f0185dde740 10 client.0 _ll_register_callbacks cb 0x7f0174018590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:33.699 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.699+0000 7f017bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.699+0000 7f017bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54758/0 (socket says 172.21.15.55:54758) 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.699+0000 7f017bfff640 1 -- 172.21.15.55:0/2800758739 learned_addr learned my addr 172.21.15.55:0/2800758739 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.699+0000 7f0180d85640 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.699+0000 7f0181586640 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.700+0000 7f017bfff640 1 -- 172.21.15.55:0/2800758739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 msgr2=0x55bdc28cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.700+0000 7f017bfff640 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.700+0000 7f017bfff640 1 -- 172.21.15.55:0/2800758739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 msgr2=0x55bdc28d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.700+0000 7f017bfff640 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28d1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:33.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.700+0000 7f017bfff640 1 -- 172.21.15.55:0/2800758739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc28d1880 con 0x55bdc28b12a0 2024-04-26T22:05:33.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.700+0000 7f0180d85640 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:05:33.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.700+0000 7f017bfff640 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28d02f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f01700ba270 tx=0x7f017003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fbbce9247289b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:33.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.700+0000 7f015ffff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:05:33.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.701+0000 7f015ffff640 1 -- 172.21.15.55:0/2800758739 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01700bde10 con 0x55bdc28b12a0 2024-04-26T22:05:33.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.701+0000 7f015ffff640 1 -- 172.21.15.55:0/2800758739 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f01700de2b0 con 0x55bdc28b12a0 2024-04-26T22:05:33.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.701+0000 7f0185dde740 1 -- 172.21.15.55:0/2800758739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01740055c0 con 0x55bdc28b12a0 2024-04-26T22:05:33.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.701+0000 7f015ffff640 1 -- 172.21.15.55:0/2800758739 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f01700c21c0 con 0x55bdc28b12a0 2024-04-26T22:05:33.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.701+0000 7f0185dde740 10 client.17427 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:33.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.701+0000 7f0185dde740 1 -- 172.21.15.55:0/2800758739 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55bdc293d830 con 0x55bdc28b12a0 2024-04-26T22:05:33.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.702+0000 7f015ffff640 1 -- 172.21.15.55:0/2800758739 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01740055c0 con 0x55bdc28b12a0 2024-04-26T22:05:33.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.704+0000 7f015ffff640 1 -- 172.21.15.55:0/2800758739 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f01700ba480 con 0x55bdc28b12a0 2024-04-26T22:05:33.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.704+0000 7f0185dde740 10 client.17427 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:33.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.704+0000 7f0185dde740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:33.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.704+0000 7f0185dde740 20 client.17427 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:33.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.704+0000 7f0185dde740 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bdc293e1a0 0x55bdc29405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:33.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.704+0000 7f0185dde740 4 client.17427 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:05:33.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.704+0000 7f0185dde740 1 -- 172.21.15.55:0/2800758739 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bdc281d580 con 0x55bdc293e1a0 2024-04-26T22:05:33.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.706+0000 7f0180d85640 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bdc293e1a0 0x55bdc29405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:33.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.709+0000 7f0180d85640 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bdc293e1a0 0x55bdc29405c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb72594592e280 server_cookie=97b4ca265db689c9 in_seq=0 out_seq=0 2024-04-26T22:05:33.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:33.709+0000 7f015ffff640 10 client.17427 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:34.527 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f015ffff640 1 -- 172.21.15.55:0/2800758739 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bdc281d580 con 0x55bdc293e1a0 2024-04-26T22:05:34.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f015ffff640 10 client.17427 handle_command_reply: tid=0 2024-04-26T22:05:34.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 client.17427 shutdown 2024-04-26T22:05:34.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 -- 172.21.15.55:0/2800758739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 msgr2=0x55bdc28d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28d02f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f01700ba270 tx=0x7f017003c000 comp rx=0 tx=0).stop 2024-04-26T22:05:34.528 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 -- 172.21.15.55:0/2800758739 shutdown_connections 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bdc28b6410 0x55bdc28d1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55bdc293e1a0 0x55bdc29405c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55bdc28b12a0 0x55bdc28d02f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 --2- 172.21.15.55:0/2800758739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bdc28b08b0 0x55bdc28cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f0185dde740 1 -- 172.21.15.55:0/2800758739 >> 172.21.15.55:0/2800758739 conn(0x55bdc28abf20 msgr2=0x55bdc28ab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.527+0000 7f015ffff640 0 client.17427 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.528+0000 7f0185dde740 1 -- 172.21.15.55:0/2800758739 shutdown_connections 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.528+0000 7f0185dde740 1 -- 172.21.15.55:0/2800758739 wait complete. 2024-04-26T22:05:34.529 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.528+0000 7f0185dde740 20 client.17427 trim_cache size 0 max 16384 2024-04-26T22:05:34.530 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:05:34.530 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:05:34.530 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "be6c01a9-b2f7-43e8-adc8-08704fe4e635", 2024-04-26T22:05:34.530 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:05:34.530 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:05:34.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.533+0000 7f0182fe9640 1 -- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01640a3e40 msgr2=0x7f01640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.533+0000 7f0182fe9640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01640a3e40 0x7f01640a6300 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f016c005fd0 tx=0x7f016c005a60 comp rx=0 tx=0).stop 2024-04-26T22:05:34.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.533+0000 7f0182fe9640 1 -- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f017c105d80 msgr2=0x7f017c07ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.533 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.533+0000 7f0182fe9640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f017c105d80 0x7f017c07ac20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f017002a950 tx=0x7f0170032cb0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.537+0000 7f0182fe9640 1 -- 172.21.15.55:0/321903268 shutdown_connections 2024-04-26T22:05:34.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.537+0000 7f0182fe9640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f01640a3e40 0x7f01640a6300 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.537+0000 7f0182fe9640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f017c106740 0x7f017c077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.537 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.537+0000 7f0182fe9640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f017c105d80 0x7f017c07ac20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.537+0000 7f0182fe9640 1 --2- 172.21.15.55:0/321903268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f017c104b90 0x7f017c07a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.538 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.537+0000 7f0182fe9640 1 -- 172.21.15.55:0/321903268 >> 172.21.15.55:0/321903268 conn(0x7f017c100120 msgr2=0x7f017c109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:34.543 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.543+0000 7f0182fe9640 1 -- 172.21.15.55:0/321903268 shutdown_connections 2024-04-26T22:05:34.545 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.544+0000 7f0182fe9640 1 -- 172.21.15.55:0/321903268 wait complete. 2024-04-26T22:05:34.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:05:34.610 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.868+0000 7f04c554e640 1 -- 172.21.15.55:0/509857350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 msgr2=0x7f04c0079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.868+0000 7f04c554e640 1 --2- 172.21.15.55:0/509857350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 0x7f04c0079770 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009900 tx=0x7f04b802a410 comp rx=0 tx=0).stop 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.868+0000 7f04c554e640 1 -- 172.21.15.55:0/509857350 shutdown_connections 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.868+0000 7f04c554e640 1 --2- 172.21.15.55:0/509857350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 0x7f04c0079770 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.868+0000 7f04c554e640 1 --2- 172.21.15.55:0/509857350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c0078530 0x7f04c0078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.868+0000 7f04c554e640 1 --2- 172.21.15.55:0/509857350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04c00772f0 0x7f04c0077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.868+0000 7f04c554e640 1 -- 172.21.15.55:0/509857350 >> 172.21.15.55:0/509857350 conn(0x7f04c01000e0 msgr2=0x7f04c0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.868+0000 7f04c554e640 1 -- 172.21.15.55:0/509857350 shutdown_connections 2024-04-26T22:05:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.869+0000 7f04c554e640 1 -- 172.21.15.55:0/509857350 wait complete. 2024-04-26T22:05:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.869+0000 7f04c554e640 1 Processor -- start 2024-04-26T22:05:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04c554e640 1 -- start start 2024-04-26T22:05:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04c554e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c00772f0 0x7f04c019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04c554e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04c0078530 0x7f04c019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04c554e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 0x7f04c01a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04c554e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0114bd0 con 0x7f04c0078ed0 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04bf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 0x7f04c01a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04bf7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 0x7f04c01a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54764/0 (socket says 172.21.15.55:54764) 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04bf7fe640 1 -- 172.21.15.55:0/4146830814 learned_addr learned my addr 172.21.15.55:0/4146830814 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0114a50 con 0x7f04c00772f0 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.870+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0114d50 con 0x7f04c0078530 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.871+0000 7f04beffd640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c00772f0 0x7f04c019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.871+0000 7f04be7fc640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04c0078530 0x7f04c019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.871+0000 7f04bf7fe640 1 -- 172.21.15.55:0/4146830814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04c0078530 msgr2=0x7f04c019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.871+0000 7f04bf7fe640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04c0078530 0x7f04c019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.871+0000 7f04bf7fe640 1 -- 172.21.15.55:0/4146830814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c00772f0 msgr2=0x7f04c019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.871+0000 7f04bf7fe640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c00772f0 0x7f04c019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.871+0000 7f04bf7fe640 1 -- 172.21.15.55:0/4146830814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c01a6f00 con 0x7f04c0078ed0 2024-04-26T22:05:34.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.871+0000 7f04bf7fe640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 0x7f04c01a6890 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f04b802a930 tx=0x7f04b8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.872+0000 7f04b7fff640 1 -- 172.21.15.55:0/4146830814 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b803f070 con 0x7f04c0078ed0 2024-04-26T22:05:34.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.872+0000 7f04b7fff640 1 -- 172.21.15.55:0/4146830814 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b803b460 con 0x7f04c0078ed0 2024-04-26T22:05:34.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.872+0000 7f04b7fff640 1 -- 172.21.15.55:0/4146830814 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8035030 con 0x7f04c0078ed0 2024-04-26T22:05:34.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.872+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c01a7190 con 0x7f04c0078ed0 2024-04-26T22:05:34.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.872+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c01a7690 con 0x7f04c0078ed0 2024-04-26T22:05:34.877 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.877+0000 7f04c825f740 1 Processor -- start 2024-04-26T22:05:34.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.878+0000 7f04b7fff640 1 -- 172.21.15.55:0/4146830814 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04b8042070 con 0x7f04c0078ed0 2024-04-26T22:05:34.878 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.878+0000 7f04c825f740 1 -- start start 2024-04-26T22:05:34.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.878+0000 7f04c825f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3dd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.878+0000 7f04c825f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 0x55d89b3de530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.878+0000 7f04c825f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3dea70 0x55d89b3e5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.878+0000 7f04c825f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d89b341d80 con 0x55d89b3dcee0 2024-04-26T22:05:34.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.878+0000 7f04c825f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d89b2fb8b0 con 0x55d89b3de0b0 2024-04-26T22:05:34.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.878+0000 7f04c825f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d89b33dd40 con 0x55d89b3dea70 2024-04-26T22:05:34.879 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.879+0000 7f04be7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3dd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.879+0000 7f04be7fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3dd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54770/0 (socket says 172.21.15.55:54770) 2024-04-26T22:05:34.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.879+0000 7f04be7fc640 1 -- 172.21.15.55:0/3527475079 learned_addr learned my addr 172.21.15.55:0/3527475079 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:34.880 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.879+0000 7f04beffd640 1 --2- 172.21.15.55:0/3527475079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 0x55d89b3de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.879+0000 7f04be7fc640 1 -- 172.21.15.55:0/3527475079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3dea70 msgr2=0x55d89b3e5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:34.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.879+0000 7f04be7fc640 1 --2- 172.21.15.55:0/3527475079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3dea70 0x55d89b3e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.879+0000 7f04be7fc640 1 -- 172.21.15.55:0/3527475079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 msgr2=0x55d89b3de530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:34.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.880+0000 7f04be7fc640 1 --2- 172.21.15.55:0/3527475079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 0x55d89b3de530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.880+0000 7f04b7fff640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04ac0a3d40 0x7f04ac0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.880+0000 7f04beffd640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04ac0a3d40 0x7f04ac0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.881 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.880+0000 7f04be7fc640 1 -- 172.21.15.55:0/3527475079 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89b2cca40 con 0x55d89b3dcee0 2024-04-26T22:05:34.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.880+0000 7f04b7fff640 1 -- 172.21.15.55:0/4146830814 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f04b80f0f30 con 0x7f04c0078ed0 2024-04-26T22:05:34.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.881+0000 7f04be7fc640 1 --2- 172.21.15.55:0/3527475079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3dd300 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009a10 tx=0x7f04a8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbd6f53b4bcf80c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.881+0000 7f04b57fa640 1 -- 172.21.15.55:0/3527475079 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a800fc30 con 0x55d89b3dcee0 2024-04-26T22:05:34.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.881+0000 7f04beffd640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04ac0a3d40 0x7f04ac0a6200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009fd0 tx=0x7f04b000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.881+0000 7f04b57fa640 1 -- 172.21.15.55:0/3527475079 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a8019070 con 0x55d89b3dcee0 2024-04-26T22:05:34.882 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.881+0000 7f04b57fa640 1 -- 172.21.15.55:0/3527475079 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a80135d0 con 0x55d89b3dcee0 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.882+0000 7f04c825f740 1 -- 172.21.15.55:0/3527475079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 msgr2=0x55d89b3dd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.882+0000 7f04c825f740 1 --2- 172.21.15.55:0/3527475079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3dd300 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009a10 tx=0x7f04a8009ed0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 -- 172.21.15.55:0/3527475079 shutdown_connections 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 --2- 172.21.15.55:0/3527475079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3dea70 0x55d89b3e5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 --2- 172.21.15.55:0/3527475079 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 0x55d89b3de530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 --2- 172.21.15.55:0/3527475079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3dd300 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 -- 172.21.15.55:0/3527475079 >> 172.21.15.55:0/3527475079 conn(0x55d89b3d83d0 msgr2=0x55d89b3d87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 -- 172.21.15.55:0/3527475079 shutdown_connections 2024-04-26T22:05:34.883 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 -- 172.21.15.55:0/3527475079 wait complete. 2024-04-26T22:05:34.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 -- start start 2024-04-26T22:05:34.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.883+0000 7f04c825f740 1 Processor -- start 2024-04-26T22:05:34.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04c825f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.884 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04c825f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 0x55d89b3e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04c825f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3dea70 0x55d89b3ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04c825f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d89b3e5a80 con 0x55d89b3dcee0 2024-04-26T22:05:34.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04c825f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d89b2c8d10 con 0x55d89b3de0b0 2024-04-26T22:05:34.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04c825f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d89b3e5c00 con 0x55d89b3dea70 2024-04-26T22:05:34.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04beffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.885 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04beffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3e73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54786/0 (socket says 172.21.15.55:54786) 2024-04-26T22:05:34.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04beffd640 1 -- 172.21.15.55:0/52121099 learned_addr learned my addr 172.21.15.55:0/52121099 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:34.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.884+0000 7f04c825f740 10 client.0 _ll_register_callbacks cb 0x55d89b2f5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:34.886 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.885+0000 7f04beffd640 1 -- 172.21.15.55:0/52121099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3dea70 msgr2=0x55d89b3ee970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.885+0000 7f04beffd640 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3dea70 0x55d89b3ee970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.885+0000 7f04beffd640 1 -- 172.21.15.55:0/52121099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 msgr2=0x55d89b3e7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.885+0000 7f04beffd640 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 0x55d89b3e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.885+0000 7f04beffd640 1 -- 172.21.15.55:0/52121099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d89b11d7d0 con 0x55d89b3dcee0 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.885+0000 7f04beffd640 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3e73f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f04b0018b50 tx=0x7f04b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4b85423d09109e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.885+0000 7f04a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.886+0000 7f04a67fc640 1 -- 172.21.15.55:0/52121099 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0028060 con 0x55d89b3dcee0 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.886+0000 7f04a67fc640 1 -- 172.21.15.55:0/52121099 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f04b0023440 con 0x55d89b3dcee0 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.886+0000 7f04c825f740 1 -- 172.21.15.55:0/52121099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a8019070 con 0x55d89b3dcee0 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.886+0000 7f04a67fc640 1 -- 172.21.15.55:0/52121099 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f04b0023640 con 0x55d89b3dcee0 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.886+0000 7f04c825f740 10 client.17439 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:34.887 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.886+0000 7f04c825f740 1 -- 172.21.15.55:0/52121099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d89b0533f0 con 0x55d89b3dcee0 2024-04-26T22:05:34.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.887+0000 7f04a67fc640 1 -- 172.21.15.55:0/52121099 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a8019070 con 0x55d89b3dcee0 2024-04-26T22:05:34.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.888+0000 7f04a67fc640 1 -- 172.21.15.55:0/52121099 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f04b00237e0 con 0x55d89b3dcee0 2024-04-26T22:05:34.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.888+0000 7f04c825f740 10 client.17439 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:34.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.888+0000 7f04c825f740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:34.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.888+0000 7f04c825f740 20 client.17439 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:34.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.889+0000 7f04c825f740 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d89b3f2fc0 0x55d89b3f5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.889+0000 7f04c825f740 4 client.17439 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:05:34.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.889+0000 7f04c825f740 1 -- 172.21.15.55:0/52121099 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d89b34f990 con 0x55d89b3f2fc0 2024-04-26T22:05:34.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.890+0000 7f04be7fc640 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d89b3f2fc0 0x55d89b3f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.892+0000 7f04be7fc640 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d89b3f2fc0 0x55d89b3f5480 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5025fb729aa79f68 server_cookie=712fa76b9a7423eb in_seq=0 out_seq=0 2024-04-26T22:05:34.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.892+0000 7f04a67fc640 10 client.17439 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04a67fc640 1 -- 172.21.15.55:0/52121099 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d89b34f990 con 0x55d89b3f2fc0 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04a67fc640 10 client.17439 handle_command_reply: tid=0 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 client.17439 shutdown 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 -- 172.21.15.55:0/52121099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 msgr2=0x55d89b3e73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3e73f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f04b0018b50 tx=0x7f04b0004a40 comp rx=0 tx=0).stop 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 -- 172.21.15.55:0/52121099 shutdown_connections 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3dea70 0x55d89b3ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3de0b0 0x55d89b3e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d89b3f2fc0 0x55d89b3f5480 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 --2- 172.21.15.55:0/52121099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3e73f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04a67fc640 0 client.17439 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:34.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 -- 172.21.15.55:0/52121099 >> 172.21.15.55:0/52121099 conn(0x55d89b3d83d0 msgr2=0x55d89b3d7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:34.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 -- 172.21.15.55:0/52121099 shutdown_connections 2024-04-26T22:05:34.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.895+0000 7f04c825f740 1 -- 172.21.15.55:0/52121099 wait complete. 2024-04-26T22:05:34.897 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.896+0000 7f04c825f740 20 client.17439 trim_cache size 0 max 16384 2024-04-26T22:05:34.958 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.956+0000 7f04c825f740 1 Processor -- start 2024-04-26T22:05:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.957+0000 7f04c825f740 1 -- start start 2024-04-26T22:05:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.957+0000 7f04c825f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.958+0000 7f04c825f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 0x55d89b3ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.958+0000 7f04c825f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 0x55d89b3db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.958+0000 7f04c825f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b2ec0 con 0x55d89b3dcee0 2024-04-26T22:05:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.958+0000 7f04c825f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b0790 con 0x55d89b3dea70 2024-04-26T22:05:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.958+0000 7f04c825f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01ae5e0 con 0x55d89b3de0b0 2024-04-26T22:05:34.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.959+0000 7f04beffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.959+0000 7f04beffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3db0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54790/0 (socket says 172.21.15.55:54790) 2024-04-26T22:05:34.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.959+0000 7f04beffd640 1 -- 172.21.15.55:0/2201230044 learned_addr learned my addr 172.21.15.55:0/2201230044 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:34.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.959+0000 7f04bf7fe640 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 0x55d89b3db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.960+0000 7f04be7fc640 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 0x55d89b3ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.961+0000 7f04bf7fe640 1 -- 172.21.15.55:0/2201230044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 msgr2=0x55d89b3ee900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:34.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.961+0000 7f04bf7fe640 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 0x55d89b3ee900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.961+0000 7f04bf7fe640 1 -- 172.21.15.55:0/2201230044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 msgr2=0x55d89b3db0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:05:34.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.961+0000 7f04bf7fe640 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3db0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.961+0000 7f04bf7fe640 1 -- 172.21.15.55:0/2201230044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89b0130f0 con 0x55d89b3dea70 2024-04-26T22:05:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.968+0000 7f04bf7fe640 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 0x55d89b3db5f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f04b80ee840 tx=0x7f04b8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b48fa977b89cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.970+0000 7f04b4ff9640 1 -- 172.21.15.55:0/2201230044 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8049040 con 0x55d89b3dea70 2024-04-26T22:05:34.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.970+0000 7f04b4ff9640 1 -- 172.21.15.55:0/2201230044 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b80eee50 con 0x55d89b3dea70 2024-04-26T22:05:34.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.970+0000 7f04b4ff9640 1 -- 172.21.15.55:0/2201230044 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b80eea90 con 0x55d89b3dea70 2024-04-26T22:05:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.973+0000 7f04c825f740 1 -- 172.21.15.55:0/2201230044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 msgr2=0x55d89b3db5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:34.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.973+0000 7f04c825f740 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 0x55d89b3db5f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f04b80ee840 tx=0x7f04b8009260 comp rx=0 tx=0).stop 2024-04-26T22:05:34.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.976+0000 7f04c825f740 1 -- 172.21.15.55:0/2201230044 shutdown_connections 2024-04-26T22:05:34.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.976+0000 7f04c825f740 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 0x55d89b3db5f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.976+0000 7f04c825f740 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 0x55d89b3ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.976+0000 7f04c825f740 1 --2- 172.21.15.55:0/2201230044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.976+0000 7f04c825f740 1 -- 172.21.15.55:0/2201230044 >> 172.21.15.55:0/2201230044 conn(0x55d89b3d83d0 msgr2=0x55d89b3d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.976+0000 7f04c825f740 1 -- 172.21.15.55:0/2201230044 shutdown_connections 2024-04-26T22:05:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.976+0000 7f04c825f740 1 -- 172.21.15.55:0/2201230044 wait complete. 2024-04-26T22:05:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.977+0000 7f04c825f740 1 -- start start 2024-04-26T22:05:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.977+0000 7f04c825f740 1 Processor -- start 2024-04-26T22:05:34.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.978+0000 7f04c825f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.979+0000 7f04c825f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 0x55d89b3ec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.979+0000 7f04c825f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 0x55d89b3ed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.979+0000 7f04beffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.979+0000 7f04beffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3ec170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54798/0 (socket says 172.21.15.55:54798) 2024-04-26T22:05:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.979+0000 7f04beffd640 1 -- 172.21.15.55:0/2862101594 learned_addr learned my addr 172.21.15.55:0/2862101594 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.980+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01aec50 con 0x55d89b3dcee0 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.980+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01b00c0 con 0x55d89b3dea70 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.980+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01af320 con 0x55d89b3de0b0 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.980+0000 7f04c825f740 10 client.0 _ll_register_callbacks cb 0x7f04b0009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.981+0000 7f04beffd640 1 -- 172.21.15.55:0/2862101594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 msgr2=0x55d89b3ec6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.981+0000 7f04beffd640 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 0x55d89b3ec6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.981+0000 7f04beffd640 1 -- 172.21.15.55:0/2862101594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 msgr2=0x55d89b3ed6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.981+0000 7f04beffd640 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 0x55d89b3ed6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:34.983 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.981+0000 7f04beffd640 1 -- 172.21.15.55:0/2862101594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d89b3024b0 con 0x55d89b3dcee0 2024-04-26T22:05:34.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.982+0000 7f04beffd640 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3ec170 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f04b0023ca0 tx=0x7f04b0023a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7aa999963edc30d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:05:34.984 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.982+0000 7f04a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:05:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.982+0000 7f04a67fc640 1 -- 172.21.15.55:0/2862101594 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b002e830 con 0x55d89b3dcee0 2024-04-26T22:05:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.982+0000 7f04a67fc640 1 -- 172.21.15.55:0/2862101594 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f04b002e9d0 con 0x55d89b3dcee0 2024-04-26T22:05:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.983+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b80eee50 con 0x55d89b3dcee0 2024-04-26T22:05:34.986 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.984+0000 7f04a67fc640 1 -- 172.21.15.55:0/2862101594 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f04b002ebd0 con 0x55d89b3dcee0 2024-04-26T22:05:34.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.986+0000 7f04c825f740 10 client.17445 fetch_fsmap learned FSMap version 23 2024-04-26T22:05:34.987 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.986+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d89b46a5d0 con 0x55d89b3dcee0 2024-04-26T22:05:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.988+0000 7f04a67fc640 1 -- 172.21.15.55:0/2862101594 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b002ed70 con 0x55d89b3dcee0 2024-04-26T22:05:34.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.988+0000 7f04a67fc640 1 -- 172.21.15.55:0/2862101594 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f04b00353d0 con 0x55d89b3dcee0 2024-04-26T22:05:34.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.988+0000 7f04c825f740 10 client.17445 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:05:34.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.988+0000 7f04c825f740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:05:34.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.990+0000 7f04c825f740 20 client.17445 populate_metadata read hostname 'smithi055' 2024-04-26T22:05:34.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.990+0000 7f04c825f740 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d89b46af20 0x55d89b46d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:05:34.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.990+0000 7f04c825f740 4 client.17445 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:05:34.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.990+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f04b80eee50 con 0x55d89b46af20 2024-04-26T22:05:34.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.994+0000 7f04be7fc640 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d89b46af20 0x55d89b46d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:05:35.001 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:34.999+0000 7f04be7fc640 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d89b46af20 0x55d89b46d340 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61fb6add7a5de947 server_cookie=4b3def4f715e2f42 in_seq=0 out_seq=0 2024-04-26T22:05:35.002 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.000+0000 7f04a67fc640 10 client.17445 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:35.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.006+0000 7f04a67fc640 1 -- 172.21.15.55:0/2862101594 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x7f04b80eee50 con 0x55d89b46af20 2024-04-26T22:05:35.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.006+0000 7f04a67fc640 10 client.17445 handle_command_reply: tid=0 2024-04-26T22:05:35.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.006+0000 7f04c825f740 1 client.17445 shutdown 2024-04-26T22:05:35.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.010+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 msgr2=0x55d89b3ec170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.010+0000 7f04c825f740 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3ec170 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f04b0023ca0 tx=0x7f04b0023a90 comp rx=0 tx=0).stop 2024-04-26T22:05:35.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.011+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 shutdown_connections 2024-04-26T22:05:35.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.011+0000 7f04c825f740 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d89b3dea70 0x55d89b3ed6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.011+0000 7f04c825f740 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d89b3de0b0 0x55d89b3ec6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.011+0000 7f04c825f740 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d89b46af20 0x55d89b46d340 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.011+0000 7f04c825f740 1 --2- 172.21.15.55:0/2862101594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d89b3dcee0 0x55d89b3ec170 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.011+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 >> 172.21.15.55:0/2862101594 conn(0x55d89b3d83d0 msgr2=0x55d89b3d6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:35.013 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.011+0000 7f04a67fc640 0 client.17445 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:05:35.015 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.015+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 shutdown_connections 2024-04-26T22:05:35.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.015+0000 7f04c825f740 1 -- 172.21.15.55:0/2862101594 wait complete. 2024-04-26T22:05:35.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.018+0000 7f04c825f740 20 client.17445 trim_cache size 0 max 16384 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:05:35.020 INFO:teuthology.orchestra.run.smithi055.stdout: "be6c01a9-b2f7-43e8-adc8-08704fe4e635": { 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "be6c01a9-b2f7-43e8-adc8-08704fe4e635", 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:05:35.021 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:05:35.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.023+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04ac0a3d40 msgr2=0x7f04ac0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.023+0000 7f04c554e640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04ac0a3d40 0x7f04ac0a6200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009fd0 tx=0x7f04b000d040 comp rx=0 tx=0).stop 2024-04-26T22:05:35.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.023+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 msgr2=0x7f04c01a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:05:35.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.023+0000 7f04c554e640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 0x7f04c01a6890 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f04b802a930 tx=0x7f04b8003cd0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.026+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 shutdown_connections 2024-04-26T22:05:35.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.026+0000 7f04c554e640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f04ac0a3d40 0x7f04ac0a6200 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.026+0000 7f04c554e640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04c0078ed0 0x7f04c01a6890 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.026+0000 7f04c554e640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04c0078530 0x7f04c019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.026+0000 7f04c554e640 1 --2- 172.21.15.55:0/4146830814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04c00772f0 0x7f04c019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:05:35.027 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.026+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 >> 172.21.15.55:0/4146830814 conn(0x7f04c01000e0 msgr2=0x7f04c01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:05:35.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.037+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 shutdown_connections 2024-04-26T22:05:35.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:05:35.042+0000 7f04c554e640 1 -- 172.21.15.55:0/4146830814 wait complete. 2024-04-26T22:05:35.075 INFO:tasks.cephfs.filesystem:scrub status for tag:be6c01a9-b2f7-43e8-adc8-08704fe4e635 - {'path': '/', 'tag': 'be6c01a9-b2f7-43e8-adc8-08704fe4e635', 'options': 'recursive,force'} 2024-04-26T22:05:35.439 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:35 smithi055 ceph-mon[29221]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 1.43k op/s 2024-04-26T22:05:35.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:35 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:05:35.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:35 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:35.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:35 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:05:35.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:35 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:05:35.440 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:35 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:35 smithi064 ceph-mon[32765]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 1.43k op/s 2024-04-26T22:05:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:35 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:05:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:35 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:35 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:05:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:35 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:05:35.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:35 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:35 smithi165 ceph-mon[33296]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 863 MiB/s wr, 1.43k op/s 2024-04-26T22:05:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:35 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:05:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:35 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:35 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:05:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:35 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:05:35.577 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:35 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:37.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:37 smithi064 ceph-mon[32765]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 768 MiB/s wr, 1.38k op/s 2024-04-26T22:05:37.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:37 smithi165 ceph-mon[33296]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 768 MiB/s wr, 1.38k op/s 2024-04-26T22:05:37.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:37 smithi055 ceph-mon[29221]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 768 MiB/s wr, 1.38k op/s 2024-04-26T22:05:39.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:39 smithi064 ceph-mon[32765]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 955 MiB/s wr, 1.79k op/s 2024-04-26T22:05:39.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:39 smithi165 ceph-mon[33296]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 955 MiB/s wr, 1.79k op/s 2024-04-26T22:05:39.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:39 smithi055 ceph-mon[29221]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 955 MiB/s wr, 1.79k op/s 2024-04-26T22:05:40.507 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:40 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:40.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:40 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:40.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:40 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:05:41.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:41 smithi165 ceph-mon[33296]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.57k op/s 2024-04-26T22:05:41.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:41 smithi055 ceph-mon[29221]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.57k op/s 2024-04-26T22:05:41.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:41 smithi064 ceph-mon[32765]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 855 MiB/s wr, 1.57k op/s 2024-04-26T22:05:42.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:42 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:05:42.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:42 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:05:42.696 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:42 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:05:42.697 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:42 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:05:42.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:42 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:05:42.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:42 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:05:43.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:43 smithi165 ceph-mon[33296]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 835 MiB/s wr, 1.54k op/s 2024-04-26T22:05:43.599 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:43 smithi055 ceph-mon[29221]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 835 MiB/s wr, 1.54k op/s 2024-04-26T22:05:43.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:43 smithi064 ceph-mon[32765]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 835 MiB/s wr, 1.54k op/s 2024-04-26T22:05:45.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:45 smithi165 ceph-mon[33296]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 894 MiB/s wr, 1.70k op/s 2024-04-26T22:05:45.689 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:45 smithi055 ceph-mon[29221]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 894 MiB/s wr, 1.70k op/s 2024-04-26T22:05:45.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:45 smithi064 ceph-mon[32765]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 894 MiB/s wr, 1.70k op/s 2024-04-26T22:05:47.576 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:47 smithi165 ceph-mon[33296]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 770 MiB/s wr, 1.38k op/s 2024-04-26T22:05:47.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:47 smithi055 ceph-mon[29221]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 770 MiB/s wr, 1.38k op/s 2024-04-26T22:05:47.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:47 smithi064 ceph-mon[32765]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 770 MiB/s wr, 1.38k op/s 2024-04-26T22:05:49.688 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:49 smithi055 ceph-mon[29221]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 969 MiB/s wr, 1.55k op/s 2024-04-26T22:05:49.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:49 smithi064 ceph-mon[32765]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 969 MiB/s wr, 1.55k op/s 2024-04-26T22:05:49.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:49 smithi165 ceph-mon[33296]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 969 MiB/s wr, 1.55k op/s 2024-04-26T22:05:51.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:51 smithi055 ceph-mon[29221]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.20k op/s 2024-04-26T22:05:51.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:51 smithi064 ceph-mon[32765]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.20k op/s 2024-04-26T22:05:51.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:51 smithi165 ceph-mon[33296]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.20k op/s 2024-04-26T22:05:53.694 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:53 smithi055 ceph-mon[29221]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.38k op/s 2024-04-26T22:05:53.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:53 smithi064 ceph-mon[32765]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.38k op/s 2024-04-26T22:05:53.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:53 smithi165 ceph-mon[33296]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 837 MiB/s wr, 1.38k op/s 2024-04-26T22:05:53.985 INFO:tasks.workunit.client.0.smithi055.stdout:OK 2024-04-26T22:05:53.985 INFO:tasks.workunit.client.0.smithi055.stderr:+ echo OK 2024-04-26T22:05:55.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:55 smithi055 ceph-mon[29221]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 1.63k op/s 2024-04-26T22:05:55.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:55 smithi064 ceph-mon[32765]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 1.63k op/s 2024-04-26T22:05:55.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:55 smithi165 ceph-mon[33296]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 885 MiB/s wr, 1.63k op/s 2024-04-26T22:05:56.500 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T22:05:56.500 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T22:05:56.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:56 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:56.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:56 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:56.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:56 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:57.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:57 smithi055 ceph-mon[29221]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 765 MiB/s wr, 1.44k op/s 2024-04-26T22:05:57.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:57.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:57.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:57.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:57.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:57.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:57 smithi064 ceph-mon[32765]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 765 MiB/s wr, 1.44k op/s 2024-04-26T22:05:57.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:57.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:57.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:57.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:57.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:57.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:57 smithi165 ceph-mon[33296]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 765 MiB/s wr, 1.44k op/s 2024-04-26T22:05:57.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:57.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:57.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:57.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:57.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:05:59.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:05:59 smithi055 ceph-mon[29221]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 925 MiB/s wr, 1.55k op/s 2024-04-26T22:05:59.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:05:59 smithi064 ceph-mon[32765]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 925 MiB/s wr, 1.55k op/s 2024-04-26T22:05:59.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:05:59 smithi165 ceph-mon[33296]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 925 MiB/s wr, 1.55k op/s 2024-04-26T22:06:01.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:01 smithi055 ceph-mon[29221]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 727 MiB/s wr, 1.29k op/s 2024-04-26T22:06:01.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:01 smithi064 ceph-mon[32765]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 727 MiB/s wr, 1.29k op/s 2024-04-26T22:06:01.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:01 smithi165 ceph-mon[33296]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 727 MiB/s wr, 1.29k op/s 2024-04-26T22:06:03.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:03 smithi055 ceph-mon[29221]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 633 MiB/s wr, 1.17k op/s 2024-04-26T22:06:03.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:03 smithi064 ceph-mon[32765]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 633 MiB/s wr, 1.17k op/s 2024-04-26T22:06:03.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:03 smithi165 ceph-mon[33296]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 633 MiB/s wr, 1.17k op/s 2024-04-26T22:06:05.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:06:05.076 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:06:05.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.190+0000 7f2034f80640 1 -- 172.21.15.55:0/3505704230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20301086f0 msgr2=0x7f203010aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.190+0000 7f2034f80640 1 --2- 172.21.15.55:0/3505704230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20301086f0 0x7f203010aae0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2028009ca0 tx=0x7f202802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.191+0000 7f2034f80640 1 -- 172.21.15.55:0/3505704230 shutdown_connections 2024-04-26T22:06:05.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.191+0000 7f2034f80640 1 --2- 172.21.15.55:0/3505704230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20301086f0 0x7f203010aae0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.191 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.191+0000 7f2034f80640 1 --2- 172.21.15.55:0/3505704230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2030105dc0 0x7f20301081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.191+0000 7f2034f80640 1 --2- 172.21.15.55:0/3505704230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2030103490 0x7f2030105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.191+0000 7f2034f80640 1 -- 172.21.15.55:0/3505704230 >> 172.21.15.55:0/3505704230 conn(0x7f20300fcde0 msgr2=0x7f20300ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.191+0000 7f2034f80640 1 -- 172.21.15.55:0/3505704230 shutdown_connections 2024-04-26T22:06:05.192 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.191+0000 7f2034f80640 1 -- 172.21.15.55:0/3505704230 wait complete. 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f2034f80640 1 Processor -- start 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f2034f80640 1 -- start start 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f2034f80640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2030103490 0x7f2030102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f2034f80640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2030105dc0 0x7f2030101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f2034f80640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20301086f0 0x7f2030101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f2034f80640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f203010e4c0 con 0x7f2030103490 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f2034f80640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f203010e340 con 0x7f2030105dc0 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f2034f80640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f203010e640 con 0x7f20301086f0 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f202e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2030103490 0x7f2030102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f202e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2030103490 0x7f2030102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52950/0 (socket says 172.21.15.55:52950) 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f202e59c640 1 -- 172.21.15.55:0/4166985041 learned_addr learned my addr 172.21.15.55:0/4166985041 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f202dd9b640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2030105dc0 0x7f2030101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.192+0000 7f202ed9d640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20301086f0 0x7f2030101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f202e59c640 1 -- 172.21.15.55:0/4166985041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20301086f0 msgr2=0x7f2030101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f202e59c640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20301086f0 0x7f2030101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f202e59c640 1 -- 172.21.15.55:0/4166985041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2030105dc0 msgr2=0x7f2030101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f202e59c640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2030105dc0 0x7f2030101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f202e59c640 1 -- 172.21.15.55:0/4166985041 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2030102210 con 0x7f2030103490 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f202e59c640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2030103490 0x7f2030102e00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f202000e9f0 tx=0x7f202000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f201f7fe640 1 -- 172.21.15.55:0/4166985041 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202000cd80 con 0x7f2030103490 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f2034f80640 1 -- 172.21.15.55:0/4166985041 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20301a2d10 con 0x7f2030103490 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f201f7fe640 1 -- 172.21.15.55:0/4166985041 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20200108e0 con 0x7f2030103490 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f201f7fe640 1 -- 172.21.15.55:0/4166985041 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020018980 con 0x7f2030103490 2024-04-26T22:06:05.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.193+0000 7f2034f80640 1 -- 172.21.15.55:0/4166985041 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20301a31a0 con 0x7f2030103490 2024-04-26T22:06:05.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.196+0000 7f201f7fe640 1 -- 172.21.15.55:0/4166985041 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2020010430 con 0x7f2030103490 2024-04-26T22:06:05.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.196+0000 7f2037d70740 1 Processor -- start 2024-04-26T22:06:05.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.196+0000 7f2037d70740 1 -- start start 2024-04-26T22:06:05.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.196+0000 7f2037d70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf7d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.196+0000 7f2037d70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf7e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f2037d70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf85300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f2037d70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560cabee1d80 con 0x560cabf7ea70 2024-04-26T22:06:05.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f2037d70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560cabe9b8b0 con 0x560cabf7cee0 2024-04-26T22:06:05.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f2037d70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560cabeddd40 con 0x560cabf7e0b0 2024-04-26T22:06:05.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf85300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf7d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf85300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52966/0 (socket says 172.21.15.55:52966) 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 -- 172.21.15.55:0/4077870050 learned_addr learned my addr 172.21.15.55:0/4077870050 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202e59c640 1 --2- 172.21.15.55:0/4077870050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf7e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f201f7fe640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20180a3c40 0x7f20180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202ed9d640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20180a3c40 0x7f20180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 -- 172.21.15.55:0/4077870050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 msgr2=0x560cabf7e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 --2- 172.21.15.55:0/4077870050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf7e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 -- 172.21.15.55:0/4077870050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 msgr2=0x560cabf7d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 --2- 172.21.15.55:0/4077870050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf7d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 -- 172.21.15.55:0/4077870050 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cabe6ca40 con 0x560cabf7ea70 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f201f7fe640 1 -- 172.21.15.55:0/4166985041 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f20200d08c0 con 0x7f2030103490 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f202dd9b640 1 --2- 172.21.15.55:0/4077870050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf85300 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f202400abb0 tx=0x7f202400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db76da264200343c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.197+0000 7f201cff9640 1 -- 172.21.15.55:0/4077870050 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202401d070 con 0x560cabf7ea70 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f202ed9d640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20180a3c40 0x7f20180a6100 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2028031b50 tx=0x7f202802a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f201cff9640 1 -- 172.21.15.55:0/4077870050 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2024018af0 con 0x560cabf7ea70 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f201cff9640 1 -- 172.21.15.55:0/4077870050 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024010a90 con 0x560cabf7ea70 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f2037d70740 1 -- 172.21.15.55:0/4077870050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 msgr2=0x560cabf85300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f2037d70740 1 --2- 172.21.15.55:0/4077870050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf85300 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f202400abb0 tx=0x7f202400f790 comp rx=0 tx=0).stop 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f2037d70740 1 -- 172.21.15.55:0/4077870050 shutdown_connections 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f2037d70740 1 --2- 172.21.15.55:0/4077870050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf85300 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f2037d70740 1 --2- 172.21.15.55:0/4077870050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf7e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f2037d70740 1 --2- 172.21.15.55:0/4077870050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf7d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.198+0000 7f2037d70740 1 -- 172.21.15.55:0/4077870050 >> 172.21.15.55:0/4077870050 conn(0x560cabf783d0 msgr2=0x560cabf787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 -- 172.21.15.55:0/4077870050 shutdown_connections 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 -- 172.21.15.55:0/4077870050 wait complete. 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 -- start start 2024-04-26T22:06:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 Processor -- start 2024-04-26T22:06:05.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf87360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf8e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x560cabf85a80 con 0x560cabf7ea70 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560cabe68d10 con 0x560cabf7cee0 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560cabf85c00 con 0x560cabf7e0b0 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f2037d70740 10 client.0 _ll_register_callbacks cb 0x560cabe95210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f202e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf87360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f202dd9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f202ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf8e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f202e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf87360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45538/0 (socket says 172.21.15.55:45538) 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f202e59c640 1 -- 172.21.15.55:0/3576948467 learned_addr learned my addr 172.21.15.55:0/3576948467 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.199+0000 7f202dd9b640 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf878a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:38194/0 (socket says 172.21.15.55:38194) 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f202e59c640 1 -- 172.21.15.55:0/3576948467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 msgr2=0x560cabf878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f202e59c640 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f202e59c640 1 -- 172.21.15.55:0/3576948467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 msgr2=0x560cabf8e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f202e59c640 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf8e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f202e59c640 1 -- 172.21.15.55:0/3576948467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cabcbd7d0 con 0x560cabf7cee0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f202ed9d640 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf8e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f202e59c640 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf87360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f202009ca40 tx=0x7f202000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a00019e24d48d2a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f2015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f2015ffb640 1 -- 172.21.15.55:0/3576948467 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202009c410 con 0x560cabf7cee0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f2037d70740 1 -- 172.21.15.55:0/3576948467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2024018af0 con 0x560cabf7cee0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f2015ffb640 1 -- 172.21.15.55:0/3576948467 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2020010a90 con 0x560cabf7cee0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f2015ffb640 1 -- 172.21.15.55:0/3576948467 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2020010720 con 0x560cabf7cee0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f2037d70740 10 client.26372 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.200+0000 7f2037d70740 1 -- 172.21.15.55:0/3576948467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560cabbf33f0 con 0x560cabf7cee0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2015ffb640 1 -- 172.21.15.55:0/3576948467 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20200b7020 con 0x560cabf7cee0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2015ffb640 1 -- 172.21.15.55:0/3576948467 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f20200bbbc0 con 0x560cabf7cee0 2024-04-26T22:06:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2037d70740 10 client.26372 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2037d70740 10 client.26372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2037d70740 20 client.26372 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2037d70740 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560cabf92d70 0x560cabf951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2037d70740 4 client.26372 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2037d70740 1 -- 172.21.15.55:0/3576948467 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cabeef990 con 0x560cabf92d70 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f202dd9b640 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560cabf92d70 0x560cabf951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f202dd9b640 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560cabf92d70 0x560cabf951e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87e0cfcf5189647 server_cookie=406789b79967fe34 in_seq=0 out_seq=0 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.201+0000 7f2015ffb640 10 client.26372 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2015ffb640 1 -- 172.21.15.55:0/3576948467 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cabeef990 con 0x560cabf92d70 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2015ffb640 10 client.26372 handle_command_reply: tid=0 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 client.26372 shutdown 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 -- 172.21.15.55:0/3576948467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 msgr2=0x560cabf87360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf87360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f202009ca40 tx=0x7f202000cf20 comp rx=0 tx=0).stop 2024-04-26T22:06:05.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 -- 172.21.15.55:0/3576948467 shutdown_connections 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560cabf92d70 0x560cabf951e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf8e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2015ffb640 0 client.26372 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 --2- 172.21.15.55:0/3576948467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf87360 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 -- 172.21.15.55:0/3576948467 >> 172.21.15.55:0/3576948467 conn(0x560cabf783d0 msgr2=0x560cabf77550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 -- 172.21.15.55:0/3576948467 shutdown_connections 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.203+0000 7f2037d70740 1 -- 172.21.15.55:0/3576948467 wait complete. 2024-04-26T22:06:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.204+0000 7f2037d70740 20 client.26372 trim_cache size 0 max 16384 2024-04-26T22:06:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f2037d70740 1 Processor -- start 2024-04-26T22:06:05.227 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f2037d70740 1 -- start start 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f2037d70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf86f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f2037d70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf8e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f2037d70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f2037d70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20301aea20 con 0x560cabf7ea70 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f2037d70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20301ac2f0 con 0x560cabf7cee0 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f2037d70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f20301aa140 con 0x560cabf7e0b0 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf874b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52994/0 (socket says 172.21.15.55:52994) 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 -- 172.21.15.55:0/1750846239 learned_addr learned my addr 172.21.15.55:0/1750846239 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202dd9b640 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf8e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202e59c640 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf86f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 -- 172.21.15.55:0/1750846239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 msgr2=0x560cabf8e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.228 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf8e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 -- 172.21.15.55:0/1750846239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 msgr2=0x560cabf86f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf86f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 -- 172.21.15.55:0/1750846239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cabbb30f0 con 0x560cabf7ea70 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.223+0000 7f202ed9d640 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf874b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2028035000 tx=0x7f2028005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3273113d77430bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2017fff640 1 -- 172.21.15.55:0/1750846239 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028051070 con 0x560cabf7ea70 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2017fff640 1 -- 172.21.15.55:0/1750846239 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2028036050 con 0x560cabf7ea70 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2017fff640 1 -- 172.21.15.55:0/1750846239 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028030940 con 0x560cabf7ea70 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 -- 172.21.15.55:0/1750846239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 msgr2=0x560cabf874b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf874b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2028035000 tx=0x7f2028005f60 comp rx=0 tx=0).stop 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 -- 172.21.15.55:0/1750846239 shutdown_connections 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7ea70 0x560cabf874b0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf8e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 --2- 172.21.15.55:0/1750846239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7cee0 0x560cabf86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.229 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 -- 172.21.15.55:0/1750846239 >> 172.21.15.55:0/1750846239 conn(0x560cabf783d0 msgr2=0x560cabf73480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 -- 172.21.15.55:0/1750846239 shutdown_connections 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.224+0000 7f2037d70740 1 -- 172.21.15.55:0/1750846239 wait complete. 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 1 -- start start 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 1 Processor -- start 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7cee0 0x560cabf8b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf8bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7ea70 0x560cabff7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f20301aa7b0 con 0x560cabf7cee0 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f20301abc20 con 0x560cabf7ea70 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f20301aae80 con 0x560cabf7e0b0 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f2037d70740 10 client.0 _ll_register_callbacks cb 0x7f202000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7cee0 0x560cabf8b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7cee0 0x560cabf8b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53000/0 (socket says 172.21.15.55:53000) 2024-04-26T22:06:05.230 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202e59c640 1 -- 172.21.15.55:0/2222686687 learned_addr learned my addr 172.21.15.55:0/2222686687 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202ed9d640 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7ea70 0x560cabff7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202dd9b640 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf8bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202e59c640 1 -- 172.21.15.55:0/2222686687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 msgr2=0x560cabf8bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202e59c640 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf8bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202e59c640 1 -- 172.21.15.55:0/2222686687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7ea70 msgr2=0x560cabff7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202e59c640 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7ea70 0x560cabff7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.225+0000 7f202e59c640 1 -- 172.21.15.55:0/2222686687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cabea24b0 con 0x560cabf7cee0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f202e59c640 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7cee0 0x560cabf8b730 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f202000c6a0 tx=0x7f20200ce6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b62c27a14125ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2015ffb640 1 -- 172.21.15.55:0/2222686687 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202009c410 con 0x560cabf7cee0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2015ffb640 1 -- 172.21.15.55:0/2222686687 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f202009bf00 con 0x560cabf7cee0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 1 -- 172.21.15.55:0/2222686687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2028036050 con 0x560cabf7cee0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2015ffb640 1 -- 172.21.15.55:0/2222686687 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f202009cab0 con 0x560cabf7cee0 2024-04-26T22:06:05.231 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 10 client.17463 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 1 -- 172.21.15.55:0/2222686687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560cabff87f0 con 0x560cabf7cee0 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2015ffb640 1 -- 172.21.15.55:0/2222686687 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2028036050 con 0x560cabf7cee0 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2015ffb640 1 -- 172.21.15.55:0/2222686687 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f20200a1f40 con 0x560cabf7cee0 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 10 client.17463 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 20 client.17463 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560cabff91a0 0x560cabffb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 4 client.17463 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f2037d70740 1 -- 172.21.15.55:0/2222686687 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560cabee9800 con 0x560cabff91a0 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.226+0000 7f202dd9b640 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560cabff91a0 0x560cabffb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.227+0000 7f202dd9b640 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560cabff91a0 0x560cabffb5c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea785f90dfb6958f server_cookie=5c90f2856e01d0a3 in_seq=0 out_seq=0 2024-04-26T22:06:05.232 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.227+0000 7f2015ffb640 10 client.17463 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:05.233 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2015ffb640 1 -- 172.21.15.55:0/2222686687 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560cabee9800 con 0x560cabff91a0 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2015ffb640 10 client.17463 handle_command_reply: tid=0 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 client.17463 shutdown 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 -- 172.21.15.55:0/2222686687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7cee0 msgr2=0x560cabf8b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7cee0 0x560cabf8b730 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f202000c6a0 tx=0x7f20200ce6b0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 -- 172.21.15.55:0/2222686687 shutdown_connections 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560cabf7ea70 0x560cabff7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cabf7e0b0 0x560cabf8bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x560cabff91a0 0x560cabffb5c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 --2- 172.21.15.55:0/2222686687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x560cabf7cee0 0x560cabf8b730 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 -- 172.21.15.55:0/2222686687 >> 172.21.15.55:0/2222686687 conn(0x560cabf783d0 msgr2=0x560cabf86f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2015ffb640 0 client.17463 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 -- 172.21.15.55:0/2222686687 shutdown_connections 2024-04-26T22:06:05.234 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 1 -- 172.21.15.55:0/2222686687 wait complete. 2024-04-26T22:06:05.235 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.229+0000 7f2037d70740 20 client.17463 trim_cache size 0 max 16384 2024-04-26T22:06:05.235 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:06:05.235 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:06:05.235 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:06:05.235 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 -- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20180a3c40 msgr2=0x7f20180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20180a3c40 0x7f20180a6100 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2028031b50 tx=0x7f202802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 -- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2030103490 msgr2=0x7f2030102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2030103490 0x7f2030102e00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f202000e9f0 tx=0x7f202000eeb0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 -- 172.21.15.55:0/4166985041 shutdown_connections 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20180a3c40 0x7f20180a6100 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20301086f0 0x7f2030101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2030105dc0 0x7f2030101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.236 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 --2- 172.21.15.55:0/4166985041 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2030103490 0x7f2030102e00 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 -- 172.21.15.55:0/4166985041 >> 172.21.15.55:0/4166985041 conn(0x7f20300fcde0 msgr2=0x7f20301052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 -- 172.21.15.55:0/4166985041 shutdown_connections 2024-04-26T22:06:05.237 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.230+0000 7f2034f80640 1 -- 172.21.15.55:0/4166985041 wait complete. 2024-04-26T22:06:05.241 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:06:05.241 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:05.346 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 -- 172.21.15.55:0/3738790214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 msgr2=0x7fe220103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 --2- 172.21.15.55:0/3738790214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 0x7fe220103ff0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe208009900 tx=0x7fe20802a410 comp rx=0 tx=0).stop 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 -- 172.21.15.55:0/3738790214 shutdown_connections 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 --2- 172.21.15.55:0/3738790214 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220104530 0x7fe22010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 --2- 172.21.15.55:0/3738790214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 0x7fe220103ff0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 --2- 172.21.15.55:0/3738790214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe220102980 0x7fe220102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 -- 172.21.15.55:0/3738790214 >> 172.21.15.55:0/3738790214 conn(0x7fe2200fdf10 msgr2=0x7fe220100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 -- 172.21.15.55:0/3738790214 shutdown_connections 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.344+0000 7fe224d14640 1 -- 172.21.15.55:0/3738790214 wait complete. 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe224d14640 1 Processor -- start 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe224d14640 1 -- start start 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe224d14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220102980 0x7fe220198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe224d14640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 0x7fe220199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe224d14640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe220104530 0x7fe2201a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe224d14640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010e4b0 con 0x7fe220103b70 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe224d14640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010e330 con 0x7fe220102980 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe224d14640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe22010e630 con 0x7fe220104530 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe217fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 0x7fe220199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe217fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 0x7fe220199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53018/0 (socket says 172.21.15.55:53018) 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe21e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220102980 0x7fe220198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe217fff640 1 -- 172.21.15.55:0/529754028 learned_addr learned my addr 172.21.15.55:0/529754028 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe217fff640 1 -- 172.21.15.55:0/529754028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe220104530 msgr2=0x7fe2201a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe217fff640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe220104530 0x7fe2201a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe217fff640 1 -- 172.21.15.55:0/529754028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220102980 msgr2=0x7fe220198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe217fff640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220102980 0x7fe220198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe217fff640 1 -- 172.21.15.55:0/529754028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2201a07f0 con 0x7fe220103b70 2024-04-26T22:06:05.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.345+0000 7fe21e59c640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220102980 0x7fe220198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:05.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.346+0000 7fe217fff640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 0x7fe220199110 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe208005940 tx=0x7fe2080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.346+0000 7fe2177fe640 1 -- 172.21.15.55:0/529754028 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20803f030 con 0x7fe220103b70 2024-04-26T22:06:05.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.346+0000 7fe2177fe640 1 -- 172.21.15.55:0/529754028 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe208003cb0 con 0x7fe220103b70 2024-04-26T22:06:05.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.346+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2201a0a80 con 0x7fe220103b70 2024-04-26T22:06:05.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.346+0000 7fe2177fe640 1 -- 172.21.15.55:0/529754028 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208035070 con 0x7fe220103b70 2024-04-26T22:06:05.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.346+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a0f70 con 0x7fe220103b70 2024-04-26T22:06:05.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.347+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe220109d70 con 0x7fe220103b70 2024-04-26T22:06:05.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.348+0000 7fe2177fe640 1 -- 172.21.15.55:0/529754028 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe208042030 con 0x7fe220103b70 2024-04-26T22:06:05.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.349+0000 7fe2177fe640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe20c0a3c40 0x7fe20c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.349+0000 7fe21e59c640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe20c0a3c40 0x7fe20c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.349+0000 7fe2177fe640 1 -- 172.21.15.55:0/529754028 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe2080f0a90 con 0x7fe220103b70 2024-04-26T22:06:05.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.349+0000 7fe21e59c640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe20c0a3c40 0x7fe20c0a6100 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe2201039d0 tx=0x7fe210005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.353+0000 7fe2177fe640 1 -- 172.21.15.55:0/529754028 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe2080bd270 con 0x7fe220103b70 2024-04-26T22:06:05.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.582+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe22002dcb0 con 0x7fe220103b70 2024-04-26T22:06:05.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.583+0000 7fe2177fe640 1 -- 172.21.15.55:0/529754028 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fe208003e60 con 0x7fe220103b70 2024-04-26T22:06:05.584 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:06:05.584 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:06:05.584 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:06:05.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.585+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe20c0a3c40 msgr2=0x7fe20c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.585+0000 7fe224d14640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe20c0a3c40 0x7fe20c0a6100 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe2201039d0 tx=0x7fe210005f90 comp rx=0 tx=0).stop 2024-04-26T22:06:05.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.585+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 msgr2=0x7fe220199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.585+0000 7fe224d14640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 0x7fe220199110 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe208005940 tx=0x7fe2080042f0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.586+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 shutdown_connections 2024-04-26T22:06:05.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.586+0000 7fe224d14640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe20c0a3c40 0x7fe20c0a6100 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.586 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.586+0000 7fe224d14640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe220104530 0x7fe2201a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.586+0000 7fe224d14640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe220103b70 0x7fe220199110 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.586+0000 7fe224d14640 1 --2- 172.21.15.55:0/529754028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe220102980 0x7fe220198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.586+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 >> 172.21.15.55:0/529754028 conn(0x7fe2200fdf10 msgr2=0x7fe220106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.586+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 shutdown_connections 2024-04-26T22:06:05.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.586+0000 7fe224d14640 1 -- 172.21.15.55:0/529754028 wait complete. 2024-04-26T22:06:05.598 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:06:05.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:05 smithi055 ceph-mon[29221]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 521 MiB/s wr, 853 op/s 2024-04-26T22:06:05.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 -- 172.21.15.55:0/443726113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 msgr2=0x7f868c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 --2- 172.21.15.55:0/443726113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 0x7f868c108130 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8680009900 tx=0x7f868002a410 comp rx=0 tx=0).stop 2024-04-26T22:06:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 -- 172.21.15.55:0/443726113 shutdown_connections 2024-04-26T22:06:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 --2- 172.21.15.55:0/443726113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f868c108670 0x7f868c10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 --2- 172.21.15.55:0/443726113 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 0x7f868c108130 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 --2- 172.21.15.55:0/443726113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f868c103410 0x7f868c105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 -- 172.21.15.55:0/443726113 >> 172.21.15.55:0/443726113 conn(0x7f868c0fce20 msgr2=0x7f868c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 -- 172.21.15.55:0/443726113 shutdown_connections 2024-04-26T22:06:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 -- 172.21.15.55:0/443726113 wait complete. 2024-04-26T22:06:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.705+0000 7f8691d9e640 1 Processor -- start 2024-04-26T22:06:05.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f8691d9e640 1 -- start start 2024-04-26T22:06:05.708 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f8691d9e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f868c103410 0x7f868c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f8691d9e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 0x7f868c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f8691d9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f868c108670 0x7f868c1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f8691d9e640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10e4d0 con 0x7f868c105d40 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f8691d9e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10e350 con 0x7f868c108670 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f8691d9e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10e650 con 0x7f868c103410 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 0x7f868c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 0x7f868c1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53032/0 (socket says 172.21.15.55:53032) 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868affd640 1 -- 172.21.15.55:0/2150146995 learned_addr learned my addr 172.21.15.55:0/2150146995 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868bfff640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f868c108670 0x7f868c1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868b7fe640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f868c103410 0x7f868c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868bfff640 1 -- 172.21.15.55:0/2150146995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f868c103410 msgr2=0x7f868c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868bfff640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f868c103410 0x7f868c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868bfff640 1 -- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 msgr2=0x7f868c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868bfff640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 0x7f868c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868bfff640 1 -- 172.21.15.55:0/2150146995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c1a0850 con 0x7f868c108670 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868affd640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 0x7f868c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.706+0000 7f868bfff640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f868c108670 0x7f868c1a01b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8684009fd0 tx=0x7f868400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.707+0000 7f8688ff9640 1 -- 172.21.15.55:0/2150146995 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868400ce20 con 0x7f868c108670 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.707+0000 7f8688ff9640 1 -- 172.21.15.55:0/2150146995 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8684005020 con 0x7f868c108670 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.707+0000 7f8688ff9640 1 -- 172.21.15.55:0/2150146995 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684010690 con 0x7f868c108670 2024-04-26T22:06:05.710 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.707+0000 7f8691d9e640 1 -- 172.21.15.55:0/2150146995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c1a0b40 con 0x7f868c108670 2024-04-26T22:06:05.711 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.707+0000 7f8691d9e640 1 -- 172.21.15.55:0/2150146995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c1a1040 con 0x7f868c108670 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8688ff9640 1 -- 172.21.15.55:0/2150146995 <== mon.2 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f86840054d0 con 0x7f868c108670 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8694b8b740 1 Processor -- start 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8694b8b740 1 -- start start 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b172f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 0x559b172fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17301c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8694b8b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559b1725dd80 con 0x559b172fb3b0 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8694b8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559b17217a20 con 0x559b172f9820 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f8694b8b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b17259d40 con 0x559b172fa9f0 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f868affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17301c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.709+0000 7f868affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b172f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.718 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b172f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45580/0 (socket says 172.21.15.55:45580) 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868affd640 1 -- 172.21.15.55:0/4176670638 learned_addr learned my addr 172.21.15.55:0/4176670638 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868b7fe640 1 --2- 172.21.15.55:0/4176670638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 0x559b172fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868affd640 1 -- 172.21.15.55:0/4176670638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 msgr2=0x559b172fae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f8688ff9640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86740a3c40 0x7f86740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868affd640 1 --2- 172.21.15.55:0/4176670638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 0x559b172fae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868affd640 1 -- 172.21.15.55:0/4176670638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 msgr2=0x559b17301c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868affd640 1 --2- 172.21.15.55:0/4176670638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17301c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868affd640 1 -- 172.21.15.55:0/4176670638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b17222c30 con 0x559b172f9820 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868bfff640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86740a3c40 0x7f86740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f868affd640 1 --2- 172.21.15.55:0/4176670638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b172f9c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8680031950 tx=0x7f86800314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c08f6fef0fc0fe18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f8688ff9640 1 -- 172.21.15.55:0/2150146995 <== mon.2 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f86840d0d20 con 0x7f868c108670 2024-04-26T22:06:05.719 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f8671ffb640 1 -- 172.21.15.55:0/4176670638 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680035050 con 0x559b172f9820 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f8671ffb640 1 -- 172.21.15.55:0/4176670638 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8680003aa0 con 0x559b172f9820 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.710+0000 7f8671ffb640 1 -- 172.21.15.55:0/4176670638 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868003b6c0 con 0x559b172f9820 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 -- 172.21.15.55:0/4176670638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 msgr2=0x559b172f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f868bfff640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86740a3c40 0x7f86740a6100 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8684062830 tx=0x7f868406b7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 --2- 172.21.15.55:0/4176670638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b172f9c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8680031950 tx=0x7f86800314a0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 -- 172.21.15.55:0/4176670638 shutdown_connections 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 --2- 172.21.15.55:0/4176670638 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17301c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 --2- 172.21.15.55:0/4176670638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 0x559b172fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 --2- 172.21.15.55:0/4176670638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b172f9c40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 -- 172.21.15.55:0/4176670638 >> 172.21.15.55:0/4176670638 conn(0x559b1723a910 msgr2=0x559b172f5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 -- 172.21.15.55:0/4176670638 shutdown_connections 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 -- 172.21.15.55:0/4176670638 wait complete. 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.711+0000 7f8694b8b740 1 -- start start 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f8694b8b740 1 Processor -- start 2024-04-26T22:06:05.720 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b17303a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 0x559b17303fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b1730aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f8694b8b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x559b17302360 con 0x559b172fb3b0 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f8694b8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559b171e4dc0 con 0x559b172f9820 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f8694b8b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b173024e0 con 0x559b172fa9f0 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f8694b8b740 10 client.0 _ll_register_callbacks cb 0x559b17200600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b1730aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b17303a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b17303a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45590/0 (socket says 172.21.15.55:45590) 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 -- 172.21.15.55:0/3997989693 learned_addr learned my addr 172.21.15.55:0/3997989693 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868b7fe640 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 0x559b17303fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 -- 172.21.15.55:0/3997989693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 msgr2=0x559b17303fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 0x559b17303fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.721 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 -- 172.21.15.55:0/3997989693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 msgr2=0x559b1730aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b1730aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 -- 172.21.15.55:0/3997989693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b16f6f3f0 con 0x559b172f9820 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f868affd640 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b17303a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8680006010 tx=0x7f8680000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3538c1c2ed8f8542 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.712+0000 7f866affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f866affd640 1 -- 172.21.15.55:0/3997989693 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680035040 con 0x559b172f9820 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f866affd640 1 -- 172.21.15.55:0/3997989693 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f86800035f0 con 0x559b172f9820 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 1 -- 172.21.15.55:0/3997989693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8680003aa0 con 0x559b172f9820 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f866affd640 1 -- 172.21.15.55:0/3997989693 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8680003e60 con 0x559b172f9820 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 10 client.26396 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 1 -- 172.21.15.55:0/3997989693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559b170397d0 con 0x559b172f9820 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f866affd640 1 -- 172.21.15.55:0/3997989693 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8680003aa0 con 0x559b172f9820 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f866affd640 1 -- 172.21.15.55:0/3997989693 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f868003d450 con 0x559b172f9820 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 10 client.26396 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 10 client.26396 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:06:05.722 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 20 client.26396 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x559b1730f4e0 0x559b173118d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 4 client.26396 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.713+0000 7f8694b8b740 1 -- 172.21.15.55:0/3997989693 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b1726b990 con 0x559b1730f4e0 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.714+0000 7f868b7fe640 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x559b1730f4e0 0x559b173118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.714+0000 7f868b7fe640 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x559b1730f4e0 0x559b173118d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa7c8f860575cacd server_cookie=9289168b822c48ad in_seq=0 out_seq=0 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.714+0000 7f866affd640 10 client.26396 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f866affd640 1 -- 172.21.15.55:0/3997989693 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b1726b990 con 0x559b1730f4e0 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f866affd640 10 client.26396 handle_command_reply: tid=0 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 client.26396 shutdown 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 -- 172.21.15.55:0/3997989693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 msgr2=0x559b17303a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b17303a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8680006010 tx=0x7f8680000bc0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 -- 172.21.15.55:0/3997989693 shutdown_connections 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x559b1730f4e0 0x559b173118d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.723 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b1730aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fa9f0 0x559b17303fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 --2- 172.21.15.55:0/3997989693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172f9820 0x559b17303a70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f866affd640 0 client.26396 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:06:05.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 -- 172.21.15.55:0/3997989693 >> 172.21.15.55:0/3997989693 conn(0x559b1723a910 msgr2=0x559b172f4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 -- 172.21.15.55:0/3997989693 shutdown_connections 2024-04-26T22:06:05.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 1 -- 172.21.15.55:0/3997989693 wait complete. 2024-04-26T22:06:05.724 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.715+0000 7f8694b8b740 20 client.26396 trim_cache size 0 max 16384 2024-04-26T22:06:05.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f8694b8b740 1 Processor -- start 2024-04-26T22:06:05.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f8694b8b740 1 -- start start 2024-04-26T22:06:05.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172f9820 0x559b172f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b1730af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fb3b0 0x559b17317060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f8694b8b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1ac7a0 con 0x559b172f9820 2024-04-26T22:06:05.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f8694b8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1aa070 con 0x559b172fa9f0 2024-04-26T22:06:05.734 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f8694b8b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a7ec0 con 0x559b172fb3b0 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172f9820 0x559b172f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172f9820 0x559b172f88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53056/0 (socket says 172.21.15.55:53056) 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 -- 172.21.15.55:0/1718047985 learned_addr learned my addr 172.21.15.55:0/1718047985 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868b7fe640 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b1730af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fb3b0 0x559b17317060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 -- 172.21.15.55:0/1718047985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fb3b0 msgr2=0x559b17317060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fb3b0 0x559b17317060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 -- 172.21.15.55:0/1718047985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 msgr2=0x559b1730af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b1730af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868affd640 1 -- 172.21.15.55:0/1718047985 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b172f08b0 con 0x559b172f9820 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.729+0000 7f868b7fe640 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b1730af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f868affd640 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172f9820 0x559b172f88a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f86800059d0 tx=0x7f86800314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9582590eebda5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.735 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f86717fa640 1 -- 172.21.15.55:0/1718047985 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680041070 con 0x559b172f9820 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f86717fa640 1 -- 172.21.15.55:0/1718047985 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8680003aa0 con 0x559b172f9820 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f86717fa640 1 -- 172.21.15.55:0/1718047985 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680035040 con 0x559b172f9820 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 -- 172.21.15.55:0/1718047985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172f9820 msgr2=0x559b172f88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172f9820 0x559b172f88a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f86800059d0 tx=0x7f86800314a0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 -- 172.21.15.55:0/1718047985 shutdown_connections 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172fb3b0 0x559b17317060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b1730af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1718047985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172f9820 0x559b172f88a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 -- 172.21.15.55:0/1718047985 >> 172.21.15.55:0/1718047985 conn(0x559b1723a910 msgr2=0x559b17303cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 -- 172.21.15.55:0/1718047985 shutdown_connections 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.730+0000 7f8694b8b740 1 -- 172.21.15.55:0/1718047985 wait complete. 2024-04-26T22:06:05.736 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 1 -- start start 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 1 Processor -- start 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172f9820 0x559b173081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b173086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17309700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a8530 con 0x559b172fb3b0 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a99a0 con 0x559b172fa9f0 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f868c1a8c00 con 0x559b172f9820 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f8694b8b740 10 client.0 _ll_register_callbacks cb 0x7f86800316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f868affd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17309700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b173086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b173086f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45602/0 (socket says 172.21.15.55:45602) 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f868b7fe640 1 -- 172.21.15.55:0/1543655664 learned_addr learned my addr 172.21.15.55:0/1543655664 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.737 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.731+0000 7f868affd640 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172f9820 0x559b173081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.732+0000 7f868b7fe640 1 -- 172.21.15.55:0/1543655664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172f9820 msgr2=0x559b173081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.732+0000 7f868b7fe640 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172f9820 0x559b173081b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.732+0000 7f868b7fe640 1 -- 172.21.15.55:0/1543655664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 msgr2=0x559b17309700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.732+0000 7f868b7fe640 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17309700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.732+0000 7f868b7fe640 1 -- 172.21.15.55:0/1543655664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b17309cf0 con 0x559b172fa9f0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.732+0000 7f868affd640 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17309700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.732+0000 7f868b7fe640 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b173086f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x559b172f5fd0 tx=0x7f867c0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1503518ac5b00a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.732+0000 7f866affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f866affd640 1 -- 172.21.15.55:0/1543655664 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c00bea0 con 0x559b172fa9f0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f866affd640 1 -- 172.21.15.55:0/1543655664 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f867c0204f0 con 0x559b172fa9f0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 1 -- 172.21.15.55:0/1543655664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8680003aa0 con 0x559b172fa9f0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f866affd640 1 -- 172.21.15.55:0/1543655664 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f867c030050 con 0x559b172fa9f0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 10 client.26402 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 1 -- 172.21.15.55:0/1543655664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559b1730b090 con 0x559b172fa9f0 2024-04-26T22:06:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f866affd640 1 -- 172.21.15.55:0/1543655664 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8680003aa0 con 0x559b172fa9f0 2024-04-26T22:06:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f866affd640 1 -- 172.21.15.55:0/1543655664 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f867c01e400 con 0x559b172fa9f0 2024-04-26T22:06:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 10 client.26402 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 10 client.26402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:06:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 20 client.26402 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x559b17375e30 0x559b17378250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 4 client.26402 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:06:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.733+0000 7f8694b8b740 1 -- 172.21.15.55:0/1543655664 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b17265800 con 0x559b17375e30 2024-04-26T22:06:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.734+0000 7f868affd640 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x559b17375e30 0x559b17378250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.734+0000 7f868affd640 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x559b17375e30 0x559b17378250 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c964be2ec47da10 server_cookie=804e49fda6806aee in_seq=0 out_seq=0 2024-04-26T22:06:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.734+0000 7f866affd640 10 client.26402 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:06:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.734+0000 7f866affd640 1 -- 172.21.15.55:0/1543655664 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b17265800 con 0x559b17375e30 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.734+0000 7f866affd640 10 client.26402 handle_command_reply: tid=0 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.734+0000 7f8694b8b740 1 client.26402 shutdown 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 -- 172.21.15.55:0/1543655664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 msgr2=0x559b173086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b173086f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x559b172f5fd0 tx=0x7f867c0039c0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 -- 172.21.15.55:0/1543655664 shutdown_connections 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x559b17375e30 0x559b17378250 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x559b172fb3b0 0x559b17309700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b172fa9f0 0x559b173086f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f866affd640 0 client.26402 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 --2- 172.21.15.55:0/1543655664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b172f9820 0x559b173081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 -- 172.21.15.55:0/1543655664 >> 172.21.15.55:0/1543655664 conn(0x559b1723a910 msgr2=0x559b172f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 -- 172.21.15.55:0/1543655664 shutdown_connections 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 1 -- 172.21.15.55:0/1543655664 wait complete. 2024-04-26T22:06:05.741 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.735+0000 7f8694b8b740 20 client.26402 trim_cache size 0 max 16384 2024-04-26T22:06:05.742 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 -- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86740a3c40 msgr2=0x7f86740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86740a3c40 0x7f86740a6100 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8684062830 tx=0x7f868406b7d0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 -- 172.21.15.55:0/2150146995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f868c108670 msgr2=0x7f868c1a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f868c108670 0x7f868c1a01b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8684009fd0 tx=0x7f868400ef10 comp rx=0 tx=0).stop 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 -- 172.21.15.55:0/2150146995 shutdown_connections 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f86740a3c40 0x7f86740a6100 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f868c108670 0x7f868c1a01b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f868c105d40 0x7f868c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 --2- 172.21.15.55:0/2150146995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f868c103410 0x7f868c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 -- 172.21.15.55:0/2150146995 >> 172.21.15.55:0/2150146995 conn(0x7f868c0fce20 msgr2=0x7f868c103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 -- 172.21.15.55:0/2150146995 shutdown_connections 2024-04-26T22:06:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.736+0000 7f8691d9e640 1 -- 172.21.15.55:0/2150146995 wait complete. 2024-04-26T22:06:05.750 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:06:05.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:05 smithi064 ceph-mon[32765]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 521 MiB/s wr, 853 op/s 2024-04-26T22:06:05.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:05 smithi165 ceph-mon[33296]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 521 MiB/s wr, 853 op/s 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.857+0000 7eff6f06d640 1 -- 172.21.15.55:0/3924428898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 msgr2=0x7eff681058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.857+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3924428898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 0x7eff681058a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7eff58009900 tx=0x7eff5802a410 comp rx=0 tx=0).stop 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 -- 172.21.15.55:0/3924428898 shutdown_connections 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3924428898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff68108710 0x7eff6810ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3924428898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff68105de0 0x7eff681081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3924428898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 0x7eff681058a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 -- 172.21.15.55:0/3924428898 >> 172.21.15.55:0/3924428898 conn(0x7eff680fcde0 msgr2=0x7eff680ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 -- 172.21.15.55:0/3924428898 shutdown_connections 2024-04-26T22:06:05.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 -- 172.21.15.55:0/3924428898 wait complete. 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 Processor -- start 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 -- start start 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 0x7eff68198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff68105de0 0x7eff681991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff68108710 0x7eff681a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810e4d0 con 0x7eff681034b0 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810e350 con 0x7eff68105de0 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6f06d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810e650 con 0x7eff68108710 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6ce09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 0x7eff68198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6ce09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 0x7eff68198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53088/0 (socket says 172.21.15.55:53088) 2024-04-26T22:06:05.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff6ce09640 1 -- 172.21.15.55:0/3373072671 learned_addr learned my addr 172.21.15.55:0/3373072671 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.858+0000 7eff67fff640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff68105de0 0x7eff681991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6d60a640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff68108710 0x7eff681a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6ce09640 1 -- 172.21.15.55:0/3373072671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff68108710 msgr2=0x7eff681a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6ce09640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff68108710 0x7eff681a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6ce09640 1 -- 172.21.15.55:0/3373072671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff68105de0 msgr2=0x7eff681991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6ce09640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff68105de0 0x7eff681991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6ce09640 1 -- 172.21.15.55:0/3373072671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff681a0870 con 0x7eff681034b0 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6ce09640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 0x7eff68198c60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7eff58005940 tx=0x7eff58031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff65ffb640 1 -- 172.21.15.55:0/3373072671 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5803f030 con 0x7eff681034b0 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff65ffb640 1 -- 172.21.15.55:0/3373072671 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff58003aa0 con 0x7eff681034b0 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff65ffb640 1 -- 172.21.15.55:0/3373072671 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58035070 con 0x7eff681034b0 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6f06d640 1 -- 172.21.15.55:0/3373072671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff681a0b00 con 0x7eff681034b0 2024-04-26T22:06:05.862 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.859+0000 7eff6f06d640 1 -- 172.21.15.55:0/3373072671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff681a0ff0 con 0x7eff681034b0 2024-04-26T22:06:05.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.862+0000 7eff71d84740 1 Processor -- start 2024-04-26T22:06:05.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.862+0000 7eff65ffb640 1 -- 172.21.15.55:0/3373072671 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7eff58042030 con 0x7eff681034b0 2024-04-26T22:06:05.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.862+0000 7eff71d84740 1 -- start start 2024-04-26T22:06:05.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff71d84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f22820 0x55a855f22c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff71d84740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f23e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff71d84740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f243b0 0x55a855f2ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff71d84740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a855e86d80 con 0x55a855f239f0 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff71d84740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a855e40a20 con 0x55a855f243b0 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff71d84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a855e82d40 con 0x55a855f22820 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff6ce09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f23e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f243b0 0x55a855f2ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f243b0 0x55a855f2ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45628/0 (socket says 172.21.15.55:45628) 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 -- 172.21.15.55:0/1979851770 learned_addr learned my addr 172.21.15.55:0/1979851770 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff6d60a640 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f22820 0x55a855f22c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 -- 172.21.15.55:0/1979851770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f22820 msgr2=0x55a855f22c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f22820 0x55a855f22c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 -- 172.21.15.55:0/1979851770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 msgr2=0x55a855f23e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f23e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.870 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 -- 172.21.15.55:0/1979851770 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a855e4bc30 con 0x55a855f243b0 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff6d60a640 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f22820 0x55a855f22c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff65ffb640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7eff500a3c40 0x7eff500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff6d60a640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7eff500a3c40 0x7eff500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff67fff640 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f243b0 0x55a855f2ac40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00b740 tx=0x7eff5c00bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eec6a784ad4ce7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff65ffb640 1 -- 172.21.15.55:0/3373072671 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7eff580f0bf0 con 0x7eff681034b0 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.863+0000 7eff4effd640 1 -- 172.21.15.55:0/1979851770 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c003dd0 con 0x55a855f243b0 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff4effd640 1 -- 172.21.15.55:0/1979851770 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c015070 con 0x55a855f243b0 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff4effd640 1 -- 172.21.15.55:0/1979851770 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004d90 con 0x55a855f243b0 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 -- 172.21.15.55:0/1979851770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f243b0 msgr2=0x55a855f2ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff6d60a640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7eff500a3c40 0x7eff500a6100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55a855f238f0 tx=0x7eff60009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f243b0 0x55a855f2ac40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00b740 tx=0x7eff5c00bc00 comp rx=0 tx=0).stop 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 -- 172.21.15.55:0/1979851770 shutdown_connections 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f243b0 0x55a855f2ac40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f23e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 --2- 172.21.15.55:0/1979851770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f22820 0x55a855f22c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 -- 172.21.15.55:0/1979851770 >> 172.21.15.55:0/1979851770 conn(0x55a855e63910 msgr2=0x55a855f1e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 -- 172.21.15.55:0/1979851770 shutdown_connections 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.864+0000 7eff71d84740 1 -- 172.21.15.55:0/1979851770 wait complete. 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 1 -- start start 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 1 Processor -- start 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f34020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55a855f2b3c0 con 0x55a855f22820 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a855e0ddc0 con 0x55a855f239f0 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a855f2b540 con 0x55a855f243b0 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff71d84740 10 client.0 _ll_register_callbacks cb 0x55a855e29600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:05.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff6ce09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff6ce09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f2caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53114/0 (socket says 172.21.15.55:53114) 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff6ce09640 1 -- 172.21.15.55:0/2700671734 learned_addr learned my addr 172.21.15.55:0/2700671734 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff67fff640 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.865+0000 7eff6d60a640 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f34020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff6ce09640 1 -- 172.21.15.55:0/2700671734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 msgr2=0x55a855f34020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff6ce09640 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f34020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff6ce09640 1 -- 172.21.15.55:0/2700671734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 msgr2=0x55a855f2cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff6ce09640 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f2cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff6ce09640 1 -- 172.21.15.55:0/2700671734 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a855b983f0 con 0x55a855f22820 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff6ce09640 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f2caa0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7eff580c5470 tx=0x7eff58003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88304efd1664a35f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff47fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff47fff640 1 -- 172.21.15.55:0/2700671734 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58049070 con 0x55a855f22820 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff47fff640 1 -- 172.21.15.55:0/2700671734 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7eff580b93c0 con 0x55a855f22820 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 1 -- 172.21.15.55:0/2700671734 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5c015070 con 0x55a855f22820 2024-04-26T22:06:05.873 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff47fff640 1 -- 172.21.15.55:0/2700671734 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7eff580bcbb0 con 0x55a855f22820 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 10 client.17499 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 1 -- 172.21.15.55:0/2700671734 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a855c627d0 con 0x55a855f22820 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff47fff640 1 -- 172.21.15.55:0/2700671734 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c015070 con 0x55a855f22820 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff47fff640 1 -- 172.21.15.55:0/2700671734 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7eff580bd040 con 0x55a855f22820 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 10 client.17499 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 10 client.17499 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 20 client.17499 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a855f38800 0x55a855f3ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 4 client.17499 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.866+0000 7eff71d84740 1 -- 172.21.15.55:0/2700671734 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a855e94990 con 0x55a855f38800 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.867+0000 7eff67fff640 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a855f38800 0x55a855f3ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.867+0000 7eff47fff640 10 client.17499 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.867+0000 7eff67fff640 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a855f38800 0x55a855f3ac30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9942783550f3698 server_cookie=54fdd779f6a5150b in_seq=0 out_seq=0 2024-04-26T22:06:05.874 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff47fff640 1 -- 172.21.15.55:0/2700671734 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a855e94990 con 0x55a855f38800 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff47fff640 10 client.17499 handle_command_reply: tid=0 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 client.17499 shutdown 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 -- 172.21.15.55:0/2700671734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 msgr2=0x55a855f2caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f2caa0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7eff580c5470 tx=0x7eff58003f50 comp rx=0 tx=0).stop 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 -- 172.21.15.55:0/2700671734 shutdown_connections 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a855f38800 0x55a855f3ac30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f34020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f2cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff47fff640 0 client.17499 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 --2- 172.21.15.55:0/2700671734 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f2caa0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 -- 172.21.15.55:0/2700671734 >> 172.21.15.55:0/2700671734 conn(0x55a855e63910 msgr2=0x55a855f1d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 -- 172.21.15.55:0/2700671734 shutdown_connections 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 1 -- 172.21.15.55:0/2700671734 wait complete. 2024-04-26T22:06:05.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.868+0000 7eff71d84740 20 client.17499 trim_cache size 0 max 16384 2024-04-26T22:06:05.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff71d84740 1 Processor -- start 2024-04-26T22:06:05.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff71d84740 1 -- start start 2024-04-26T22:06:05.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff71d84740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f22820 0x55a855f2cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff71d84740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f33fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff71d84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f1d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.888 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff71d84740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7eff681ac770 con 0x55a855f239f0 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff71d84740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff681aa040 con 0x55a855f22820 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff71d84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff681a7e90 con 0x55a855f243b0 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff67fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f33fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff67fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f33fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53126/0 (socket says 172.21.15.55:53126) 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff6ce09640 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f22820 0x55a855f2cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.882+0000 7eff67fff640 1 -- 172.21.15.55:0/1749722970 learned_addr learned my addr 172.21.15.55:0/1749722970 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff6d60a640 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f1d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff67fff640 1 -- 172.21.15.55:0/1749722970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 msgr2=0x55a855f1d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff67fff640 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f1d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff67fff640 1 -- 172.21.15.55:0/1749722970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f22820 msgr2=0x55a855f2cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff67fff640 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f22820 0x55a855f2cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff67fff640 1 -- 172.21.15.55:0/1749722970 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a855f1e370 con 0x55a855f239f0 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff6ce09640 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f22820 0x55a855f2cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:05.889 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff6d60a640 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f1d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff67fff640 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f33fe0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0039c0 tx=0x7eff5c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c10a75be292717 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff4e7fc640 1 -- 172.21.15.55:0/1749722970 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c02f070 con 0x55a855f239f0 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff4e7fc640 1 -- 172.21.15.55:0/1749722970 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c015040 con 0x55a855f239f0 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.883+0000 7eff4e7fc640 1 -- 172.21.15.55:0/1749722970 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c02a940 con 0x55a855f239f0 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 -- 172.21.15.55:0/1749722970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 msgr2=0x55a855f33fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f33fe0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0039c0 tx=0x7eff5c00be50 comp rx=0 tx=0).stop 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 -- 172.21.15.55:0/1749722970 shutdown_connections 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f1d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f239f0 0x55a855f33fe0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 --2- 172.21.15.55:0/1749722970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f22820 0x55a855f2cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 -- 172.21.15.55:0/1749722970 >> 172.21.15.55:0/1749722970 conn(0x55a855e63910 msgr2=0x55a855f1c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 -- 172.21.15.55:0/1749722970 shutdown_connections 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 -- 172.21.15.55:0/1749722970 wait complete. 2024-04-26T22:06:05.890 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 -- start start 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.884+0000 7eff71d84740 1 Processor -- start 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f31820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f31d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f32d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7eff681a8500 con 0x55a855f22820 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff681a9970 con 0x55a855f239f0 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff681a8bd0 con 0x55a855f243b0 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 10 client.0 _ll_register_callbacks cb 0x7eff58005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f31820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f31820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53130/0 (socket says 172.21.15.55:53130) 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 -- 172.21.15.55:0/1499915460 learned_addr learned my addr 172.21.15.55:0/1499915460 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff67fff640 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f31d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6d60a640 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f32d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.891 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 -- 172.21.15.55:0/1499915460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 msgr2=0x55a855f32d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f32d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 -- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 msgr2=0x55a855f31d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f31d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 -- 172.21.15.55:0/1499915460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a855f33360 con 0x55a855f22820 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff67fff640 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f31d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff6ce09640 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f31820 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7eff5803d040 tx=0x7eff58009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625cbd30639fed03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff47fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff47fff640 1 -- 172.21.15.55:0/1499915460 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff580da020 con 0x55a855f22820 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff47fff640 1 -- 172.21.15.55:0/1499915460 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7eff580bd040 con 0x55a855f22820 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 1 -- 172.21.15.55:0/1499915460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5c015040 con 0x55a855f22820 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff47fff640 1 -- 172.21.15.55:0/1499915460 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7eff580a2050 con 0x55a855f22820 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 10 client.17505 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.885+0000 7eff71d84740 1 -- 172.21.15.55:0/1499915460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a855f9ef00 con 0x55a855f22820 2024-04-26T22:06:05.892 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff47fff640 1 -- 172.21.15.55:0/1499915460 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c015040 con 0x55a855f22820 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff47fff640 1 -- 172.21.15.55:0/1499915460 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7eff580b93c0 con 0x55a855f22820 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff71d84740 10 client.17505 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff71d84740 10 client.17505 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff71d84740 20 client.17505 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff71d84740 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a855f9f770 0x55a855fa1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff71d84740 4 client.17505 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff71d84740 1 -- 172.21.15.55:0/1499915460 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a855e8e800 con 0x55a855f9f770 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff67fff640 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a855f9f770 0x55a855fa1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff47fff640 10 client.17505 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.886+0000 7eff67fff640 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a855f9f770 0x55a855fa1b90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17e63853b75f9e73 server_cookie=e1cf4f61db8f9071 in_seq=0 out_seq=0 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff47fff640 1 -- 172.21.15.55:0/1499915460 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a855e8e800 con 0x55a855f9f770 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff47fff640 10 client.17505 handle_command_reply: tid=0 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 client.17505 shutdown 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 -- 172.21.15.55:0/1499915460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 msgr2=0x55a855f31820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.893 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f31820 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7eff5803d040 tx=0x7eff58009880 comp rx=0 tx=0).stop 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 -- 172.21.15.55:0/1499915460 shutdown_connections 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55a855f9f770 0x55a855fa1b90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a855f243b0 0x55a855f32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a855f239f0 0x55a855f31d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 --2- 172.21.15.55:0/1499915460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a855f22820 0x55a855f31820 secure :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7eff5803d040 tx=0x7eff58009880 comp rx=0 tx=0).stop 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff47fff640 0 client.17505 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 -- 172.21.15.55:0/1499915460 >> 172.21.15.55:0/1499915460 conn(0x55a855e63910 msgr2=0x55a855f2cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.887+0000 7eff71d84740 1 -- 172.21.15.55:0/1499915460 shutdown_connections 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.888+0000 7eff71d84740 1 -- 172.21.15.55:0/1499915460 wait complete. 2024-04-26T22:06:05.894 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.888+0000 7eff71d84740 20 client.17505 trim_cache size 0 max 16384 2024-04-26T22:06:05.895 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:06:05.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 -- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7eff500a3c40 msgr2=0x7eff500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7eff500a3c40 0x7eff500a6100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55a855f238f0 tx=0x7eff60009500 comp rx=0 tx=0).stop 2024-04-26T22:06:05.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 -- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 msgr2=0x7eff68198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:05.895 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 0x7eff68198c60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7eff58005940 tx=0x7eff58031a50 comp rx=0 tx=0).stop 2024-04-26T22:06:05.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 -- 172.21.15.55:0/3373072671 shutdown_connections 2024-04-26T22:06:05.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7eff500a3c40 0x7eff500a6100 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff68108710 0x7eff681a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff68105de0 0x7eff681991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 --2- 172.21.15.55:0/3373072671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff681034b0 0x7eff68198c60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:05.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 -- 172.21.15.55:0/3373072671 >> 172.21.15.55:0/3373072671 conn(0x7eff680fcde0 msgr2=0x7eff681052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:05.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 -- 172.21.15.55:0/3373072671 shutdown_connections 2024-04-26T22:06:05.896 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:05.889+0000 7eff6f06d640 1 -- 172.21.15.55:0/3373072671 wait complete. 2024-04-26T22:06:05.901 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:06:06.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 -- 172.21.15.55:0/2279645294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4103440 msgr2=0x7f9cf4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/2279645294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4103440 0x7f9cf4109cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009ca0 tx=0x7f9cec02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 -- 172.21.15.55:0/2279645294 shutdown_connections 2024-04-26T22:06:06.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/2279645294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4103440 0x7f9cf4109cd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.006 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/2279645294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cf4102a80 0x7f9cf4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/2279645294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cf4101890 0x7f9cf4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 -- 172.21.15.55:0/2279645294 >> 172.21.15.55:0/2279645294 conn(0x7f9cf40fce20 msgr2=0x7f9cf40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:06.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 -- 172.21.15.55:0/2279645294 shutdown_connections 2024-04-26T22:06:06.007 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.005+0000 7f9cf8c20640 1 -- 172.21.15.55:0/2279645294 wait complete. 2024-04-26T22:06:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf8c20640 1 Processor -- start 2024-04-26T22:06:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf8c20640 1 -- start start 2024-04-26T22:06:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf8c20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cf4101890 0x7f9cf4105820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf8c20640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4102a80 0x7f9cf4103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf8c20640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cf4103440 0x7f9cf41043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf8c20640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf410d3c0 con 0x7f9cf4102a80 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf8c20640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf410d240 con 0x7f9cf4101890 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf8c20640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf410d540 con 0x7f9cf4103440 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4102a80 0x7f9cf4103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cf4101890 0x7f9cf4105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.006+0000 7f9cf1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4102a80 0x7f9cf4103e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53158/0 (socket says 172.21.15.55:53158) 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf1d9b640 1 -- 172.21.15.55:0/3332041446 learned_addr learned my addr 172.21.15.55:0/3332041446 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cf4103440 0x7f9cf41043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf1d9b640 1 -- 172.21.15.55:0/3332041446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cf4103440 msgr2=0x7f9cf41043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cf4103440 0x7f9cf41043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf1d9b640 1 -- 172.21.15.55:0/3332041446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cf4101890 msgr2=0x7f9cf4105820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cf4101890 0x7f9cf4105820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf1d9b640 1 -- 172.21.15.55:0/3332041446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf4104c30 con 0x7f9cf4102a80 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf259c640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cf4101890 0x7f9cf4105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4102a80 0x7f9cf4103e60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800e9f0 tx=0x7f9ce800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9ce37fe640 1 -- 172.21.15.55:0/3332041446 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800cd80 con 0x7f9cf4102a80 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9ce37fe640 1 -- 172.21.15.55:0/3332041446 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce80108e0 con 0x7f9cf4102a80 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf8c20640 1 -- 172.21.15.55:0/3332041446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf41a2cc0 con 0x7f9cf4102a80 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9ce37fe640 1 -- 172.21.15.55:0/3332041446 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8018980 con 0x7f9cf4102a80 2024-04-26T22:06:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.007+0000 7f9cf8c20640 1 -- 172.21.15.55:0/3332041446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf41a31b0 con 0x7f9cf4102a80 2024-04-26T22:06:06.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.009+0000 7f9ce37fe640 1 -- 172.21.15.55:0/3332041446 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ce8010430 con 0x7f9cf4102a80 2024-04-26T22:06:06.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cfba13740 1 Processor -- start 2024-04-26T22:06:06.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cfba13740 1 -- start start 2024-04-26T22:06:06.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405af820 0x55d2405afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b09f0 0x55d2405b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b13b0 0x55d2405b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cfba13740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d240513d80 con 0x55d2405b13b0 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cfba13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d2404cda20 con 0x55d2405af820 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cfba13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d24050fd40 con 0x55d2405b09f0 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9ce37fe640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9cdc0a3cc0 0x7f9cdc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cf2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405af820 0x55d2405afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cf1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b13b0 0x55d2405b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cf2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405af820 0x55d2405afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45694/0 (socket says 172.21.15.55:45694) 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cf2d9d640 1 -- 172.21.15.55:0/1090537238 learned_addr learned my addr 172.21.15.55:0/1090537238 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.010+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9cdc0a3cc0 0x7f9cdc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9cf259c640 1 --2- 172.21.15.55:0/1090537238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b09f0 0x55d2405b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9ce37fe640 1 -- 172.21.15.55:0/3332041446 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9ce80d0840 con 0x7f9cf4102a80 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9cf2d9d640 1 -- 172.21.15.55:0/1090537238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b09f0 msgr2=0x55d2405b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/1090537238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b09f0 0x55d2405b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9cf2d9d640 1 -- 172.21.15.55:0/1090537238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b13b0 msgr2=0x55d2405b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/1090537238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b13b0 0x55d2405b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9cf2d9d640 1 -- 172.21.15.55:0/1090537238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2404d8c30 con 0x55d2405af820 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/1090537238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405af820 0x55d2405afc40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009430 tx=0x7f9cec031580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ceafaac9a34a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9cdc0a3cc0 0x7f9cdc0a6180 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9cec02ade0 tx=0x7f9cec03b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9ce0ff9640 1 -- 172.21.15.55:0/1090537238 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec04e070 con 0x55d2405af820 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9ce0ff9640 1 -- 172.21.15.55:0/1090537238 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cec036040 con 0x55d2405af820 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.011+0000 7f9ce0ff9640 1 -- 172.21.15.55:0/1090537238 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec02f8c0 con 0x55d2405af820 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 -- 172.21.15.55:0/1090537238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405af820 msgr2=0x55d2405afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 --2- 172.21.15.55:0/1090537238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405af820 0x55d2405afc40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009430 tx=0x7f9cec031580 comp rx=0 tx=0).stop 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 -- 172.21.15.55:0/1090537238 shutdown_connections 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 --2- 172.21.15.55:0/1090537238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b13b0 0x55d2405b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 --2- 172.21.15.55:0/1090537238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b09f0 0x55d2405b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 --2- 172.21.15.55:0/1090537238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405af820 0x55d2405afc40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 -- 172.21.15.55:0/1090537238 >> 172.21.15.55:0/1090537238 conn(0x55d2404f0910 msgr2=0x55d2405ab120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 -- 172.21.15.55:0/1090537238 shutdown_connections 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 -- 172.21.15.55:0/1090537238 wait complete. 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 -- start start 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.012+0000 7f9cfba13740 1 Processor -- start 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cfba13740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d2405b81e0 con 0x55d2405af820 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cfba13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d24049adc0 con 0x55d2405b09f0 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cfba13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d2405b8360 con 0x55d2405b13b0 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cfba13740 10 client.0 _ll_register_callbacks cb 0x55d2404b6600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:06.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf1d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405b99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53172/0 (socket says 172.21.15.55:53172) 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 -- 172.21.15.55:0/3330832394 learned_addr learned my addr 172.21.15.55:0/3330832394 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 -- 172.21.15.55:0/3330832394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 msgr2=0x55d2405c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 -- 172.21.15.55:0/3330832394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 msgr2=0x55d2405b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 -- 172.21.15.55:0/3330832394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2402253f0 con 0x55d2405af820 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cf259c640 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405b99e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4011760 tx=0x7f9ce4011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc8fae0887a0a4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:06.022 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.013+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/3330832394 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce401f060 con 0x55d2405af820 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 1 -- 172.21.15.55:0/3330832394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cec036040 con 0x55d2405af820 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/3330832394 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9ce4010b40 con 0x55d2405af820 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/3330832394 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9ce4010d40 con 0x55d2405af820 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 10 client.17517 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 1 -- 172.21.15.55:0/3330832394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d2402ef7d0 con 0x55d2405af820 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/3330832394 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cec036040 con 0x55d2405af820 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/3330832394 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9ce40243d0 con 0x55d2405af820 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 10 client.17517 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 20 client.17517 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d2405c57d0 0x55d2405c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 4 client.17517 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cfba13740 1 -- 172.21.15.55:0/3330832394 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d240521990 con 0x55d2405c57d0 2024-04-26T22:06:06.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.014+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d2405c57d0 0x55d2405c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.015+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d2405c57d0 0x55d2405c7c70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f3f299c5acb8d6 server_cookie=7a3a7a55c183247b in_seq=0 out_seq=0 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.015+0000 7f9cd9ffb640 10 client.17517 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/3330832394 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d240521990 con 0x55d2405c57d0 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cd9ffb640 10 client.17517 handle_command_reply: tid=0 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 client.17517 shutdown 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 -- 172.21.15.55:0/3330832394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 msgr2=0x55d2405b99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405b99e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4011760 tx=0x7f9ce4011c20 comp rx=0 tx=0).stop 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 -- 172.21.15.55:0/3330832394 shutdown_connections 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d2405c57d0 0x55d2405c7c70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 --2- 172.21.15.55:0/3330832394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405b99e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 -- 172.21.15.55:0/3330832394 >> 172.21.15.55:0/3330832394 conn(0x55d2404f0910 msgr2=0x55d2405aa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:06.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cd9ffb640 0 client.17517 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:06.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 -- 172.21.15.55:0/3330832394 shutdown_connections 2024-04-26T22:06:06.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 1 -- 172.21.15.55:0/3330832394 wait complete. 2024-04-26T22:06:06.025 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.017+0000 7f9cfba13740 20 client.17517 trim_cache size 0 max 16384 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cfba13740 1 Processor -- start 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cfba13740 1 -- start start 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405aa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cfba13740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf41ae9e0 con 0x55d2405af820 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cfba13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf41ac2b0 con 0x55d2405b09f0 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cfba13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf41aa100 con 0x55d2405b13b0 2024-04-26T22:06:06.036 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cf259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cf259c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405aa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53188/0 (socket says 172.21.15.55:53188) 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cf259c640 1 -- 172.21.15.55:0/4250316587 learned_addr learned my addr 172.21.15.55:0/4250316587 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.031+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cf259c640 1 -- 172.21.15.55:0/4250316587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 msgr2=0x55d2405c0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cf259c640 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cf259c640 1 -- 172.21.15.55:0/4250316587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 msgr2=0x55d2405aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cf259c640 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cf259c640 1 -- 172.21.15.55:0/4250316587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2405abdd0 con 0x55d2405af820 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cf259c640 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405aa100 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4006010 tx=0x7f9ce40108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6b1ccc7ab597f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cdbfff640 1 -- 172.21.15.55:0/4250316587 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4024e60 con 0x55d2405af820 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cdbfff640 1 -- 172.21.15.55:0/4250316587 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce4010b40 con 0x55d2405af820 2024-04-26T22:06:06.037 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cdbfff640 1 -- 172.21.15.55:0/4250316587 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce401bc60 con 0x55d2405af820 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cfba13740 1 -- 172.21.15.55:0/4250316587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 msgr2=0x55d2405aa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cfba13740 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405aa100 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4006010 tx=0x7f9ce40108c0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cfba13740 1 -- 172.21.15.55:0/4250316587 shutdown_connections 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cfba13740 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405b13b0 0x55d2405c0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cfba13740 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b09f0 0x55d2405aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cfba13740 1 --2- 172.21.15.55:0/4250316587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405af820 0x55d2405aa100 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.032+0000 7f9cfba13740 1 -- 172.21.15.55:0/4250316587 >> 172.21.15.55:0/4250316587 conn(0x55d2404f0910 msgr2=0x55d2405a99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 -- 172.21.15.55:0/4250316587 shutdown_connections 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 -- 172.21.15.55:0/4250316587 wait complete. 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 -- start start 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 Processor -- start 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405af820 0x55d2405be0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b09f0 0x55d2405be630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.038 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b13b0 0x55d2405bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf41aa770 con 0x55d2405b09f0 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf41abbe0 con 0x55d2405b13b0 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf41aae40 con 0x55d2405af820 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cfba13740 10 client.0 _ll_register_callbacks cb 0x7f9ce400bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cf1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b09f0 0x55d2405be630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cf1d9b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b09f0 0x55d2405be630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53190/0 (socket says 172.21.15.55:53190) 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cf1d9b640 1 -- 172.21.15.55:0/2259240515 learned_addr learned my addr 172.21.15.55:0/2259240515 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cf2d9d640 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b13b0 0x55d2405bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.033+0000 7f9cf259c640 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405af820 0x55d2405be0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cf1d9b640 1 -- 172.21.15.55:0/2259240515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405af820 msgr2=0x55d2405be0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405af820 0x55d2405be0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cf1d9b640 1 -- 172.21.15.55:0/2259240515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b13b0 msgr2=0x55d2405bf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b13b0 0x55d2405bf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.039 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cf1d9b640 1 -- 172.21.15.55:0/2259240515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2405bfc30 con 0x55d2405b09f0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cf1d9b640 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b09f0 0x55d2405be630 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8098e90 tx=0x7f9ce806c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b077596f9546aaee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/2259240515 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce80ce400 con 0x55d2405b09f0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/2259240515 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9ce8010a90 con 0x55d2405b09f0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cfba13740 1 -- 172.21.15.55:0/2259240515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ce4010b40 con 0x55d2405b09f0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/2259240515 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9ce80b8dd0 con 0x55d2405b09f0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cfba13740 10 client.17523 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cfba13740 1 -- 172.21.15.55:0/2259240515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d2405c0fa0 con 0x55d2405b09f0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/2259240515 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce4010b40 con 0x55d2405b09f0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/2259240515 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9ce80a51f0 con 0x55d2405b09f0 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cfba13740 10 client.17523 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cfba13740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cfba13740 20 client.17523 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:06.040 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.034+0000 7f9cfba13740 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d24062c380 0x55d24062e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:06.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.035+0000 7f9cfba13740 4 client.17523 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:06:06.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.035+0000 7f9cfba13740 1 -- 172.21.15.55:0/2259240515 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d24051b800 con 0x55d24062c380 2024-04-26T22:06:06.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.035+0000 7f9cf259c640 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d24062c380 0x55d24062e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:06.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.035+0000 7f9cf259c640 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d24062c380 0x55d24062e7a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c47b392dda08d57 server_cookie=e763eb69045c86d6 in_seq=0 out_seq=0 2024-04-26T22:06:06.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.035+0000 7f9cd9ffb640 10 client.17523 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:06.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cd9ffb640 1 -- 172.21.15.55:0/2259240515 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d24051b800 con 0x55d24062c380 2024-04-26T22:06:06.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cd9ffb640 10 client.17523 handle_command_reply: tid=0 2024-04-26T22:06:06.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 client.17523 shutdown 2024-04-26T22:06:06.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 -- 172.21.15.55:0/2259240515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b09f0 msgr2=0x55d2405be630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b09f0 0x55d2405be630 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8098e90 tx=0x7f9ce806c420 comp rx=0 tx=0).stop 2024-04-26T22:06:06.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 -- 172.21.15.55:0/2259240515 shutdown_connections 2024-04-26T22:06:06.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d2405b13b0 0x55d2405bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d24062c380 0x55d24062e7a0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d2405b09f0 0x55d2405be630 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 --2- 172.21.15.55:0/2259240515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d2405af820 0x55d2405be0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cd9ffb640 0 client.17523 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 -- 172.21.15.55:0/2259240515 >> 172.21.15.55:0/2259240515 conn(0x55d2404f0910 msgr2=0x55d2405aa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 -- 172.21.15.55:0/2259240515 shutdown_connections 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 1 -- 172.21.15.55:0/2259240515 wait complete. 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.038+0000 7f9cfba13740 20 client.17523 trim_cache size 0 max 16384 2024-04-26T22:06:06.043 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:06:06.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 -- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9cdc0a3cc0 msgr2=0x7f9cdc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9cdc0a3cc0 0x7f9cdc0a6180 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9cec02ade0 tx=0x7f9cec03b000 comp rx=0 tx=0).stop 2024-04-26T22:06:06.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 -- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4102a80 msgr2=0x7f9cf4103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:06.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4102a80 0x7f9cf4103e60 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800e9f0 tx=0x7f9ce800eeb0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 -- 172.21.15.55:0/3332041446 shutdown_connections 2024-04-26T22:06:06.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9cdc0a3cc0 0x7f9cdc0a6180 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9cf4103440 0x7f9cf41043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cf4102a80 0x7f9cf4103e60 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 --2- 172.21.15.55:0/3332041446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cf4101890 0x7f9cf4105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:06.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 -- 172.21.15.55:0/3332041446 >> 172.21.15.55:0/3332041446 conn(0x7f9cf40fce20 msgr2=0x7f9cf40fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:06.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 -- 172.21.15.55:0/3332041446 shutdown_connections 2024-04-26T22:06:06.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:06.039+0000 7f9cf8c20640 1 -- 172.21.15.55:0/3332041446 wait complete. 2024-04-26T22:06:06.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:06 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/529754028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:06:06.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:06 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/529754028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:06:06.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:06 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/529754028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:06:07.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:06:07.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:06:07.052 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:06:07.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 -- 172.21.15.55:0/140558130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818103470 msgr2=0x7f5818105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 --2- 172.21.15.55:0/140558130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818103470 0x7f5818105860 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5810009900 tx=0x7f581002a410 comp rx=0 tx=0).stop 2024-04-26T22:06:07.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 -- 172.21.15.55:0/140558130 shutdown_connections 2024-04-26T22:06:07.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 --2- 172.21.15.55:0/140558130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58181086d0 0x7f581810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 --2- 172.21.15.55:0/140558130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5818105da0 0x7f5818108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 --2- 172.21.15.55:0/140558130 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818103470 0x7f5818105860 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 -- 172.21.15.55:0/140558130 >> 172.21.15.55:0/140558130 conn(0x7f58180fce20 msgr2=0x7f58180ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 -- 172.21.15.55:0/140558130 shutdown_connections 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 -- 172.21.15.55:0/140558130 wait complete. 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.183+0000 7f581da09640 1 Processor -- start 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f581da09640 1 -- start start 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f581da09640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5818103470 0x7f5818198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f581da09640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818105da0 0x7f5818199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f581da09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58181086d0 0x7f58181a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f581da09640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f581810e4b0 con 0x7f5818105da0 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f581da09640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f581810e330 con 0x7f58181086d0 2024-04-26T22:06:07.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f581da09640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f581810e630 con 0x7f5818103470 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818105da0 0x7f5818199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818105da0 0x7f5818199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53206/0 (socket says 172.21.15.55:53206) 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 -- 172.21.15.55:0/2755032935 learned_addr learned my addr 172.21.15.55:0/2755032935 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f5816ffd640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5818103470 0x7f5818198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58177fe640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58181086d0 0x7f58181a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 -- 172.21.15.55:0/2755032935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5818103470 msgr2=0x7f5818198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5818103470 0x7f5818198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 -- 172.21.15.55:0/2755032935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58181086d0 msgr2=0x7f58181a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58181086d0 0x7f58181a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 -- 172.21.15.55:0/2755032935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58181a0800 con 0x7f5818105da0 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.184+0000 7f58167fc640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818105da0 0x7f5818199120 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5800005fd0 tx=0x7f580000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.185+0000 7f581ca07640 1 -- 172.21.15.55:0/2755032935 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800017070 con 0x7f5818105da0 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.185+0000 7f581ca07640 1 -- 172.21.15.55:0/2755032935 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5800009c30 con 0x7f5818105da0 2024-04-26T22:06:07.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.185+0000 7f581da09640 1 -- 172.21.15.55:0/2755032935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58181a0af0 con 0x7f5818105da0 2024-04-26T22:06:07.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.185+0000 7f581ca07640 1 -- 172.21.15.55:0/2755032935 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800003d30 con 0x7f5818105da0 2024-04-26T22:06:07.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.185+0000 7f581da09640 1 -- 172.21.15.55:0/2755032935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a1040 con 0x7f5818105da0 2024-04-26T22:06:07.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f581ca07640 1 -- 172.21.15.55:0/2755032935 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5800003ed0 con 0x7f5818105da0 2024-04-26T22:06:07.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f58207fe740 1 Processor -- start 2024-04-26T22:06:07.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f58207fe740 1 -- start start 2024-04-26T22:06:07.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f58207fe740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f58207fe740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 0x5596e88521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f58207fe740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 0x5596e8854cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f58207fe740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5596e87adb00 con 0x5596e884ccc0 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f58207fe740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5596e8749920 con 0x5596e8852820 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.187+0000 7f58207fe740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5596e87a9ac0 con 0x5596e884d6b0 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58177fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58177fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53214/0 (socket says 172.21.15.55:53214) 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58177fe640 1 -- 172.21.15.55:0/2308110329 learned_addr learned my addr 172.21.15.55:0/2308110329 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58167fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 0x5596e8854cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f581ca07640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58040a3c40 0x7f58040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f5816ffd640 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 0x5596e88521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58177fe640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58040a3c40 0x7f58040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f581ca07640 1 -- 172.21.15.55:0/2755032935 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f58000d0240 con 0x7f5818105da0 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58167fc640 1 -- 172.21.15.55:0/2308110329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 msgr2=0x5596e88521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.194 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58167fc640 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 0x5596e88521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58167fc640 1 -- 172.21.15.55:0/2308110329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 msgr2=0x5596e884d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58167fc640 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58167fc640 1 -- 172.21.15.55:0/2308110329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e85877e0 con 0x5596e8852820 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58177fe640 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58167fc640 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 0x5596e8854cb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58000cdb50 tx=0x7f580000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe96d68ec769099e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f58177fe640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58040a3c40 0x7f58040a6100 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5596e884df10 tx=0x7f5808004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.188+0000 7f580dffb640 1 -- 172.21.15.55:0/2308110329 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800009de0 con 0x5596e8852820 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f580dffb640 1 -- 172.21.15.55:0/2308110329 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580009c320 con 0x5596e8852820 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f580dffb640 1 -- 172.21.15.55:0/2308110329 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800098af0 con 0x5596e8852820 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 -- 172.21.15.55:0/2308110329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 msgr2=0x5596e8854cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 0x5596e8854cb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f58000cdb50 tx=0x7f580000bf60 comp rx=0 tx=0).stop 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 -- 172.21.15.55:0/2308110329 shutdown_connections 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 0x5596e8854cb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.195 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 0x5596e88521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 --2- 172.21.15.55:0/2308110329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 -- 172.21.15.55:0/2308110329 >> 172.21.15.55:0/2308110329 conn(0x5596e8848330 msgr2=0x5596e8848730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 -- 172.21.15.55:0/2308110329 shutdown_connections 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 -- 172.21.15.55:0/2308110329 wait complete. 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.189+0000 7f58207fe740 1 -- start start 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58207fe740 1 Processor -- start 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58207fe740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884ccc0 0x5596e88570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58207fe740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e88575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58207fe740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e8852820 0x5596e885e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58207fe740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5596e8855770 con 0x5596e8852820 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58207fe740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5596e87475d0 con 0x5596e884d6b0 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58207fe740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5596e88558f0 con 0x5596e884ccc0 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58207fe740 10 client.0 _ll_register_callbacks cb 0x5596e8735080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58177fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e8852820 0x5596e885e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.196 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58167fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e88575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58177fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e8852820 0x5596e885e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53218/0 (socket says 172.21.15.55:53218) 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58177fe640 1 -- 172.21.15.55:0/2683658614 learned_addr learned my addr 172.21.15.55:0/2683658614 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f5816ffd640 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884ccc0 0x5596e88570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58177fe640 1 -- 172.21.15.55:0/2683658614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884ccc0 msgr2=0x5596e88570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58177fe640 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884ccc0 0x5596e88570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58177fe640 1 -- 172.21.15.55:0/2683658614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 msgr2=0x5596e88575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58177fe640 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e88575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58177fe640 1 -- 172.21.15.55:0/2683658614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e85e7f70 con 0x5596e8852820 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f58167fc640 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e88575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.190+0000 7f5816ffd640 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884ccc0 0x5596e88570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58177fe640 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e8852820 0x5596e885e630 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5808007be0 tx=0x7f5808004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7613e83c52cd10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f57f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f57f6ffd640 1 -- 172.21.15.55:0/2683658614 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808019070 con 0x5596e8852820 2024-04-26T22:06:07.197 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f57f6ffd640 1 -- 172.21.15.55:0/2683658614 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5808020810 con 0x5596e8852820 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 1 -- 172.21.15.55:0/2683658614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580009c320 con 0x5596e8852820 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f57f6ffd640 1 -- 172.21.15.55:0/2683658614 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5808020a10 con 0x5596e8852820 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 10 client.17535 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 1 -- 172.21.15.55:0/2683658614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5596e847f170 con 0x5596e8852820 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f57f6ffd640 1 -- 172.21.15.55:0/2683658614 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580009c320 con 0x5596e8852820 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f57f6ffd640 1 -- 172.21.15.55:0/2683658614 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f5808028b20 con 0x5596e8852820 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 10 client.17535 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 20 client.17535 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5596e8862cd0 0x5596e8865120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 4 client.17535 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.191+0000 7f58207fe740 1 -- 172.21.15.55:0/2683658614 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e87bb710 con 0x5596e8862cd0 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.192+0000 7f5816ffd640 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5596e8862cd0 0x5596e8865120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.198 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.192+0000 7f5816ffd640 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5596e8862cd0 0x5596e8865120 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7e441db95e2b46 server_cookie=70b5488c588899f8 in_seq=0 out_seq=0 2024-04-26T22:06:07.199 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.192+0000 7f57f6ffd640 10 client.17535 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f57f6ffd640 1 -- 172.21.15.55:0/2683658614 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596e87bb710 con 0x5596e8862cd0 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f57f6ffd640 10 client.17535 handle_command_reply: tid=0 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 client.17535 shutdown 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 -- 172.21.15.55:0/2683658614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e8852820 msgr2=0x5596e885e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e8852820 0x5596e885e630 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f5808007be0 tx=0x7f5808004a40 comp rx=0 tx=0).stop 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 -- 172.21.15.55:0/2683658614 shutdown_connections 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5596e8862cd0 0x5596e8865120 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e8852820 0x5596e885e630 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f57f6ffd640 0 client.17535 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e88575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.200 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 --2- 172.21.15.55:0/2683658614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884ccc0 0x5596e88570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 -- 172.21.15.55:0/2683658614 >> 172.21.15.55:0/2683658614 conn(0x5596e8848330 msgr2=0x5596e88430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 -- 172.21.15.55:0/2683658614 shutdown_connections 2024-04-26T22:06:07.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 1 -- 172.21.15.55:0/2683658614 wait complete. 2024-04-26T22:06:07.201 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.194+0000 7f58207fe740 20 client.17535 trim_cache size 0 max 16384 2024-04-26T22:06:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f58207fe740 1 Processor -- start 2024-04-26T22:06:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f58207fe740 1 -- start start 2024-04-26T22:06:07.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f58207fe740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f58207fe740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e885e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f58207fe740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e8852820 0x5596e884b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f58207fe740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f58181aa5e0 con 0x5596e884ccc0 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f58207fe740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58181a7eb0 con 0x5596e884d6b0 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f58207fe740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58181a5d00 con 0x5596e8852820 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f5816ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f5816ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53228/0 (socket says 172.21.15.55:53228) 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.212+0000 7f5816ffd640 1 -- 172.21.15.55:0/3014442244 learned_addr learned my addr 172.21.15.55:0/3014442244 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f58167fc640 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e885e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f58177fe640 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e8852820 0x5596e884b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f5816ffd640 1 -- 172.21.15.55:0/3014442244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e8852820 msgr2=0x5596e884b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f5816ffd640 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e8852820 0x5596e884b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f5816ffd640 1 -- 172.21.15.55:0/3014442244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 msgr2=0x5596e885e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f5816ffd640 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e885e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f5816ffd640 1 -- 172.21.15.55:0/3014442244 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e8848160 con 0x5596e884ccc0 2024-04-26T22:06:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f5816ffd640 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884b010 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f581002a930 tx=0x7f58100317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5c105e5afdc591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f580d7fa640 1 -- 172.21.15.55:0/3014442244 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810035040 con 0x5596e884ccc0 2024-04-26T22:06:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f580d7fa640 1 -- 172.21.15.55:0/3014442244 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5810031930 con 0x5596e884ccc0 2024-04-26T22:06:07.215 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.213+0000 7f580d7fa640 1 -- 172.21.15.55:0/3014442244 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58100038e0 con 0x5596e884ccc0 2024-04-26T22:06:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 -- 172.21.15.55:0/3014442244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 msgr2=0x5596e884b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884b010 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f581002a930 tx=0x7f58100317c0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 -- 172.21.15.55:0/3014442244 shutdown_connections 2024-04-26T22:06:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e8852820 0x5596e884b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e884d6b0 0x5596e885e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 --2- 172.21.15.55:0/3014442244 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e884b010 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 -- 172.21.15.55:0/3014442244 >> 172.21.15.55:0/3014442244 conn(0x5596e8848330 msgr2=0x5596e884eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 -- 172.21.15.55:0/3014442244 shutdown_connections 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.214+0000 7f58207fe740 1 -- 172.21.15.55:0/3014442244 wait complete. 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 1 -- start start 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 1 Processor -- start 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e885bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 0x5596e885c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 0x5596e885d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f58181a6370 con 0x5596e884ccc0 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58181a77e0 con 0x5596e8852820 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58181a6a40 con 0x5596e884d6b0 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.215+0000 7f58207fe740 10 client.0 _ll_register_callbacks cb 0x7f5808011f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e885bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e885bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53238/0 (socket says 172.21.15.55:53238) 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 -- 172.21.15.55:0/4232897274 learned_addr learned my addr 172.21.15.55:0/4232897274 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 -- 172.21.15.55:0/4232897274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 msgr2=0x5596e885c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 0x5596e885c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 -- 172.21.15.55:0/4232897274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 msgr2=0x5596e885d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 0x5596e885d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 -- 172.21.15.55:0/4232897274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e885d6e0 con 0x5596e884ccc0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f5816ffd640 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e885bba0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5810000c00 tx=0x7f5810031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6185e1d1b6cad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.216+0000 7f57f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f57f6ffd640 1 -- 172.21.15.55:0/4232897274 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810035030 con 0x5596e884ccc0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f57f6ffd640 1 -- 172.21.15.55:0/4232897274 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f58100468d0 con 0x5596e884ccc0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f58207fe740 1 -- 172.21.15.55:0/4232897274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5810031930 con 0x5596e884ccc0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f57f6ffd640 1 -- 172.21.15.55:0/4232897274 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f581003c440 con 0x5596e884ccc0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f58207fe740 10 client.17541 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f58207fe740 1 -- 172.21.15.55:0/4232897274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5596e88c9260 con 0x5596e884ccc0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f57f6ffd640 1 -- 172.21.15.55:0/4232897274 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5810031930 con 0x5596e884ccc0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f57f6ffd640 1 -- 172.21.15.55:0/4232897274 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f58100587c0 con 0x5596e884ccc0 2024-04-26T22:06:07.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.217+0000 7f58207fe740 10 client.17541 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.218+0000 7f58207fe740 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.218+0000 7f58207fe740 20 client.17541 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.218+0000 7f58207fe740 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5596e88c9a80 0x5596e88cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.218+0000 7f58207fe740 4 client.17541 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:06:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.218+0000 7f58207fe740 1 -- 172.21.15.55:0/4232897274 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5596e87b5580 con 0x5596e88c9a80 2024-04-26T22:06:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.218+0000 7f58167fc640 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5596e88c9a80 0x5596e88cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.219+0000 7f58167fc640 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5596e88c9a80 0x5596e88cbec0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c5f4dfef8ad06a server_cookie=69cd470020ad3ac5 in_seq=0 out_seq=0 2024-04-26T22:06:07.221 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.219+0000 7f57f6ffd640 10 client.17541 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:07.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f57f6ffd640 1 -- 172.21.15.55:0/4232897274 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5596e87b5580 con 0x5596e88c9a80 2024-04-26T22:06:07.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f57f6ffd640 10 client.17541 handle_command_reply: tid=0 2024-04-26T22:06:07.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 client.17541 shutdown 2024-04-26T22:06:07.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 -- 172.21.15.55:0/4232897274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 msgr2=0x5596e885bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e885bba0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f5810000c00 tx=0x7f5810031630 comp rx=0 tx=0).stop 2024-04-26T22:06:07.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 -- 172.21.15.55:0/4232897274 shutdown_connections 2024-04-26T22:06:07.472 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596e8852820 0x5596e885d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596e884d6b0 0x5596e885c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5596e88c9a80 0x5596e88cbec0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 --2- 172.21.15.55:0/4232897274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5596e884ccc0 0x5596e885bba0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f57f6ffd640 0 client.17541 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 -- 172.21.15.55:0/4232897274 >> 172.21.15.55:0/4232897274 conn(0x5596e8848330 msgr2=0x5596e8842850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 -- 172.21.15.55:0/4232897274 shutdown_connections 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 1 -- 172.21.15.55:0/4232897274 wait complete. 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.471+0000 7f58207fe740 20 client.17541 trim_cache size 0 max 16384 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:06:07.473 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "a1e66644-e64b-4178-8efc-0164a49e86a2", 2024-04-26T22:06:07.474 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:06:07.474 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:06:07.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 -- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58040a3c40 msgr2=0x7f58040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.474 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58040a3c40 0x7f58040a6100 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x5596e884df10 tx=0x7f5808004ab0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 -- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818105da0 msgr2=0x7f5818199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818105da0 0x7f5818199120 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f5800005fd0 tx=0x7f580000dde0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 -- 172.21.15.55:0/2755032935 shutdown_connections 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f58040a3c40 0x7f58040a6100 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58181086d0 0x7f58181a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5818105da0 0x7f5818199120 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 --2- 172.21.15.55:0/2755032935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5818103470 0x7f5818198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 -- 172.21.15.55:0/2755032935 >> 172.21.15.55:0/2755032935 conn(0x7f58180fce20 msgr2=0x7f5818103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 -- 172.21.15.55:0/2755032935 shutdown_connections 2024-04-26T22:06:07.475 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.472+0000 7f581da09640 1 -- 172.21.15.55:0/2755032935 wait complete. 2024-04-26T22:06:07.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:06:07.484 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 -- 172.21.15.55:0/3772958853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d34103f80 msgr2=0x7f3d341043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 --2- 172.21.15.55:0/3772958853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d34103f80 0x7f3d341043a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009900 tx=0x7f3d2402a410 comp rx=0 tx=0).stop 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 -- 172.21.15.55:0/3772958853 shutdown_connections 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 --2- 172.21.15.55:0/3772958853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d341013b0 0x7f3d3410af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 --2- 172.21.15.55:0/3772958853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d341009f0 0x7f3d34100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 --2- 172.21.15.55:0/3772958853 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d34103f80 0x7f3d341043a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 -- 172.21.15.55:0/3772958853 >> 172.21.15.55:0/3772958853 conn(0x7f3d340fc570 msgr2=0x7f3d340fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 -- 172.21.15.55:0/3772958853 shutdown_connections 2024-04-26T22:06:07.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.589+0000 7f3d3902a640 1 -- 172.21.15.55:0/3772958853 wait complete. 2024-04-26T22:06:07.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d3902a640 1 Processor -- start 2024-04-26T22:06:07.592 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d3902a640 1 -- start start 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d3902a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d341009f0 0x7f3d34103680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d3902a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d341013b0 0x7f3d34101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d3902a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d34103f80 0x7f3d34102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d3902a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410e490 con 0x7f3d341009f0 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d3902a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410e310 con 0x7f3d34103f80 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d3902a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3410e610 con 0x7f3d341013b0 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d32d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d341009f0 0x7f3d34103680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d32d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d341009f0 0x7f3d34103680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53264/0 (socket says 172.21.15.55:53264) 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.590+0000 7f3d32d9d640 1 -- 172.21.15.55:0/988767520 learned_addr learned my addr 172.21.15.55:0/988767520 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d3359e640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d34103f80 0x7f3d34102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d341013b0 0x7f3d34101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d32d9d640 1 -- 172.21.15.55:0/988767520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d341013b0 msgr2=0x7f3d34101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d341013b0 0x7f3d34101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d32d9d640 1 -- 172.21.15.55:0/988767520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d34103f80 msgr2=0x7f3d34102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d34103f80 0x7f3d34102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d32d9d640 1 -- 172.21.15.55:0/988767520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d34102a90 con 0x7f3d341009f0 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d341013b0 0x7f3d34101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d341009f0 0x7f3d34103680 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006010 tx=0x7f3d240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d30d99640 1 -- 172.21.15.55:0/988767520 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2403f030 con 0x7f3d341009f0 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d30d99640 1 -- 172.21.15.55:0/988767520 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d24031910 con 0x7f3d341009f0 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d3902a640 1 -- 172.21.15.55:0/988767520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d341a2c90 con 0x7f3d341009f0 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d30d99640 1 -- 172.21.15.55:0/988767520 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24035070 con 0x7f3d341009f0 2024-04-26T22:06:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.591+0000 7f3d3902a640 1 -- 172.21.15.55:0/988767520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d341a3190 con 0x7f3d341009f0 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.593+0000 7f3d30d99640 1 -- 172.21.15.55:0/988767520 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3d24042030 con 0x7f3d341009f0 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.593+0000 7f3d3be29740 1 Processor -- start 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3be29740 1 -- start start 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d97300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1d98530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1d9f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3be29740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f5e1cfbd80 con 0x55f5e1d980b0 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3be29740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f5e1cb58b0 con 0x55f5e1d96ee0 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3be29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f5e1cf7d40 con 0x55f5e1d98a70 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d32d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1d98530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.600 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d97300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d97300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:45800/0 (socket says 172.21.15.55:45800) 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d32d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1d98530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53266/0 (socket says 172.21.15.55:53266) 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 -- 172.21.15.55:0/1199174071 learned_addr learned my addr 172.21.15.55:0/1199174071 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/1199174071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1d9f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d30d99640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3d200a3c40 0x7f3d200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 -- 172.21.15.55:0/1199174071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 msgr2=0x55f5e1d9f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 --2- 172.21.15.55:0/1199174071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1d9f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 -- 172.21.15.55:0/1199174071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 msgr2=0x55f5e1d98530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 --2- 172.21.15.55:0/1199174071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1d98530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 -- 172.21.15.55:0/1199174071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e1c86a40 con 0x55f5e1d96ee0 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d30d99640 1 -- 172.21.15.55:0/988767520 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3d240f0b70 con 0x7f3d341009f0 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.594+0000 7f3d3359e640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3d200a3c40 0x7f3d200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.595+0000 7f3d3359e640 1 --2- 172.21.15.55:0/1199174071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d97300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c011590 tx=0x7f3d2c011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6de85413a86d8cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.601 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.595+0000 7f3d297fa640 1 -- 172.21.15.55:0/1199174071 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c017050 con 0x55f5e1d96ee0 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.595+0000 7f3d3359e640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3d200a3c40 0x7f3d200a6100 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c022000 tx=0x7f3d2c023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.595+0000 7f3d297fa640 1 -- 172.21.15.55:0/1199174071 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d2c00fcf0 con 0x55f5e1d96ee0 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.595+0000 7f3d297fa640 1 -- 172.21.15.55:0/1199174071 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c01baf0 con 0x55f5e1d96ee0 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- 172.21.15.55:0/1199174071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 msgr2=0x55f5e1d97300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 --2- 172.21.15.55:0/1199174071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d97300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c011590 tx=0x7f3d2c011a50 comp rx=0 tx=0).stop 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- 172.21.15.55:0/1199174071 shutdown_connections 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 --2- 172.21.15.55:0/1199174071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1d9f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 --2- 172.21.15.55:0/1199174071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1d98530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 --2- 172.21.15.55:0/1199174071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d97300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- 172.21.15.55:0/1199174071 >> 172.21.15.55:0/1199174071 conn(0x55f5e1d923d0 msgr2=0x55f5e1d927d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- 172.21.15.55:0/1199174071 shutdown_connections 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- 172.21.15.55:0/1199174071 wait complete. 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- start start 2024-04-26T22:06:07.602 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 Processor -- start 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1da8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55f5e1d9fa80 con 0x55f5e1d980b0 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f5e1c82d10 con 0x55f5e1d96ee0 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.596+0000 7f3d3be29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f5e1d9fc00 con 0x55f5e1d98a70 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d3be29740 10 client.0 _ll_register_callbacks cb 0x55f5e1caf210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d32d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53272/0 (socket says 172.21.15.55:53272) 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 -- 172.21.15.55:0/4191468932 learned_addr learned my addr 172.21.15.55:0/4191468932 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d3359e640 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1da8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 -- 172.21.15.55:0/4191468932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 msgr2=0x55f5e1da8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.603 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1da8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 -- 172.21.15.55:0/4191468932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 msgr2=0x55f5e1da13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 -- 172.21.15.55:0/4191468932 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e1ad77d0 con 0x55f5e1d980b0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da1930 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009c80 tx=0x7f3d1c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f52cf196df40a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/4191468932 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c012860 con 0x55f5e1d980b0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/4191468932 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3d1c012a00 con 0x55f5e1d980b0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d3be29740 1 -- 172.21.15.55:0/4191468932 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d2c00fcf0 con 0x55f5e1d980b0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/4191468932 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3d1c012c00 con 0x55f5e1d980b0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d3be29740 10 client.17553 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.597+0000 7f3d3be29740 1 -- 172.21.15.55:0/4191468932 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f5e1a0d3f0 con 0x55f5e1d980b0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/4191468932 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d2c00fcf0 con 0x55f5e1d980b0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/4191468932 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f3d1c01a020 con 0x55f5e1d980b0 2024-04-26T22:06:07.604 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d3be29740 10 client.17553 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:07.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d3be29740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:07.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d3be29740 20 client.17553 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:07.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d3be29740 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f5e1dacf20 0x55f5e1daf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d3be29740 4 client.17553 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:07.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d3be29740 1 -- 172.21.15.55:0/4191468932 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5e1d09990 con 0x55f5e1dacf20 2024-04-26T22:06:07.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f5e1dacf20 0x55f5e1daf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f5e1dacf20 0x55f5e1daf3a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a4a0dd6deaecf9 server_cookie=dd7226b4ecd142f5 in_seq=0 out_seq=0 2024-04-26T22:06:07.605 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.598+0000 7f3d1a7fc640 10 client.17553 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:07.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.600+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/4191468932 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5e1d09990 con 0x55f5e1dacf20 2024-04-26T22:06:07.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.600+0000 7f3d1a7fc640 10 client.17553 handle_command_reply: tid=0 2024-04-26T22:06:07.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 client.17553 shutdown 2024-04-26T22:06:07.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 -- 172.21.15.55:0/4191468932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 msgr2=0x55f5e1da1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da1930 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009c80 tx=0x7f3d1c0039c0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 -- 172.21.15.55:0/4191468932 shutdown_connections 2024-04-26T22:06:07.606 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1da8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f5e1dacf20 0x55f5e1daf3a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da1930 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 --2- 172.21.15.55:0/4191468932 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d1a7fc640 0 client.17553 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:07.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 -- 172.21.15.55:0/4191468932 >> 172.21.15.55:0/4191468932 conn(0x55f5e1d923d0 msgr2=0x55f5e1d91550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 -- 172.21.15.55:0/4191468932 shutdown_connections 2024-04-26T22:06:07.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 1 -- 172.21.15.55:0/4191468932 wait complete. 2024-04-26T22:06:07.607 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.601+0000 7f3d3be29740 20 client.17553 trim_cache size 0 max 16384 2024-04-26T22:06:07.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d3be29740 1 Processor -- start 2024-04-26T22:06:07.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d3be29740 1 -- start start 2024-04-26T22:06:07.621 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d9bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d980b0 0x55f5e1d9c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1d950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d3be29740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341ae9d0 con 0x55f5e1d96ee0 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d3be29740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341ac2a0 con 0x55f5e1d980b0 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d3be29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341aa0f0 con 0x55f5e1d98a70 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.619+0000 7f3d32d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d9bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d32d9d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d9bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53274/0 (socket says 172.21.15.55:53274) 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d32d9d640 1 -- 172.21.15.55:0/1218731008 learned_addr learned my addr 172.21.15.55:0/1218731008 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d980b0 0x55f5e1d9c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d3359e640 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1d950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d32d9d640 1 -- 172.21.15.55:0/1218731008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 msgr2=0x55f5e1d950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1d950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.622 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d32d9d640 1 -- 172.21.15.55:0/1218731008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d980b0 msgr2=0x55f5e1d9c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d980b0 0x55f5e1d9c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d32d9d640 1 -- 172.21.15.55:0/1218731008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5e19cd0f0 con 0x55f5e1d96ee0 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d9bbf0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3d240f4460 tx=0x7f3d240dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f0e3e3fc40da3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d28ff9640 1 -- 172.21.15.55:0/1218731008 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24049040 con 0x55f5e1d96ee0 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d28ff9640 1 -- 172.21.15.55:0/1218731008 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d240bd3b0 con 0x55f5e1d96ee0 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.620+0000 7f3d28ff9640 1 -- 172.21.15.55:0/1218731008 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d240c5920 con 0x55f5e1d96ee0 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 -- 172.21.15.55:0/1218731008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d96ee0 msgr2=0x55f5e1d9bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d9bbf0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3d240f4460 tx=0x7f3d240dd040 comp rx=0 tx=0).stop 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 -- 172.21.15.55:0/1218731008 shutdown_connections 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1d950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d980b0 0x55f5e1d9c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.623 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 --2- 172.21.15.55:0/1218731008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d96ee0 0x55f5e1d9bbf0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 -- 172.21.15.55:0/1218731008 >> 172.21.15.55:0/1218731008 conn(0x55f5e1d923d0 msgr2=0x55f5e1d8d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 -- 172.21.15.55:0/1218731008 shutdown_connections 2024-04-26T22:06:07.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 -- 172.21.15.55:0/1218731008 wait complete. 2024-04-26T22:06:07.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 -- start start 2024-04-26T22:06:07.624 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 Processor -- start 2024-04-26T22:06:07.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.621+0000 7f3d3be29740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1da66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d3be29740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341aa760 con 0x55f5e1d980b0 2024-04-26T22:06:07.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d3be29740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341abbd0 con 0x55f5e1d96ee0 2024-04-26T22:06:07.627 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d3be29740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3d341aae30 con 0x55f5e1d98a70 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d3be29740 10 client.0 _ll_register_callbacks cb 0x7f3d1c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d32d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53276/0 (socket says 172.21.15.55:53276) 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 -- 172.21.15.55:0/2605568315 learned_addr learned my addr 172.21.15.55:0/2605568315 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d3359e640 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1da66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 -- 172.21.15.55:0/2605568315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 msgr2=0x55f5e1da66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1da66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 -- 172.21.15.55:0/2605568315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 msgr2=0x55f5e1da51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 -- 172.21.15.55:0/2605568315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5e1cbc4b0 con 0x55f5e1d980b0 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d2bfff640 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da56f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x55f5e1d8ca00 tx=0x7f3d1c012660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1b3a1a4e9b1c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:07.628 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/2605568315 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c035060 con 0x55f5e1d980b0 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/2605568315 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3d1c005240 con 0x55f5e1d980b0 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d3be29740 1 -- 172.21.15.55:0/2605568315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d240bd3b0 con 0x55f5e1d980b0 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.622+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/2605568315 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3d1c02b050 con 0x55f5e1d980b0 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d3be29740 10 client.17559 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d3be29740 1 -- 172.21.15.55:0/2605568315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f5e1da7e80 con 0x55f5e1d980b0 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/2605568315 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d240bd3b0 con 0x55f5e1d980b0 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/2605568315 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f3d1c01a040 con 0x55f5e1d980b0 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d3be29740 10 client.17559 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d3be29740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d3be29740 20 client.17559 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d3be29740 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f5e1e13920 0x55f5e1e15d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d3be29740 4 client.17559 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d3be29740 1 -- 172.21.15.55:0/2605568315 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5e1d03800 con 0x55f5e1e13920 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f5e1e13920 0x55f5e1e15d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:07.629 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d1a7fc640 10 client.17559 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.623+0000 7f3d32d9d640 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f5e1e13920 0x55f5e1e15d40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f8585e250302ed server_cookie=fe112ceb3af75552 in_seq=0 out_seq=0 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.625+0000 7f3d1a7fc640 1 -- 172.21.15.55:0/2605568315 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55f5e1d03800 con 0x55f5e1e13920 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.625+0000 7f3d1a7fc640 10 client.17559 handle_command_reply: tid=0 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.625+0000 7f3d3be29740 1 client.17559 shutdown 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 -- 172.21.15.55:0/2605568315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 msgr2=0x55f5e1da56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da56f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x55f5e1d8ca00 tx=0x7f3d1c012660 comp rx=0 tx=0).stop 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 -- 172.21.15.55:0/2605568315 shutdown_connections 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f5e1d98a70 0x55f5e1da66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55f5e1e13920 0x55f5e1e15d40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55f5e1d980b0 0x55f5e1da56f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d1a7fc640 0 client.17559 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 --2- 172.21.15.55:0/2605568315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f5e1d96ee0 0x55f5e1da51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 -- 172.21.15.55:0/2605568315 >> 172.21.15.55:0/2605568315 conn(0x55f5e1d923d0 msgr2=0x55f5e1da13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 -- 172.21.15.55:0/2605568315 shutdown_connections 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 1 -- 172.21.15.55:0/2605568315 wait complete. 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3be29740 20 client.17559 trim_cache size 0 max 16384 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout: "a1e66644-e64b-4178-8efc-0164a49e86a2": { 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "a1e66644-e64b-4178-8efc-0164a49e86a2", 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:06:07.631 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:06:07.632 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:06:07.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 -- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3d200a3c40 msgr2=0x7f3d200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3d200a3c40 0x7f3d200a6100 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c022000 tx=0x7f3d2c023000 comp rx=0 tx=0).stop 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 -- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d341009f0 msgr2=0x7f3d34103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d341009f0 0x7f3d34103680 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006010 tx=0x7f3d240042f0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 -- 172.21.15.55:0/988767520 shutdown_connections 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3d200a3c40 0x7f3d200a6100 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3d34103f80 0x7f3d34102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d341013b0 0x7f3d34101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 --2- 172.21.15.55:0/988767520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d341009f0 0x7f3d34103680 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.626+0000 7f3d3902a640 1 -- 172.21.15.55:0/988767520 >> 172.21.15.55:0/988767520 conn(0x7f3d340fc570 msgr2=0x7f3d34107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.627+0000 7f3d3902a640 1 -- 172.21.15.55:0/988767520 shutdown_connections 2024-04-26T22:06:07.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:07.627+0000 7f3d3902a640 1 -- 172.21.15.55:0/988767520 wait complete. 2024-04-26T22:06:07.639 INFO:tasks.cephfs.filesystem:scrub status for tag:a1e66644-e64b-4178-8efc-0164a49e86a2 - {'path': '/', 'tag': 'a1e66644-e64b-4178-8efc-0164a49e86a2', 'options': 'recursive,force'} 2024-04-26T22:06:07.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:07 smithi064 ceph-mon[32765]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 353 MiB/s wr, 415 op/s 2024-04-26T22:06:07.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:07 smithi165 ceph-mon[33296]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 353 MiB/s wr, 415 op/s 2024-04-26T22:06:07.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:07 smithi055 ceph-mon[29221]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 353 MiB/s wr, 415 op/s 2024-04-26T22:06:08.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:08 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:06:08.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:08 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:08.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:08 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:06:08.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:08 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:06:08.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:08 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:08.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:08 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:06:08.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:08 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:08.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:08 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:06:08.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:08 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:06:08.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:08 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:08.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:08 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:06:08.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:08 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:08.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:08 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:06:08.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:08 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:06:08.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:08 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:09.728 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:09 smithi165 ceph-mon[33296]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 324 MiB/s wr, 306 op/s 2024-04-26T22:06:09.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:09 smithi064 ceph-mon[32765]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 324 MiB/s wr, 306 op/s 2024-04-26T22:06:09.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:09 smithi055 ceph-mon[29221]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 324 MiB/s wr, 306 op/s 2024-04-26T22:06:10.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:10 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:10.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:10 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:10.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:10 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:11.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:11 smithi064 ceph-mon[32765]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 124 MiB/s wr, 131 op/s 2024-04-26T22:06:11.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:11 smithi165 ceph-mon[33296]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 124 MiB/s wr, 131 op/s 2024-04-26T22:06:11.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:11 smithi055 ceph-mon[29221]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 124 MiB/s wr, 131 op/s 2024-04-26T22:06:13.738 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:13 smithi055 ceph-mon[29221]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 86 MiB/s wr, 94 op/s 2024-04-26T22:06:13.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:13 smithi064 ceph-mon[32765]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 86 MiB/s wr, 94 op/s 2024-04-26T22:06:13.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:13 smithi165 ceph-mon[33296]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 86 MiB/s wr, 94 op/s 2024-04-26T22:06:15.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:15 smithi064 ceph-mon[32765]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 84 MiB/s wr, 103 op/s 2024-04-26T22:06:15.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:15 smithi165 ceph-mon[33296]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 84 MiB/s wr, 103 op/s 2024-04-26T22:06:15.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:15 smithi055 ceph-mon[29221]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 84 MiB/s wr, 103 op/s 2024-04-26T22:06:17.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:17 smithi064 ceph-mon[32765]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 70 MiB/s wr, 90 op/s 2024-04-26T22:06:17.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:17 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:06:17.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:17 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:06:17.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:17 smithi165 ceph-mon[33296]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 70 MiB/s wr, 90 op/s 2024-04-26T22:06:17.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:17 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:06:17.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:17 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:06:17.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:17 smithi055 ceph-mon[29221]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 70 MiB/s wr, 90 op/s 2024-04-26T22:06:17.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:17 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:06:17.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:17 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:06:19.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:19 smithi064 ceph-mon[32765]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 82 MiB/s wr, 136 op/s 2024-04-26T22:06:19.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:19 smithi165 ceph-mon[33296]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 82 MiB/s wr, 136 op/s 2024-04-26T22:06:19.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:19 smithi055 ceph-mon[29221]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 82 MiB/s wr, 136 op/s 2024-04-26T22:06:21.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:21 smithi064 ceph-mon[32765]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 64 MiB/s wr, 162 op/s 2024-04-26T22:06:21.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:21 smithi165 ceph-mon[33296]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 64 MiB/s wr, 162 op/s 2024-04-26T22:06:21.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:21 smithi055 ceph-mon[29221]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 64 MiB/s wr, 162 op/s 2024-04-26T22:06:23.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:23 smithi064 ceph-mon[32765]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 66 MiB/s wr, 207 op/s 2024-04-26T22:06:23.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:23 smithi165 ceph-mon[33296]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 66 MiB/s wr, 207 op/s 2024-04-26T22:06:23.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:23 smithi055 ceph-mon[29221]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 66 MiB/s wr, 207 op/s 2024-04-26T22:06:25.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:25 smithi064 ceph-mon[32765]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 75 MiB/s wr, 282 op/s 2024-04-26T22:06:25.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:25 smithi165 ceph-mon[33296]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 75 MiB/s wr, 282 op/s 2024-04-26T22:06:25.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:25 smithi055 ceph-mon[29221]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 75 MiB/s wr, 282 op/s 2024-04-26T22:06:27.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:27 smithi064 ceph-mon[32765]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 67 MiB/s wr, 286 op/s 2024-04-26T22:06:27.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:27 smithi165 ceph-mon[33296]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 67 MiB/s wr, 286 op/s 2024-04-26T22:06:27.938 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:27 smithi055 ceph-mon[29221]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 67 MiB/s wr, 286 op/s 2024-04-26T22:06:29.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:29 smithi064 ceph-mon[32765]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 86 MiB/s wr, 420 op/s 2024-04-26T22:06:29.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:29 smithi165 ceph-mon[33296]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 86 MiB/s wr, 420 op/s 2024-04-26T22:06:29.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:29 smithi055 ceph-mon[29221]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 86 MiB/s wr, 420 op/s 2024-04-26T22:06:31.756 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:31 smithi064 ceph-mon[32765]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 80 MiB/s wr, 478 op/s 2024-04-26T22:06:31.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:31 smithi165 ceph-mon[33296]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 80 MiB/s wr, 478 op/s 2024-04-26T22:06:31.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:31 smithi055 ceph-mon[29221]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 80 MiB/s wr, 478 op/s 2024-04-26T22:06:33.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:33 smithi064 ceph-mon[32765]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 79 MiB/s wr, 530 op/s 2024-04-26T22:06:33.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:33 smithi165 ceph-mon[33296]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 79 MiB/s wr, 530 op/s 2024-04-26T22:06:33.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:33 smithi055 ceph-mon[29221]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 79 MiB/s wr, 530 op/s 2024-04-26T22:06:35.758 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:35 smithi064 ceph-mon[32765]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 74 MiB/s wr, 540 op/s 2024-04-26T22:06:35.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:35 smithi165 ceph-mon[33296]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 74 MiB/s wr, 540 op/s 2024-04-26T22:06:35.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:35 smithi055 ceph-mon[29221]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 74 MiB/s wr, 540 op/s 2024-04-26T22:06:37.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:06:37.641 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:06:37.757 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:37 smithi064 ceph-mon[32765]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 56 MiB/s wr, 457 op/s 2024-04-26T22:06:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.807+0000 7f30f002c640 1 -- 172.21.15.55:0/69677880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e81024e0 msgr2=0x7f30e8102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.807+0000 7f30f002c640 1 --2- 172.21.15.55:0/69677880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e81024e0 0x7f30e8102960 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009ca0 tx=0x7f30e002a8e0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.807+0000 7f30f002c640 1 -- 172.21.15.55:0/69677880 shutdown_connections 2024-04-26T22:06:37.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.807+0000 7f30f002c640 1 --2- 172.21.15.55:0/69677880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e81024e0 0x7f30e8102960 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.807+0000 7f30f002c640 1 --2- 172.21.15.55:0/69677880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e8101b40 0x7f30e8101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.807+0000 7f30f002c640 1 --2- 172.21.15.55:0/69677880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e81011e0 0x7f30e8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.807+0000 7f30f002c640 1 -- 172.21.15.55:0/69677880 >> 172.21.15.55:0/69677880 conn(0x7f30e80fcde0 msgr2=0x7f30e80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.808+0000 7f30f002c640 1 -- 172.21.15.55:0/69677880 shutdown_connections 2024-04-26T22:06:37.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.808+0000 7f30f002c640 1 -- 172.21.15.55:0/69677880 wait complete. 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.808+0000 7f30f002c640 1 Processor -- start 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.808+0000 7f30f002c640 1 -- start start 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30f002c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e81011e0 0x7f30e819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30f002c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e8101b40 0x7f30e819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30f002c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e81024e0 0x7f30e81a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30f002c640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30e8110750 con 0x7f30e8101b40 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30f002c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30e81105d0 con 0x7f30e81024e0 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30f002c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30e81108d0 con 0x7f30e81011e0 2024-04-26T22:06:37.811 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e8101b40 0x7f30e819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e8101b40 0x7f30e819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60428/0 (socket says 172.21.15.55:60428) 2024-04-26T22:06:37.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 -- 172.21.15.55:0/3954444039 learned_addr learned my addr 172.21.15.55:0/3954444039 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:37.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 -- 172.21.15.55:0/3954444039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e81011e0 msgr2=0x7f30e819ad60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:06:37.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e81011e0 0x7f30e819ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 -- 172.21.15.55:0/3954444039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e81024e0 msgr2=0x7f30e81a2310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T22:06:37.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e81024e0 0x7f30e81a2310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.812 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 -- 172.21.15.55:0/3954444039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e81a2980 con 0x7f30e8101b40 2024-04-26T22:06:37.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.809+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e8101b40 0x7f30e819b2a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f30d800c920 tx=0x7f30d800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.810+0000 7f30deffd640 1 -- 172.21.15.55:0/3954444039 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8007c70 con 0x7f30e8101b40 2024-04-26T22:06:37.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.810+0000 7f30deffd640 1 -- 172.21.15.55:0/3954444039 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30d8007e10 con 0x7f30e8101b40 2024-04-26T22:06:37.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.810+0000 7f30f002c640 1 -- 172.21.15.55:0/3954444039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e81a2c70 con 0x7f30e8101b40 2024-04-26T22:06:37.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.810+0000 7f30deffd640 1 -- 172.21.15.55:0/3954444039 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8018820 con 0x7f30e8101b40 2024-04-26T22:06:37.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.810+0000 7f30f002c640 1 -- 172.21.15.55:0/3954444039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e81a3240 con 0x7f30e8101b40 2024-04-26T22:06:37.813 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.813+0000 7f30deffd640 1 -- 172.21.15.55:0/3954444039 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f30d800f450 con 0x7f30e8101b40 2024-04-26T22:06:37.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30deffd640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30d40a3cc0 0x7f30d40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30deffd640 1 -- 172.21.15.55:0/3954444039 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f30d80d0760 con 0x7f30e8101b40 2024-04-26T22:06:37.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30eddc8640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30d40a3cc0 0x7f30d40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30f2e20740 1 Processor -- start 2024-04-26T22:06:37.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30f2e20740 1 -- start start 2024-04-26T22:06:37.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec7a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec7b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec82300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30f2e20740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558cbebded80 con 0x558cbec79ee0 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30f2e20740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558cbeb988b0 con 0x558cbec7b0b0 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.814+0000 7f30f2e20740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558cbebdad40 con 0x558cbec7ba70 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30d40a3cc0 0x7f30d40a6180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009670 tx=0x7f30e4006bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec7a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec7a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60432/0 (socket says 172.21.15.55:60432) 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 -- 172.21.15.55:0/3076528978 learned_addr learned my addr 172.21.15.55:0/3076528978 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/3076528978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec7b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 -- 172.21.15.55:0/3076528978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 msgr2=0x558cbec82300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 --2- 172.21.15.55:0/3076528978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec82300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 -- 172.21.15.55:0/3076528978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 msgr2=0x558cbec7b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 --2- 172.21.15.55:0/3076528978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec7b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 -- 172.21.15.55:0/3076528978 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbeb69a40 con 0x558cbec79ee0 2024-04-26T22:06:37.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.815+0000 7f30eddc8640 1 --2- 172.21.15.55:0/3076528978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec7a300 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f30e4013840 tx=0x7f30e400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633ca878b9c00a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.816+0000 7f30d3fff640 1 -- 172.21.15.55:0/3076528978 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e400bae0 con 0x558cbec79ee0 2024-04-26T22:06:37.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.816+0000 7f30d3fff640 1 -- 172.21.15.55:0/3076528978 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e4022070 con 0x558cbec79ee0 2024-04-26T22:06:37.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.816+0000 7f30d3fff640 1 -- 172.21.15.55:0/3076528978 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e40125d0 con 0x558cbec79ee0 2024-04-26T22:06:37.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.816+0000 7f30f2e20740 1 -- 172.21.15.55:0/3076528978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 msgr2=0x558cbec7a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.816+0000 7f30f2e20740 1 --2- 172.21.15.55:0/3076528978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec7a300 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f30e4013840 tx=0x7f30e400e000 comp rx=0 tx=0).stop 2024-04-26T22:06:37.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 -- 172.21.15.55:0/3076528978 shutdown_connections 2024-04-26T22:06:37.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 --2- 172.21.15.55:0/3076528978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec82300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.822 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 --2- 172.21.15.55:0/3076528978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec7b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 --2- 172.21.15.55:0/3076528978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec7a300 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 -- 172.21.15.55:0/3076528978 >> 172.21.15.55:0/3076528978 conn(0x558cbec753d0 msgr2=0x558cbec757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 -- 172.21.15.55:0/3076528978 shutdown_connections 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 -- 172.21.15.55:0/3076528978 wait complete. 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 -- start start 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 Processor -- start 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec79ee0 0x558cbec8cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec8d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.817+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec94120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30f2e20740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x558cbec82a80 con 0x558cbec7b0b0 2024-04-26T22:06:37.823 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30f2e20740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558cbeb65d10 con 0x558cbec79ee0 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30f2e20740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558cbec82c00 con 0x558cbec7ba70 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30f2e20740 10 client.0 _ll_register_callbacks cb 0x558cbeb92210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec8d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec8d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60444/0 (socket says 172.21.15.55:60444) 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 -- 172.21.15.55:0/756746926 learned_addr learned my addr 172.21.15.55:0/756746926 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 -- 172.21.15.55:0/756746926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 msgr2=0x558cbec94120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec94120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 -- 172.21.15.55:0/756746926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec79ee0 msgr2=0x558cbec8cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec79ee0 0x558cbec8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 -- 172.21.15.55:0/756746926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbe9ba7d0 con 0x558cbec7b0b0 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec8d0e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f30d8099270 tx=0x7f30d809c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c100e377416480 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:37.824 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30d17fa640 1 -- 172.21.15.55:0/756746926 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80a5510 con 0x558cbec7b0b0 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30d17fa640 1 -- 172.21.15.55:0/756746926 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f30d80ce320 con 0x558cbec7b0b0 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.818+0000 7f30f2e20740 1 -- 172.21.15.55:0/756746926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e4022070 con 0x558cbec7b0b0 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30d17fa640 1 -- 172.21.15.55:0/756746926 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f30d8098f10 con 0x558cbec7b0b0 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30f2e20740 10 client.17571 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30f2e20740 1 -- 172.21.15.55:0/756746926 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558cbe8f03f0 con 0x558cbec7b0b0 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30d17fa640 1 -- 172.21.15.55:0/756746926 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30e4022070 con 0x558cbec7b0b0 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30d17fa640 1 -- 172.21.15.55:0/756746926 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f30d80a5080 con 0x558cbec7b0b0 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30f2e20740 10 client.17571 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30f2e20740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30f2e20740 20 client.17571 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:37.825 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30f2e20740 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558cbec986e0 0x558cbec9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30f2e20740 4 client.17571 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.819+0000 7f30f2e20740 1 -- 172.21.15.55:0/756746926 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cbebec990 con 0x558cbec986e0 2024-04-26T22:06:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.820+0000 7f30eddc8640 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558cbec986e0 0x558cbec9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.821+0000 7f30eddc8640 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558cbec986e0 0x558cbec9aba0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21a74074d054e4fe server_cookie=670179a007abba27 in_seq=0 out_seq=0 2024-04-26T22:06:37.826 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.821+0000 7f30d17fa640 10 client.17571 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:37.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:37 smithi165 ceph-mon[33296]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 56 MiB/s wr, 457 op/s 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30d17fa640 1 -- 172.21.15.55:0/756746926 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558cbebec990 con 0x558cbec986e0 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30d17fa640 10 client.17571 handle_command_reply: tid=0 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 client.17571 shutdown 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 -- 172.21.15.55:0/756746926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 msgr2=0x558cbec8d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec8d0e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f30d8099270 tx=0x7f30d809c960 comp rx=0 tx=0).stop 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 -- 172.21.15.55:0/756746926 shutdown_connections 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec94120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558cbec986e0 0x558cbec9aba0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec8d0e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 --2- 172.21.15.55:0/756746926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec79ee0 0x558cbec8cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30d17fa640 0 client.17571 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:37.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 -- 172.21.15.55:0/756746926 >> 172.21.15.55:0/756746926 conn(0x558cbec753d0 msgr2=0x558cbec74550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 -- 172.21.15.55:0/756746926 shutdown_connections 2024-04-26T22:06:37.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 1 -- 172.21.15.55:0/756746926 wait complete. 2024-04-26T22:06:37.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.827+0000 7f30f2e20740 20 client.17571 trim_cache size 0 max 16384 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30f2e20740 1 Processor -- start 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30f2e20740 1 -- start start 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec79ee0 0x558cbec70240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7ba70 0x558cbec743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30f2e20740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30e81aa5e0 con 0x558cbec7b0b0 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30f2e20740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30e81a7eb0 con 0x558cbec7ba70 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30f2e20740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30e81a5d00 con 0x558cbec79ee0 2024-04-26T22:06:37.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30ed5c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30ed5c7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec940e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60448/0 (socket says 172.21.15.55:60448) 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30ed5c7640 1 -- 172.21.15.55:0/2854264169 learned_addr learned my addr 172.21.15.55:0/2854264169 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30eddc8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec79ee0 0x558cbec70240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.851+0000 7f30ee5c9640 1 --2- 172.21.15.55:0/2854264169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7ba70 0x558cbec743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30ed5c7640 1 -- 172.21.15.55:0/2854264169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec79ee0 msgr2=0x558cbec70240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/2854264169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec79ee0 0x558cbec70240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30ed5c7640 1 -- 172.21.15.55:0/2854264169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7ba70 msgr2=0x558cbec743d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/2854264169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7ba70 0x558cbec743d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30ed5c7640 1 -- 172.21.15.55:0/2854264169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cbec7f270 con 0x558cbec7b0b0 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/2854264169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec940e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f30d8098d00 tx=0x7f30d800fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7dfe197f75f38e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30d37fe640 1 -- 172.21.15.55:0/2854264169 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80a5510 con 0x558cbec7b0b0 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30d37fe640 1 -- 172.21.15.55:0/2854264169 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30d800de00 con 0x558cbec7b0b0 2024-04-26T22:06:37.857 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.852+0000 7f30d37fe640 1 -- 172.21.15.55:0/2854264169 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80a5080 con 0x558cbec7b0b0 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 -- 172.21.15.55:0/2854264169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 msgr2=0x558cbec940e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 --2- 172.21.15.55:0/2854264169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec940e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f30d8098d00 tx=0x7f30d800fe60 comp rx=0 tx=0).stop 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 -- 172.21.15.55:0/2854264169 shutdown_connections 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 --2- 172.21.15.55:0/2854264169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7ba70 0x558cbec743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 --2- 172.21.15.55:0/2854264169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec7b0b0 0x558cbec940e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 --2- 172.21.15.55:0/2854264169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec79ee0 0x558cbec70240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 -- 172.21.15.55:0/2854264169 >> 172.21.15.55:0/2854264169 conn(0x558cbec753d0 msgr2=0x558cbec73cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 -- 172.21.15.55:0/2854264169 shutdown_connections 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 -- 172.21.15.55:0/2854264169 wait complete. 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 -- start start 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 Processor -- start 2024-04-26T22:06:37.858 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.853+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec91dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec92310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30f2e20740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec93320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30f2e20740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f30e81a6370 con 0x558cbec79ee0 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30f2e20740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f30e81a77e0 con 0x558cbec7b0b0 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30f2e20740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f30e81a6a40 con 0x558cbec7ba70 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30f2e20740 10 client.0 _ll_register_callbacks cb 0x7f30d8098b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec91dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30ee5c9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec93320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30ed5c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec92310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec91dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60460/0 (socket says 172.21.15.55:60460) 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30ee5c9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec93320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:48986/0 (socket says 172.21.15.55:48986) 2024-04-26T22:06:37.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30ed5c7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec92310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43420/0 (socket says 172.21.15.55:43420) 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 -- 172.21.15.55:0/1579098900 learned_addr learned my addr 172.21.15.55:0/1579098900 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 -- 172.21.15.55:0/1579098900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 msgr2=0x558cbec93320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec93320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 -- 172.21.15.55:0/1579098900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 msgr2=0x558cbec92310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec92310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 -- 172.21.15.55:0/1579098900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cbe8b00f0 con 0x558cbec79ee0 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30eddc8640 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec91dd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f30e400b7b0 tx=0x7f30e401dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e15014df1bdab0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30d17fa640 1 -- 172.21.15.55:0/1579098900 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4038060 con 0x558cbec79ee0 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30d17fa640 1 -- 172.21.15.55:0/1579098900 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f30e401d680 con 0x558cbec79ee0 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.854+0000 7f30f2e20740 1 -- 172.21.15.55:0/1579098900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d800de00 con 0x558cbec79ee0 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30d17fa640 1 -- 172.21.15.55:0/1579098900 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f30e401d880 con 0x558cbec79ee0 2024-04-26T22:06:37.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30f2e20740 10 client.17577 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30f2e20740 1 -- 172.21.15.55:0/1579098900 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558cbeb9f4b0 con 0x558cbec79ee0 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30d17fa640 1 -- 172.21.15.55:0/1579098900 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30d800de00 con 0x558cbec79ee0 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30d17fa640 1 -- 172.21.15.55:0/1579098900 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f30e4012070 con 0x558cbec79ee0 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30f2e20740 10 client.17577 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30f2e20740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30f2e20740 20 client.17577 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30f2e20740 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558cbecff6a0 0x558cbed01ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30f2e20740 4 client.17577 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30f2e20740 1 -- 172.21.15.55:0/1579098900 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558cbebe6800 con 0x558cbecff6a0 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558cbecff6a0 0x558cbed01ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30ed5c7640 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558cbecff6a0 0x558cbed01ac0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eeedc482bc73507 server_cookie=fef7b325f44a12a7 in_seq=0 out_seq=0 2024-04-26T22:06:37.861 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.855+0000 7f30d17fa640 10 client.17577 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:37.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30d17fa640 1 -- 172.21.15.55:0/1579098900 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558cbebe6800 con 0x558cbecff6a0 2024-04-26T22:06:37.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30d17fa640 10 client.17577 handle_command_reply: tid=0 2024-04-26T22:06:37.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 client.17577 shutdown 2024-04-26T22:06:37.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 -- 172.21.15.55:0/1579098900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 msgr2=0x558cbec91dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec91dd0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f30e400b7b0 tx=0x7f30e401dcb0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 -- 172.21.15.55:0/1579098900 shutdown_connections 2024-04-26T22:06:37.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558cbec7ba70 0x558cbec93320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.863 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558cbec7b0b0 0x558cbec92310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x558cbecff6a0 0x558cbed01ac0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 --2- 172.21.15.55:0/1579098900 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558cbec79ee0 0x558cbec91dd0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30d17fa640 0 client.17577 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 -- 172.21.15.55:0/1579098900 >> 172.21.15.55:0/1579098900 conn(0x558cbec753d0 msgr2=0x558cbec78b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 -- 172.21.15.55:0/1579098900 shutdown_connections 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 1 -- 172.21.15.55:0/1579098900 wait complete. 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.858+0000 7f30f2e20740 20 client.17577 trim_cache size 0 max 16384 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:06:37.864 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:06:37.865 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:06:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 -- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30d40a3cc0 msgr2=0x7f30d40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30d40a3cc0 0x7f30d40a6180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009670 tx=0x7f30e4006bd0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 -- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e8101b40 msgr2=0x7f30e819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e8101b40 0x7f30e819b2a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f30d800c920 tx=0x7f30d800cde0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 -- 172.21.15.55:0/3954444039 shutdown_connections 2024-04-26T22:06:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f30d40a3cc0 0x7f30d40a6180 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f30e81024e0 0x7f30e81a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30e8101b40 0x7f30e819b2a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 --2- 172.21.15.55:0/3954444039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e81011e0 0x7f30e819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 -- 172.21.15.55:0/3954444039 >> 172.21.15.55:0/3954444039 conn(0x7f30e80fcde0 msgr2=0x7f30e80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 -- 172.21.15.55:0/3954444039 shutdown_connections 2024-04-26T22:06:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.859+0000 7f30f002c640 1 -- 172.21.15.55:0/3954444039 wait complete. 2024-04-26T22:06:37.869 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:37 smithi055 ceph-mon[29221]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 56 MiB/s wr, 457 op/s 2024-04-26T22:06:37.870 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:06:37.870 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:06:37.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 -- 172.21.15.55:0/3841459898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90103b90 msgr2=0x7fdf90104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 --2- 172.21.15.55:0/3841459898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90103b90 0x7fdf90104010 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009900 tx=0x7fdf8402a410 comp rx=0 tx=0).stop 2024-04-26T22:06:37.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 -- 172.21.15.55:0/3841459898 shutdown_connections 2024-04-26T22:06:37.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 --2- 172.21.15.55:0/3841459898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf90104550 0x7fdf9010ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 --2- 172.21.15.55:0/3841459898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90103b90 0x7fdf90104010 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 --2- 172.21.15.55:0/3841459898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf901029a0 0x7fdf90102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 -- 172.21.15.55:0/3841459898 >> 172.21.15.55:0/3841459898 conn(0x7fdf900fdf10 msgr2=0x7fdf90100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:37.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 -- 172.21.15.55:0/3841459898 shutdown_connections 2024-04-26T22:06:37.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.973+0000 7fdf95047640 1 -- 172.21.15.55:0/3841459898 wait complete. 2024-04-26T22:06:37.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.974+0000 7fdf95047640 1 Processor -- start 2024-04-26T22:06:37.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.974+0000 7fdf95047640 1 -- start start 2024-04-26T22:06:37.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf95047640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf901029a0 0x7fdf90198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf95047640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf90103b90 0x7fdf90199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf95047640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90104550 0x7fdf901a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf95047640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9010e4d0 con 0x7fdf90104550 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf95047640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9010e350 con 0x7fdf90103b90 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf95047640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9010e650 con 0x7fdf901029a0 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90104550 0x7fdf901a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf90103b90 0x7fdf90199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90104550 0x7fdf901a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60464/0 (socket says 172.21.15.55:60464) 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf90103b90 0x7fdf90199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43438/0 (socket says 172.21.15.55:43438) 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 -- 172.21.15.55:0/17472623 learned_addr learned my addr 172.21.15.55:0/17472623 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8ed9d640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf901029a0 0x7fdf90198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 -- 172.21.15.55:0/17472623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf901029a0 msgr2=0x7fdf90198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.978 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf901029a0 0x7fdf90198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 -- 172.21.15.55:0/17472623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf90103b90 msgr2=0x7fdf90199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf90103b90 0x7fdf90199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 -- 172.21.15.55:0/17472623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf901a0800 con 0x7fdf90104550 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8e59c640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf90103b90 0x7fdf90199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf8f59e640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90104550 0x7fdf901a0190 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000eb10 tx=0x7fdf8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.975+0000 7fdf77fff640 1 -- 172.21.15.55:0/17472623 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80004b20 con 0x7fdf90104550 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.976+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf901a0af0 con 0x7fdf90104550 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.976+0000 7fdf77fff640 1 -- 172.21.15.55:0/17472623 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf80004cc0 con 0x7fdf90104550 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.976+0000 7fdf77fff640 1 -- 172.21.15.55:0/17472623 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80010690 con 0x7fdf90104550 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.976+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf901a1060 con 0x7fdf90104550 2024-04-26T22:06:37.979 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.976+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf90109db0 con 0x7fdf90104550 2024-04-26T22:06:37.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.978+0000 7fdf77fff640 1 -- 172.21.15.55:0/17472623 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdf80005000 con 0x7fdf90104550 2024-04-26T22:06:37.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.978+0000 7fdf77fff640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdf780a3c40 0x7fdf780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:37.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.978+0000 7fdf77fff640 1 -- 172.21.15.55:0/17472623 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fdf800d0d20 con 0x7fdf90104550 2024-04-26T22:06:37.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.979+0000 7fdf8ed9d640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdf780a3c40 0x7fdf780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:37.980 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.979+0000 7fdf8ed9d640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdf780a3c40 0x7fdf780a6100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf901039f0 tx=0x7fdf88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:37.982 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:37.982+0000 7fdf77fff640 1 -- 172.21.15.55:0/17472623 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdf80054060 con 0x7fdf90104550 2024-04-26T22:06:38.213 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.213+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf9002dad0 con 0x7fdf90104550 2024-04-26T22:06:38.214 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.214+0000 7fdf77fff640 1 -- 172.21.15.55:0/17472623 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fdf8009d560 con 0x7fdf90104550 2024-04-26T22:06:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:06:38.214 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T21:58:11.153423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24502,24509,24551]},"id":1}]} 2024-04-26T22:06:38.217 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 23 2024-04-26T22:06:38.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdf780a3c40 msgr2=0x7fdf780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.218 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdf780a3c40 0x7fdf780a6100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf901039f0 tx=0x7fdf88005f90 comp rx=0 tx=0).stop 2024-04-26T22:06:38.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90104550 msgr2=0x7fdf901a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90104550 0x7fdf901a0190 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000eb10 tx=0x7fdf8000c2d0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 shutdown_connections 2024-04-26T22:06:38.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fdf780a3c40 0x7fdf780a6100 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdf90104550 0x7fdf901a0190 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdf90103b90 0x7fdf90199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.219 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 --2- 172.21.15.55:0/17472623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf901029a0 0x7fdf90198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 >> 172.21.15.55:0/17472623 conn(0x7fdf900fdf10 msgr2=0x7fdf90106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 shutdown_connections 2024-04-26T22:06:38.220 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.216+0000 7fdf95047640 1 -- 172.21.15.55:0/17472623 wait complete. 2024-04-26T22:06:38.228 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:06:38.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.335+0000 7f4253dda640 1 -- 172.21.15.55:0/3324421442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 msgr2=0x7f424c102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.335+0000 7f4253dda640 1 --2- 172.21.15.55:0/3324421442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 0x7f424c102ee0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f4240009900 tx=0x7f424002a410 comp rx=0 tx=0).stop 2024-04-26T22:06:38.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 -- 172.21.15.55:0/3324421442 shutdown_connections 2024-04-26T22:06:38.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 --2- 172.21.15.55:0/3324421442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424c103420 0x7f424c109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 --2- 172.21.15.55:0/3324421442 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 0x7f424c102ee0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.338 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 --2- 172.21.15.55:0/3324421442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f424c101870 0x7f424c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 -- 172.21.15.55:0/3324421442 >> 172.21.15.55:0/3324421442 conn(0x7f424c0fcde0 msgr2=0x7f424c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 -- 172.21.15.55:0/3324421442 shutdown_connections 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 -- 172.21.15.55:0/3324421442 wait complete. 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 Processor -- start 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 -- start start 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f424c101870 0x7f424c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 0x7f424c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424c103420 0x7f424c1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f424c10d3a0 con 0x7f424c102a60 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f424c10d220 con 0x7f424c103420 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.336+0000 7f4253dda640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f424c10d520 con 0x7f424c101870 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 0x7f424c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 0x7f424c199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60488/0 (socket says 172.21.15.55:60488) 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 -- 172.21.15.55:0/2913293004 learned_addr learned my addr 172.21.15.55:0/2913293004 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4252377640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424c103420 0x7f424c1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251b76640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f424c101870 0x7f424c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 -- 172.21.15.55:0/2913293004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f424c101870 msgr2=0x7f424c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f424c101870 0x7f424c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 -- 172.21.15.55:0/2913293004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424c103420 msgr2=0x7f424c1a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424c103420 0x7f424c1a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 -- 172.21.15.55:0/2913293004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c1a0850 con 0x7f424c102a60 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251b76640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f424c101870 0x7f424c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4251375640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 0x7f424c199180 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f4240005940 tx=0x7f4240031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f424affd640 1 -- 172.21.15.55:0/2913293004 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424003f030 con 0x7f424c102a60 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f424affd640 1 -- 172.21.15.55:0/2913293004 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4240003aa0 con 0x7f424c102a60 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4253dda640 1 -- 172.21.15.55:0/2913293004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424c1a0ae0 con 0x7f424c102a60 2024-04-26T22:06:38.340 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f424affd640 1 -- 172.21.15.55:0/2913293004 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240035070 con 0x7f424c102a60 2024-04-26T22:06:38.341 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.337+0000 7f4253dda640 1 -- 172.21.15.55:0/2913293004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424c1a0fd0 con 0x7f424c102a60 2024-04-26T22:06:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.339+0000 7f424affd640 1 -- 172.21.15.55:0/2913293004 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4240042030 con 0x7f424c102a60 2024-04-26T22:06:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f4256bc8740 1 Processor -- start 2024-04-26T22:06:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f4256bc8740 1 -- start start 2024-04-26T22:06:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f424affd640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f42340a3c40 0x7f42340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f63038c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63039e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 0x561f63040c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f4256bc8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561f62f9cd80 con 0x561f630399f0 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f4256bc8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561f62f56a20 con 0x561f63038820 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.340+0000 7f4256bc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561f62f98d40 con 0x561f6303a3b0 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4252377640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f42340a3c40 0x7f42340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f424affd640 1 -- 172.21.15.55:0/2913293004 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f42400f0d10 con 0x7f424c102a60 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63039e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251b76640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f63038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63039e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60500/0 (socket says 172.21.15.55:60500) 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251b76640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f63038c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43468/0 (socket says 172.21.15.55:43468) 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 -- 172.21.15.55:0/2313022940 learned_addr learned my addr 172.21.15.55:0/2313022940 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4252377640 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 0x561f63040c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 -- 172.21.15.55:0/2313022940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 msgr2=0x561f63040c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 0x561f63040c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 -- 172.21.15.55:0/2313022940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 msgr2=0x561f63038c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f63038c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 -- 172.21.15.55:0/2313022940 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f62f61c30 con 0x561f630399f0 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251b76640 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f63038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4252377640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f42340a3c40 0x7f42340a6100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4244009fa0 tx=0x7f4244013830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4251375640 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63039e70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f42400b9730 tx=0x7f42400bcc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7ceead7e42973c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4233fff640 1 -- 172.21.15.55:0/2313022940 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240049030 con 0x561f630399f0 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4233fff640 1 -- 172.21.15.55:0/2313022940 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42400c0070 con 0x561f630399f0 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.341+0000 7f4233fff640 1 -- 172.21.15.55:0/2313022940 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400bcdf0 con 0x561f630399f0 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 -- 172.21.15.55:0/2313022940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 msgr2=0x561f63039e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63039e70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f42400b9730 tx=0x7f42400bcc80 comp rx=0 tx=0).stop 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 -- 172.21.15.55:0/2313022940 shutdown_connections 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 0x561f63040c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63039e70 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.349 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 --2- 172.21.15.55:0/2313022940 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f63038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 -- 172.21.15.55:0/2313022940 >> 172.21.15.55:0/2313022940 conn(0x561f62f79910 msgr2=0x561f63034120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 -- 172.21.15.55:0/2313022940 shutdown_connections 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 -- 172.21.15.55:0/2313022940 wait complete. 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 -- start start 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.342+0000 7f4256bc8740 1 Processor -- start 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f63038820 0x561f630427b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63042cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f6303a3b0 0x561f63049d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4256bc8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x561f63041180 con 0x561f630399f0 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4256bc8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561f62f23dc0 con 0x561f6303a3b0 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4256bc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561f63041300 con 0x561f63038820 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4256bc8740 10 client.0 _ll_register_callbacks cb 0x561f62f3f600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63042cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63042cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60502/0 (socket says 172.21.15.55:60502) 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 -- 172.21.15.55:0/1098533373 learned_addr learned my addr 172.21.15.55:0/1098533373 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4252377640 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f6303a3b0 0x561f63049d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251b76640 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f63038820 0x561f630427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 -- 172.21.15.55:0/1098533373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f63038820 msgr2=0x561f630427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f63038820 0x561f630427b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 -- 172.21.15.55:0/1098533373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f6303a3b0 msgr2=0x561f63049d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f6303a3b0 0x561f63049d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 -- 172.21.15.55:0/1098533373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f62cae3f0 con 0x561f630399f0 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4252377640 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f6303a3b0 0x561f63049d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.343+0000 7f4251375640 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63042cf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f42400b9170 tx=0x7f42400bccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bc2953bf8a5183 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f42317fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f42317fa640 1 -- 172.21.15.55:0/1098533373 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240049040 con 0x561f630399f0 2024-04-26T22:06:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 1 -- 172.21.15.55:0/1098533373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42400c0070 con 0x561f630399f0 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f42317fa640 1 -- 172.21.15.55:0/1098533373 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f42400309c0 con 0x561f630399f0 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f42317fa640 1 -- 172.21.15.55:0/1098533373 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f424004c760 con 0x561f630399f0 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 10 client.17595 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 1 -- 172.21.15.55:0/1098533373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561f62d787d0 con 0x561f630399f0 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f42317fa640 1 -- 172.21.15.55:0/1098533373 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42400c0070 con 0x561f630399f0 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f42317fa640 1 -- 172.21.15.55:0/1098533373 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f42400a7040 con 0x561f630399f0 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 10 client.17595 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 10 client.17595 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 20 client.17595 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561f6304e440 0x561f63050860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 4 client.17595 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.344+0000 7f4256bc8740 1 -- 172.21.15.55:0/1098533373 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f62faa990 con 0x561f6304e440 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.345+0000 7f4251b76640 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561f6304e440 0x561f63050860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.345+0000 7f4251b76640 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561f6304e440 0x561f63050860 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=535418f19f142661 server_cookie=3bf3b8af0bf5b07 in_seq=0 out_seq=0 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.345+0000 7f42317fa640 10 client.17595 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f42317fa640 1 -- 172.21.15.55:0/1098533373 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f62faa990 con 0x561f6304e440 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f42317fa640 10 client.17595 handle_command_reply: tid=0 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 client.17595 shutdown 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 -- 172.21.15.55:0/1098533373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 msgr2=0x561f63042cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63042cf0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f42400b9170 tx=0x7f42400bccf0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 -- 172.21.15.55:0/1098533373 shutdown_connections 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561f6304e440 0x561f63050860 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f6303a3b0 0x561f63049d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f630399f0 0x561f63042cf0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f42317fa640 0 client.17595 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1098533373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f63038820 0x561f630427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 -- 172.21.15.55:0/1098533373 >> 172.21.15.55:0/1098533373 conn(0x561f62f79910 msgr2=0x561f63033280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 -- 172.21.15.55:0/1098533373 shutdown_connections 2024-04-26T22:06:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 1 -- 172.21.15.55:0/1098533373 wait complete. 2024-04-26T22:06:38.354 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.346+0000 7f4256bc8740 20 client.17595 trim_cache size 0 max 16384 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4256bc8740 1 Processor -- start 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4256bc8740 1 -- start start 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f6302f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f630399f0 0x561f63049cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f6303a3b0 0x561f63036fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4256bc8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1ac740 con 0x561f6303a3b0 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4256bc8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1aa010 con 0x561f63038820 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4256bc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1a7e60 con 0x561f630399f0 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4252377640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f6303a3b0 0x561f63036fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.366 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4252377640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f6303a3b0 0x561f63036fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60512/0 (socket says 172.21.15.55:60512) 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4252377640 1 -- 172.21.15.55:0/1056682848 learned_addr learned my addr 172.21.15.55:0/1056682848 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4251375640 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f630399f0 0x561f63049cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4251b76640 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f6302f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4252377640 1 -- 172.21.15.55:0/1056682848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f630399f0 msgr2=0x561f63049cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4252377640 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f630399f0 0x561f63049cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4252377640 1 -- 172.21.15.55:0/1056682848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 msgr2=0x561f6302f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4252377640 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f6302f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.360+0000 7f4252377640 1 -- 172.21.15.55:0/1056682848 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6302f8b0 con 0x561f6303a3b0 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4252377640 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f6303a3b0 0x561f63036fe0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4244004470 tx=0x7f4244005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f000bf91c242de86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f42337fe640 1 -- 172.21.15.55:0/1056682848 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244003cf0 con 0x561f6303a3b0 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f42337fe640 1 -- 172.21.15.55:0/1056682848 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f424400f990 con 0x561f6303a3b0 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f42337fe640 1 -- 172.21.15.55:0/1056682848 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42440108a0 con 0x561f6303a3b0 2024-04-26T22:06:38.367 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 -- 172.21.15.55:0/1056682848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f6303a3b0 msgr2=0x561f63036fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f6303a3b0 0x561f63036fe0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4244004470 tx=0x7f4244005f90 comp rx=0 tx=0).stop 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 -- 172.21.15.55:0/1056682848 shutdown_connections 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f6303a3b0 0x561f63036fe0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f630399f0 0x561f63049cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 --2- 172.21.15.55:0/1056682848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f63038820 0x561f6302f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 -- 172.21.15.55:0/1056682848 >> 172.21.15.55:0/1056682848 conn(0x561f62f79910 msgr2=0x561f6303d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 -- 172.21.15.55:0/1056682848 shutdown_connections 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.361+0000 7f4256bc8740 1 -- 172.21.15.55:0/1056682848 wait complete. 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 1 -- start start 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 1 Processor -- start 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f63038820 0x561f63047330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f630399f0 0x561f63047870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 0x561f63048880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.368 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1a84d0 con 0x561f63038820 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1a9940 con 0x561f630399f0 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1a8ba0 con 0x561f6303a3b0 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4256bc8740 10 client.0 _ll_register_callbacks cb 0x7f423c009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4251b76640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f63038820 0x561f63047330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4251375640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f630399f0 0x561f63047870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4251b76640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f63038820 0x561f63047330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60516/0 (socket says 172.21.15.55:60516) 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4251b76640 1 -- 172.21.15.55:0/617129656 learned_addr learned my addr 172.21.15.55:0/617129656 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.362+0000 7f4252377640 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 0x561f63048880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4251375640 1 -- 172.21.15.55:0/617129656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 msgr2=0x561f63048880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4251375640 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 0x561f63048880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4251375640 1 -- 172.21.15.55:0/617129656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f63038820 msgr2=0x561f63047330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4251375640 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f63038820 0x561f63047330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.369 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4251375640 1 -- 172.21.15.55:0/617129656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f63048e70 con 0x561f630399f0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4251b76640 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f63038820 0x561f63047330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4251375640 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f630399f0 0x561f63047870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f42400bcaf0 tx=0x7f42400bcc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114771ae8dd37b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f42317fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f42317fa640 1 -- 172.21.15.55:0/617129656 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240049050 con 0x561f630399f0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4256bc8740 1 -- 172.21.15.55:0/617129656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f424400f990 con 0x561f630399f0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f42317fa640 1 -- 172.21.15.55:0/617129656 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f42400ee870 con 0x561f630399f0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f42317fa640 1 -- 172.21.15.55:0/617129656 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f42400bd4f0 con 0x561f630399f0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4256bc8740 10 client.26504 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f4256bc8740 1 -- 172.21.15.55:0/617129656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561f63048e70 con 0x561f630399f0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.363+0000 7f42317fa640 1 -- 172.21.15.55:0/617129656 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f424400f990 con 0x561f630399f0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f42317fa640 1 -- 172.21.15.55:0/617129656 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f42400a7040 con 0x561f630399f0 2024-04-26T22:06:38.370 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4256bc8740 10 client.26504 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4256bc8740 10 client.26504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4256bc8740 20 client.26504 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4256bc8740 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561f630b4e90 0x561f630b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4256bc8740 4 client.26504 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4256bc8740 1 -- 172.21.15.55:0/617129656 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f62fa4800 con 0x561f630b4e90 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4251b76640 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561f630b4e90 0x561f630b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f42317fa640 10 client.26504 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4251b76640 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561f630b4e90 0x561f630b72b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89214be82d7a0144 server_cookie=5c4ca9eee91fd15c in_seq=0 out_seq=0 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f42317fa640 1 -- 172.21.15.55:0/617129656 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f62fa4800 con 0x561f630b4e90 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f42317fa640 10 client.26504 handle_command_reply: tid=0 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.364+0000 7f4256bc8740 1 client.26504 shutdown 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 -- 172.21.15.55:0/617129656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f630399f0 msgr2=0x561f63047870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.371 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f630399f0 0x561f63047870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f42400bcaf0 tx=0x7f42400bcc80 comp rx=0 tx=0).stop 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 -- 172.21.15.55:0/617129656 shutdown_connections 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x561f630b4e90 0x561f630b72b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561f6303a3b0 0x561f63048880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561f630399f0 0x561f63047870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 --2- 172.21.15.55:0/617129656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x561f63038820 0x561f63047330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f42317fa640 0 client.26504 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 -- 172.21.15.55:0/617129656 >> 172.21.15.55:0/617129656 conn(0x561f62f79910 msgr2=0x561f630329f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 -- 172.21.15.55:0/617129656 shutdown_connections 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 1 -- 172.21.15.55:0/617129656 wait complete. 2024-04-26T22:06:38.372 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4256bc8740 20 client.26504 trim_cache size 0 max 16384 2024-04-26T22:06:38.373 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 -- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f42340a3c40 msgr2=0x7f42340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f42340a3c40 0x7f42340a6100 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4244009fa0 tx=0x7f4244013830 comp rx=0 tx=0).stop 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 -- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 msgr2=0x7f424c199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 0x7f424c199180 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f4240005940 tx=0x7f4240031a50 comp rx=0 tx=0).stop 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 -- 172.21.15.55:0/2913293004 shutdown_connections 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f42340a3c40 0x7f42340a6100 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f424c103420 0x7f424c1a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f424c102a60 0x7f424c199180 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 --2- 172.21.15.55:0/2913293004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f424c101870 0x7f424c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 -- 172.21.15.55:0/2913293004 >> 172.21.15.55:0/2913293004 conn(0x7f424c0fcde0 msgr2=0x7f424c069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.374 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 -- 172.21.15.55:0/2913293004 shutdown_connections 2024-04-26T22:06:38.375 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.365+0000 7f4253dda640 1 -- 172.21.15.55:0/2913293004 wait complete. 2024-04-26T22:06:38.377 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:06:38.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.481+0000 7f8315a74640 1 -- 172.21.15.55:0/2408221495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 msgr2=0x7f8310101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.481+0000 7f8315a74640 1 --2- 172.21.15.55:0/2408221495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 0x7f8310101c90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8300009900 tx=0x7f830002a410 comp rx=0 tx=0).stop 2024-04-26T22:06:38.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.481+0000 7f8315a74640 1 -- 172.21.15.55:0/2408221495 shutdown_connections 2024-04-26T22:06:38.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.481+0000 7f8315a74640 1 --2- 172.21.15.55:0/2408221495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8310103420 0x7f8310109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.481+0000 7f8315a74640 1 --2- 172.21.15.55:0/2408221495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310102a60 0x7f8310102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.481+0000 7f8315a74640 1 --2- 172.21.15.55:0/2408221495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 0x7f8310101c90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.493 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.481+0000 7f8315a74640 1 -- 172.21.15.55:0/2408221495 >> 172.21.15.55:0/2408221495 conn(0x7f83100fcde0 msgr2=0x7f83100ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.481+0000 7f8315a74640 1 -- 172.21.15.55:0/2408221495 shutdown_connections 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 -- 172.21.15.55:0/2408221495 wait complete. 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 Processor -- start 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 -- start start 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 0x7f8310198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8310102a60 0x7f8310199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310103420 0x7f83101a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f831010e4c0 con 0x7f8310101870 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f831010e340 con 0x7f8310102a60 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f8315a74640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f831010e640 con 0x7f8310103420 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.482+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 0x7f8310198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 0x7f8310198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60530/0 (socket says 172.21.15.55:60530) 2024-04-26T22:06:38.494 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830f7fe640 1 -- 172.21.15.55:0/719022756 learned_addr learned my addr 172.21.15.55:0/719022756 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830effd640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8310102a60 0x7f8310199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830ffff640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310103420 0x7f83101a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830f7fe640 1 -- 172.21.15.55:0/719022756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310103420 msgr2=0x7f83101a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830f7fe640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310103420 0x7f83101a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830f7fe640 1 -- 172.21.15.55:0/719022756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8310102a60 msgr2=0x7f8310199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830f7fe640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8310102a60 0x7f8310199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830f7fe640 1 -- 172.21.15.55:0/719022756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83101a0860 con 0x7f8310101870 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830effd640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8310102a60 0x7f8310199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830ffff640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310103420 0x7f83101a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830f7fe640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 0x7f8310198c50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8300006010 tx=0x7f83000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830cff9640 1 -- 172.21.15.55:0/719022756 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830003f030 con 0x7f8310101870 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830cff9640 1 -- 172.21.15.55:0/719022756 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000318f0 con 0x7f8310101870 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f8315a74640 1 -- 172.21.15.55:0/719022756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83101a0af0 con 0x7f8310101870 2024-04-26T22:06:38.495 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f830cff9640 1 -- 172.21.15.55:0/719022756 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300035070 con 0x7f8310101870 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.483+0000 7f8315a74640 1 -- 172.21.15.55:0/719022756 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83101a0fe0 con 0x7f8310101870 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.486+0000 7f830cff9640 1 -- 172.21.15.55:0/719022756 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8300042030 con 0x7f8310101870 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.486+0000 7f831878a740 1 Processor -- start 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.486+0000 7f831878a740 1 -- start start 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f831878a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df086e820 0x563df086ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f831878a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df086fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f831878a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df08703b0 0x563df0876c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f831878a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563df07d2d80 con 0x563df086f9f0 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f831878a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563df078ca20 con 0x563df086e820 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f831878a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563df07ced40 con 0x563df08703b0 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df086fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.496 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df086e820 0x563df086ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df086e820 0x563df086ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43522/0 (socket says 172.21.15.55:43522) 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 -- 172.21.15.55:0/2488003584 learned_addr learned my addr 172.21.15.55:0/2488003584 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830cff9640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82f80a3c40 0x7f82f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830effd640 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df08703b0 0x563df0876c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82f80a3c40 0x7f82f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 -- 172.21.15.55:0/2488003584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df08703b0 msgr2=0x563df0876c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df08703b0 0x563df0876c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 -- 172.21.15.55:0/2488003584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 msgr2=0x563df086fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830cff9640 1 -- 172.21.15.55:0/719022756 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f83000f0c50 con 0x7f8310101870 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df086fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 -- 172.21.15.55:0/2488003584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df0797c30 con 0x563df086e820 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830f7fe640 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df086fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.487+0000 7f830ffff640 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df086e820 0x563df086ec40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8308005c40 tx=0x7f830800e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66718769b51dd252 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.497 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f830ffff640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82f80a3c40 0x7f82f80a6100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f830801c000 tx=0x7f8308020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f82f5ffb640 1 -- 172.21.15.55:0/2488003584 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830801fc90 con 0x563df086e820 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f82f5ffb640 1 -- 172.21.15.55:0/2488003584 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8308023070 con 0x563df086e820 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f82f5ffb640 1 -- 172.21.15.55:0/2488003584 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308010c00 con 0x563df086e820 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 -- 172.21.15.55:0/2488003584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df086e820 msgr2=0x563df086ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df086e820 0x563df086ec40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8308005c40 tx=0x7f830800e350 comp rx=0 tx=0).stop 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 -- 172.21.15.55:0/2488003584 shutdown_connections 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df08703b0 0x563df0876c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df086fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 --2- 172.21.15.55:0/2488003584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df086e820 0x563df086ec40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 -- 172.21.15.55:0/2488003584 >> 172.21.15.55:0/2488003584 conn(0x563df07af910 msgr2=0x563df086a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 -- 172.21.15.55:0/2488003584 shutdown_connections 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.488+0000 7f831878a740 1 -- 172.21.15.55:0/2488003584 wait complete. 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 1 -- start start 2024-04-26T22:06:38.498 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 1 Processor -- start 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df0878fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x563df0877360 con 0x563df086e820 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563df0759dc0 con 0x563df08703b0 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563df08774e0 con 0x563df086f9f0 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f831878a740 10 client.0 _ll_register_callbacks cb 0x563df0775600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60554/0 (socket says 172.21.15.55:60554) 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f830f7fe640 1 -- 172.21.15.55:0/905121768 learned_addr learned my addr 172.21.15.55:0/905121768 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f830ffff640 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.489+0000 7f830effd640 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df0878fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.499 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f830f7fe640 1 -- 172.21.15.55:0/905121768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 msgr2=0x563df0878fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f830f7fe640 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df0878fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f830f7fe640 1 -- 172.21.15.55:0/905121768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 msgr2=0x563df087fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f830f7fe640 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f830f7fe640 1 -- 172.21.15.55:0/905121768 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df04e43f0 con 0x563df086e820 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f830f7fe640 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878a70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f83000ee590 tx=0x7f830003b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708f5188e028684e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f82eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f82eeffd640 1 -- 172.21.15.55:0/905121768 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300049040 con 0x563df086e820 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f82eeffd640 1 -- 172.21.15.55:0/905121768 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f83000b89f0 con 0x563df086e820 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 1 -- 172.21.15.55:0/905121768 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8308023070 con 0x563df086e820 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f82eeffd640 1 -- 172.21.15.55:0/905121768 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f83000bc870 con 0x563df086e820 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 10 client.17613 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 1 -- 172.21.15.55:0/905121768 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563df05ae7d0 con 0x563df086e820 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f82eeffd640 1 -- 172.21.15.55:0/905121768 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8308023070 con 0x563df086e820 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f82eeffd640 1 -- 172.21.15.55:0/905121768 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f83000bcf90 con 0x563df086e820 2024-04-26T22:06:38.500 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 10 client.17613 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 10 client.17613 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 20 client.17613 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x563df0884870 0x563df0886cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 4 client.17613 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.490+0000 7f831878a740 1 -- 172.21.15.55:0/905121768 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563df07e0990 con 0x563df0884870 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.491+0000 7f830effd640 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x563df0884870 0x563df0886cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.491+0000 7f830effd640 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x563df0884870 0x563df0886cc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d92fb1acafb85247 server_cookie=a40dca907aa734bd in_seq=0 out_seq=0 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.491+0000 7f82eeffd640 10 client.17613 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f82eeffd640 1 -- 172.21.15.55:0/905121768 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563df07e0990 con 0x563df0884870 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f82eeffd640 10 client.17613 handle_command_reply: tid=0 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 client.17613 shutdown 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 -- 172.21.15.55:0/905121768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 msgr2=0x563df0878a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878a70 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f83000ee590 tx=0x7f830003b5d0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.501 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 -- 172.21.15.55:0/905121768 shutdown_connections 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x563df0884870 0x563df0886cc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f82eeffd640 0 client.17613 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df0878fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 --2- 172.21.15.55:0/905121768 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878a70 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 -- 172.21.15.55:0/905121768 >> 172.21.15.55:0/905121768 conn(0x563df07af910 msgr2=0x563df0869280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 -- 172.21.15.55:0/905121768 shutdown_connections 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 1 -- 172.21.15.55:0/905121768 wait complete. 2024-04-26T22:06:38.502 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.492+0000 7f831878a740 20 client.17613 trim_cache size 0 max 16384 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.505+0000 7f831878a740 1 Processor -- start 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.505+0000 7f831878a740 1 -- start start 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f831878a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f831878a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df087ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f831878a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df08689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f831878a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f83101ac710 con 0x563df086e820 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f831878a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f83101a9fe0 con 0x563df08703b0 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f831878a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f83101a7e30 con 0x563df086f9f0 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60558/0 (socket says 172.21.15.55:60558) 2024-04-26T22:06:38.508 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 -- 172.21.15.55:0/3868258616 learned_addr learned my addr 172.21.15.55:0/3868258616 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830effd640 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df087ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830ffff640 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df08689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 -- 172.21.15.55:0/3868258616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 msgr2=0x563df087ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df087ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 -- 172.21.15.55:0/3868258616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 msgr2=0x563df08689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df08689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 -- 172.21.15.55:0/3868258616 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563df0887310 con 0x563df086e820 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830effd640 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df087ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f830f7fe640 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878d90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f83000badb0 tx=0x7f8300004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f52e0ecb70f0fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f82f57fa640 1 -- 172.21.15.55:0/3868258616 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300049070 con 0x563df086e820 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f82f57fa640 1 -- 172.21.15.55:0/3868258616 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000f4020 con 0x563df086e820 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.506+0000 7f82f57fa640 1 -- 172.21.15.55:0/3868258616 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000bae70 con 0x563df086e820 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 -- 172.21.15.55:0/3868258616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 msgr2=0x563df0878d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878d90 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f83000badb0 tx=0x7f8300004610 comp rx=0 tx=0).stop 2024-04-26T22:06:38.509 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 -- 172.21.15.55:0/3868258616 shutdown_connections 2024-04-26T22:06:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df08689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086f9f0 0x563df087ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 --2- 172.21.15.55:0/3868258616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086e820 0x563df0878d90 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 -- 172.21.15.55:0/3868258616 >> 172.21.15.55:0/3868258616 conn(0x563df07af910 msgr2=0x563df0878680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 -- 172.21.15.55:0/3868258616 shutdown_connections 2024-04-26T22:06:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.507+0000 7f831878a740 1 -- 172.21.15.55:0/3868258616 wait complete. 2024-04-26T22:06:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 1 -- start start 2024-04-26T22:06:38.510 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 1 Processor -- start 2024-04-26T22:06:38.511 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086e820 0x563df087d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df087dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f83101a84a0 con 0x563df086f9f0 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f83101a9910 con 0x563df08703b0 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f83101a8b70 con 0x563df086e820 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f831878a740 10 client.0 _ll_register_callbacks cb 0x7f83000b7670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086e820 0x563df087d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f830effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df087dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f830f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086e820 0x563df087d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.55:49122/0 (socket says 172.21.15.55:49122) 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f830effd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df087dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60560/0 (socket says 172.21.15.55:60560) 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f830f7fe640 1 -- 172.21.15.55:0/3396678153 learned_addr learned my addr 172.21.15.55:0/3396678153 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.512 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.508+0000 7f830ffff640 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f830f7fe640 1 -- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 msgr2=0x563df087ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f830f7fe640 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f830f7fe640 1 -- 172.21.15.55:0/3396678153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 msgr2=0x563df087dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f830f7fe640 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df087dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f830f7fe640 1 -- 172.21.15.55:0/3396678153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563df087f470 con 0x563df086e820 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f830effd640 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df087dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f830ffff640 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f830f7fe640 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086e820 0x563df087d980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83000c5470 tx=0x7f83000bad40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67bbeb3fa5b53181 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.509+0000 7f82eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.510+0000 7f82eeffd640 1 -- 172.21.15.55:0/3396678153 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300049050 con 0x563df086e820 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.510+0000 7f82eeffd640 1 -- 172.21.15.55:0/3396678153 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f83000c5710 con 0x563df086e820 2024-04-26T22:06:38.513 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.510+0000 7f831878a740 1 -- 172.21.15.55:0/3396678153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83000f4020 con 0x563df086e820 2024-04-26T22:06:38.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.515+0000 7f82eeffd640 1 -- 172.21.15.55:0/3396678153 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f83000bc690 con 0x563df086e820 2024-04-26T22:06:38.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.515+0000 7f831878a740 10 client.26527 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:38.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.515+0000 7f831878a740 1 -- 172.21.15.55:0/3396678153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563df087f470 con 0x563df086e820 2024-04-26T22:06:38.516 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f82eeffd640 1 -- 172.21.15.55:0/3396678153 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000f4020 con 0x563df086e820 2024-04-26T22:06:38.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f82eeffd640 1 -- 172.21.15.55:0/3396678153 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f83000eeb70 con 0x563df086e820 2024-04-26T22:06:38.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f831878a740 10 client.26527 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:38.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f831878a740 10 client.26527 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:06:38.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f831878a740 20 client.26527 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:38.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f831878a740 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x563df08eb500 0x563df08ed8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f831878a740 4 client.26527 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:06:38.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f831878a740 1 -- 172.21.15.55:0/3396678153 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563df07da800 con 0x563df08eb500 2024-04-26T22:06:38.517 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.516+0000 7f830effd640 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x563df08eb500 0x563df08ed8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f82eeffd640 10 client.26527 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:06:38.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f830effd640 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x563df08eb500 0x563df08ed8f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f069cacca15f0d5a server_cookie=fe9bf10b656fe3e in_seq=0 out_seq=0 2024-04-26T22:06:38.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f82eeffd640 1 -- 172.21.15.55:0/3396678153 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563df07da800 con 0x563df08eb500 2024-04-26T22:06:38.518 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f82eeffd640 10 client.26527 handle_command_reply: tid=0 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 client.26527 shutdown 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 -- 172.21.15.55:0/3396678153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086e820 msgr2=0x563df087d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086e820 0x563df087d980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83000c5470 tx=0x7f83000bad40 comp rx=0 tx=0).stop 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 -- 172.21.15.55:0/3396678153 shutdown_connections 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x563df08eb500 0x563df08ed8f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563df08703b0 0x563df087ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x563df086f9f0 0x563df087dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 --2- 172.21.15.55:0/3396678153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563df086e820 0x563df087d980 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f82eeffd640 0 client.26527 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 -- 172.21.15.55:0/3396678153 >> 172.21.15.55:0/3396678153 conn(0x563df07af910 msgr2=0x563df086d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 -- 172.21.15.55:0/3396678153 shutdown_connections 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 1 -- 172.21.15.55:0/3396678153 wait complete. 2024-04-26T22:06:38.519 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.517+0000 7f831878a740 20 client.26527 trim_cache size 0 max 16384 2024-04-26T22:06:38.520 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 -- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82f80a3c40 msgr2=0x7f82f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82f80a3c40 0x7f82f80a6100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f830801c000 tx=0x7f8308020000 comp rx=0 tx=0).stop 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 -- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 msgr2=0x7f8310198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 0x7f8310198c50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8300006010 tx=0x7f83000042f0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 -- 172.21.15.55:0/719022756 shutdown_connections 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f82f80a3c40 0x7f82f80a6100 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8310103420 0x7f83101a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8310102a60 0x7f8310199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 --2- 172.21.15.55:0/719022756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8310101870 0x7f8310198c50 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 -- 172.21.15.55:0/719022756 >> 172.21.15.55:0/719022756 conn(0x7f83100fcde0 msgr2=0x7f8310105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.521 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 -- 172.21.15.55:0/719022756 shutdown_connections 2024-04-26T22:06:38.522 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.518+0000 7f8315a74640 1 -- 172.21.15.55:0/719022756 wait complete. 2024-04-26T22:06:38.529 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 -- 172.21.15.55:0/4263118750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950105d40 msgr2=0x7f2950108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 --2- 172.21.15.55:0/4263118750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950105d40 0x7f2950108130 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2938009900 tx=0x7f293802a410 comp rx=0 tx=0).stop 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 -- 172.21.15.55:0/4263118750 shutdown_connections 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 --2- 172.21.15.55:0/4263118750 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2950108670 0x7f295010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 --2- 172.21.15.55:0/4263118750 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950105d40 0x7f2950108130 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 --2- 172.21.15.55:0/4263118750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2950103410 0x7f2950105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 -- 172.21.15.55:0/4263118750 >> 172.21.15.55:0/4263118750 conn(0x7f29500fce20 msgr2=0x7f29500ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 -- 172.21.15.55:0/4263118750 shutdown_connections 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.635+0000 7f2954a85640 1 -- 172.21.15.55:0/4263118750 wait complete. 2024-04-26T22:06:38.638 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f2954a85640 1 Processor -- start 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f2954a85640 1 -- start start 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f2954a85640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950103410 0x7f2950198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f2954a85640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2950105d40 0x7f29501991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f2954a85640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2950108670 0x7f29501a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f2954a85640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f295010e4d0 con 0x7f2950103410 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f2954a85640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f295010e350 con 0x7f2950105d40 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f2954a85640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f295010e650 con 0x7f2950108670 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f294e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950103410 0x7f2950198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f294e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950103410 0x7f2950198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60590/0 (socket says 172.21.15.55:60590) 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.636+0000 7f294e59c640 1 -- 172.21.15.55:0/2872858033 learned_addr learned my addr 172.21.15.55:0/2872858033 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f2947fff640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2950105d40 0x7f29501991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f294ed9d640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2950108670 0x7f29501a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f294e59c640 1 -- 172.21.15.55:0/2872858033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2950108670 msgr2=0x7f29501a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f294e59c640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2950108670 0x7f29501a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.639 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f294e59c640 1 -- 172.21.15.55:0/2872858033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2950105d40 msgr2=0x7f29501991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f294e59c640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2950105d40 0x7f29501991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f294e59c640 1 -- 172.21.15.55:0/2872858033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29501a0850 con 0x7f2950103410 2024-04-26T22:06:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f294e59c640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950103410 0x7f2950198c90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f294000d770 tx=0x7f294000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f29477fe640 1 -- 172.21.15.55:0/2872858033 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940017070 con 0x7f2950103410 2024-04-26T22:06:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f29477fe640 1 -- 172.21.15.55:0/2872858033 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29400054d0 con 0x7f2950103410 2024-04-26T22:06:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f2954a85640 1 -- 172.21.15.55:0/2872858033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29501a0b40 con 0x7f2950103410 2024-04-26T22:06:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f29477fe640 1 -- 172.21.15.55:0/2872858033 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940004c30 con 0x7f2950103410 2024-04-26T22:06:38.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.637+0000 7f2954a85640 1 -- 172.21.15.55:0/2872858033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29501a1090 con 0x7f2950103410 2024-04-26T22:06:38.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.640+0000 7f29477fe640 1 -- 172.21.15.55:0/2872858033 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2940004e10 con 0x7f2950103410 2024-04-26T22:06:38.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.640+0000 7f295779f740 1 Processor -- start 2024-04-26T22:06:38.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f295779f740 1 -- start start 2024-04-26T22:06:38.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f295779f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f12c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f295779f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f13e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f295779f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f295779f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e612e76d80 con 0x55e612f139f0 2024-04-26T22:06:38.646 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f295779f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e612e30a20 con 0x55e612f12820 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f295779f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e612e72d40 con 0x55e612f143b0 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f13e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f12c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f13e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60592/0 (socket says 172.21.15.55:60592) 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 -- 172.21.15.55:0/427472635 learned_addr learned my addr 172.21.15.55:0/427472635 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f2947fff640 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 -- 172.21.15.55:0/427472635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 msgr2=0x55e612f1ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f1ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 -- 172.21.15.55:0/427472635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 msgr2=0x55e612f12c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f12c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 -- 172.21.15.55:0/427472635 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e612e3bc30 con 0x55e612f139f0 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f2947fff640 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f1ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294ed9d640 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f12c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f29477fe640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f293c0a3cc0 0x7f293c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.647 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294ed9d640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f293c0a3cc0 0x7f293c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.641+0000 7f294e59c640 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f13e70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f29400cd8a0 tx=0x7f29400d86f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d63c65a8d594db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f29477fe640 1 -- 172.21.15.55:0/2872858033 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f29400cfff0 con 0x7f2950103410 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f2944ff9640 1 -- 172.21.15.55:0/427472635 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940020030 con 0x55e612f139f0 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f2944ff9640 1 -- 172.21.15.55:0/427472635 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2940005680 con 0x55e612f139f0 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f2944ff9640 1 -- 172.21.15.55:0/427472635 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294009bfb0 con 0x55e612f139f0 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f295779f740 1 -- 172.21.15.55:0/427472635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 msgr2=0x55e612f13e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f294ed9d640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f293c0a3cc0 0x7f293c0a6180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55e612f138f0 tx=0x7f2948009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f295779f740 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f13e70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f29400cd8a0 tx=0x7f29400d86f0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f295779f740 1 -- 172.21.15.55:0/427472635 shutdown_connections 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f295779f740 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f1ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f295779f740 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f13e70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f295779f740 1 --2- 172.21.15.55:0/427472635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f12c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.642+0000 7f295779f740 1 -- 172.21.15.55:0/427472635 >> 172.21.15.55:0/427472635 conn(0x55e612e53910 msgr2=0x55e612f0e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 -- 172.21.15.55:0/427472635 shutdown_connections 2024-04-26T22:06:38.648 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 -- 172.21.15.55:0/427472635 wait complete. 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 -- start start 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 Processor -- start 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 0x55e612f1caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f1d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f24070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e612f1b3c0 con 0x55e612f139f0 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e612dfddc0 con 0x55e612f143b0 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e612f1b540 con 0x55e612f12820 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f295779f740 10 client.0 _ll_register_callbacks cb 0x55e612e19600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f2947fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f1d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f2947fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f1d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60606/0 (socket says 172.21.15.55:60606) 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f2947fff640 1 -- 172.21.15.55:0/3264734199 learned_addr learned my addr 172.21.15.55:0/3264734199 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.643+0000 7f294ed9d640 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f24070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f294e59c640 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 0x55e612f1caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.649 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2947fff640 1 -- 172.21.15.55:0/3264734199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 msgr2=0x55e612f1caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2947fff640 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 0x55e612f1caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2947fff640 1 -- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 msgr2=0x55e612f24070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2947fff640 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f24070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2947fff640 1 -- 172.21.15.55:0/3264734199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e612b883f0 con 0x55e612f139f0 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f294ed9d640 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f24070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2947fff640 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f1d030 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2938009900 tx=0x7f2938031e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630b7d3d026a1879 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2935ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2935ffb640 1 -- 172.21.15.55:0/3264734199 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938035070 con 0x55e612f139f0 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2935ffb640 1 -- 172.21.15.55:0/3264734199 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f2938004240 con 0x55e612f139f0 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f295779f740 1 -- 172.21.15.55:0/3264734199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2940005680 con 0x55e612f139f0 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2935ffb640 1 -- 172.21.15.55:0/3264734199 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f29380318f0 con 0x55e612f139f0 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f295779f740 10 client.17631 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f295779f740 1 -- 172.21.15.55:0/3264734199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e612c527d0 con 0x55e612f139f0 2024-04-26T22:06:38.650 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.644+0000 7f2935ffb640 1 -- 172.21.15.55:0/3264734199 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2940005680 con 0x55e612f139f0 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f2935ffb640 1 -- 172.21.15.55:0/3264734199 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f293803b4c0 con 0x55e612f139f0 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f295779f740 10 client.17631 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f295779f740 10 client.17631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f295779f740 20 client.17631 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f295779f740 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e612f28a60 0x55e612f2aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f295779f740 4 client.17631 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f295779f740 1 -- 172.21.15.55:0/3264734199 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e612e84990 con 0x55e612f28a60 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f294e59c640 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e612f28a60 0x55e612f2aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f2935ffb640 10 client.17631 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:38.651 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.645+0000 7f294e59c640 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e612f28a60 0x55e612f2aea0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa8b5d2bbd2dd2 server_cookie=31183eaf9916a2e3 in_seq=0 out_seq=0 2024-04-26T22:06:38.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f2935ffb640 1 -- 172.21.15.55:0/3264734199 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e612e84990 con 0x55e612f28a60 2024-04-26T22:06:38.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f2935ffb640 10 client.17631 handle_command_reply: tid=0 2024-04-26T22:06:38.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 client.17631 shutdown 2024-04-26T22:06:38.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 -- 172.21.15.55:0/3264734199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 msgr2=0x55e612f1d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.652 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f1d030 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2938009900 tx=0x7f2938031e30 comp rx=0 tx=0).stop 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 -- 172.21.15.55:0/3264734199 shutdown_connections 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f24070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e612f28a60 0x55e612f2aea0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f1d030 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 --2- 172.21.15.55:0/3264734199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 0x55e612f1caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.647+0000 7f295779f740 1 -- 172.21.15.55:0/3264734199 >> 172.21.15.55:0/3264734199 conn(0x55e612e53910 msgr2=0x55e612f0d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.648+0000 7f2935ffb640 0 client.17631 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.648+0000 7f295779f740 1 -- 172.21.15.55:0/3264734199 shutdown_connections 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.648+0000 7f295779f740 1 -- 172.21.15.55:0/3264734199 wait complete. 2024-04-26T22:06:38.653 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.648+0000 7f295779f740 20 client.17631 trim_cache size 0 max 16384 2024-04-26T22:06:38.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f295779f740 1 Processor -- start 2024-04-26T22:06:38.667 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f295779f740 1 -- start start 2024-04-26T22:06:38.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f295779f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 0x55e612f0d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f295779f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f0d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f295779f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f2f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f295779f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f29501ac780 con 0x55e612f139f0 2024-04-26T22:06:38.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f295779f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29501aa050 con 0x55e612f143b0 2024-04-26T22:06:38.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f295779f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29501a7ea0 con 0x55e612f12820 2024-04-26T22:06:38.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f2947fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f0d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.668 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f2947fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f0d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60616/0 (socket says 172.21.15.55:60616) 2024-04-26T22:06:38.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f2947fff640 1 -- 172.21.15.55:0/2730946686 learned_addr learned my addr 172.21.15.55:0/2730946686 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f294ed9d640 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f2f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f294e59c640 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 0x55e612f0d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f2947fff640 1 -- 172.21.15.55:0/2730946686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 msgr2=0x55e612f0d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f2947fff640 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 0x55e612f0d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f2947fff640 1 -- 172.21.15.55:0/2730946686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 msgr2=0x55e612f2f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f2947fff640 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f2f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.669 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.661+0000 7f2947fff640 1 -- 172.21.15.55:0/2730946686 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e612f0e370 con 0x55e612f139f0 2024-04-26T22:06:38.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f2947fff640 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f0d640 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f293803b9b0 tx=0x7f293803bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cebdd346e78389 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f2937fff640 1 -- 172.21.15.55:0/2730946686 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938053030 con 0x55e612f139f0 2024-04-26T22:06:38.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f2937fff640 1 -- 172.21.15.55:0/2730946686 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2938005850 con 0x55e612f139f0 2024-04-26T22:06:38.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f2937fff640 1 -- 172.21.15.55:0/2730946686 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938035040 con 0x55e612f139f0 2024-04-26T22:06:38.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 -- 172.21.15.55:0/2730946686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 msgr2=0x55e612f0d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f0d640 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f293803b9b0 tx=0x7f293803bd30 comp rx=0 tx=0).stop 2024-04-26T22:06:38.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 -- 172.21.15.55:0/2730946686 shutdown_connections 2024-04-26T22:06:38.670 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f143b0 0x55e612f2f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f0d640 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 --2- 172.21.15.55:0/2730946686 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f12820 0x55e612f0d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 -- 172.21.15.55:0/2730946686 >> 172.21.15.55:0/2730946686 conn(0x55e612e53910 msgr2=0x55e612f0c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 -- 172.21.15.55:0/2730946686 shutdown_connections 2024-04-26T22:06:38.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.662+0000 7f295779f740 1 -- 172.21.15.55:0/2730946686 wait complete. 2024-04-26T22:06:38.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 1 -- start start 2024-04-26T22:06:38.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 1 Processor -- start 2024-04-26T22:06:38.671 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f8a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f8ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f91b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f29501a8510 con 0x55e612f139f0 2024-04-26T22:06:38.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29501a9980 con 0x55e612f12820 2024-04-26T22:06:38.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f29501a8be0 con 0x55e612f143b0 2024-04-26T22:06:38.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f295779f740 10 client.0 _ll_register_callbacks cb 0x7f2938009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:38.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f2947fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f8ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f8a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f8a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.55:43600/0 (socket says 172.21.15.55:43600) 2024-04-26T22:06:38.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f91b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294e59c640 1 -- 172.21.15.55:0/83443288 learned_addr learned my addr 172.21.15.55:0/83443288 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:38.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294e59c640 1 -- 172.21.15.55:0/83443288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 msgr2=0x55e612f91b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294e59c640 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f91b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294e59c640 1 -- 172.21.15.55:0/83443288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 msgr2=0x55e612f8ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:38.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294e59c640 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f8ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f294e59c640 1 -- 172.21.15.55:0/83443288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e612f92140 con 0x55e612f12820 2024-04-26T22:06:38.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.663+0000 7f2947fff640 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f8ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:38.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f294e59c640 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f8a5d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29400cdee0 tx=0x7f29400d3b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e21a8154ebff40d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:38.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f2935ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-04-26T22:06:38.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f2935ffb640 1 -- 172.21.15.55:0/83443288 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940020040 con 0x55e612f12820 2024-04-26T22:06:38.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 1 -- 172.21.15.55:0/83443288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2938005850 con 0x55e612f12820 2024-04-26T22:06:38.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f2935ffb640 1 -- 172.21.15.55:0/83443288 <== mon.2 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f29400cdbb0 con 0x55e612f12820 2024-04-26T22:06:38.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f2935ffb640 1 -- 172.21.15.55:0/83443288 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f29400d3c30 con 0x55e612f12820 2024-04-26T22:06:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 10 client.26540 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 1 -- 172.21.15.55:0/83443288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e612f93490 con 0x55e612f12820 2024-04-26T22:06:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f2935ffb640 1 -- 172.21.15.55:0/83443288 <== mon.2 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2938005850 con 0x55e612f12820 2024-04-26T22:06:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f2935ffb640 1 -- 172.21.15.55:0/83443288 <== mon.2 v2:172.21.15.64:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f29400bc070 con 0x55e612f12820 2024-04-26T22:06:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 10 client.26540 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 10 client.26540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 20 client.26540 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:38.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e612f939c0 0x55e612f95de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:38.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 4 client.26540 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:06:38.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.664+0000 7f295779f740 1 -- 172.21.15.55:0/83443288 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e612e7e800 con 0x55e612f939c0 2024-04-26T22:06:38.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.665+0000 7f2947fff640 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e612f939c0 0x55e612f95de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:38.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.665+0000 7f2947fff640 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e612f939c0 0x55e612f95de0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15f8674676de26e server_cookie=ccefdb8ffab38a54 in_seq=0 out_seq=0 2024-04-26T22:06:38.676 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.665+0000 7f2935ffb640 10 client.26540 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f2935ffb640 1 -- 172.21.15.55:0/83443288 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e612e7e800 con 0x55e612f939c0 2024-04-26T22:06:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f2935ffb640 10 client.26540 handle_command_reply: tid=0 2024-04-26T22:06:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 client.26540 shutdown 2024-04-26T22:06:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 -- 172.21.15.55:0/83443288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 msgr2=0x55e612f8a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f8a5d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f29400cdee0 tx=0x7f29400d3b40 comp rx=0 tx=0).stop 2024-04-26T22:06:38.678 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 -- 172.21.15.55:0/83443288 shutdown_connections 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e612f143b0 0x55e612f91b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55e612f939c0 0x55e612f95de0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e612f139f0 0x55e612f8ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 --2- 172.21.15.55:0/83443288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e612f12820 0x55e612f8a5d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f2935ffb640 0 client.26540 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 -- 172.21.15.55:0/83443288 >> 172.21.15.55:0/83443288 conn(0x55e612e53910 msgr2=0x55e612f0c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 -- 172.21.15.55:0/83443288 shutdown_connections 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 1 -- 172.21.15.55:0/83443288 wait complete. 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f295779f740 20 client.26540 trim_cache size 0 max 16384 2024-04-26T22:06:38.679 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f2954a85640 1 -- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f293c0a3cc0 msgr2=0x7f293c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f2954a85640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f293c0a3cc0 0x7f293c0a6180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x55e612f138f0 tx=0x7f2948009500 comp rx=0 tx=0).stop 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f2954a85640 1 -- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950103410 msgr2=0x7f2950198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.668+0000 7f2954a85640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950103410 0x7f2950198c90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f294000d770 tx=0x7f294000dc30 comp rx=0 tx=0).stop 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.669+0000 7f2954a85640 1 -- 172.21.15.55:0/2872858033 shutdown_connections 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.669+0000 7f2954a85640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f293c0a3cc0 0x7f293c0a6180 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.669+0000 7f2954a85640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2950108670 0x7f29501a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.669+0000 7f2954a85640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2950105d40 0x7f29501991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.669+0000 7f2954a85640 1 --2- 172.21.15.55:0/2872858033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2950103410 0x7f2950198c90 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.669+0000 7f2954a85640 1 -- 172.21.15.55:0/2872858033 >> 172.21.15.55:0/2872858033 conn(0x7f29500fce20 msgr2=0x7f2950103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.669+0000 7f2954a85640 1 -- 172.21.15.55:0/2872858033 shutdown_connections 2024-04-26T22:06:38.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:38.669+0000 7f2954a85640 1 -- 172.21.15.55:0/2872858033 wait complete. 2024-04-26T22:06:38.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:38 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/17472623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:06:38.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:38 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/17472623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:06:39.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:38 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/17472623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:06:39.682 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-26T22:06:39.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:06:39.683 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-26T22:06:39.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- 172.21.15.55:0/3577649867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410ef30 msgr2=0x7f97141157c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:39.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 --2- 172.21.15.55:0/3577649867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410ef30 0x7f97141157c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f970c009ca0 tx=0x7f970c02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- 172.21.15.55:0/3577649867 shutdown_connections 2024-04-26T22:06:39.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 --2- 172.21.15.55:0/3577649867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410ef30 0x7f97141157c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 --2- 172.21.15.55:0/3577649867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f971410e570 0x7f971410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 --2- 172.21.15.55:0/3577649867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971410d380 0x7f971410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.794 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- 172.21.15.55:0/3577649867 >> 172.21.15.55:0/3577649867 conn(0x7f971406d360 msgr2=0x7f971406d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- 172.21.15.55:0/3577649867 shutdown_connections 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- 172.21.15.55:0/3577649867 wait complete. 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 Processor -- start 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- start start 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410d380 0x7f97141a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f971410e570 0x7f97141a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971410ef30 0x7f97141aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9714118e90 con 0x7f971410d380 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9714118d10 con 0x7f971410ef30 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.792+0000 7f971ac62640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9714119010 con 0x7f971410e570 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410d380 0x7f97141a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410d380 0x7f97141a3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60636/0 (socket says 172.21.15.55:60636) 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 -- 172.21.15.55:0/2408344815 learned_addr learned my addr 172.21.15.55:0/2408344815 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f9713fff640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f971410e570 0x7f97141a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.795 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97191ff640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971410ef30 0x7f97141aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 -- 172.21.15.55:0/2408344815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f971410e570 msgr2=0x7f97141a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f971410e570 0x7f97141a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 -- 172.21.15.55:0/2408344815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971410ef30 msgr2=0x7f97141aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971410ef30 0x7f97141aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 -- 172.21.15.55:0/2408344815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97141ab180 con 0x7f971410d380 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f97189fe640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410d380 0x7f97141a3560 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f970800e9c0 tx=0x7f970800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f9711ffb640 1 -- 172.21.15.55:0/2408344815 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800cde0 con 0x7f971410d380 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f971ac62640 1 -- 172.21.15.55:0/2408344815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97141ab470 con 0x7f971410d380 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f9711ffb640 1 -- 172.21.15.55:0/2408344815 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97080108e0 con 0x7f971410d380 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.793+0000 7f9711ffb640 1 -- 172.21.15.55:0/2408344815 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708004a40 con 0x7f971410d380 2024-04-26T22:06:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.794+0000 7f971ac62640 1 -- 172.21.15.55:0/2408344815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97141ab9c0 con 0x7f971410d380 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.796+0000 7f9711ffb640 1 -- 172.21.15.55:0/2408344815 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f970801f020 con 0x7f971410d380 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.796+0000 7f971d97b740 1 Processor -- start 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f971d97b740 1 -- start start 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f971d97b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f971d97b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a92a0 0x55d4899adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f971d97b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f971d97b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d48990db00 con 0x55d4899a92a0 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f971d97b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d4898a9920 con 0x55d4899a88b0 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f971d97b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d489909ac0 con 0x55d4899ae410 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a92a0 0x55d4899adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.802 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a92a0 0x55d4899adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60638/0 (socket says 172.21.15.55:60638) 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97191ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 -- 172.21.15.55:0/603180256 learned_addr learned my addr 172.21.15.55:0/603180256 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f9711ffb640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f97040a3c40 0x7f97040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f9713fff640 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97191ff640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f97040a3c40 0x7f97040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f9711ffb640 1 -- 172.21.15.55:0/2408344815 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f97080d0870 con 0x7f971410d380 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 -- 172.21.15.55:0/603180256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 msgr2=0x55d4899b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 -- 172.21.15.55:0/603180256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 msgr2=0x55d4899a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899a8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 -- 172.21.15.55:0/603180256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4896e77e0 con 0x55d4899a92a0 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f9713fff640 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97191ff640 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:39.803 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.797+0000 7f97189fe640 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a92a0 0x55d4899adda0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f970809c8e0 tx=0x7f970800efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526076772c8615a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f96feffd640 1 -- 172.21.15.55:0/603180256 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970809c490 con 0x55d4899a92a0 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f97191ff640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f97040a3c40 0x7f97040a6100 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f970c031b50 tx=0x7f970c02a8c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f96feffd640 1 -- 172.21.15.55:0/603180256 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97080d3e50 con 0x55d4899a92a0 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f96feffd640 1 -- 172.21.15.55:0/603180256 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970809c2b0 con 0x55d4899a92a0 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 -- 172.21.15.55:0/603180256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a92a0 msgr2=0x55d4899adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a92a0 0x55d4899adda0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f970809c8e0 tx=0x7f970800efb0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 -- 172.21.15.55:0/603180256 shutdown_connections 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a92a0 0x55d4899adda0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 --2- 172.21.15.55:0/603180256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 -- 172.21.15.55:0/603180256 >> 172.21.15.55:0/603180256 conn(0x55d4899a3f20 msgr2=0x55d4899a4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 -- 172.21.15.55:0/603180256 shutdown_connections 2024-04-26T22:06:39.804 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.798+0000 7f971d97b740 1 -- 172.21.15.55:0/603180256 wait complete. 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 1 -- start start 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 1 Processor -- start 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a92a0 0x55d4899b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55d4899b1300 con 0x55d4899a88b0 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d4898a75d0 con 0x55d4899a92a0 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d4899b1480 con 0x55d4899ae410 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f971d97b740 10 client.0 _ll_register_callbacks cb 0x55d489895080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f97189fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f97189fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60648/0 (socket says 172.21.15.55:60648) 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f97189fe640 1 -- 172.21.15.55:0/2882959846 learned_addr learned my addr 172.21.15.55:0/2882959846 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.799+0000 7f9713fff640 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a92a0 0x55d4899b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f97191ff640 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.805 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f97189fe640 1 -- 172.21.15.55:0/2882959846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 msgr2=0x55d4899ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f97189fe640 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f97189fe640 1 -- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a92a0 msgr2=0x55d4899b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f97189fe640 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a92a0 0x55d4899b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f97189fe640 1 -- 172.21.15.55:0/2882959846 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d489747f70 con 0x55d4899a88b0 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f9713fff640 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a92a0 0x55d4899b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f97189fe640 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2c20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f97080a5090 tx=0x7f970809c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b594f16e349c1ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f96f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f96f3fff640 1 -- 172.21.15.55:0/2882959846 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970809c2b0 con 0x55d4899a88b0 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f96f3fff640 1 -- 172.21.15.55:0/2882959846 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f97080ce3d0 con 0x55d4899a88b0 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f971d97b740 1 -- 172.21.15.55:0/2882959846 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97080d3e50 con 0x55d4899a88b0 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f96f3fff640 1 -- 172.21.15.55:0/2882959846 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f970809c490 con 0x55d4899a88b0 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f971d97b740 10 client.17649 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f971d97b740 1 -- 172.21.15.55:0/2882959846 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d4895df170 con 0x55d4899a88b0 2024-04-26T22:06:39.806 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f96f3fff640 1 -- 172.21.15.55:0/2882959846 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97080d3e50 con 0x55d4899a88b0 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f96f3fff640 1 -- 172.21.15.55:0/2882959846 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f97080870f0 con 0x55d4899a88b0 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f971d97b740 10 client.17649 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.800+0000 7f971d97b740 10 client.17649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.801+0000 7f971d97b740 20 client.17649 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.801+0000 7f971d97b740 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d4899be300 0x55d4899c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.801+0000 7f971d97b740 4 client.17649 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.801+0000 7f971d97b740 1 -- 172.21.15.55:0/2882959846 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d48991b710 con 0x55d4899be300 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.801+0000 7f9713fff640 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d4899be300 0x55d4899c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.801+0000 7f9713fff640 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d4899be300 0x55d4899c06f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abb4877d03a20d3 server_cookie=33bc558cd4389a20 in_seq=0 out_seq=0 2024-04-26T22:06:39.807 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.801+0000 7f96f3fff640 10 client.17649 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:39.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.804+0000 7f96f3fff640 1 -- 172.21.15.55:0/2882959846 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d48991b710 con 0x55d4899be300 2024-04-26T22:06:39.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.804+0000 7f96f3fff640 10 client.17649 handle_command_reply: tid=0 2024-04-26T22:06:39.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.804+0000 7f971d97b740 1 client.17649 shutdown 2024-04-26T22:06:39.808 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 -- 172.21.15.55:0/2882959846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 msgr2=0x55d4899b2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2c20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f97080a5090 tx=0x7f970809c9c0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 -- 172.21.15.55:0/2882959846 shutdown_connections 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899ae410 0x55d4899ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a92a0 0x55d4899b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d4899be300 0x55d4899c06f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 --2- 172.21.15.55:0/2882959846 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2c20 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f96f3fff640 0 client.17649 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 -- 172.21.15.55:0/2882959846 >> 172.21.15.55:0/2882959846 conn(0x55d4899a3f20 msgr2=0x55d4899a30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 -- 172.21.15.55:0/2882959846 shutdown_connections 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 1 -- 172.21.15.55:0/2882959846 wait complete. 2024-04-26T22:06:39.809 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.805+0000 7f971d97b740 20 client.17649 trim_cache size 0 max 16384 2024-04-26T22:06:39.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.822+0000 7f971d97b740 1 Processor -- start 2024-04-26T22:06:39.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.822+0000 7f971d97b740 1 -- start start 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f971d97b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f971d97b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f971d97b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899ae410 0x55d4899a2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f971d97b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b7260 con 0x55d4899a88b0 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f971d97b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b4b30 con 0x55d4899ae410 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f971d97b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b2980 con 0x55d4899a92a0 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60662/0 (socket says 172.21.15.55:60662) 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97191ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899ae410 0x55d4899a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 -- 172.21.15.55:0/3279270651 learned_addr learned my addr 172.21.15.55:0/3279270651 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f9713fff640 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 -- 172.21.15.55:0/3279270651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 msgr2=0x55d4899ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 -- 172.21.15.55:0/3279270651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899ae410 msgr2=0x55d4899a2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899ae410 0x55d4899a2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 -- 172.21.15.55:0/3279270651 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48999f8e0 con 0x55d4899a88b0 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f9713fff640 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f97189fe640 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2f40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9708005b60 tx=0x7f970800efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8697097631b705b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f96fe7fc640 1 -- 172.21.15.55:0/3279270651 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708098bf0 con 0x55d4899a88b0 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f96fe7fc640 1 -- 172.21.15.55:0/3279270651 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970809d050 con 0x55d4899a88b0 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.823+0000 7f96fe7fc640 1 -- 172.21.15.55:0/3279270651 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97080ce3d0 con 0x55d4899a88b0 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 -- 172.21.15.55:0/3279270651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 msgr2=0x55d4899b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2f40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9708005b60 tx=0x7f970800efb0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 -- 172.21.15.55:0/3279270651 shutdown_connections 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899ae410 0x55d4899a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 --2- 172.21.15.55:0/3279270651 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899a88b0 0x55d4899b2f40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 -- 172.21.15.55:0/3279270651 >> 172.21.15.55:0/3279270651 conn(0x55d4899a3f20 msgr2=0x55d4899b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 -- 172.21.15.55:0/3279270651 shutdown_connections 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 -- 172.21.15.55:0/3279270651 wait complete. 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 -- start start 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.824+0000 7f971d97b740 1 Processor -- start 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f971d97b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f971d97b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f971d97b740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899ae410 0x55d4899b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f971d97b740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b2ff0 con 0x55d4899ae410 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f971d97b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b4460 con 0x55d4899a88b0 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f971d97b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f97141b36c0 con 0x55d4899a92a0 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f971d97b740 10 client.0 _ll_register_callbacks cb 0x7f97080a55c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899ae410 0x55d4899b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97189fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899ae410 0x55d4899b9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60672/0 (socket says 172.21.15.55:60672) 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 -- 172.21.15.55:0/3273482568 learned_addr learned my addr 172.21.15.55:0/3273482568 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f9713fff640 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 -- 172.21.15.55:0/3273482568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 msgr2=0x55d4899b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 -- 172.21.15.55:0/3273482568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 msgr2=0x55d4899b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 -- 172.21.15.55:0/3273482568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4899b9880 con 0x55d4899ae410 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97189fe640 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f97191ff640 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899ae410 0x55d4899b9290 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f970c009ca0 tx=0x7f970c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4e5e2786d6c5dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.825+0000 7f96f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f96f3fff640 1 -- 172.21.15.55:0/3273482568 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c02f860 con 0x55d4899ae410 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f96f3fff640 1 -- 172.21.15.55:0/3273482568 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f970c02fa00 con 0x55d4899ae410 2024-04-26T22:06:39.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 1 -- 172.21.15.55:0/3273482568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f970809d050 con 0x55d4899ae410 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f96f3fff640 1 -- 172.21.15.55:0/3273482568 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f970c003e20 con 0x55d4899ae410 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 10 client.17655 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 1 -- 172.21.15.55:0/3273482568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d489a24b20 con 0x55d4899ae410 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f96f3fff640 1 -- 172.21.15.55:0/3273482568 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f970809d050 con 0x55d4899ae410 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f96f3fff640 1 -- 172.21.15.55:0/3273482568 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f970c04a040 con 0x55d4899ae410 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 10 client.17655 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 20 client.17655 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d489a25550 0x55d489a27970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 4 client.17655 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f971d97b740 1 -- 172.21.15.55:0/3273482568 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d489915580 con 0x55d489a25550 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.826+0000 7f97189fe640 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d489a25550 0x55d489a27970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.827+0000 7f96f3fff640 10 client.17655 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:39.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:39.827+0000 7f97189fe640 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d489a25550 0x55d489a27970 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d20b5e528bc2ad server_cookie=33b96aa6552cbffc in_seq=0 out_seq=0 2024-04-26T22:06:39.833 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:39 smithi165 ceph-mon[33296]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 58 MiB/s wr, 469 op/s 2024-04-26T22:06:39.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:39 smithi055 ceph-mon[29221]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 58 MiB/s wr, 469 op/s 2024-04-26T22:06:40.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:39 smithi064 ceph-mon[32765]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 58 MiB/s wr, 469 op/s 2024-04-26T22:06:40.868 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f96f3fff640 1 -- 172.21.15.55:0/3273482568 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d489915580 con 0x55d489a25550 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f96f3fff640 10 client.17655 handle_command_reply: tid=0 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 client.17655 shutdown 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 -- 172.21.15.55:0/3273482568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899ae410 msgr2=0x55d4899b9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899ae410 0x55d4899b9290 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f970c009ca0 tx=0x7f970c0095c0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 -- 172.21.15.55:0/3273482568 shutdown_connections 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55d489a25550 0x55d489a27970 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55d4899ae410 0x55d4899b9290 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d4899a92a0 0x55d4899b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 --2- 172.21.15.55:0/3273482568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d4899a88b0 0x55d4899b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f96f3fff640 0 client.17655 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 -- 172.21.15.55:0/3273482568 >> 172.21.15.55:0/3273482568 conn(0x55d4899a3f20 msgr2=0x55d4899a6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 -- 172.21.15.55:0/3273482568 shutdown_connections 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.867+0000 7f971d97b740 1 -- 172.21.15.55:0/3273482568 wait complete. 2024-04-26T22:06:40.869 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971d97b740 20 client.17655 trim_cache size 0 max 16384 2024-04-26T22:06:40.870 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:06:40.870 INFO:teuthology.orchestra.run.smithi055.stdout: "return_code": 0, 2024-04-26T22:06:40.870 INFO:teuthology.orchestra.run.smithi055.stdout: "scrub_tag": "539226db-c7ed-4649-b481-026ce8ddc93a", 2024-04-26T22:06:40.870 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": "asynchronous" 2024-04-26T22:06:40.870 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 -- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f97040a3c40 msgr2=0x7f97040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f97040a3c40 0x7f97040a6100 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f970c031b50 tx=0x7f970c02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 -- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410d380 msgr2=0x7f97141a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410d380 0x7f97141a3560 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f970800e9c0 tx=0x7f970800ee80 comp rx=0 tx=0).stop 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 -- 172.21.15.55:0/2408344815 shutdown_connections 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f97040a3c40 0x7f97040a6100 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f971410ef30 0x7f97141aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f971410e570 0x7f97141a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 --2- 172.21.15.55:0/2408344815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f971410d380 0x7f97141a3560 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.868+0000 7f971ac62640 1 -- 172.21.15.55:0/2408344815 >> 172.21.15.55:0/2408344815 conn(0x7f971406d360 msgr2=0x7f9714111910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:40.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.869+0000 7f971ac62640 1 -- 172.21.15.55:0/2408344815 shutdown_connections 2024-04-26T22:06:40.872 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:40.869+0000 7f971ac62640 1 -- 172.21.15.55:0/2408344815 wait complete. 2024-04-26T22:06:40.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:06:40.886 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:06:41.008 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 -- 172.21.15.55:0/3440724102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8102a60 msgr2=0x7fb1f8102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 --2- 172.21.15.55:0/3440724102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8102a60 0x7fb1f8102ee0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009900 tx=0x7fb1ec02a410 comp rx=0 tx=0).stop 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 -- 172.21.15.55:0/3440724102 shutdown_connections 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 --2- 172.21.15.55:0/3440724102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8103420 0x7fb1f8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 --2- 172.21.15.55:0/3440724102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8102a60 0x7fb1f8102ee0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 --2- 172.21.15.55:0/3440724102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1f8101870 0x7fb1f8101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 -- 172.21.15.55:0/3440724102 >> 172.21.15.55:0/3440724102 conn(0x7fb1f80fcde0 msgr2=0x7fb1f80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 -- 172.21.15.55:0/3440724102 shutdown_connections 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.006+0000 7fb1fde46640 1 -- 172.21.15.55:0/3440724102 wait complete. 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1fde46640 1 Processor -- start 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1fde46640 1 -- start start 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1fde46640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8101870 0x7fb1f819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1fde46640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1f8102a60 0x7fb1f819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1fde46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8103420 0x7fb1f81a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.009 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1fde46640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f81106a0 con 0x7fb1f8101870 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1fde46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f8110520 con 0x7fb1f8103420 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1fde46640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f8110820 con 0x7fb1f8102a60 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8101870 0x7fb1f819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8101870 0x7fb1f819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42582/0 (socket says 172.21.15.55:42582) 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1f77fe640 1 -- 172.21.15.55:0/878557902 learned_addr learned my addr 172.21.15.55:0/878557902 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1f7fff640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8103420 0x7fb1f81a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.007+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1f8102a60 0x7fb1f819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f77fe640 1 -- 172.21.15.55:0/878557902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1f8102a60 msgr2=0x7fb1f819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1f8102a60 0x7fb1f819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f77fe640 1 -- 172.21.15.55:0/878557902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8103420 msgr2=0x7fb1f81a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8103420 0x7fb1f81a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f77fe640 1 -- 172.21.15.55:0/878557902 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f81a2a10 con 0x7fb1f8101870 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8101870 0x7fb1f819ae40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800e9c0 tx=0x7fb1e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:41.010 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f4ff9640 1 -- 172.21.15.55:0/878557902 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e800cd80 con 0x7fb1f8101870 2024-04-26T22:06:41.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1fde46640 1 -- 172.21.15.55:0/878557902 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f81a2d00 con 0x7fb1f8101870 2024-04-26T22:06:41.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f4ff9640 1 -- 172.21.15.55:0/878557902 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e80035f0 con 0x7fb1f8101870 2024-04-26T22:06:41.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1f4ff9640 1 -- 172.21.15.55:0/878557902 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8010690 con 0x7fb1f8101870 2024-04-26T22:06:41.011 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.008+0000 7fb1fde46640 1 -- 172.21.15.55:0/878557902 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f81a31d0 con 0x7fb1f8101870 2024-04-26T22:06:41.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.010+0000 7fb1f4ff9640 1 -- 172.21.15.55:0/878557902 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb1e8004000 con 0x7fb1f8101870 2024-04-26T22:06:41.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb200c33740 1 Processor -- start 2024-04-26T22:06:41.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb200c33740 1 -- start start 2024-04-26T22:06:41.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb200c33740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd653e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.016 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb200c33740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb200c33740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653fa70 0x55afd6546300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb200c33740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55afd64a2d80 con 0x55afd653dee0 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb200c33740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55afd645c8b0 con 0x55afd653f0b0 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb200c33740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55afd649ed40 con 0x55afd653fa70 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd653e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f7fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd653e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42592/0 (socket says 172.21.15.55:42592) 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f7fff640 1 -- 172.21.15.55:0/293713680 learned_addr learned my addr 172.21.15.55:0/293713680 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653fa70 0x55afd6546300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f4ff9640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb1e00a3c20 0x7fb1e00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f77fe640 1 -- 172.21.15.55:0/293713680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653fa70 msgr2=0x55afd6546300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653fa70 0x55afd6546300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f77fe640 1 -- 172.21.15.55:0/293713680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 msgr2=0x55afd653e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.011+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd653e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.017 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1f77fe640 1 -- 172.21.15.55:0/293713680 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd642da40 con 0x55afd653f0b0 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1f7fff640 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd653e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653fa70 0x55afd6546300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1f7fff640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb1e00a3c20 0x7fb1e00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1f4ff9640 1 -- 172.21.15.55:0/878557902 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb1e80d0b10 con 0x7fb1f8101870 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653f530 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80ce3f0 tx=0x7fb1e801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb134f8835914b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1ddffb640 1 -- 172.21.15.55:0/293713680 <== mon.2 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e809c9c0 con 0x55afd653f0b0 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1f7fff640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb1e00a3c20 0x7fb1e00a60e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55afd653efb0 tx=0x7fb1f0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1ddffb640 1 -- 172.21.15.55:0/293713680 <== mon.2 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e8087070 con 0x55afd653f0b0 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.012+0000 7fb1ddffb640 1 -- 172.21.15.55:0/293713680 <== mon.2 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e807ce30 con 0x55afd653f0b0 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 -- 172.21.15.55:0/293713680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 msgr2=0x55afd653f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653f530 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80ce3f0 tx=0x7fb1e801d000 comp rx=0 tx=0).stop 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 -- 172.21.15.55:0/293713680 shutdown_connections 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653fa70 0x55afd6546300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.018 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653f530 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 --2- 172.21.15.55:0/293713680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd653e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 -- 172.21.15.55:0/293713680 >> 172.21.15.55:0/293713680 conn(0x55afd65393d0 msgr2=0x55afd65397d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 -- 172.21.15.55:0/293713680 shutdown_connections 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 -- 172.21.15.55:0/293713680 wait complete. 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 -- start start 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.013+0000 7fb200c33740 1 Processor -- start 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb200c33740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd65482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb200c33740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 0x55afd65487e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb200c33740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653fa70 0x55afd654f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb200c33740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55afd65469c0 con 0x55afd653dee0 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb200c33740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55afd6429d10 con 0x55afd653fa70 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb200c33740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55afd6546b40 con 0x55afd653f0b0 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb200c33740 10 client.0 _ll_register_callbacks cb 0x55afd6456210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:41.019 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd65482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd65482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42596/0 (socket says 172.21.15.55:42596) 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 -- 172.21.15.55:0/544248951 learned_addr learned my addr 172.21.15.55:0/544248951 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f7fff640 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653fa70 0x55afd654f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 0x55afd65487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 -- 172.21.15.55:0/544248951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 msgr2=0x55afd65487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 0x55afd65487e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 -- 172.21.15.55:0/544248951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653fa70 msgr2=0x55afd654f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653fa70 0x55afd654f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 -- 172.21.15.55:0/544248951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd627e7d0 con 0x55afd653dee0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd65482a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80ce420 tx=0x7fb1e800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4811c53f6d13d361 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/544248951 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e809c550 con 0x55afd653dee0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.014+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/544248951 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb1e80037a0 con 0x55afd653dee0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 1 -- 172.21.15.55:0/544248951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e8087070 con 0x55afd653dee0 2024-04-26T22:06:41.020 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/544248951 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb1e809c730 con 0x55afd653dee0 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 10 client.17667 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 1 -- 172.21.15.55:0/544248951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55afd61b43f0 con 0x55afd653dee0 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/544248951 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e8087070 con 0x55afd653dee0 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/544248951 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb1e80ce670 con 0x55afd653dee0 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 10 client.17667 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 20 client.17667 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55afd6553980 0x55afd6555d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 4 client.17667 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb200c33740 1 -- 172.21.15.55:0/544248951 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afd64b0990 con 0x55afd6553980 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.015+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55afd6553980 0x55afd6555d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.016+0000 7fb1d6ffd640 10 client.17667 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:41.021 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.016+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55afd6553980 0x55afd6555d70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=973b6dc4205b6d53 server_cookie=a0a5bae3be8d223c in_seq=0 out_seq=0 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/544248951 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afd64b0990 con 0x55afd6553980 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb1d6ffd640 10 client.17667 handle_command_reply: tid=0 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 client.17667 shutdown 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 -- 172.21.15.55:0/544248951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 msgr2=0x55afd65482a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd65482a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80ce420 tx=0x7fb1e800c840 comp rx=0 tx=0).stop 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 -- 172.21.15.55:0/544248951 shutdown_connections 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653fa70 0x55afd654f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 0x55afd65487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55afd6553980 0x55afd6555d70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 --2- 172.21.15.55:0/544248951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd65482a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb1d6ffd640 0 client.17667 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:41.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 -- 172.21.15.55:0/544248951 >> 172.21.15.55:0/544248951 conn(0x55afd65393d0 msgr2=0x55afd6538550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:41.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 -- 172.21.15.55:0/544248951 shutdown_connections 2024-04-26T22:06:41.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 1 -- 172.21.15.55:0/544248951 wait complete. 2024-04-26T22:06:41.024 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.018+0000 7fb200c33740 20 client.17667 trim_cache size 0 max 16384 2024-04-26T22:06:41.041 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb200c33740 1 Processor -- start 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb200c33740 1 -- start start 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb200c33740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd65393d0 0x55afd65485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb200c33740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb200c33740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb200c33740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f81aea30 con 0x55afd653dee0 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb200c33740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f81ac300 con 0x55afd653f0b0 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb200c33740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f81aa150 con 0x55afd65393d0 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42612/0 (socket says 172.21.15.55:42612) 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f6ffd640 1 -- 172.21.15.55:0/422275263 learned_addr learned my addr 172.21.15.55:0/422275263 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f7fff640 1 --2- 172.21.15.55:0/422275263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f6ffd640 1 -- 172.21.15.55:0/422275263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd65393d0 msgr2=0x55afd65485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:41.042 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/422275263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd65393d0 0x55afd65485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f6ffd640 1 -- 172.21.15.55:0/422275263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 msgr2=0x55afd653c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/422275263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.037+0000 7fb1f6ffd640 1 -- 172.21.15.55:0/422275263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd61740f0 con 0x55afd653dee0 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/422275263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654f7e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0096c0 tx=0x7fb1ec0315a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b3c0243549468d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb1dd7fa640 1 -- 172.21.15.55:0/422275263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec049020 con 0x55afd653dee0 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb1dd7fa640 1 -- 172.21.15.55:0/422275263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1ec0453b0 con 0x55afd653dee0 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb1dd7fa640 1 -- 172.21.15.55:0/422275263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec035040 con 0x55afd653dee0 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 -- 172.21.15.55:0/422275263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 msgr2=0x55afd654f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 --2- 172.21.15.55:0/422275263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654f7e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0096c0 tx=0x7fb1ec0315a0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 -- 172.21.15.55:0/422275263 shutdown_connections 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 --2- 172.21.15.55:0/422275263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd653f0b0 0x55afd653c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 --2- 172.21.15.55:0/422275263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654f7e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 --2- 172.21.15.55:0/422275263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd65393d0 0x55afd65485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.043 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 -- 172.21.15.55:0/422275263 >> 172.21.15.55:0/422275263 conn(0x55afd65347f0 msgr2=0x55afd6547eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 -- 172.21.15.55:0/422275263 shutdown_connections 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.038+0000 7fb200c33740 1 -- 172.21.15.55:0/422275263 wait complete. 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 1 -- start start 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 1 Processor -- start 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd65393d0 0x55afd654cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 0x55afd654e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f81aa7c0 con 0x55afd653dee0 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f81abc30 con 0x55afd65393d0 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f81aae90 con 0x55afd653f0b0 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb200c33740 10 client.0 _ll_register_callbacks cb 0x7fb1f0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb1f6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb1f6ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42614/0 (socket says 172.21.15.55:42614) 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb1f6ffd640 1 -- 172.21.15.55:0/807627340 learned_addr learned my addr 172.21.15.55:0/807627340 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.039+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd65393d0 0x55afd654cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.044 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1f7fff640 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 0x55afd654e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1f6ffd640 1 -- 172.21.15.55:0/807627340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 msgr2=0x55afd654e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 0x55afd654e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1f6ffd640 1 -- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd65393d0 msgr2=0x55afd654cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd65393d0 0x55afd654cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1f6ffd640 1 -- 172.21.15.55:0/807627340 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd64634b0 con 0x55afd653dee0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd65393d0 0x55afd654cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1f6ffd640 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654d260 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec031850 tx=0x7fb1ec031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea5a2acfe22f177 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/807627340 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec035030 con 0x55afd653dee0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/807627340 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb1ec058470 con 0x55afd653dee0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb200c33740 1 -- 172.21.15.55:0/807627340 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1ec0453b0 con 0x55afd653dee0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/807627340 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb1ec058670 con 0x55afd653dee0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb200c33740 10 client.17673 fetch_fsmap learned FSMap version 23 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb200c33740 1 -- 172.21.15.55:0/807627340 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55afd654fa40 con 0x55afd653dee0 2024-04-26T22:06:41.045 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.040+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/807627340 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1ec0453b0 con 0x55afd653dee0 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/807627340 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb1ec04e940 con 0x55afd653dee0 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb200c33740 10 client.17673 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb200c33740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb200c33740 20 client.17673 populate_metadata read hostname 'smithi055' 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb200c33740 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55afd653fa70 0x55afd65bc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb200c33740 4 client.17673 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb200c33740 1 -- 172.21.15.55:0/807627340 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55afd64aa800 con 0x55afd653fa70 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55afd653fa70 0x55afd65bc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb1f77fe640 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55afd653fa70 0x55afd65bc520 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892ca151fd75eff4 server_cookie=6aee4a46d8e58374 in_seq=0 out_seq=0 2024-04-26T22:06:41.046 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.041+0000 7fb1d6ffd640 10 client.17673 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:41.047 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb1d6ffd640 1 -- 172.21.15.55:0/807627340 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55afd64aa800 con 0x55afd653fa70 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb1d6ffd640 10 client.17673 handle_command_reply: tid=0 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 client.17673 shutdown 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 -- 172.21.15.55:0/807627340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 msgr2=0x55afd654d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654d260 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec031850 tx=0x7fb1ec031880 comp rx=0 tx=0).stop 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 -- 172.21.15.55:0/807627340 shutdown_connections 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55afd653f0b0 0x55afd654e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x55afd653fa70 0x55afd65bc520 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55afd653dee0 0x55afd654d260 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 --2- 172.21.15.55:0/807627340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55afd65393d0 0x55afd654cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 -- 172.21.15.55:0/807627340 >> 172.21.15.55:0/807627340 conn(0x55afd65347f0 msgr2=0x55afd6537cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb1d6ffd640 0 client.17673 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.043+0000 7fb200c33740 1 -- 172.21.15.55:0/807627340 shutdown_connections 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb200c33740 1 -- 172.21.15.55:0/807627340 wait complete. 2024-04-26T22:06:41.048 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb200c33740 20 client.17673 trim_cache size 0 max 16384 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": { 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout: "539226db-c7ed-4649-b481-026ce8ddc93a": { 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout: "path": "/", 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout: "tag": "539226db-c7ed-4649-b481-026ce8ddc93a", 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout: "options": "recursive,force" 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:06:41.049 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:06:41.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 -- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb1e00a3c20 msgr2=0x7fb1e00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb1e00a3c20 0x7fb1e00a60e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55afd653efb0 tx=0x7fb1f0004a40 comp rx=0 tx=0).stop 2024-04-26T22:06:41.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 -- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8101870 msgr2=0x7fb1f819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:06:41.050 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8101870 0x7fb1f819ae40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800e9c0 tx=0x7fb1e800ee80 comp rx=0 tx=0).stop 2024-04-26T22:06:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 -- 172.21.15.55:0/878557902 shutdown_connections 2024-04-26T22:06:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb1e00a3c20 0x7fb1e00a60e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb1f8103420 0x7fb1f81a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1f8102a60 0x7fb1f819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 --2- 172.21.15.55:0/878557902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb1f8101870 0x7fb1f819ae40 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:06:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 -- 172.21.15.55:0/878557902 >> 172.21.15.55:0/878557902 conn(0x7fb1f80fcde0 msgr2=0x7fb1f8105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:06:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 -- 172.21.15.55:0/878557902 shutdown_connections 2024-04-26T22:06:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:06:41.044+0000 7fb1fde46640 1 -- 172.21.15.55:0/878557902 wait complete. 2024-04-26T22:06:41.056 INFO:tasks.cephfs.filesystem:scrub status for tag:539226db-c7ed-4649-b481-026ce8ddc93a - {'path': '/', 'tag': '539226db-c7ed-4649-b481-026ce8ddc93a', 'options': 'recursive,force'} 2024-04-26T22:06:41.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:41 smithi165 ceph-mon[33296]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 45 MiB/s wr, 373 op/s 2024-04-26T22:06:41.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:41 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:06:41.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:41 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:41.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:41 smithi055 ceph-mon[29221]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 45 MiB/s wr, 373 op/s 2024-04-26T22:06:41.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:41 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:06:41.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:41 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:42.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:41 smithi064 ceph-mon[32765]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 45 MiB/s wr, 373 op/s 2024-04-26T22:06:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:41 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:06:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:41 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:42.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:42 smithi165 ceph-mon[33296]: scrub queued for path: / 2024-04-26T22:06:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:42 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:42.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:42 smithi165 ceph-mon[33296]: scrub summary: active paths [/] 2024-04-26T22:06:42.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:42 smithi055 ceph-mon[29221]: scrub queued for path: / 2024-04-26T22:06:42.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:42 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:42.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:42 smithi055 ceph-mon[29221]: scrub summary: active paths [/] 2024-04-26T22:06:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:42 smithi064 ceph-mon[32765]: scrub queued for path: / 2024-04-26T22:06:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:42 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:43.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:42 smithi064 ceph-mon[32765]: scrub summary: active paths [/] 2024-04-26T22:06:43.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:43 smithi165 ceph-mon[33296]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 45 MiB/s wr, 317 op/s 2024-04-26T22:06:43.837 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:43 smithi055 ceph-mon[29221]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 45 MiB/s wr, 317 op/s 2024-04-26T22:06:44.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:43 smithi064 ceph-mon[32765]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 45 MiB/s wr, 317 op/s 2024-04-26T22:06:45.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:45 smithi165 ceph-mon[33296]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 43 MiB/s wr, 242 op/s 2024-04-26T22:06:45.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:45 smithi055 ceph-mon[29221]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 43 MiB/s wr, 242 op/s 2024-04-26T22:06:46.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:45 smithi064 ceph-mon[32765]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 43 MiB/s wr, 242 op/s 2024-04-26T22:06:46.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:46 smithi165 ceph-mon[33296]: client.24916 does not advance its oldest_flush_tid (700245), 100000 completed flushes recorded in session 2024-04-26T22:06:46.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:46 smithi055 ceph-mon[29221]: client.24916 does not advance its oldest_flush_tid (700245), 100000 completed flushes recorded in session 2024-04-26T22:06:47.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:46 smithi064 ceph-mon[32765]: client.24916 does not advance its oldest_flush_tid (700245), 100000 completed flushes recorded in session 2024-04-26T22:06:47.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:47 smithi165 ceph-mon[33296]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 35 MiB/s wr, 174 op/s 2024-04-26T22:06:47.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:47 smithi055 ceph-mon[29221]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 35 MiB/s wr, 174 op/s 2024-04-26T22:06:48.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:47 smithi064 ceph-mon[32765]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 35 MiB/s wr, 174 op/s 2024-04-26T22:06:49.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:49 smithi165 ceph-mon[33296]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 37 MiB/s wr, 176 op/s 2024-04-26T22:06:49.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:49 smithi055 ceph-mon[29221]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 37 MiB/s wr, 176 op/s 2024-04-26T22:06:50.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:49 smithi064 ceph-mon[32765]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 37 MiB/s wr, 176 op/s 2024-04-26T22:06:50.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:50 smithi165 ceph-mon[33296]: Health check failed: 1 clients failing to advance oldest client/flush tid (MDS_CLIENT_OLDEST_TID) 2024-04-26T22:06:50.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:50 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T22:06:50.827 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:50 smithi165 ceph-mon[33296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:06:50.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:50 smithi055 ceph-mon[29221]: Health check failed: 1 clients failing to advance oldest client/flush tid (MDS_CLIENT_OLDEST_TID) 2024-04-26T22:06:50.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:50 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T22:06:50.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:50 smithi055 ceph-mon[29221]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:06:51.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:50 smithi064 ceph-mon[32765]: Health check failed: 1 clients failing to advance oldest client/flush tid (MDS_CLIENT_OLDEST_TID) 2024-04-26T22:06:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:50 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T22:06:51.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:50 smithi064 ceph-mon[32765]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:06:51.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:51 smithi165 ceph-mon[33296]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 29 MiB/s wr, 138 op/s 2024-04-26T22:06:51.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:51 smithi055 ceph-mon[29221]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 29 MiB/s wr, 138 op/s 2024-04-26T22:06:52.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:51 smithi064 ceph-mon[32765]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 29 MiB/s wr, 138 op/s 2024-04-26T22:06:53.826 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:53 smithi165 ceph-mon[33296]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 95 op/s 2024-04-26T22:06:53.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:53 smithi055 ceph-mon[29221]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 95 op/s 2024-04-26T22:06:54.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:53 smithi064 ceph-mon[32765]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 95 op/s 2024-04-26T22:06:55.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:55 smithi055 ceph-mon[29221]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 17 MiB/s wr, 80 op/s 2024-04-26T22:06:56.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:55 smithi064 ceph-mon[32765]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 17 MiB/s wr, 80 op/s 2024-04-26T22:06:56.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:55 smithi165 ceph-mon[33296]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 17 MiB/s wr, 80 op/s 2024-04-26T22:06:57.611 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-04-26T22:06:57.612 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T22:06:57.901 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:57 smithi055 ceph-mon[29221]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 66 op/s 2024-04-26T22:06:57.901 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:57 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:57.901 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:57 smithi055 ceph-mon[29221]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:57.902 DEBUG:teuthology.parallel:result is None 2024-04-26T22:06:57.902 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T22:06:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:57 smithi064 ceph-mon[32765]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 66 op/s 2024-04-26T22:06:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:57 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:58.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:57 smithi064 ceph-mon[32765]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:58.014 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T22:06:58.014 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-26T22:06:58.025 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-26T22:06:58.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:57 smithi165 ceph-mon[33296]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 66 op/s 2024-04-26T22:06:58.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:57 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:58.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:57 smithi165 ceph-mon[33296]: scrub summary: idle+waiting paths [/] 2024-04-26T22:06:58.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:58 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:58.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:58 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:58.937 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:58 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:06:59.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:58 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:59.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:58 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:59.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:58 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:06:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:58 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:58 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:59.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:58 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:06:59.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:06:59 smithi055 ceph-mon[29221]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 27 MiB/s wr, 128 op/s 2024-04-26T22:07:00.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:06:59 smithi064 ceph-mon[32765]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 27 MiB/s wr, 128 op/s 2024-04-26T22:07:00.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:06:59 smithi165 ceph-mon[33296]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 27 MiB/s wr, 128 op/s 2024-04-26T22:07:01.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:01 smithi055 ceph-mon[29221]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 34 MiB/s wr, 223 op/s 2024-04-26T22:07:01.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:01 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:07:01.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:01 smithi055 ceph-mon[29221]: MDS health message cleared (mds.?): Client smithi055:0 failing to advance its oldest client/flush tid. 2024-04-26T22:07:02.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:01 smithi064 ceph-mon[32765]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 34 MiB/s wr, 223 op/s 2024-04-26T22:07:02.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:01 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:07:02.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:01 smithi064 ceph-mon[32765]: MDS health message cleared (mds.?): Client smithi055:0 failing to advance its oldest client/flush tid. 2024-04-26T22:07:02.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:01 smithi165 ceph-mon[33296]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 34 MiB/s wr, 223 op/s 2024-04-26T22:07:02.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:01 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:07:02.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:01 smithi165 ceph-mon[33296]: MDS health message cleared (mds.?): Client smithi055:0 failing to advance its oldest client/flush tid. 2024-04-26T22:07:03.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:02 smithi064 ceph-mon[32765]: scrub summary: idle 2024-04-26T22:07:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:02 smithi064 ceph-mon[32765]: Health check cleared: MDS_CLIENT_OLDEST_TID (was: 1 clients failing to advance oldest client/flush tid) 2024-04-26T22:07:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:02 smithi064 ceph-mon[32765]: Cluster is now healthy 2024-04-26T22:07:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:02 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T22:07:03.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:02 smithi064 ceph-mon[32765]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:07:03.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:02 smithi165 ceph-mon[33296]: scrub summary: idle 2024-04-26T22:07:03.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:02 smithi165 ceph-mon[33296]: Health check cleared: MDS_CLIENT_OLDEST_TID (was: 1 clients failing to advance oldest client/flush tid) 2024-04-26T22:07:03.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:02 smithi165 ceph-mon[33296]: Cluster is now healthy 2024-04-26T22:07:03.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:02 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T22:07:03.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:02 smithi165 ceph-mon[33296]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:07:03.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:02 smithi055 ceph-mon[29221]: scrub summary: idle 2024-04-26T22:07:03.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:02 smithi055 ceph-mon[29221]: Health check cleared: MDS_CLIENT_OLDEST_TID (was: 1 clients failing to advance oldest client/flush tid) 2024-04-26T22:07:03.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:02 smithi055 ceph-mon[29221]: Cluster is now healthy 2024-04-26T22:07:03.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:02 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] up:active 2024-04-26T22:07:03.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:02 smithi055 ceph-mon[29221]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T22:07:04.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:03 smithi064 ceph-mon[32765]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 45 MiB/s wr, 335 op/s 2024-04-26T22:07:04.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:03 smithi165 ceph-mon[33296]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 45 MiB/s wr, 335 op/s 2024-04-26T22:07:04.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:03 smithi055 ceph-mon[29221]: pgmap v775: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 45 MiB/s wr, 335 op/s 2024-04-26T22:07:06.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:05 smithi064 ceph-mon[32765]: pgmap v776: 97 pgs: 97 active+clean; 7.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 52 MiB/s wr, 467 op/s 2024-04-26T22:07:06.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:05 smithi165 ceph-mon[33296]: pgmap v776: 97 pgs: 97 active+clean; 7.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 52 MiB/s wr, 467 op/s 2024-04-26T22:07:06.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:05 smithi055 ceph-mon[29221]: pgmap v776: 97 pgs: 97 active+clean; 7.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 52 MiB/s wr, 467 op/s 2024-04-26T22:07:08.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:07 smithi064 ceph-mon[32765]: pgmap v777: 97 pgs: 97 active+clean; 6.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 45 MiB/s wr, 450 op/s 2024-04-26T22:07:08.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:07 smithi165 ceph-mon[33296]: pgmap v777: 97 pgs: 97 active+clean; 6.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 45 MiB/s wr, 450 op/s 2024-04-26T22:07:08.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:07 smithi055 ceph-mon[29221]: pgmap v777: 97 pgs: 97 active+clean; 6.8 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 45 MiB/s wr, 450 op/s 2024-04-26T22:07:10.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:09 smithi064 ceph-mon[32765]: pgmap v778: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 41 MiB/s wr, 487 op/s 2024-04-26T22:07:10.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:09 smithi165 ceph-mon[33296]: pgmap v778: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 41 MiB/s wr, 487 op/s 2024-04-26T22:07:10.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:09 smithi055 ceph-mon[29221]: pgmap v778: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 41 MiB/s wr, 487 op/s 2024-04-26T22:07:11.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-26T22:07:11.057 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 -- 172.21.15.55:0/3545160512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec104540 msgr2=0x7f9fec10add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 --2- 172.21.15.55:0/3545160512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec104540 0x7f9fec10add0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009ca0 tx=0x7f9fe402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 -- 172.21.15.55:0/3545160512 shutdown_connections 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 --2- 172.21.15.55:0/3545160512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec104540 0x7f9fec10add0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 --2- 172.21.15.55:0/3545160512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fec103b80 0x7f9fec104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 --2- 172.21.15.55:0/3545160512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec102990 0x7f9fec102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 -- 172.21.15.55:0/3545160512 >> 172.21.15.55:0/3545160512 conn(0x7f9fec0fdf00 msgr2=0x7f9fec100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 -- 172.21.15.55:0/3545160512 shutdown_connections 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.169+0000 7f9ff459d640 1 -- 172.21.15.55:0/3545160512 wait complete. 2024-04-26T22:07:11.173 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff459d640 1 Processor -- start 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff459d640 1 -- start start 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff459d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec102990 0x7f9fec19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff459d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fec103b80 0x7f9fec19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff459d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec104540 0x7f9fec1a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff459d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec10e4a0 con 0x7f9fec102990 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff459d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec10e320 con 0x7f9fec104540 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff459d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec10e620 con 0x7f9fec103b80 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2339640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec102990 0x7f9fec19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2339640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec102990 0x7f9fec19ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53230/0 (socket says 172.21.15.55:53230) 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2339640 1 -- 172.21.15.55:0/2211655708 learned_addr learned my addr 172.21.15.55:0/2211655708 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec104540 0x7f9fec1a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fec103b80 0x7f9fec19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.174 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2339640 1 -- 172.21.15.55:0/2211655708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fec103b80 msgr2=0x7f9fec19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2339640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fec103b80 0x7f9fec19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2339640 1 -- 172.21.15.55:0/2211655708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec104540 msgr2=0x7f9fec1a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2339640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec104540 0x7f9fec1a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.170+0000 7f9ff2339640 1 -- 172.21.15.55:0/2211655708 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec1a2950 con 0x7f9fec102990 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.171+0000 7f9ff2339640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec102990 0x7f9fec19ade0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800d770 tx=0x7f9fe800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.171+0000 7f9fe37fe640 1 -- 172.21.15.55:0/2211655708 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8017070 con 0x7f9fec102990 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.171+0000 7f9fe37fe640 1 -- 172.21.15.55:0/2211655708 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe8009c30 con 0x7f9fec102990 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.171+0000 7f9ff459d640 1 -- 172.21.15.55:0/2211655708 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec1a2c40 con 0x7f9fec102990 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.171+0000 7f9fe37fe640 1 -- 172.21.15.55:0/2211655708 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8003cc0 con 0x7f9fec102990 2024-04-26T22:07:11.175 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.171+0000 7f9ff459d640 1 -- 172.21.15.55:0/2211655708 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec1a3190 con 0x7f9fec102990 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.173+0000 7f9fe37fe640 1 -- 172.21.15.55:0/2211655708 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9fe8003e60 con 0x7f9fec102990 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.173+0000 7f9ff738e740 1 Processor -- start 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff738e740 1 -- start start 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa780ee0 0x5650aa781300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa782530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa782a70 0x5650aa789300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff738e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5650aa6e5d80 con 0x5650aa7820b0 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff738e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5650aa69f8b0 con 0x5650aa780ee0 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff738e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5650aa6e1d40 con 0x5650aa782a70 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9fe37fe640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9fd80a3cc0 0x7f9fd80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff2339640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa782530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff2339640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa782530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53238/0 (socket says 172.21.15.55:53238) 2024-04-26T22:07:11.182 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff2339640 1 -- 172.21.15.55:0/2093152570 learned_addr learned my addr 172.21.15.55:0/2093152570 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9fd80a3cc0 0x7f9fd80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa782a70 0x5650aa789300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa780ee0 0x5650aa781300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9fe37fe640 1 -- 172.21.15.55:0/2211655708 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9fe80cff20 con 0x7f9fec102990 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff2339640 1 -- 172.21.15.55:0/2093152570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa782a70 msgr2=0x5650aa789300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff2339640 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa782a70 0x5650aa789300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.174+0000 7f9ff2339640 1 -- 172.21.15.55:0/2093152570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa780ee0 msgr2=0x5650aa781300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.175+0000 7f9ff2339640 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa780ee0 0x5650aa781300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.175+0000 7f9ff2339640 1 -- 172.21.15.55:0/2093152570 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650aa670a40 con 0x5650aa7820b0 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.175+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa782a70 0x5650aa789300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.175+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9fd80a3cc0 0x7f9fd80a6180 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4031b50 tx=0x7f9fe4009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.183 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.175+0000 7f9ff2339640 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa782530 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8021000 tx=0x7f9fe8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ee834622211c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.175+0000 7f9fe0bfe640 1 -- 172.21.15.55:0/2093152570 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8020070 con 0x5650aa7820b0 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.175+0000 7f9fe0bfe640 1 -- 172.21.15.55:0/2093152570 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe8009de0 con 0x5650aa7820b0 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.175+0000 7f9fe0bfe640 1 -- 172.21.15.55:0/2093152570 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80cda80 con 0x5650aa7820b0 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- 172.21.15.55:0/2093152570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 msgr2=0x5650aa782530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa782530 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8021000 tx=0x7f9fe8000bc0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- 172.21.15.55:0/2093152570 shutdown_connections 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa782a70 0x5650aa789300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa782530 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 --2- 172.21.15.55:0/2093152570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa780ee0 0x5650aa781300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- 172.21.15.55:0/2093152570 >> 172.21.15.55:0/2093152570 conn(0x5650aa77c3d0 msgr2=0x5650aa77c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- 172.21.15.55:0/2093152570 shutdown_connections 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- 172.21.15.55:0/2093152570 wait complete. 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- start start 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 Processor -- start 2024-04-26T22:07:11.184 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa780ee0 0x5650aa78b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa78b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa782a70 0x5650aa792880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x5650aa7899c0 con 0x5650aa7820b0 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5650aa66cd10 con 0x5650aa782a70 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5650aa789b40 con 0x5650aa780ee0 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff738e740 10 client.0 _ll_register_callbacks cb 0x5650aa699210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.176+0000 7f9ff1b38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa78b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff1b38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa78b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53254/0 (socket says 172.21.15.55:53254) 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff1b38640 1 -- 172.21.15.55:0/1651360350 learned_addr learned my addr 172.21.15.55:0/1651360350 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff2339640 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa780ee0 0x5650aa78b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa782a70 0x5650aa792880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff1b38640 1 -- 172.21.15.55:0/1651360350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa780ee0 msgr2=0x5650aa78b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa780ee0 0x5650aa78b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.185 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff1b38640 1 -- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa782a70 msgr2=0x5650aa792880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa782a70 0x5650aa792880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff1b38640 1 -- 172.21.15.55:0/1651360350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650aa4c17d0 con 0x5650aa7820b0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa782a70 0x5650aa792880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa78b8b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00bfc0 tx=0x7f9fdc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb91a0a99d52f144 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/1651360350 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc004020 con 0x5650aa7820b0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/1651360350 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9fdc0041c0 con 0x5650aa7820b0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff738e740 1 -- 172.21.15.55:0/1651360350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe8009de0 con 0x5650aa7820b0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/1651360350 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9fdc004400 con 0x5650aa7820b0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff738e740 10 client.17685 fetch_fsmap learned FSMap version 25 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.177+0000 7f9ff738e740 1 -- 172.21.15.55:0/1651360350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650aa3f73f0 con 0x5650aa7820b0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/1651360350 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe8009de0 con 0x5650aa7820b0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/1651360350 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9fdc003ad0 con 0x5650aa7820b0 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9ff738e740 10 client.17685 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:07:11.186 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9ff738e740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9ff738e740 20 client.17685 populate_metadata read hostname 'smithi055' 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9ff738e740 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5650aa796ee0 0x5650aa799320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9ff738e740 4 client.17685 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9ff738e740 1 -- 172.21.15.55:0/1651360350 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650aa6f3990 con 0x5650aa796ee0 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.178+0000 7f9ff2339640 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5650aa796ee0 0x5650aa799320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9fd5ffb640 10 client.17685 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff2339640 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5650aa796ee0 0x5650aa799320 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3648ba8ae8a4cff server_cookie=f93d03114438a95a in_seq=0 out_seq=0 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/1651360350 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650aa6f3990 con 0x5650aa796ee0 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9fd5ffb640 10 client.17685 handle_command_reply: tid=0 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 client.17685 shutdown 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 -- 172.21.15.55:0/1651360350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 msgr2=0x5650aa78b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa78b8b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00bfc0 tx=0x7f9fdc0118d0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.187 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 -- 172.21.15.55:0/1651360350 shutdown_connections 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa782a70 0x5650aa792880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5650aa796ee0 0x5650aa799320 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa7820b0 0x5650aa78b8b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 --2- 172.21.15.55:0/1651360350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa780ee0 0x5650aa78b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9fd5ffb640 0 client.17685 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.179+0000 7f9ff738e740 1 -- 172.21.15.55:0/1651360350 >> 172.21.15.55:0/1651360350 conn(0x5650aa77c3d0 msgr2=0x5650aa77b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.180+0000 7f9ff738e740 1 -- 172.21.15.55:0/1651360350 shutdown_connections 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.180+0000 7f9ff738e740 1 -- 172.21.15.55:0/1651360350 wait complete. 2024-04-26T22:07:11.188 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.180+0000 7f9ff738e740 20 client.17685 trim_cache size 0 max 16384 2024-04-26T22:07:11.203 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff738e740 1 Processor -- start 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff738e740 1 -- start start 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa77c3d0 0x5650aa785c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa7861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa7820b0 0x5650aa792840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff738e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec1ae960 con 0x5650aa780ee0 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff738e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec1ac230 con 0x5650aa7820b0 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff738e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec1aa080 con 0x5650aa77c3d0 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff1b38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa7861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff1b38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa7861b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53260/0 (socket says 172.21.15.55:53260) 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.197+0000 7f9ff1b38640 1 -- 172.21.15.55:0/2265674444 learned_addr learned my addr 172.21.15.55:0/2265674444 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9ff2339640 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa77c3d0 0x5650aa785c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa7820b0 0x5650aa792840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9ff1b38640 1 -- 172.21.15.55:0/2265674444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa77c3d0 msgr2=0x5650aa785c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.204 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa77c3d0 0x5650aa785c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9ff1b38640 1 -- 172.21.15.55:0/2265674444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa7820b0 msgr2=0x5650aa792840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa7820b0 0x5650aa792840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9ff1b38640 1 -- 172.21.15.55:0/2265674444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650aa3b70f0 con 0x5650aa780ee0 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa7861b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc003f10 tx=0x7f9fdc00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f99f1822917625b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9fd7fff640 1 -- 172.21.15.55:0/2265674444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc029040 con 0x5650aa780ee0 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9fd7fff640 1 -- 172.21.15.55:0/2265674444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fdc003aa0 con 0x5650aa780ee0 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.198+0000 7f9fd7fff640 1 -- 172.21.15.55:0/2265674444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc010c80 con 0x5650aa780ee0 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- 172.21.15.55:0/2265674444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 msgr2=0x5650aa7861b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa7861b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc003f10 tx=0x7f9fdc00b8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- 172.21.15.55:0/2265674444 shutdown_connections 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa7820b0 0x5650aa792840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa7861b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 --2- 172.21.15.55:0/2265674444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa77c3d0 0x5650aa785c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.205 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- 172.21.15.55:0/2265674444 >> 172.21.15.55:0/2265674444 conn(0x5650aa7777f0 msgr2=0x5650aa7850c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- 172.21.15.55:0/2265674444 shutdown_connections 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- 172.21.15.55:0/2265674444 wait complete. 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- start start 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 Processor -- start 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa77c3d0 0x5650aa78dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa78e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa7820b0 0x5650aa7913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec1aa6f0 con 0x5650aa780ee0 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec1abb60 con 0x5650aa77c3d0 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec1aadc0 con 0x5650aa7820b0 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff738e740 10 client.0 _ll_register_callbacks cb 0x7f9fdc00bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff1b38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa78e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.206 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff1b38640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa78e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53264/0 (socket says 172.21.15.55:53264) 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.199+0000 7f9ff1b38640 1 -- 172.21.15.55:0/3035949047 learned_addr learned my addr 172.21.15.55:0/3035949047 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa7820b0 0x5650aa7913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff1b38640 1 -- 172.21.15.55:0/3035949047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa7820b0 msgr2=0x5650aa7913f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff2339640 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa77c3d0 0x5650aa78dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa7820b0 0x5650aa7913f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff1b38640 1 -- 172.21.15.55:0/3035949047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa77c3d0 msgr2=0x5650aa78dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa77c3d0 0x5650aa78dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff1b38640 1 -- 172.21.15.55:0/3035949047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650aa6a64b0 con 0x5650aa780ee0 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff2b3a640 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa7820b0 0x5650aa7913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff1b38640 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa78e420 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc010800 tx=0x7f9fdc003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fe74076ef0cfbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/3035949047 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc029040 con 0x5650aa780ee0 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/3035949047 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9fdc0035f0 con 0x5650aa780ee0 2024-04-26T22:07:11.207 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff738e740 1 -- 172.21.15.55:0/3035949047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fdc003aa0 con 0x5650aa780ee0 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/3035949047 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9fdc004230 con 0x5650aa780ee0 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff738e740 10 client.17691 fetch_fsmap learned FSMap version 25 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9ff738e740 1 -- 172.21.15.55:0/3035949047 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650aa792b20 con 0x5650aa780ee0 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.200+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/3035949047 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fdc003aa0 con 0x5650aa780ee0 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/3035949047 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9fdc02c070 con 0x5650aa780ee0 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 10 client.17691 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 20 client.17691 populate_metadata read hostname 'smithi055' 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5650aa782a70 0x5650aa7ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 4 client.17691 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 1 -- 172.21.15.55:0/3035949047 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650aa6ed800 con 0x5650aa782a70 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff2339640 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5650aa782a70 0x5650aa7ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9fd5ffb640 10 client.17691 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff2339640 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5650aa782a70 0x5650aa7ffbd0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf89e04a2bd0c80 server_cookie=948f7a97dd379bde in_seq=0 out_seq=0 2024-04-26T22:07:11.208 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9fd5ffb640 1 -- 172.21.15.55:0/3035949047 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5650aa6ed800 con 0x5650aa782a70 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9fd5ffb640 10 client.17691 handle_command_reply: tid=0 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 1 client.17691 shutdown 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 1 -- 172.21.15.55:0/3035949047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 msgr2=0x5650aa78e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.201+0000 7f9ff738e740 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa78e420 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc010800 tx=0x7f9fdc003c50 comp rx=0 tx=0).stop 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 1 -- 172.21.15.55:0/3035949047 shutdown_connections 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5650aa7820b0 0x5650aa7913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x5650aa782a70 0x5650aa7ffbd0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5650aa780ee0 0x5650aa78e420 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9fd5ffb640 0 client.17691 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 1 --2- 172.21.15.55:0/3035949047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5650aa77c3d0 0x5650aa78dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 1 -- 172.21.15.55:0/3035949047 >> 172.21.15.55:0/3035949047 conn(0x5650aa7777f0 msgr2=0x5650aa77acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 1 -- 172.21.15.55:0/3035949047 shutdown_connections 2024-04-26T22:07:11.209 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 1 -- 172.21.15.55:0/3035949047 wait complete. 2024-04-26T22:07:11.210 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff738e740 20 client.17691 trim_cache size 0 max 16384 2024-04-26T22:07:11.210 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:07:11.210 INFO:teuthology.orchestra.run.smithi055.stdout: "status": "no active scrubs running", 2024-04-26T22:07:11.210 INFO:teuthology.orchestra.run.smithi055.stdout: "scrubs": {} 2024-04-26T22:07:11.210 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 -- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9fd80a3cc0 msgr2=0x7f9fd80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9fd80a3cc0 0x7f9fd80a6180 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4031b50 tx=0x7f9fe4009800 comp rx=0 tx=0).stop 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 -- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec102990 msgr2=0x7f9fec19ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec102990 0x7f9fec19ade0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800d770 tx=0x7f9fe800dc30 comp rx=0 tx=0).stop 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 -- 172.21.15.55:0/2211655708 shutdown_connections 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9fd80a3cc0 0x7f9fd80a6180 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fec104540 0x7f9fec1a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fec103b80 0x7f9fec19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.211 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 --2- 172.21.15.55:0/2211655708 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fec102990 0x7f9fec19ade0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 -- 172.21.15.55:0/2211655708 >> 172.21.15.55:0/2211655708 conn(0x7f9fec0fdf00 msgr2=0x7f9fec074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 -- 172.21.15.55:0/2211655708 shutdown_connections 2024-04-26T22:07:11.212 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.202+0000 7f9ff459d640 1 -- 172.21.15.55:0/2211655708 wait complete. 2024-04-26T22:07:11.214 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-26T22:07:11.214 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 -- 172.21.15.55:0/3945496357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c104550 msgr2=0x7f5e0c10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/3945496357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c104550 0x7f5e0c10ade0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009ca0 tx=0x7f5e0402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 -- 172.21.15.55:0/3945496357 shutdown_connections 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/3945496357 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c104550 0x7f5e0c10ade0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/3945496357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c103b90 0x7f5e0c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/3945496357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e0c1029a0 0x7f5e0c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 -- 172.21.15.55:0/3945496357 >> 172.21.15.55:0/3945496357 conn(0x7f5e0c0fdf30 msgr2=0x7f5e0c100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 -- 172.21.15.55:0/3945496357 shutdown_connections 2024-04-26T22:07:11.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.315+0000 7f5e11d5f640 1 -- 172.21.15.55:0/3945496357 wait complete. 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e11d5f640 1 Processor -- start 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e11d5f640 1 -- start start 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e11d5f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c1029a0 0x7f5e0c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e11d5f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e0c103b90 0x7f5e0c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e11d5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c104550 0x7f5e0c1a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e11d5f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10e4b0 con 0x7f5e0c1029a0 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e11d5f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10e330 con 0x7f5e0c104550 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e11d5f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10e630 con 0x7f5e0c103b90 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c1029a0 0x7f5e0c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c1029a0 0x7f5e0c19adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53278/0 (socket says 172.21.15.55:53278) 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0b7fe640 1 -- 172.21.15.55:0/1586252334 learned_addr learned my addr 172.21.15.55:0/1586252334 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0bfff640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c104550 0x7f5e0c1a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.320 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0affd640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e0c103b90 0x7f5e0c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0b7fe640 1 -- 172.21.15.55:0/1586252334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e0c103b90 msgr2=0x7f5e0c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0b7fe640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e0c103b90 0x7f5e0c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0b7fe640 1 -- 172.21.15.55:0/1586252334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c104550 msgr2=0x7f5e0c1a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0b7fe640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c104550 0x7f5e0c1a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.316+0000 7f5e0b7fe640 1 -- 172.21.15.55:0/1586252334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c1a29e0 con 0x7f5e0c1029a0 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.317+0000 7f5e0b7fe640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c1029a0 0x7f5e0c19adc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00e9c0 tx=0x7f5dfc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.317+0000 7f5e08ff9640 1 -- 172.21.15.55:0/1586252334 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00cd80 con 0x7f5e0c1029a0 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.317+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c1a2cd0 con 0x7f5e0c1029a0 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.317+0000 7f5e08ff9640 1 -- 172.21.15.55:0/1586252334 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc0035f0 con 0x7f5e0c1029a0 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.317+0000 7f5e08ff9640 1 -- 172.21.15.55:0/1586252334 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc010690 con 0x7f5e0c1029a0 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.317+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a31d0 con 0x7f5e0c1029a0 2024-04-26T22:07:11.321 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.318+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c1a3400 con 0x7f5e0c1029a0 2024-04-26T22:07:11.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.319+0000 7f5e08ff9640 1 -- 172.21.15.55:0/1586252334 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5dfc0037a0 con 0x7f5e0c1029a0 2024-04-26T22:07:11.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.319+0000 7f5e08ff9640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5df40a3c40 0x7f5df40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.320+0000 7f5e0affd640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5df40a3c40 0x7f5df40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.322 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.320+0000 7f5e08ff9640 1 -- 172.21.15.55:0/1586252334 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5dfc0d0a40 con 0x7f5e0c1029a0 2024-04-26T22:07:11.323 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.320+0000 7f5e0affd640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5df40a3c40 0x7f5df40a6100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c19c2d0 tx=0x7f5e00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.325 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.324+0000 7f5e08ff9640 1 -- 172.21.15.55:0/1586252334 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5dfc09d220 con 0x7f5e0c1029a0 2024-04-26T22:07:11.550 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.549+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e0c1a35d0 con 0x7f5e0c1029a0 2024-04-26T22:07:11.551 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.550+0000 7f5e08ff9640 1 -- 172.21.15.55:0/1586252334 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7f5dfc09cb20 con 0x7f5e0c1029a0 2024-04-26T22:07:11.551 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:07:11.551 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T22:07:01.422175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":244,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24509,24551,24502]},"id":1}]} 2024-04-26T22:07:11.552 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 25 2024-04-26T22:07:11.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5df40a3c40 msgr2=0x7f5df40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5df40a3c40 0x7f5df40a6100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c19c2d0 tx=0x7f5e00005f90 comp rx=0 tx=0).stop 2024-04-26T22:07:11.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c1029a0 msgr2=0x7f5e0c19adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c1029a0 0x7f5e0c19adc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00e9c0 tx=0x7f5dfc00ee80 comp rx=0 tx=0).stop 2024-04-26T22:07:11.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 shutdown_connections 2024-04-26T22:07:11.553 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5df40a3c40 0x7f5df40a6100 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c104550 0x7f5e0c1a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e0c103b90 0x7f5e0c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 --2- 172.21.15.55:0/1586252334 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5e0c1029a0 0x7f5e0c19adc0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 >> 172.21.15.55:0/1586252334 conn(0x7f5e0c0fdf30 msgr2=0x7f5e0c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 shutdown_connections 2024-04-26T22:07:11.554 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.552+0000 7f5e11d5f640 1 -- 172.21.15.55:0/1586252334 wait complete. 2024-04-26T22:07:11.565 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-26T22:07:11.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 -- 172.21.15.55:0/1439222195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830103440 msgr2=0x7f4830109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 --2- 172.21.15.55:0/1439222195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830103440 0x7f4830109cd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4828009ca0 tx=0x7f482802a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 -- 172.21.15.55:0/1439222195 shutdown_connections 2024-04-26T22:07:11.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 --2- 172.21.15.55:0/1439222195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830103440 0x7f4830109cd0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 --2- 172.21.15.55:0/1439222195 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4830102a80 0x7f4830102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.672 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 --2- 172.21.15.55:0/1439222195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830101890 0x7f4830101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 -- 172.21.15.55:0/1439222195 >> 172.21.15.55:0/1439222195 conn(0x7f48300fce20 msgr2=0x7f48300ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 -- 172.21.15.55:0/1439222195 shutdown_connections 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.668+0000 7f4837f14640 1 -- 172.21.15.55:0/1439222195 wait complete. 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4837f14640 1 Processor -- start 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4837f14640 1 -- start start 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4837f14640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4830101890 0x7f4830198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4837f14640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830102a80 0x7f4830199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4837f14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103440 0x7f48301a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4837f14640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f483010d460 con 0x7f4830102a80 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4837f14640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f483010d2e0 con 0x7f4830103440 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4837f14640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f483010d5e0 con 0x7f4830101890 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48354af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830102a80 0x7f4830199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48354af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830102a80 0x7f4830199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53286/0 (socket says 172.21.15.55:53286) 2024-04-26T22:07:11.673 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48354af640 1 -- 172.21.15.55:0/1829671744 learned_addr learned my addr 172.21.15.55:0/1829671744 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48364b1640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103440 0x7f48301a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4835cb0640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4830101890 0x7f4830198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48354af640 1 -- 172.21.15.55:0/1829671744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4830101890 msgr2=0x7f4830198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48354af640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4830101890 0x7f4830198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48354af640 1 -- 172.21.15.55:0/1829671744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103440 msgr2=0x7f48301a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48354af640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103440 0x7f48301a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48354af640 1 -- 172.21.15.55:0/1829671744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48301a0830 con 0x7f4830102a80 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f48364b1640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103440 0x7f48301a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.669+0000 7f4835cb0640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4830101890 0x7f4830198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.670+0000 7f48354af640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830102a80 0x7f4830199160 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f482000d770 tx=0x7f482000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.670+0000 7f4826ffd640 1 -- 172.21.15.55:0/1829671744 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820017070 con 0x7f4830102a80 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.670+0000 7f4826ffd640 1 -- 172.21.15.55:0/1829671744 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48200054d0 con 0x7f4830102a80 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.670+0000 7f4837f14640 1 -- 172.21.15.55:0/1829671744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48301a0b20 con 0x7f4830102a80 2024-04-26T22:07:11.674 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.670+0000 7f4826ffd640 1 -- 172.21.15.55:0/1829671744 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820004c30 con 0x7f4830102a80 2024-04-26T22:07:11.675 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.670+0000 7f4837f14640 1 -- 172.21.15.55:0/1829671744 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48301a1070 con 0x7f4830102a80 2024-04-26T22:07:11.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f4826ffd640 1 -- 172.21.15.55:0/1829671744 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4820004e10 con 0x7f4830102a80 2024-04-26T22:07:11.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f483ac2f740 1 Processor -- start 2024-04-26T22:07:11.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f483ac2f740 1 -- start start 2024-04-26T22:07:11.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.681 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe61f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 0x55dfe61fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f483ac2f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dfe6157d80 con 0x55dfe61f3820 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f483ac2f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dfe6111a20 con 0x55dfe61f53b0 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.672+0000 7f483ac2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfe6153d40 con 0x55dfe61f49f0 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53294/0 (socket says 172.21.15.55:53294) 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 -- 172.21.15.55:0/3629502542 learned_addr learned my addr 172.21.15.55:0/3629502542 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48354af640 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 0x55dfe61fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f4835cb0640 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe61f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f4826ffd640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f481c0a3cc0 0x7f481c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 -- 172.21.15.55:0/3629502542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 msgr2=0x55dfe61f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe61f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 -- 172.21.15.55:0/3629502542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 msgr2=0x55dfe61fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.682 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 0x55dfe61fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 -- 172.21.15.55:0/3629502542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe611cc30 con 0x55dfe61f3820 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f4835cb0640 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe61f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f4826ffd640 1 -- 172.21.15.55:0/1829671744 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f48200d01b0 con 0x7f4830102a80 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f481c0a3cc0 0x7f481c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f48364b1640 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61f3c40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f48280099e0 tx=0x7f4828009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8964673b17db6087 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f481bfff640 1 -- 172.21.15.55:0/3629502542 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828004100 con 0x55dfe61f3820 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f481bfff640 1 -- 172.21.15.55:0/3629502542 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4828036040 con 0x55dfe61f3820 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.673+0000 7f481bfff640 1 -- 172.21.15.55:0/3629502542 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482802fc90 con 0x55dfe61f3820 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 -- 172.21.15.55:0/3629502542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 msgr2=0x55dfe61f3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f48364b1640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f481c0a3cc0 0x7f481c0a6180 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f482804c000 tx=0x7f4828049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61f3c40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f48280099e0 tx=0x7f4828009a10 comp rx=0 tx=0).stop 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 -- 172.21.15.55:0/3629502542 shutdown_connections 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 0x55dfe61fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.683 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe61f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3629502542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61f3c40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 -- 172.21.15.55:0/3629502542 >> 172.21.15.55:0/3629502542 conn(0x55dfe6134910 msgr2=0x55dfe61ef120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 -- 172.21.15.55:0/3629502542 shutdown_connections 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.674+0000 7f483ac2f740 1 -- 172.21.15.55:0/3629502542 wait complete. 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 1 -- start start 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 1 Processor -- start 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f3820 0x55dfe61fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f49f0 0x55dfe61fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f53b0 0x55dfe6204ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55dfe61fc240 con 0x55dfe61f53b0 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dfe60dedc0 con 0x55dfe61f49f0 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfe61fc3c0 con 0x55dfe61f3820 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f483ac2f740 10 client.0 _ll_register_callbacks cb 0x55dfe60fa600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:07:11.684 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f48364b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f53b0 0x55dfe6204ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f48364b1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f53b0 0x55dfe6204ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53306/0 (socket says 172.21.15.55:53306) 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f48364b1640 1 -- 172.21.15.55:0/2837947499 learned_addr learned my addr 172.21.15.55:0/2837947499 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f4835cb0640 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f3820 0x55dfe61fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.675+0000 7f48354af640 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f49f0 0x55dfe61fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48364b1640 1 -- 172.21.15.55:0/2837947499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f3820 msgr2=0x55dfe61fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48364b1640 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f3820 0x55dfe61fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48364b1640 1 -- 172.21.15.55:0/2837947499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f49f0 msgr2=0x55dfe61fdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48364b1640 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f49f0 0x55dfe61fdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48364b1640 1 -- 172.21.15.55:0/2837947499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe5e693f0 con 0x55dfe61f53b0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48364b1640 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f53b0 0x55dfe6204ef0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4828004330 tx=0x7f482802fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e833ece7b517e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48197fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48197fa640 1 -- 172.21.15.55:0/2837947499 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482802f660 con 0x55dfe61f53b0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48197fa640 1 -- 172.21.15.55:0/2837947499 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f482804e050 con 0x55dfe61f53b0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f483ac2f740 1 -- 172.21.15.55:0/2837947499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4828036040 con 0x55dfe61f53b0 2024-04-26T22:07:11.685 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48197fa640 1 -- 172.21.15.55:0/2837947499 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4828004720 con 0x55dfe61f53b0 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f483ac2f740 10 client.17709 fetch_fsmap learned FSMap version 25 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f483ac2f740 1 -- 172.21.15.55:0/2837947499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfe5f337d0 con 0x55dfe61f53b0 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48197fa640 1 -- 172.21.15.55:0/2837947499 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4828036040 con 0x55dfe61f53b0 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.676+0000 7f48197fa640 1 -- 172.21.15.55:0/2837947499 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4828052480 con 0x55dfe61f53b0 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f483ac2f740 10 client.17709 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f483ac2f740 10 client.17709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f483ac2f740 20 client.17709 populate_metadata read hostname 'smithi055' 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55dfe6209410 0x55dfe620b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f483ac2f740 4 client.17709 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f483ac2f740 1 -- 172.21.15.55:0/2837947499 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfe6165990 con 0x55dfe6209410 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f4835cb0640 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55dfe6209410 0x55dfe620b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f4835cb0640 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55dfe6209410 0x55dfe620b800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad7a0cf319ead660 server_cookie=e02eeaf5cfcf438e in_seq=0 out_seq=0 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.677+0000 7f48197fa640 10 client.17709 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:07:11.686 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f48197fa640 1 -- 172.21.15.55:0/2837947499 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfe6165990 con 0x55dfe6209410 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f48197fa640 10 client.17709 handle_command_reply: tid=0 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 client.17709 shutdown 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 -- 172.21.15.55:0/2837947499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f53b0 msgr2=0x55dfe6204ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f53b0 0x55dfe6204ef0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4828004330 tx=0x7f482802fdc0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 -- 172.21.15.55:0/2837947499 shutdown_connections 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55dfe6209410 0x55dfe620b800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f53b0 0x55dfe6204ef0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f49f0 0x55dfe61fdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2837947499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f3820 0x55dfe61fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f48197fa640 0 client.17709 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 -- 172.21.15.55:0/2837947499 >> 172.21.15.55:0/2837947499 conn(0x55dfe6134910 msgr2=0x55dfe61ee280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 -- 172.21.15.55:0/2837947499 shutdown_connections 2024-04-26T22:07:11.687 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 1 -- 172.21.15.55:0/2837947499 wait complete. 2024-04-26T22:07:11.688 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.678+0000 7f483ac2f740 20 client.17709 trim_cache size 0 max 16384 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f483ac2f740 1 Processor -- start 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f483ac2f740 1 -- start start 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe6204eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 0x55dfe61f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f483ac2f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48301ac740 con 0x55dfe61f3820 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f483ac2f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48301aa010 con 0x55dfe61f53b0 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f483ac2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48301a7e60 con 0x55dfe61f49f0 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61ea0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53314/0 (socket says 172.21.15.55:53314) 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 -- 172.21.15.55:0/2335748762 learned_addr learned my addr 172.21.15.55:0/2335748762 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f48354af640 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe6204eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.695 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f48364b1640 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 0x55dfe61f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 -- 172.21.15.55:0/2335748762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 msgr2=0x55dfe6204eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe6204eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 -- 172.21.15.55:0/2335748762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 msgr2=0x55dfe61f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 0x55dfe61f1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 -- 172.21.15.55:0/2335748762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfe61ea8b0 con 0x55dfe61f3820 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.692+0000 7f4835cb0640 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61ea0e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f482c00ca80 tx=0x7f482c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2a740891a6969c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f481b7fe640 1 -- 172.21.15.55:0/2335748762 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c01b310 con 0x55dfe61f3820 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f481b7fe640 1 -- 172.21.15.55:0/2335748762 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482c01f030 con 0x55dfe61f3820 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f481b7fe640 1 -- 172.21.15.55:0/2335748762 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c014b10 con 0x55dfe61f3820 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 -- 172.21.15.55:0/2335748762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 msgr2=0x55dfe61ea0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61ea0e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f482c00ca80 tx=0x7f482c008040 comp rx=0 tx=0).stop 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 -- 172.21.15.55:0/2335748762 shutdown_connections 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f53b0 0x55dfe61f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.696 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f49f0 0x55dfe6204eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 --2- 172.21.15.55:0/2335748762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f3820 0x55dfe61ea0e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 -- 172.21.15.55:0/2335748762 >> 172.21.15.55:0/2335748762 conn(0x55dfe6134910 msgr2=0x55dfe61fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 -- 172.21.15.55:0/2335748762 shutdown_connections 2024-04-26T22:07:11.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 -- 172.21.15.55:0/2335748762 wait complete. 2024-04-26T22:07:11.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.693+0000 7f483ac2f740 1 -- start start 2024-04-26T22:07:11.697 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f483ac2f740 1 Processor -- start 2024-04-26T22:07:11.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f3820 0x55dfe62023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.700 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f49f0 0x55dfe6202920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f483ac2f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f53b0 0x55dfe6203930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f483ac2f740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f48301a84d0 con 0x55dfe61f49f0 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f483ac2f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48301a9940 con 0x55dfe61f3820 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f483ac2f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f48301a8ba0 con 0x55dfe61f53b0 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f483ac2f740 10 client.0 _ll_register_callbacks cb 0x7f482804b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48354af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f49f0 0x55dfe6202920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48354af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f49f0 0x55dfe6202920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53330/0 (socket says 172.21.15.55:53330) 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48354af640 1 -- 172.21.15.55:0/3359223465 learned_addr learned my addr 172.21.15.55:0/3359223465 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48364b1640 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f53b0 0x55dfe6203930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48354af640 1 -- 172.21.15.55:0/3359223465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f53b0 msgr2=0x55dfe6203930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f4835cb0640 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f3820 0x55dfe62023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48354af640 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f53b0 0x55dfe6203930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.701 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48354af640 1 -- 172.21.15.55:0/3359223465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f3820 msgr2=0x55dfe62023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48354af640 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f3820 0x55dfe62023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.694+0000 7f48354af640 1 -- 172.21.15.55:0/3359223465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfe6203f20 con 0x55dfe61f49f0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f48354af640 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f49f0 0x55dfe6202920 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4820009610 tx=0x7f482009be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563a01e89bbfe5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f48197fa640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f48197fa640 1 -- 172.21.15.55:0/3359223465 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820020040 con 0x55dfe61f49f0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f48197fa640 1 -- 172.21.15.55:0/3359223465 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f48200a4a70 con 0x55dfe61f49f0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 1 -- 172.21.15.55:0/3359223465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482c01f030 con 0x55dfe61f49f0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f48197fa640 1 -- 172.21.15.55:0/3359223465 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48200ba050 con 0x55dfe61f49f0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 10 client.17715 fetch_fsmap learned FSMap version 25 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 1 -- 172.21.15.55:0/3359223465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfe6205290 con 0x55dfe61f49f0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f48197fa640 1 -- 172.21.15.55:0/3359223465 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482c01f030 con 0x55dfe61f49f0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f48197fa640 1 -- 172.21.15.55:0/3359223465 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f482009b6e0 con 0x55dfe61f49f0 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 10 client.17715 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 10 client.17715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-26T22:07:11.702 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 20 client.17715 populate_metadata read hostname 'smithi055' 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55dfe62701a0 0x55dfe62725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 4 client.17715 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f483ac2f740 1 -- 172.21.15.55:0/3359223465 --> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfe615f800 con 0x55dfe62701a0 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.695+0000 7f4835cb0640 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55dfe62701a0 0x55dfe62725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f4835cb0640 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55dfe62701a0 0x55dfe62725c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd9f54177819b8b3 server_cookie=d077b4c90420836d in_seq=0 out_seq=0 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f48197fa640 10 client.17715 ms_handle_connect on v2:172.21.15.165:6838/1115440142 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f48197fa640 1 -- 172.21.15.55:0/3359223465 <== mds.2 v2:172.21.15.165:6838/1115440142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfe615f800 con 0x55dfe62701a0 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f48197fa640 10 client.17715 handle_command_reply: tid=0 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 client.17715 shutdown 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 -- 172.21.15.55:0/3359223465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f49f0 msgr2=0x55dfe6202920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f49f0 0x55dfe6202920 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4820009610 tx=0x7f482009be90 comp rx=0 tx=0).stop 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 -- 172.21.15.55:0/3359223465 shutdown_connections 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.165:6838/1115440142,v1:172.21.15.165:6839/1115440142] conn(0x55dfe62701a0 0x55dfe62725c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfe61f53b0 0x55dfe6203930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.703 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f48197fa640 0 client.17715 ms_handle_reset on v2:172.21.15.165:6838/1115440142 2024-04-26T22:07:11.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dfe61f49f0 0x55dfe6202920 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 --2- 172.21.15.55:0/3359223465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dfe61f3820 0x55dfe62023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 -- 172.21.15.55:0/3359223465 >> 172.21.15.55:0/3359223465 conn(0x55dfe6134910 msgr2=0x55dfe61ed9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 -- 172.21.15.55:0/3359223465 shutdown_connections 2024-04-26T22:07:11.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.696+0000 7f483ac2f740 1 -- 172.21.15.55:0/3359223465 wait complete. 2024-04-26T22:07:11.704 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f483ac2f740 20 client.17715 trim_cache size 0 max 16384 2024-04-26T22:07:11.704 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:07:11.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 -- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f481c0a3cc0 msgr2=0x7f481c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f481c0a3cc0 0x7f481c0a6180 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f482804c000 tx=0x7f4828049000 comp rx=0 tx=0).stop 2024-04-26T22:07:11.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 -- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830102a80 msgr2=0x7f4830199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830102a80 0x7f4830199160 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f482000d770 tx=0x7f482000dc30 comp rx=0 tx=0).stop 2024-04-26T22:07:11.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 -- 172.21.15.55:0/1829671744 shutdown_connections 2024-04-26T22:07:11.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f481c0a3cc0 0x7f481c0a6180 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.705 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4830103440 0x7f48301a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4830102a80 0x7f4830199160 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 --2- 172.21.15.55:0/1829671744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4830101890 0x7f4830198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 -- 172.21.15.55:0/1829671744 >> 172.21.15.55:0/1829671744 conn(0x7f48300fce20 msgr2=0x7f48300fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 -- 172.21.15.55:0/1829671744 shutdown_connections 2024-04-26T22:07:11.706 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.697+0000 7f4837f14640 1 -- 172.21.15.55:0/1829671744 wait complete. 2024-04-26T22:07:11.709 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-26T22:07:11.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 -- 172.21.15.55:0/2465170286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954104b90 msgr2=0x7f9954104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 --2- 172.21.15.55:0/2465170286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954104b90 0x7f9954104fb0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9948009900 tx=0x7f994802a410 comp rx=0 tx=0).stop 2024-04-26T22:07:11.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 -- 172.21.15.55:0/2465170286 shutdown_connections 2024-04-26T22:07:11.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 --2- 172.21.15.55:0/2465170286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9954106740 0x7f995410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 --2- 172.21.15.55:0/2465170286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9954105d80 0x7f9954106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 --2- 172.21.15.55:0/2465170286 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954104b90 0x7f9954104fb0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 -- 172.21.15.55:0/2465170286 >> 172.21.15.55:0/2465170286 conn(0x7f9954100120 msgr2=0x7f9954102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.817 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 -- 172.21.15.55:0/2465170286 shutdown_connections 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.812+0000 7f995a6bd640 1 -- 172.21.15.55:0/2465170286 wait complete. 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f995a6bd640 1 Processor -- start 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f995a6bd640 1 -- start start 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f995a6bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9954104b90 0x7f995419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f995a6bd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9954105d80 0x7f995419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f995a6bd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954106740 0x7f99541a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f995a6bd640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f99541106b0 con 0x7f9954106740 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f995a6bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9954110530 con 0x7f9954104b90 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f995a6bd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9954110830 con 0x7f9954105d80 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f9958c5a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954106740 0x7f99541a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f9958c5a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954106740 0x7f99541a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53348/0 (socket says 172.21.15.55:53348) 2024-04-26T22:07:11.818 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f9958c5a640 1 -- 172.21.15.55:0/1519219205 learned_addr learned my addr 172.21.15.55:0/1519219205 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.813+0000 7f99537fe640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9954105d80 0x7f995419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f9953fff640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9954104b90 0x7f995419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f9958c5a640 1 -- 172.21.15.55:0/1519219205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9954105d80 msgr2=0x7f995419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f9958c5a640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9954105d80 0x7f995419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f9958c5a640 1 -- 172.21.15.55:0/1519219205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9954104b90 msgr2=0x7f995419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f9958c5a640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9954104b90 0x7f995419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f9958c5a640 1 -- 172.21.15.55:0/1519219205 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99541a2a20 con 0x7f9954106740 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f9953fff640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9954104b90 0x7f995419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f9958c5a640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954106740 0x7f99541a23b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f994c00eaa0 tx=0x7f994c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f99517fa640 1 -- 172.21.15.55:0/1519219205 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c00cdb0 con 0x7f9954106740 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f995a6bd640 1 -- 172.21.15.55:0/1519219205 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99541a2d10 con 0x7f9954106740 2024-04-26T22:07:11.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f99517fa640 1 -- 172.21.15.55:0/1519219205 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994c005020 con 0x7f9954106740 2024-04-26T22:07:11.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f99517fa640 1 -- 172.21.15.55:0/1519219205 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c0116d0 con 0x7f9954106740 2024-04-26T22:07:11.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.814+0000 7f995a6bd640 1 -- 172.21.15.55:0/1519219205 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99541a31b0 con 0x7f9954106740 2024-04-26T22:07:11.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.816+0000 7f99517fa640 1 -- 172.21.15.55:0/1519219205 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f994c0051d0 con 0x7f9954106740 2024-04-26T22:07:11.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.816+0000 7f995d3cf740 1 Processor -- start 2024-04-26T22:07:11.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f995d3cf740 1 -- start start 2024-04-26T22:07:11.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66ba820 0x55e3b66bac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66bb9f0 0x55e3b66bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f995d3cf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e3b661ed80 con 0x55e3b66ba820 2024-04-26T22:07:11.827 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f995d3cf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e3b65d8a20 con 0x55e3b66bb9f0 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f995d3cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e3b661ad40 con 0x55e3b66bc3b0 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66ba820 0x55e3b66bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66ba820 0x55e3b66bac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53356/0 (socket says 172.21.15.55:53356) 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 -- 172.21.15.55:0/2854416410 learned_addr learned my addr 172.21.15.55:0/2854416410 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f9953fff640 1 --2- 172.21.15.55:0/2854416410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66bb9f0 0x55e3b66bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99517fa640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f993c0a3c40 0x7f993c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 -- 172.21.15.55:0/2854416410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 msgr2=0x55e3b66c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 --2- 172.21.15.55:0/2854416410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 -- 172.21.15.55:0/2854416410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66bb9f0 msgr2=0x55e3b66bbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 --2- 172.21.15.55:0/2854416410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66bb9f0 0x55e3b66bbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 -- 172.21.15.55:0/2854416410 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b65e3c30 con 0x55e3b66ba820 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f9958c5a640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f993c0a3c40 0x7f993c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99537fe640 1 --2- 172.21.15.55:0/2854416410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66ba820 0x55e3b66bac40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d40 tx=0x7f9944013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada533bdbb41021 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.828 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.817+0000 7f99517fa640 1 -- 172.21.15.55:0/1519219205 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f994c0d07d0 con 0x7f9954106740 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f993a7fc640 1 -- 172.21.15.55:0/2854416410 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944004050 con 0x55e3b66ba820 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f993a7fc640 1 -- 172.21.15.55:0/2854416410 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9944017030 con 0x55e3b66ba820 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f993a7fc640 1 -- 172.21.15.55:0/2854416410 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944010630 con 0x55e3b66ba820 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 -- 172.21.15.55:0/2854416410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66ba820 msgr2=0x55e3b66bac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f9958c5a640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f993c0a3c40 0x7f993c0a6100 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f994c098d70 tx=0x7f994c0c7a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 --2- 172.21.15.55:0/2854416410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66ba820 0x55e3b66bac40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9944009d40 tx=0x7f9944013830 comp rx=0 tx=0).stop 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 -- 172.21.15.55:0/2854416410 shutdown_connections 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 --2- 172.21.15.55:0/2854416410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 --2- 172.21.15.55:0/2854416410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66bb9f0 0x55e3b66bbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 --2- 172.21.15.55:0/2854416410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66ba820 0x55e3b66bac40 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 -- 172.21.15.55:0/2854416410 >> 172.21.15.55:0/2854416410 conn(0x55e3b65fb910 msgr2=0x55e3b66b6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 -- 172.21.15.55:0/2854416410 shutdown_connections 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.818+0000 7f995d3cf740 1 -- 172.21.15.55:0/2854416410 wait complete. 2024-04-26T22:07:11.829 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 1 -- start start 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 1 Processor -- start 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x55e3b66c33c0 con 0x55e3b66bb9f0 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e3b65a5dc0 con 0x55e3b66ba820 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e3b66c3540 con 0x55e3b66bc3b0 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f995d3cf740 10 client.0 _ll_register_callbacks cb 0x55e3b65c1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f9953fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f9953fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53358/0 (socket says 172.21.15.55:53358) 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f9953fff640 1 -- 172.21.15.55:0/3962497144 learned_addr learned my addr 172.21.15.55:0/3962497144 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f99537fe640 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.830 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.819+0000 7f99537fe640 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f9953fff640 1 -- 172.21.15.55:0/3962497144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 msgr2=0x55e3b66cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f9953fff640 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f9953fff640 1 -- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 msgr2=0x55e3b66c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f9953fff640 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f9953fff640 1 -- 172.21.15.55:0/3962497144 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b63303f0 con 0x55e3b66bb9f0 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f99537fe640 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f9953fff640 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c4fb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9948009900 tx=0x7f9948032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382da23d1736d98e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f99337fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f99337fe640 1 -- 172.21.15.55:0/3962497144 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948040070 con 0x55e3b66bb9f0 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f99337fe640 1 -- 172.21.15.55:0/3962497144 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9948004040 con 0x55e3b66bb9f0 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f995d3cf740 1 -- 172.21.15.55:0/3962497144 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9944017030 con 0x55e3b66bb9f0 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f99337fe640 1 -- 172.21.15.55:0/3962497144 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9948035040 con 0x55e3b66bb9f0 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f995d3cf740 10 client.17727 fetch_fsmap learned FSMap version 25 2024-04-26T22:07:11.831 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f995d3cf740 1 -- 172.21.15.55:0/3962497144 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3b63fa7d0 con 0x55e3b66bb9f0 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f99337fe640 1 -- 172.21.15.55:0/3962497144 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9944017030 con 0x55e3b66bb9f0 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f99337fe640 1 -- 172.21.15.55:0/3962497144 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f99480048c0 con 0x55e3b66bb9f0 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f995d3cf740 10 client.17727 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.820+0000 7f995d3cf740 10 client.17727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.821+0000 7f995d3cf740 20 client.17727 populate_metadata read hostname 'smithi055' 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.821+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55e3b66d08b0 0x55e3b66d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.821+0000 7f995d3cf740 4 client.17727 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.821+0000 7f995d3cf740 1 -- 172.21.15.55:0/3962497144 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b662c990 con 0x55e3b66d08b0 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.821+0000 7f99537fe640 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55e3b66d08b0 0x55e3b66d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.821+0000 7f99337fe640 10 client.17727 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.821+0000 7f99537fe640 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55e3b66d08b0 0x55e3b66d2d30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b70057fc13274cc server_cookie=b03896dfe5950bf in_seq=0 out_seq=0 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f99337fe640 1 -- 172.21.15.55:0/3962497144 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3b662c990 con 0x55e3b66d08b0 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f99337fe640 10 client.17727 handle_command_reply: tid=0 2024-04-26T22:07:11.832 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 client.17727 shutdown 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 -- 172.21.15.55:0/3962497144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 msgr2=0x55e3b66c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c4fb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9948009900 tx=0x7f9948032dc0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 -- 172.21.15.55:0/3962497144 shutdown_connections 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55e3b66d08b0 0x55e3b66d2d30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f99337fe640 0 client.17727 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c4fb0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3962497144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 -- 172.21.15.55:0/3962497144 >> 172.21.15.55:0/3962497144 conn(0x55e3b65fb910 msgr2=0x55e3b66b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 -- 172.21.15.55:0/3962497144 shutdown_connections 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 1 -- 172.21.15.55:0/3962497144 wait complete. 2024-04-26T22:07:11.833 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.822+0000 7f995d3cf740 20 client.17727 trim_cache size 0 max 16384 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f995d3cf740 1 Processor -- start 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f995d3cf740 1 -- start start 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f995d3cf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f99541ae9c0 con 0x55e3b66bb9f0 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f995d3cf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99541ac290 con 0x55e3b66ba820 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f995d3cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99541aa0e0 con 0x55e3b66bc3b0 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f9953fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f9953fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53368/0 (socket says 172.21.15.55:53368) 2024-04-26T22:07:11.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f9953fff640 1 -- 172.21.15.55:0/2505249457 learned_addr learned my addr 172.21.15.55:0/2505249457 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f99537fe640 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f99537fe640 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f9953fff640 1 -- 172.21.15.55:0/2505249457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 msgr2=0x55e3b66cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f9953fff640 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f9953fff640 1 -- 172.21.15.55:0/2505249457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 msgr2=0x55e3b66b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f9953fff640 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f9953fff640 1 -- 172.21.15.55:0/2505249457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b66bfae0 con 0x55e3b66bb9f0 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.836+0000 7f99537fe640 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f9953fff640 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66b5640 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f994802a450 tx=0x7f99480095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21908a15aa719071 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f9939ffb640 1 -- 172.21.15.55:0/2505249457 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948040070 con 0x55e3b66bb9f0 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f9939ffb640 1 -- 172.21.15.55:0/2505249457 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9948031870 con 0x55e3b66bb9f0 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f9939ffb640 1 -- 172.21.15.55:0/2505249457 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948005230 con 0x55e3b66bb9f0 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f995d3cf740 1 -- 172.21.15.55:0/2505249457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 msgr2=0x55e3b66b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.844 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f995d3cf740 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66b5640 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f994802a450 tx=0x7f99480095f0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f995d3cf740 1 -- 172.21.15.55:0/2505249457 shutdown_connections 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f995d3cf740 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f995d3cf740 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66b5640 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f995d3cf740 1 --2- 172.21.15.55:0/2505249457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.837+0000 7f995d3cf740 1 -- 172.21.15.55:0/2505249457 >> 172.21.15.55:0/2505249457 conn(0x55e3b65fb910 msgr2=0x55e3b66b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 -- 172.21.15.55:0/2505249457 shutdown_connections 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 -- 172.21.15.55:0/2505249457 wait complete. 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 -- start start 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 Processor -- start 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f99541aa750 con 0x55e3b66bb9f0 2024-04-26T22:07:11.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99541abbc0 con 0x55e3b66ba820 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99541aae20 con 0x55e3b66bc3b0 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f995d3cf740 10 client.0 _ll_register_callbacks cb 0x7f9948009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f9953fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f9953fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53382/0 (socket says 172.21.15.55:53382) 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.838+0000 7f9953fff640 1 -- 172.21.15.55:0/3439807335 learned_addr learned my addr 172.21.15.55:0/3439807335 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f99537fe640 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f99537fe640 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f9953fff640 1 -- 172.21.15.55:0/3439807335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 msgr2=0x55e3b66cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f9953fff640 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f9953fff640 1 -- 172.21.15.55:0/3439807335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 msgr2=0x55e3b66c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f9953fff640 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f9953fff640 1 -- 172.21.15.55:0/3439807335 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b66cb240 con 0x55e3b66bb9f0 2024-04-26T22:07:11.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f99537fe640 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f9953fff640 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c9c40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9948004ea0 tx=0x7f9948009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25eb45765a8e204d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f99337fe640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f99337fe640 1 -- 172.21.15.55:0/3439807335 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948040070 con 0x55e3b66bb9f0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f99337fe640 1 -- 172.21.15.55:0/3439807335 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f99480050f0 con 0x55e3b66bb9f0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f995d3cf740 1 -- 172.21.15.55:0/3439807335 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9948031870 con 0x55e3b66bb9f0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f99337fe640 1 -- 172.21.15.55:0/3439807335 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99480052f0 con 0x55e3b66bb9f0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f995d3cf740 10 client.17733 fetch_fsmap learned FSMap version 25 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.839+0000 7f995d3cf740 1 -- 172.21.15.55:0/3439807335 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3b6736e70 con 0x55e3b66bb9f0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f99337fe640 1 -- 172.21.15.55:0/3439807335 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9948031870 con 0x55e3b66bb9f0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f99337fe640 1 -- 172.21.15.55:0/3439807335 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9948060060 con 0x55e3b66bb9f0 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f995d3cf740 10 client.17733 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f995d3cf740 10 client.17733 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f995d3cf740 20 client.17733 populate_metadata read hostname 'smithi055' 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55e3b67376e0 0x55e3b6739b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.847 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f995d3cf740 4 client.17733 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f995d3cf740 1 -- 172.21.15.55:0/3439807335 --> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3b6626800 con 0x55e3b67376e0 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f99537fe640 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55e3b67376e0 0x55e3b6739b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f99337fe640 10 client.17733 ms_handle_connect on v2:172.21.15.64:6836/4014142220 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.840+0000 7f99537fe640 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55e3b67376e0 0x55e3b6739b00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=938b2bdaddaee63 server_cookie=cd0e6fbe295de9bf in_seq=0 out_seq=0 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f99337fe640 1 -- 172.21.15.55:0/3439807335 <== mds.1 v2:172.21.15.64:6836/4014142220 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3b6626800 con 0x55e3b67376e0 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f99337fe640 10 client.17733 handle_command_reply: tid=0 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 client.17733 shutdown 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 -- 172.21.15.55:0/3439807335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 msgr2=0x55e3b66c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c9c40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9948004ea0 tx=0x7f9948009660 comp rx=0 tx=0).stop 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 -- 172.21.15.55:0/3439807335 shutdown_connections 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.64:6836/4014142220,v1:172.21.15.64:6837/4014142220] conn(0x55e3b67376e0 0x55e3b6739b00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e3b66bc3b0 0x55e3b66cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55e3b66bb9f0 0x55e3b66c9c40 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.848 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f99337fe640 0 client.17733 ms_handle_reset on v2:172.21.15.64:6836/4014142220 2024-04-26T22:07:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 --2- 172.21.15.55:0/3439807335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e3b66ba820 0x55e3b66c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 -- 172.21.15.55:0/3439807335 >> 172.21.15.55:0/3439807335 conn(0x55e3b65fb910 msgr2=0x55e3b66b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 -- 172.21.15.55:0/3439807335 shutdown_connections 2024-04-26T22:07:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 1 -- 172.21.15.55:0/3439807335 wait complete. 2024-04-26T22:07:11.849 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995d3cf740 20 client.17733 trim_cache size 0 max 16384 2024-04-26T22:07:11.849 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 -- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f993c0a3c40 msgr2=0x7f993c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f993c0a3c40 0x7f993c0a6100 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f994c098d70 tx=0x7f994c0c7a20 comp rx=0 tx=0).stop 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 -- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954106740 msgr2=0x7f99541a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954106740 0x7f99541a23b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f994c00eaa0 tx=0x7f994c00ef60 comp rx=0 tx=0).stop 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 -- 172.21.15.55:0/1519219205 shutdown_connections 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f993c0a3c40 0x7f993c0a6100 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9954106740 0x7f99541a23b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9954105d80 0x7f995419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.850 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 --2- 172.21.15.55:0/1519219205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9954104b90 0x7f995419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.841+0000 7f995a6bd640 1 -- 172.21.15.55:0/1519219205 >> 172.21.15.55:0/1519219205 conn(0x7f9954100120 msgr2=0x7f99541091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.842+0000 7f995a6bd640 1 -- 172.21.15.55:0/1519219205 shutdown_connections 2024-04-26T22:07:11.851 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.842+0000 7f995a6bd640 1 -- 172.21.15.55:0/1519219205 wait complete. 2024-04-26T22:07:11.852 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:11 smithi055 ceph-mon[29221]: pgmap v779: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 27 MiB/s wr, 426 op/s 2024-04-26T22:07:11.853 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:11 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1586252334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:11.854 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-26T22:07:11.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 -- 172.21.15.55:0/2074339515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540103b80 msgr2=0x7f5540104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 --2- 172.21.15.55:0/2074339515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540103b80 0x7f5540104000 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5530009900 tx=0x7f553002a410 comp rx=0 tx=0).stop 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 -- 172.21.15.55:0/2074339515 shutdown_connections 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 --2- 172.21.15.55:0/2074339515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540104540 0x7f554010add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 --2- 172.21.15.55:0/2074339515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540103b80 0x7f5540104000 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 --2- 172.21.15.55:0/2074339515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5540102990 0x7f5540102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 -- 172.21.15.55:0/2074339515 >> 172.21.15.55:0/2074339515 conn(0x7f55400fdf00 msgr2=0x7f5540100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 -- 172.21.15.55:0/2074339515 shutdown_connections 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.957+0000 7f554843f640 1 -- 172.21.15.55:0/2074339515 wait complete. 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f554843f640 1 Processor -- start 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f554843f640 1 -- start start 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f554843f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5540102990 0x7f5540198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f554843f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540103b80 0x7f5540199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f554843f640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540104540 0x7f55401a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f554843f640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f554010e4c0 con 0x7f5540104540 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f554843f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f554010e340 con 0x7f5540103b80 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f554843f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f554010e640 con 0x7f5540102990 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f55469dc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540104540 0x7f55401a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f55469dc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540104540 0x7f55401a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53402/0 (socket says 172.21.15.55:53402) 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f55469dc640 1 -- 172.21.15.55:0/658521565 learned_addr learned my addr 172.21.15.55:0/658521565 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f55459da640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540103b80 0x7f5540199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.958+0000 7f55461db640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5540102990 0x7f5540198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55469dc640 1 -- 172.21.15.55:0/658521565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5540102990 msgr2=0x7f5540198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55469dc640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5540102990 0x7f5540198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55469dc640 1 -- 172.21.15.55:0/658521565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540103b80 msgr2=0x7f5540199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55469dc640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540103b80 0x7f5540199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55469dc640 1 -- 172.21.15.55:0/658521565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55401a07e0 con 0x7f5540104540 2024-04-26T22:07:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55469dc640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540104540 0x7f55401a0170 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f553800eaa0 tx=0x7f553800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55377fe640 1 -- 172.21.15.55:0/658521565 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553800cdb0 con 0x7f5540104540 2024-04-26T22:07:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55377fe640 1 -- 172.21.15.55:0/658521565 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5538005020 con 0x7f5540104540 2024-04-26T22:07:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f554843f640 1 -- 172.21.15.55:0/658521565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55401a0ad0 con 0x7f5540104540 2024-04-26T22:07:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f55377fe640 1 -- 172.21.15.55:0/658521565 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55380116d0 con 0x7f5540104540 2024-04-26T22:07:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.959+0000 7f554843f640 1 -- 172.21.15.55:0/658521565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55401a0fa0 con 0x7f5540104540 2024-04-26T22:07:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.961+0000 7f55377fe640 1 -- 172.21.15.55:0/658521565 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f55380051d0 con 0x7f5540104540 2024-04-26T22:07:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f554b231740 1 Processor -- start 2024-04-26T22:07:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f554b231740 1 -- start start 2024-04-26T22:07:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f554b231740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cfec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f554b231740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cff9f0 0x564fe0cffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f55377fe640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f552c0a3c40 0x7f552c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f554b231740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d06c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f554b231740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564fe0c62d80 con 0x564fe0d003b0 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f554b231740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fe0c1ca20 con 0x564fe0cff9f0 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f554b231740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564fe0c5ed40 con 0x564fe0cfe820 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f55469dc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d06c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f55459da640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f552c0a3c40 0x7f552c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f55469dc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d06c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53404/0 (socket says 172.21.15.55:53404) 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f55469dc640 1 -- 172.21.15.55:0/3233547364 learned_addr learned my addr 172.21.15.55:0/3233547364 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f55461db640 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cff9f0 0x564fe0cffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f55377fe640 1 -- 172.21.15.55:0/658521565 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f55380d0780 con 0x7f5540104540 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.962+0000 7f55459da640 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cfec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f55469dc640 1 -- 172.21.15.55:0/3233547364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 msgr2=0x564fe0cfec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f55469dc640 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cfec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f55469dc640 1 -- 172.21.15.55:0/3233547364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cff9f0 msgr2=0x564fe0cffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f55469dc640 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cff9f0 0x564fe0cffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f55469dc640 1 -- 172.21.15.55:0/3233547364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe0c27c30 con 0x564fe0d003b0 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f55461db640 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cff9f0 0x564fe0cffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f55459da640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f552c0a3c40 0x7f552c0a6100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5530000c00 tx=0x7f5530009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f55469dc640 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d06c40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f55380992e0 tx=0x7f553809c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977b4f6f634d6ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f5534bfe640 1 -- 172.21.15.55:0/3233547364 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553801f070 con 0x564fe0d003b0 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f5534bfe640 1 -- 172.21.15.55:0/3233547364 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553809c910 con 0x564fe0d003b0 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.963+0000 7f5534bfe640 1 -- 172.21.15.55:0/3233547364 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55380a5280 con 0x564fe0d003b0 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 -- 172.21.15.55:0/3233547364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 msgr2=0x564fe0d06c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d06c40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f55380992e0 tx=0x7f553809c270 comp rx=0 tx=0).stop 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 -- 172.21.15.55:0/3233547364 shutdown_connections 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d06c40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cff9f0 0x564fe0cffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 --2- 172.21.15.55:0/3233547364 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cfec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 -- 172.21.15.55:0/3233547364 >> 172.21.15.55:0/3233547364 conn(0x564fe0c3f910 msgr2=0x564fe0cfa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 -- 172.21.15.55:0/3233547364 shutdown_connections 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 -- 172.21.15.55:0/3233547364 wait complete. 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 -- start start 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.964+0000 7f554b231740 1 Processor -- start 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f554b231740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cfe820 0x564fe0d08af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f554b231740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cff9f0 0x564fe0d09030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f554b231740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d10070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f554b231740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x564fe0d07300 con 0x564fe0d003b0 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f554b231740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fe0be9dc0 con 0x564fe0cfe820 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f554b231740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564fe0d07480 con 0x564fe0cff9f0 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f554b231740 10 client.0 _ll_register_callbacks cb 0x564fe0c05600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d10070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d10070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53420/0 (socket says 172.21.15.55:53420) 2024-04-26T22:07:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 -- 172.21.15.55:0/1590977275 learned_addr learned my addr 172.21.15.55:0/1590977275 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55459da640 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cff9f0 0x564fe0d09030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55461db640 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cfe820 0x564fe0d08af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 -- 172.21.15.55:0/1590977275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cff9f0 msgr2=0x564fe0d09030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cff9f0 0x564fe0d09030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 -- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cfe820 msgr2=0x564fe0d08af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cfe820 0x564fe0d08af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 -- 172.21.15.55:0/1590977275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe09743f0 con 0x564fe0d003b0 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55461db640 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cfe820 0x564fe0d08af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f55469dc640 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d10070 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f553800efd0 tx=0x7f55380a3e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bb01632fc6b55e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f5529ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.965+0000 7f5529ffb640 1 -- 172.21.15.55:0/1590977275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553801f040 con 0x564fe0d003b0 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f5529ffb640 1 -- 172.21.15.55:0/1590977275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f553807ee50 con 0x564fe0d003b0 2024-04-26T22:07:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 1 -- 172.21.15.55:0/1590977275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f553809c910 con 0x564fe0d003b0 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f5529ffb640 1 -- 172.21.15.55:0/1590977275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55380a0c10 con 0x564fe0d003b0 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 10 client.17745 fetch_fsmap learned FSMap version 25 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 1 -- 172.21.15.55:0/1590977275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564fe0a3e7d0 con 0x564fe0d003b0 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f5529ffb640 1 -- 172.21.15.55:0/1590977275 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553809c910 con 0x564fe0d003b0 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f5529ffb640 1 -- 172.21.15.55:0/1590977275 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f553800f040 con 0x564fe0d003b0 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 10 client.17745 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 20 client.17745 populate_metadata read hostname 'smithi055' 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564fe0d14560 0x564fe0d16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 4 client.17745 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f554b231740 1 -- 172.21.15.55:0/1590977275 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fe0c70990 con 0x564fe0d14560 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.966+0000 7f55461db640 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564fe0d14560 0x564fe0d16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.967+0000 7f55461db640 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564fe0d14560 0x564fe0d16950 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5961dbed69d57d6f server_cookie=b968b17d2b195227 in_seq=0 out_seq=0 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.967+0000 7f5529ffb640 10 client.17745 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:07:11.975 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f5529ffb640 1 -- 172.21.15.55:0/1590977275 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fe0c70990 con 0x564fe0d14560 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f5529ffb640 10 client.17745 handle_command_reply: tid=0 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 client.17745 shutdown 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 -- 172.21.15.55:0/1590977275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 msgr2=0x564fe0d10070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d10070 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f553800efd0 tx=0x7f55380a3e30 comp rx=0 tx=0).stop 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 -- 172.21.15.55:0/1590977275 shutdown_connections 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564fe0d14560 0x564fe0d16950 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d003b0 0x564fe0d10070 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f5529ffb640 0 client.17745 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cff9f0 0x564fe0d09030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 --2- 172.21.15.55:0/1590977275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0cfe820 0x564fe0d08af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 -- 172.21.15.55:0/1590977275 >> 172.21.15.55:0/1590977275 conn(0x564fe0c3f910 msgr2=0x564fe0cf9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 -- 172.21.15.55:0/1590977275 shutdown_connections 2024-04-26T22:07:11.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 1 -- 172.21.15.55:0/1590977275 wait complete. 2024-04-26T22:07:11.977 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.968+0000 7f554b231740 20 client.17745 trim_cache size 0 max 16384 2024-04-26T22:07:11.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f554b231740 1 Processor -- start 2024-04-26T22:07:11.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f554b231740 1 -- start start 2024-04-26T22:07:11.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f554b231740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d06d30 0x564fe0cf9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f554b231740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0c3f910 0x564fe0d10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f554b231740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cf9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f554b231740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f55401ac720 con 0x564fe0d06d30 2024-04-26T22:07:11.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f554b231740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a9ff0 con 0x564fe0c3f910 2024-04-26T22:07:11.988 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f554b231740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a7e40 con 0x564fe0cfe820 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d06d30 0x564fe0cf9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d06d30 0x564fe0cf9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53430/0 (socket says 172.21.15.55:53430) 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 -- 172.21.15.55:0/2463746034 learned_addr learned my addr 172.21.15.55:0/2463746034 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55469dc640 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cf9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 -- 172.21.15.55:0/2463746034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 msgr2=0x564fe0cf9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cf9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55459da640 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0c3f910 0x564fe0d10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 -- 172.21.15.55:0/2463746034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0c3f910 msgr2=0x564fe0d10030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0c3f910 0x564fe0d10030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 -- 172.21.15.55:0/2463746034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe0cfadd0 con 0x564fe0d06d30 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55469dc640 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cf9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f55461db640 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d06d30 0x564fe0cf9ba0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f553c005fd0 tx=0x7f553c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebed2ef691be5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f552bfff640 1 -- 172.21.15.55:0/2463746034 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c005220 con 0x564fe0d06d30 2024-04-26T22:07:11.989 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f552bfff640 1 -- 172.21.15.55:0/2463746034 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553c023070 con 0x564fe0d06d30 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.982+0000 7f552bfff640 1 -- 172.21.15.55:0/2463746034 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c01f4a0 con 0x564fe0d06d30 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 -- 172.21.15.55:0/2463746034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d06d30 msgr2=0x564fe0cf9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d06d30 0x564fe0cf9ba0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f553c005fd0 tx=0x7f553c00b390 comp rx=0 tx=0).stop 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 -- 172.21.15.55:0/2463746034 shutdown_connections 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0cfe820 0x564fe0cf9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0c3f910 0x564fe0d10030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 --2- 172.21.15.55:0/2463746034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0d06d30 0x564fe0cf9ba0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 -- 172.21.15.55:0/2463746034 >> 172.21.15.55:0/2463746034 conn(0x564fe0d086b0 msgr2=0x564fe0cf89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 -- 172.21.15.55:0/2463746034 shutdown_connections 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 -- 172.21.15.55:0/2463746034 wait complete. 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 -- start start 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.983+0000 7f554b231740 1 Processor -- start 2024-04-26T22:07:11.990 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f554b231740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0c3f910 0x564fe0d0dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f554b231740 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0cfe820 0x564fe0d0e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f554b231740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0d06d30 0x564fe0d0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f554b231740 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a84b0 con 0x564fe0cfe820 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f554b231740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a9920 con 0x564fe0d06d30 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f554b231740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55401a8b80 con 0x564fe0c3f910 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f554b231740 10 client.0 _ll_register_callbacks cb 0x7f553c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0cfe820 0x564fe0d0e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0cfe820 0x564fe0d0e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53440/0 (socket says 172.21.15.55:53440) 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 -- 172.21.15.55:0/678808081 learned_addr learned my addr 172.21.15.55:0/678808081 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55469dc640 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0d06d30 0x564fe0d0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55461db640 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0c3f910 0x564fe0d0dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 -- 172.21.15.55:0/678808081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0c3f910 msgr2=0x564fe0d0dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0c3f910 0x564fe0d0dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.991 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 -- 172.21.15.55:0/678808081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0d06d30 msgr2=0x564fe0d0f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0d06d30 0x564fe0d0f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 -- 172.21.15.55:0/678808081 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe0d0fad0 con 0x564fe0cfe820 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f55459da640 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0cfe820 0x564fe0d0e520 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5530009900 tx=0x7f5530047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c06ab28039f020 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f5529ffb640 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f5529ffb640 1 -- 172.21.15.55:0/678808081 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530035040 con 0x564fe0cfe820 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f5529ffb640 1 -- 172.21.15.55:0/678808081 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f55300042a0 con 0x564fe0cfe820 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.984+0000 7f554b231740 1 -- 172.21.15.55:0/678808081 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f553c023070 con 0x564fe0cfe820 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f5529ffb640 1 -- 172.21.15.55:0/678808081 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55300044a0 con 0x564fe0cfe820 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f554b231740 10 client.17751 fetch_fsmap learned FSMap version 25 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f554b231740 1 -- 172.21.15.55:0/678808081 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564fe0d7b1a0 con 0x564fe0cfe820 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f5529ffb640 1 -- 172.21.15.55:0/678808081 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553c023070 con 0x564fe0cfe820 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f5529ffb640 1 -- 172.21.15.55:0/678808081 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f553003d870 con 0x564fe0cfe820 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f554b231740 10 client.17751 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T22:07:11.992 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f554b231740 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f554b231740 20 client.17751 populate_metadata read hostname 'smithi055' 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f554b231740 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564fe0cff9f0 0x564fe0d7dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f554b231740 4 client.17751 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f554b231740 1 -- 172.21.15.55:0/678808081 --> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fe0c6a800 con 0x564fe0cff9f0 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.985+0000 7f55461db640 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564fe0cff9f0 0x564fe0d7dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f55461db640 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564fe0cff9f0 0x564fe0d7dc40 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d067a5b947e8d26 server_cookie=40ac8bf1fd7e92e4 in_seq=0 out_seq=0 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f5529ffb640 10 client.17751 ms_handle_connect on v2:172.21.15.64:6838/463662731 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f5529ffb640 1 -- 172.21.15.55:0/678808081 <== mds.0 v2:172.21.15.64:6838/463662731 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fe0c6a800 con 0x564fe0cff9f0 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f5529ffb640 10 client.17751 handle_command_reply: tid=0 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 client.17751 shutdown 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 -- 172.21.15.55:0/678808081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0cfe820 msgr2=0x564fe0d0e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0cfe820 0x564fe0d0e520 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5530009900 tx=0x7f5530047000 comp rx=0 tx=0).stop 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 -- 172.21.15.55:0/678808081 shutdown_connections 2024-04-26T22:07:11.993 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564fe0d06d30 0x564fe0d0f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.64:6838/463662731,v1:172.21.15.64:6839/463662731] conn(0x564fe0cff9f0 0x564fe0d7dc40 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x564fe0cfe820 0x564fe0d0e520 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 --2- 172.21.15.55:0/678808081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564fe0c3f910 0x564fe0d0dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f5529ffb640 0 client.17751 ms_handle_reset on v2:172.21.15.64:6838/463662731 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 -- 172.21.15.55:0/678808081 >> 172.21.15.55:0/678808081 conn(0x564fe0d086b0 msgr2=0x564fe0cfd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 -- 172.21.15.55:0/678808081 shutdown_connections 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 1 -- 172.21.15.55:0/678808081 wait complete. 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554b231740 20 client.17751 trim_cache size 0 max 16384 2024-04-26T22:07:11.994 INFO:teuthology.orchestra.run.smithi055.stdout:[] 2024-04-26T22:07:11.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.986+0000 7f554843f640 1 -- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f552c0a3c40 msgr2=0x7f552c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f552c0a3c40 0x7f552c0a6100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5530000c00 tx=0x7f5530009600 comp rx=0 tx=0).stop 2024-04-26T22:07:11.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 -- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540104540 msgr2=0x7f55401a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:11.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540104540 0x7f55401a0170 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f553800eaa0 tx=0x7f553800ef60 comp rx=0 tx=0).stop 2024-04-26T22:07:11.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 -- 172.21.15.55:0/658521565 shutdown_connections 2024-04-26T22:07:11.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f552c0a3c40 0x7f552c0a6100 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540104540 0x7f55401a0170 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540103b80 0x7f5540199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 --2- 172.21.15.55:0/658521565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5540102990 0x7f5540198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:11.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 -- 172.21.15.55:0/658521565 >> 172.21.15.55:0/658521565 conn(0x7f55400fdf00 msgr2=0x7f5540106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:11.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 -- 172.21.15.55:0/658521565 shutdown_connections 2024-04-26T22:07:11.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:11.987+0000 7f554843f640 1 -- 172.21.15.55:0/658521565 wait complete. 2024-04-26T22:07:12.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:11 smithi064 ceph-mon[32765]: pgmap v779: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 27 MiB/s wr, 426 op/s 2024-04-26T22:07:12.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:11 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1586252334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:12.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:11 smithi165 ceph-mon[33296]: pgmap v779: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 27 MiB/s wr, 426 op/s 2024-04-26T22:07:12.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:11 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1586252334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:12.999 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-26T22:07:13.000 INFO:tasks.fwd_scrub:done joining 2024-04-26T22:07:13.000 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T22:07:13.011 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:07:13.131 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 -- 172.21.15.55:0/2237829886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8103b90 msgr2=0x7f18f8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 --2- 172.21.15.55:0/2237829886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8103b90 0x7f18f8104010 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009900 tx=0x7f18ec02a410 comp rx=0 tx=0).stop 2024-04-26T22:07:13.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 -- 172.21.15.55:0/2237829886 shutdown_connections 2024-04-26T22:07:13.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 --2- 172.21.15.55:0/2237829886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18f8104550 0x7f18f810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 --2- 172.21.15.55:0/2237829886 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8103b90 0x7f18f8104010 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 --2- 172.21.15.55:0/2237829886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18f81029a0 0x7f18f8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.132 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 -- 172.21.15.55:0/2237829886 >> 172.21.15.55:0/2237829886 conn(0x7f18f80fdf10 msgr2=0x7f18f8100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:13.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 -- 172.21.15.55:0/2237829886 shutdown_connections 2024-04-26T22:07:13.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 -- 172.21.15.55:0/2237829886 wait complete. 2024-04-26T22:07:13.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 Processor -- start 2024-04-26T22:07:13.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.126+0000 7f18fe532640 1 -- start start 2024-04-26T22:07:13.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fe532640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18f81029a0 0x7f18f8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.133 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fe532640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18f8103b90 0x7f18f8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fe532640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8104550 0x7f18f81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fe532640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810e4b0 con 0x7f18f8104550 2024-04-26T22:07:13.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fe532640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810e330 con 0x7f18f81029a0 2024-04-26T22:07:13.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fe532640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810e630 con 0x7f18f8103b90 2024-04-26T22:07:13.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8104550 0x7f18f81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.134 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8104550 0x7f18f81a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53468/0 (socket says 172.21.15.55:53468) 2024-04-26T22:07:13.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 -- 172.21.15.55:0/3210643852 learned_addr learned my addr 172.21.15.55:0/3210643852 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:13.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18f77fe640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18f8103b90 0x7f18f8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18f7fff640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18f81029a0 0x7f18f8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 -- 172.21.15.55:0/3210643852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18f8103b90 msgr2=0x7f18f8199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.135 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18f8103b90 0x7f18f8199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 -- 172.21.15.55:0/3210643852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18f81029a0 msgr2=0x7f18f8198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18f81029a0 0x7f18f8198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 -- 172.21.15.55:0/3210643852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f81a0830 con 0x7f18f8104550 2024-04-26T22:07:13.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.127+0000 7f18fcacf640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8104550 0x7f18f81a0190 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f18f000eaa0 tx=0x7f18f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.128+0000 7f18f57fa640 1 -- 172.21.15.55:0/3210643852 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f000cdb0 con 0x7f18f8104550 2024-04-26T22:07:13.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.128+0000 7f18f57fa640 1 -- 172.21.15.55:0/3210643852 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f0005020 con 0x7f18f8104550 2024-04-26T22:07:13.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.128+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f81a0b20 con 0x7f18f8104550 2024-04-26T22:07:13.136 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.128+0000 7f18f57fa640 1 -- 172.21.15.55:0/3210643852 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f00116d0 con 0x7f18f8104550 2024-04-26T22:07:13.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.128+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f81a1020 con 0x7f18f8104550 2024-04-26T22:07:13.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.129+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f8109d60 con 0x7f18f8104550 2024-04-26T22:07:13.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.130+0000 7f18f57fa640 1 -- 172.21.15.55:0/3210643852 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f18f00051d0 con 0x7f18f8104550 2024-04-26T22:07:13.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.130+0000 7f18f57fa640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18e00a3c40 0x7f18e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.137 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.131+0000 7f18f7fff640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18e00a3c40 0x7f18e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.131+0000 7f18f57fa640 1 -- 172.21.15.55:0/3210643852 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f18f00d0b60 con 0x7f18f8104550 2024-04-26T22:07:13.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.131+0000 7f18f7fff640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18e00a3c40 0x7f18e00a6100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f18e8005d00 tx=0x7f18e8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.138 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.134+0000 7f18f57fa640 1 -- 172.21.15.55:0/3210643852 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f18f00d0e20 con 0x7f18f8104550 2024-04-26T22:07:13.324 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.324+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18f802dcc0 con 0x7f18f8104550 2024-04-26T22:07:13.326 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.326+0000 7f18f57fa640 1 -- 172.21.15.55:0/3210643852 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22760 (secure 0 0 0) 0x7f18f0016030 con 0x7f18f8104550 2024-04-26T22:07:13.326 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:07:13.326 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:52:51.546766+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:07:13.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18e00a3c40 msgr2=0x7f18e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18e00a3c40 0x7f18e00a6100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f18e8005d00 tx=0x7f18e8005c70 comp rx=0 tx=0).stop 2024-04-26T22:07:13.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8104550 msgr2=0x7f18f81a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.328 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8104550 0x7f18f81a0190 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f18f000eaa0 tx=0x7f18f000ef60 comp rx=0 tx=0).stop 2024-04-26T22:07:13.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 shutdown_connections 2024-04-26T22:07:13.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18e00a3c40 0x7f18e00a6100 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18f8104550 0x7f18f81a0190 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18f8103b90 0x7f18f8199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 --2- 172.21.15.55:0/3210643852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18f81029a0 0x7f18f8198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 >> 172.21.15.55:0/3210643852 conn(0x7f18f80fdf10 msgr2=0x7f18f8106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:13.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 shutdown_connections 2024-04-26T22:07:13.329 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.328+0000 7f18fe532640 1 -- 172.21.15.55:0/3210643852 wait complete. 2024-04-26T22:07:13.340 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:07:13.341 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 -- 172.21.15.55:0/1248271011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb454103450 msgr2=0x7fb454105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1248271011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb454103450 0x7fb454105840 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb448009900 tx=0x7fb44802a410 comp rx=0 tx=0).stop 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 -- 172.21.15.55:0/1248271011 shutdown_connections 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1248271011 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4541086b0 0x7fb45410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1248271011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb454105d80 0x7fb454108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1248271011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb454103450 0x7fb454105840 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 -- 172.21.15.55:0/1248271011 >> 172.21.15.55:0/1248271011 conn(0x7fb4540fcde0 msgr2=0x7fb4540ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 -- 172.21.15.55:0/1248271011 shutdown_connections 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 -- 172.21.15.55:0/1248271011 wait complete. 2024-04-26T22:07:13.449 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 Processor -- start 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.446+0000 7fb45b21d640 1 -- start start 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb45b21d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb454103450 0x7fb454198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb45b21d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb454105d80 0x7fb454199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb45b21d640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4541086b0 0x7fb4541a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb45b21d640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb45410e4c0 con 0x7fb4541086b0 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb45b21d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb45410e340 con 0x7fb454103450 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb45b21d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb45410e640 con 0x7fb454105d80 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4541086b0 0x7fb4541a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4541086b0 0x7fb4541a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53498/0 (socket says 172.21.15.55:53498) 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 -- 172.21.15.55:0/1880255239 learned_addr learned my addr 172.21.15.55:0/1880255239 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb453fff640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb454105d80 0x7fb454199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb458fb9640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb454103450 0x7fb454198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 -- 172.21.15.55:0/1880255239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb454105d80 msgr2=0x7fb454199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb454105d80 0x7fb454199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 -- 172.21.15.55:0/1880255239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb454103450 msgr2=0x7fb454198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb454103450 0x7fb454198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 -- 172.21.15.55:0/1880255239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4541a0850 con 0x7fb4541086b0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb4597ba640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4541086b0 0x7fb4541a01b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00eb10 tx=0x7fb44c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb451ffb640 1 -- 172.21.15.55:0/1880255239 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c004b20 con 0x7fb4541086b0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb451ffb640 1 -- 172.21.15.55:0/1880255239 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c004cc0 con 0x7fb4541086b0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4541a0b40 con 0x7fb4541086b0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.447+0000 7fb451ffb640 1 -- 172.21.15.55:0/1880255239 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c010670 con 0x7fb4541086b0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.448+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4541a10e0 con 0x7fb4541086b0 2024-04-26T22:07:13.451 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.449+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4540fe8d0 con 0x7fb4541086b0 2024-04-26T22:07:13.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.450+0000 7fb451ffb640 1 -- 172.21.15.55:0/1880255239 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb44c005000 con 0x7fb4541086b0 2024-04-26T22:07:13.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.450+0000 7fb451ffb640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb4440a3cc0 0x7fb4440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.451+0000 7fb451ffb640 1 -- 172.21.15.55:0/1880255239 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb44c0d0d90 con 0x7fb4541086b0 2024-04-26T22:07:13.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.451+0000 7fb458fb9640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb4440a3cc0 0x7fb4440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.451+0000 7fb458fb9640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb4440a3cc0 0x7fb4440a6180 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb448009230 tx=0x7fb4480314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.454+0000 7fb451ffb640 1 -- 172.21.15.55:0/1880255239 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb44c054060 con 0x7fb4541086b0 2024-04-26T22:07:13.640 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.640+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb45402dd20 con 0x7fb4541086b0 2024-04-26T22:07:13.641 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.640+0000 7fb451ffb640 1 -- 172.21.15.55:0/1880255239 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fb44c09d570 con 0x7fb4541086b0 2024-04-26T22:07:13.641 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T22:07:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb4440a3cc0 msgr2=0x7fb4440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb4440a3cc0 0x7fb4440a6180 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb448009230 tx=0x7fb4480314a0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4541086b0 msgr2=0x7fb4541a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4541086b0 0x7fb4541a01b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00eb10 tx=0x7fb44c00c2d0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 shutdown_connections 2024-04-26T22:07:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb4440a3cc0 0x7fb4440a6180 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4541086b0 0x7fb4541a01b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb454105d80 0x7fb454199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 --2- 172.21.15.55:0/1880255239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb454103450 0x7fb454198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.642+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 >> 172.21.15.55:0/1880255239 conn(0x7fb4540fcde0 msgr2=0x7fb4541052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:13.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.643+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 shutdown_connections 2024-04-26T22:07:13.644 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.643+0000 7fb45b21d640 1 -- 172.21.15.55:0/1880255239 wait complete. 2024-04-26T22:07:13.654 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:07:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 -- 172.21.15.55:0/1388102655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464103440 msgr2=0x7ff464109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 --2- 172.21.15.55:0/1388102655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464103440 0x7ff464109cd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009ca0 tx=0x7ff45c02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 -- 172.21.15.55:0/1388102655 shutdown_connections 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 --2- 172.21.15.55:0/1388102655 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464103440 0x7ff464109cd0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 --2- 172.21.15.55:0/1388102655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff464102a80 0x7ff464102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 --2- 172.21.15.55:0/1388102655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff464101890 0x7ff464101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 -- 172.21.15.55:0/1388102655 >> 172.21.15.55:0/1388102655 conn(0x7ff4640fce20 msgr2=0x7ff4640ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 -- 172.21.15.55:0/1388102655 shutdown_connections 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 -- 172.21.15.55:0/1388102655 wait complete. 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.761+0000 7ff4698ea640 1 Processor -- start 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4698ea640 1 -- start start 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4698ea640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464101890 0x7ff46406bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4698ea640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff464102a80 0x7ff46419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4698ea640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff464103440 0x7ff46419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4698ea640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410d3c0 con 0x7ff464101890 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4698ea640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410d240 con 0x7ff464103440 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4698ea640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410d540 con 0x7ff464102a80 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464101890 0x7ff46406bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464101890 0x7ff46406bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53516/0 (socket says 172.21.15.55:53516) 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 -- 172.21.15.55:0/676463425 learned_addr learned my addr 172.21.15.55:0/676463425 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4627fc640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff464102a80 0x7ff46419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4637fe640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff464103440 0x7ff46419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 -- 172.21.15.55:0/676463425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff464102a80 msgr2=0x7ff46419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff464102a80 0x7ff46419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 -- 172.21.15.55:0/676463425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff464103440 msgr2=0x7ff46419ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff464103440 0x7ff46419ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 -- 172.21.15.55:0/676463425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4641a07c0 con 0x7ff464101890 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4637fe640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff464103440 0x7ff46419ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff462ffd640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464101890 0x7ff46406bac0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff45400e9c0 tx=0x7ff45400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4688e8640 1 -- 172.21.15.55:0/676463425 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45400cd80 con 0x7ff464101890 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4688e8640 1 -- 172.21.15.55:0/676463425 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4540035f0 con 0x7ff464101890 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.762+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4641a0ab0 con 0x7ff464101890 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.763+0000 7ff4688e8640 1 -- 172.21.15.55:0/676463425 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454010690 con 0x7ff464101890 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.763+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641a0fb0 con 0x7ff464101890 2024-04-26T22:07:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.763+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff464108bb0 con 0x7ff464101890 2024-04-26T22:07:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.765+0000 7ff4688e8640 1 -- 172.21.15.55:0/676463425 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff454004000 con 0x7ff464101890 2024-04-26T22:07:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.766+0000 7ff4688e8640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff44c0a3c40 0x7ff44c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.766+0000 7ff4627fc640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff44c0a3c40 0x7ff44c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.766+0000 7ff4688e8640 1 -- 172.21.15.55:0/676463425 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7ff4540d0d50 con 0x7ff464101890 2024-04-26T22:07:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.766+0000 7ff4627fc640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff44c0a3c40 0x7ff44c0a6100 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff46406b830 tx=0x7ff45800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.769+0000 7ff4688e8640 1 -- 172.21.15.55:0/676463425 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff45409d530 con 0x7ff464101890 2024-04-26T22:07:13.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:13 smithi055 ceph-mon[29221]: pgmap v780: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 18 MiB/s wr, 327 op/s 2024-04-26T22:07:13.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:13 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3210643852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:13.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:13 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1880255239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:07:13.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.957+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff46402dc80 con 0x7ff464101890 2024-04-26T22:07:13.957 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.957+0000 7ff4688e8640 1 -- 172.21.15.55:0/676463425 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7ff45409ce30 con 0x7ff464101890 2024-04-26T22:07:13.957 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T22:07:13.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff44c0a3c40 msgr2=0x7ff44c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.959 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff44c0a3c40 0x7ff44c0a6100 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff46406b830 tx=0x7ff45800b390 comp rx=0 tx=0).stop 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464101890 msgr2=0x7ff46406bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464101890 0x7ff46406bac0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff45400e9c0 tx=0x7ff45400ee80 comp rx=0 tx=0).stop 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 shutdown_connections 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7ff44c0a3c40 0x7ff44c0a6100 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff464103440 0x7ff46419ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff464102a80 0x7ff46419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 --2- 172.21.15.55:0/676463425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff464101890 0x7ff46406bac0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 >> 172.21.15.55:0/676463425 conn(0x7ff4640fce20 msgr2=0x7ff464105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 shutdown_connections 2024-04-26T22:07:13.960 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:13.959+0000 7ff4698ea640 1 -- 172.21.15.55:0/676463425 wait complete. 2024-04-26T22:07:13.970 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:07:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:13 smithi064 ceph-mon[32765]: pgmap v780: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 18 MiB/s wr, 327 op/s 2024-04-26T22:07:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:13 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3210643852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:14.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:13 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1880255239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:07:14.078 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 -- 172.21.15.55:0/1119426490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 msgr2=0x7fd748109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 --2- 172.21.15.55:0/1119426490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 0x7fd748109cb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fd740009ca0 tx=0x7fd74002a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 -- 172.21.15.55:0/1119426490 shutdown_connections 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 --2- 172.21.15.55:0/1119426490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 0x7fd748109cb0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 --2- 172.21.15.55:0/1119426490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd748102a60 0x7fd748102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 --2- 172.21.15.55:0/1119426490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748101870 0x7fd748101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 -- 172.21.15.55:0/1119426490 >> 172.21.15.55:0/1119426490 conn(0x7fd7480fcde0 msgr2=0x7fd7480ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 -- 172.21.15.55:0/1119426490 shutdown_connections 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 -- 172.21.15.55:0/1119426490 wait complete. 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 Processor -- start 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.076+0000 7fd74fa07640 1 -- start start 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74fa07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748101870 0x7fd748198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74fa07640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd748102a60 0x7fd748199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74fa07640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 0x7fd7481a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74fa07640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd74810e4c0 con 0x7fd748103420 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74fa07640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd74810e340 con 0x7fd748101870 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74fa07640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd74810e640 con 0x7fd748102a60 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 0x7fd7481a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 0x7fd7481a00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53522/0 (socket says 172.21.15.55:53522) 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 -- 172.21.15.55:0/2349458615 learned_addr learned my addr 172.21.15.55:0/2349458615 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74d7a3640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748101870 0x7fd748198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74cfa2640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd748102a60 0x7fd748199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 -- 172.21.15.55:0/2349458615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd748102a60 msgr2=0x7fd748199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd748102a60 0x7fd748199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 -- 172.21.15.55:0/2349458615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748101870 msgr2=0x7fd748198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748101870 0x7fd748198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 -- 172.21.15.55:0/2349458615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7481a0710 con 0x7fd748103420 2024-04-26T22:07:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74dfa4640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 0x7fd7481a00a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd7400097b0 tx=0x7fd740003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd73e7fc640 1 -- 172.21.15.55:0/2349458615 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740004240 con 0x7fd748103420 2024-04-26T22:07:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd73e7fc640 1 -- 172.21.15.55:0/2349458615 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7400317c0 con 0x7fd748103420 2024-04-26T22:07:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7481a09a0 con 0x7fd748103420 2024-04-26T22:07:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd73e7fc640 1 -- 172.21.15.55:0/2349458615 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74002fc70 con 0x7fd748103420 2024-04-26T22:07:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.077+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7481a0ef0 con 0x7fd748103420 2024-04-26T22:07:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.078+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd748108b90 con 0x7fd748103420 2024-04-26T22:07:14.082 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:13 smithi165 ceph-mon[33296]: pgmap v780: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 18 MiB/s wr, 327 op/s 2024-04-26T22:07:14.082 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:13 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3210643852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:14.082 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:13 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1880255239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:07:14.082 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.080+0000 7fd73e7fc640 1 -- 172.21.15.55:0/2349458615 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd7400043e0 con 0x7fd748103420 2024-04-26T22:07:14.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.080+0000 7fd73e7fc640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd7340a3c40 0x7fd7340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:14.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.080+0000 7fd74d7a3640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd7340a3c40 0x7fd7340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:14.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.080+0000 7fd73e7fc640 1 -- 172.21.15.55:0/2349458615 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fd7400f1e80 con 0x7fd748103420 2024-04-26T22:07:14.083 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.081+0000 7fd74d7a3640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd7340a3c40 0x7fd7340a6100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd7481028c0 tx=0x7fd74400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:14.084 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.084+0000 7fd73e7fc640 1 -- 172.21.15.55:0/2349458615 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd7400be7d0 con 0x7fd748103420 2024-04-26T22:07:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.269+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd74802dc60 con 0x7fd748103420 2024-04-26T22:07:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.270+0000 7fd73e7fc640 1 -- 172.21.15.55:0/2349458615 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fd7400098b0 con 0x7fd748103420 2024-04-26T22:07:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T22:07:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd7340a3c40 msgr2=0x7fd7340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd7340a3c40 0x7fd7340a6100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd7481028c0 tx=0x7fd74400b390 comp rx=0 tx=0).stop 2024-04-26T22:07:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 msgr2=0x7fd7481a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 0x7fd7481a00a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd7400097b0 tx=0x7fd740003c20 comp rx=0 tx=0).stop 2024-04-26T22:07:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 shutdown_connections 2024-04-26T22:07:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd7340a3c40 0x7fd7340a6100 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd748103420 0x7fd7481a00a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd748102a60 0x7fd748199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 --2- 172.21.15.55:0/2349458615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd748101870 0x7fd748198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 >> 172.21.15.55:0/2349458615 conn(0x7fd7480fcde0 msgr2=0x7fd748105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 shutdown_connections 2024-04-26T22:07:14.273 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.272+0000 7fd74fa07640 1 -- 172.21.15.55:0/2349458615 wait complete. 2024-04-26T22:07:14.283 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 -- 172.21.15.55:0/3172951735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 msgr2=0x7f1884102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 --2- 172.21.15.55:0/3172951735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 0x7f1884102dc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1878009900 tx=0x7f187802a3f0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 -- 172.21.15.55:0/3172951735 shutdown_connections 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 --2- 172.21.15.55:0/3172951735 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1884104550 0x7f188410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 --2- 172.21.15.55:0/3172951735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884103b90 0x7f1884104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 --2- 172.21.15.55:0/3172951735 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 0x7f1884102dc0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 -- 172.21.15.55:0/3172951735 >> 172.21.15.55:0/3172951735 conn(0x7f18840fdf30 msgr2=0x7f1884100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 -- 172.21.15.55:0/3172951735 shutdown_connections 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.385+0000 7f188aac8640 1 -- 172.21.15.55:0/3172951735 wait complete. 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f188aac8640 1 Processor -- start 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f188aac8640 1 -- start start 2024-04-26T22:07:14.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f188aac8640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 0x7f1884198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f188aac8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1884103b90 0x7f1884199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f188aac8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884104550 0x7f18841a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f188aac8640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f188410e4d0 con 0x7f18841029a0 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f188aac8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f188410e350 con 0x7f1884104550 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f188aac8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f188410e650 con 0x7f1884103b90 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1888864640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 0x7f1884198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1888864640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 0x7f1884198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53546/0 (socket says 172.21.15.55:53546) 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1888864640 1 -- 172.21.15.55:0/3794922982 learned_addr learned my addr 172.21.15.55:0/3794922982 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1883fff640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1884103b90 0x7f1884199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1889065640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884104550 0x7f18841a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1888864640 1 -- 172.21.15.55:0/3794922982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1884103b90 msgr2=0x7f1884199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1888864640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1884103b90 0x7f1884199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1888864640 1 -- 172.21.15.55:0/3794922982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884104550 msgr2=0x7f18841a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:14.389 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1888864640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884104550 0x7f18841a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1888864640 1 -- 172.21.15.55:0/3794922982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18841a0760 con 0x7f18841029a0 2024-04-26T22:07:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.386+0000 7f1883fff640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1884103b90 0x7f1884199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:07:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.387+0000 7f1888864640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 0x7f1884198b40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1878006010 tx=0x7f18780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.387+0000 7f1881ffb640 1 -- 172.21.15.55:0/3794922982 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187803f030 con 0x7f18841029a0 2024-04-26T22:07:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.387+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18841a09f0 con 0x7f18841029a0 2024-04-26T22:07:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.387+0000 7f1881ffb640 1 -- 172.21.15.55:0/3794922982 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1878003cb0 con 0x7f18841029a0 2024-04-26T22:07:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.387+0000 7f1881ffb640 1 -- 172.21.15.55:0/3794922982 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1878035070 con 0x7f18841029a0 2024-04-26T22:07:14.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.387+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841a0f30 con 0x7f18841029a0 2024-04-26T22:07:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.388+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1884109cc0 con 0x7f18841029a0 2024-04-26T22:07:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.389+0000 7f1881ffb640 1 -- 172.21.15.55:0/3794922982 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1878042030 con 0x7f18841029a0 2024-04-26T22:07:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.390+0000 7f1881ffb640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18740a3c40 0x7f18740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.390+0000 7f1881ffb640 1 -- 172.21.15.55:0/3794922982 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f18780f0cc0 con 0x7f18841029a0 2024-04-26T22:07:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.390+0000 7f1883fff640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18740a3c40 0x7f18740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:14.391 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.390+0000 7f1883fff640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18740a3c40 0x7f18740a6100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f188419a050 tx=0x7f1870005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:14.395 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.395+0000 7f1881ffb640 1 -- 172.21.15.55:0/3794922982 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f18780bd500 con 0x7f18841029a0 2024-04-26T22:07:14.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.634+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f1884063530 con 0x7f18841029a0 2024-04-26T22:07:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.643+0000 7f1881ffb640 1 -- 172.21.15.55:0/3794922982 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v17) ==== 81+0+745776 (secure 0 0 0) 0x7f18780bce00 con 0x7f18841029a0 2024-04-26T22:07:14.658 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:07:14.658 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:07:14.658 INFO:teuthology.orchestra.run.smithi055.stdout: "epoch": 17, 2024-04-26T22:07:14.658 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.658 INFO:teuthology.orchestra.run.smithi055.stdout: "active_gid": 14152, 2024-04-26T22:07:14.658 INFO:teuthology.orchestra.run.smithi055.stdout: "active_name": "x", 2024-04-26T22:07:14.659 INFO:teuthology.orchestra.run.smithi055.stdout: "active_addrs": { 2024-04-26T22:07:14.659 INFO:teuthology.orchestra.run.smithi055.stdout: "addrvec": [ 2024-04-26T22:07:14.659 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.659 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v2", 2024-04-26T22:07:14.659 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.55:6800", 2024-04-26T22:07:14.659 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 3284422558 2024-04-26T22:07:14.659 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.660 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.660 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v1", 2024-04-26T22:07:14.660 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.55:6801", 2024-04-26T22:07:14.660 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 3284422558 2024-04-26T22:07:14.660 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.660 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:14.661 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.661 INFO:teuthology.orchestra.run.smithi055.stdout: "active_addr": "172.21.15.55:6801/3284422558", 2024-04-26T22:07:14.661 INFO:teuthology.orchestra.run.smithi055.stdout: "active_change": "2024-04-26T21:43:49.252329+0000", 2024-04-26T22:07:14.661 INFO:teuthology.orchestra.run.smithi055.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T22:07:14.661 INFO:teuthology.orchestra.run.smithi055.stdout: "available": true, 2024-04-26T22:07:14.661 INFO:teuthology.orchestra.run.smithi055.stdout: "standbys": [ 2024-04-26T22:07:14.662 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.662 INFO:teuthology.orchestra.run.smithi055.stdout: "gid": 24109, 2024-04-26T22:07:14.662 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "z", 2024-04-26T22:07:14.662 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr_features": 4540701547738038271, 2024-04-26T22:07:14.662 INFO:teuthology.orchestra.run.smithi055.stdout: "available_modules": [ 2024-04-26T22:07:14.662 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.663 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "alerts", 2024-04-26T22:07:14.663 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.663 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.663 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.663 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:14.663 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:14.663 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.664 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.664 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.664 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:14.664 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.664 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.664 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.664 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T22:07:14.665 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.665 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.665 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.665 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.665 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.665 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.666 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.666 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.666 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.666 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.666 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.666 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.667 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.667 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.667 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.667 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.667 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.667 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.667 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.668 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.668 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.668 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.668 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.668 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.668 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.669 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.669 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.669 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.669 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.669 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.669 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.669 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.670 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.670 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.670 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.670 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.670 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.670 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.671 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.671 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.671 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.671 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.671 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.671 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.671 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.671 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.672 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.673 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.674 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.674 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.674 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.674 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.674 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.674 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.674 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_destination": { 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_destination", 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.675 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Email address to send alerts to", 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_from_name": { 2024-04-26T22:07:14.676 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_from_name", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "Ceph", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Email From: name", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_host": { 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_host", 2024-04-26T22:07:14.677 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP server", 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_password": { 2024-04-26T22:07:14.678 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_password", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Password to authenticate with", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_port": { 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_port", 2024-04-26T22:07:14.679 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "465", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP port", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_sender": { 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_sender", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.680 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP envelope sender", 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_ssl": { 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_ssl", 2024-04-26T22:07:14.681 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.682 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_user": { 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_user", 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "User to authenticate as", 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.683 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "balancer", 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "active": { 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "active", 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.684 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "begin_time": { 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "begin_time", 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0000", 2024-04-26T22:07:14.685 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.686 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.686 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.686 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T22:07:14.686 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:07:14.686 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.686 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.686 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.686 INFO:teuthology.orchestra.run.smithi055.stdout: "begin_weekday": { 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "begin_weekday", 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0", 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "6", 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.687 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_max_iterations": { 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_max_iterations", 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "25", 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "250", 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.688 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_metrics": { 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_metrics", 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.689 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_step": { 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_step", 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0.5", 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0.001", 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "0.999", 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "aggressiveness of optimization", 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.690 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "end_time": { 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "end_time", 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "2359", 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.691 INFO:teuthology.orchestra.run.smithi055.stdout: "end_weekday": { 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "end_weekday", 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0", 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "6", 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.692 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.693 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.694 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.695 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.696 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.697 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "min_score": { 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "min_score", 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:14.698 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": { 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mode", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.699 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "upmap", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "crush-compat", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "none", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "read", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap-read" 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Balancer mode", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.700 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "pool_ids": { 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pool_ids", 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:14.701 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap_max_deviation": { 2024-04-26T22:07:14.702 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "upmap_max_deviation", 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap_max_optimizations": { 2024-04-26T22:07:14.703 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "upmap_max_optimizations", 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.704 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephadm", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_down_multiplier": { 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_down_multiplier", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3.0", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.705 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_refresh_rate": { 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_refresh_rate", 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "20", 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.706 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_starting_port": { 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_starting_port", 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "4721", 2024-04-26T22:07:14.707 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_ptrace": { 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_ptrace", 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.708 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "autotune_interval": { 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "autotune_interval", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.709 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "autotune_memory_target_ratio": { 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0.7", 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.710 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm_log_destination": { 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephadm_log_destination", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.711 INFO:teuthology.orchestra.run.smithi055.stdout: "file", 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "file,syslog", 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "syslog" 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "cgroups_split": { 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cgroups_split", 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.712 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "config_checks_enabled": { 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_checks_enabled", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.713 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "config_dashboard": { 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_dashboard", 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.714 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_alertmanager": { 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_alertmanager", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.715 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_base": { 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_base", 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Container image name, without the tag", 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.716 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_elasticsearch": { 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_elasticsearch", 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "elasticsearch container image", 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.717 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_grafana": { 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_grafana", 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.718 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_haproxy": { 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_haproxy", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.719 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "HAproxy container image", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_agent": { 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_agent", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger agent container image", 2024-04-26T22:07:14.720 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_collector": { 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_collector", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger collector container image", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.721 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_query": { 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_query", 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger query container image", 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.722 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_keepalived": { 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_keepalived", 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Keepalived container image", 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.723 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_loki": { 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_loki", 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Loki container image", 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_node_exporter": { 2024-04-26T22:07:14.724 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_node_exporter", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_nvmeof": { 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_nvmeof", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.725 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Nvme-of container image", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_prometheus": { 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_prometheus", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.726 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_promtail": { 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_promtail", 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.727 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Promtail container image", 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.728 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_samba": { 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_samba", 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Samba/SMB container image", 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.729 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_snmp_gateway": { 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_snmp_gateway", 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.730 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SNMP Gateway container image", 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "container_init": { 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_init", 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.731 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.732 INFO:teuthology.orchestra.run.smithi055.stdout: "daemon_cache_timeout": { 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "daemon_cache_timeout", 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.733 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "default_cephadm_command_timeout": { 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "900", 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T22:07:14.734 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "default_registry": { 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "default_registry", 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io", 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T22:07:14.735 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "device_cache_timeout": { 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_cache_timeout", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1800", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache device inventory", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.736 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "device_enhanced_scan": { 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_enhanced_scan", 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.737 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "facts_cache_timeout": { 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "facts_cache_timeout", 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.738 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache host facts data", 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "grafana_dashboards_path": { 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "grafana_dashboards_path", 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.739 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "host_check_interval": { 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "host_check_interval", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.740 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to perform a host check", 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "hw_monitoring": { 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "hw_monitoring", 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.741 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "inventory_list_all": { 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "inventory_list_all", 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.742 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.743 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.744 INFO:teuthology.orchestra.run.smithi055.stdout: "log_refresh_metadata": { 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_refresh_metadata", 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.745 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.746 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.747 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.748 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.749 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.750 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "*", 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.751 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "max_count_per_host": { 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_count_per_host", 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "max number of daemons per service per host", 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.752 INFO:teuthology.orchestra.run.smithi055.stdout: "max_osd_draining_count": { 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_osd_draining_count", 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.753 INFO:teuthology.orchestra.run.smithi055.stdout: "migration_current": { 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "migration_current", 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "internal - do not modify", 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.754 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": { 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mode", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "root", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm-package", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "root" 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.755 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "oob_default_addr": { 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "oob_default_addr", 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "169.254.1.1", 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.756 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "prometheus_alerts_path": { 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prometheus_alerts_path", 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.757 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_insecure": { 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_insecure", 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.758 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_password": { 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_password", 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.759 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_url": { 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_url", 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.760 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_username": { 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_username", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.761 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "secure_monitoring_stack": { 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "secure_monitoring_stack", 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.762 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "service_discovery_port": { 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "service_discovery_port", 2024-04-26T22:07:14.763 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8765", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "cephadm service discovery port", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "ssh_config_file": { 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssh_config_file", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.764 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "use_agent": { 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "use_agent", 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.765 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "use_repo_digest": { 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "use_repo_digest", 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.766 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_failed_host_check": { 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_failed_host_check", 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.767 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_stray_daemons": { 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_stray_daemons", 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.768 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.769 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_stray_hosts": { 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_stray_hosts", 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.770 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crash", 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.771 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.772 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.773 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.774 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.775 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "retain_interval": { 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "retain_interval", 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "31536000", 2024-04-26T22:07:14.776 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_recent_interval": { 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_recent_interval", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1209600", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.777 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "dashboard", 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.778 INFO:teuthology.orchestra.run.smithi055.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.779 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.780 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.781 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "AUDIT_API_ENABLED": { 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.782 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.783 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.784 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.785 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.786 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.787 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.788 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.789 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T22:07:14.790 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.791 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.792 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.793 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.794 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.795 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_URL": { 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_URL", 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.796 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.797 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.798 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.799 INFO:teuthology.orchestra.run.smithi055.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.800 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.801 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "[]", 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.802 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "{}", 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.803 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.804 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.805 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.806 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.807 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.808 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.809 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.810 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.811 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.812 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.813 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.814 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T22:07:14.815 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T22:07:14.816 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.817 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "45", 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.818 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.819 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.820 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T22:07:14.821 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T22:07:14.822 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T22:07:14.823 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.823 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.823 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.823 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.823 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.823 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.823 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.824 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.825 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.826 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "cross_origin_url": { 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cross_origin_url", 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.827 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "crt_file": { 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crt_file", 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.828 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "debug": { 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "debug", 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.829 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable/disable debug options", 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "jwt_token_ttl": { 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "jwt_token_ttl", 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "28800", 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.830 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "key_file": { 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "key_file", 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.831 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.832 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.833 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.834 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.835 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.836 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.837 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "motd": { 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "motd", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "The message of the day", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.838 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "redirect_resolve_ip_addr": { 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.839 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "::", 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.840 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8080", 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.841 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl": { 2024-04-26T22:07:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl", 2024-04-26T22:07:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.842 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl_server_port": { 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl_server_port", 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.843 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8443", 2024-04-26T22:07:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.845 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_behaviour": { 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_behaviour", 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "redirect", 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.846 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "redirect" 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_error_status_code": { 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_error_status_code", 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.847 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "500", 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "400", 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "599", 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "url_prefix": { 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "url_prefix", 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.848 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "devicehealth", 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.849 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "enable_monitoring": { 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enable_monitoring", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "monitor device health metrics", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.850 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.851 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.852 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.853 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.854 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.855 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: "mark_out_threshold": { 2024-04-26T22:07:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mark_out_threshold", 2024-04-26T22:07:14.856 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "2419200", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "pool_name": { 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pool_name", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.857 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "device_health_metrics", 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "retention_period": { 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "retention_period", 2024-04-26T22:07:14.858 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15552000", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long to retain device health metrics", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "scrape_frequency": { 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "scrape_frequency", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "86400", 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "self_heal": { 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "self_heal", 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_threshold": { 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_threshold", 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "7257600", 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "diskprediction_local", 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.864 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.865 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.866 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.867 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.868 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.869 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "predict_interval": { 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "predict_interval", 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "86400", 2024-04-26T22:07:14.870 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "predictor_model": { 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "predictor_model", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "prophetstor", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.871 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.872 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.873 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "influx", 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": false, 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "influxdb python module not found", 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "batch_size": { 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "batch_size", 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5000", 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.874 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "database": { 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "database", 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "ceph", 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.875 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": { 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "hostname", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB server hostname", 2024-04-26T22:07:14.876 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "30", 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "5", 2024-04-26T22:07:14.877 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.878 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.879 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.880 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.881 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.882 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "password": { 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "password", 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "password of InfluxDB server user", 2024-04-26T22:07:14.883 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "port": { 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "port", 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8086", 2024-04-26T22:07:14.884 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB server port", 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl": { 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl", 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "false", 2024-04-26T22:07:14.885 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "threads": { 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "threads", 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.886 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "32", 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "username": { 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "username", 2024-04-26T22:07:14.887 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "username of InfluxDB server user", 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "verify_ssl": { 2024-04-26T22:07:14.888 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "verify_ssl", 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "true", 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.889 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "insights", 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.890 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.891 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.892 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.893 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.894 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.895 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "iostat", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.896 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.897 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.898 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.899 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.900 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "k8sevents", 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_event_retention_days": { 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ceph_event_retention_days", 2024-04-26T22:07:14.901 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "7", 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "config_check_secs": { 2024-04-26T22:07:14.902 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_check_secs", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "10", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.903 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.904 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.905 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.906 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.907 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.908 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "localpool", 2024-04-26T22:07:14.909 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "failure_domain": { 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "failure_domain", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "host", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "failure domain for any created local pool", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T22:07:14.910 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.911 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.912 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.913 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.914 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "min_size": { 2024-04-26T22:07:14.915 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "min_size", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default min_size for any created local pool", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "num_rep": { 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "num_rep", 2024-04-26T22:07:14.916 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default replica count for any created local pool", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "pg_num": { 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pg_num", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.917 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "128", 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "prefix": { 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prefix", 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.918 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "name prefix for any created local pool", 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "subtree": { 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "subtree", 2024-04-26T22:07:14.919 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "rack", 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.920 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mds_autoscaler", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.921 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.922 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.923 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.924 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.925 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mirroring", 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.926 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.927 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.929 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.930 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.931 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "nfs", 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.932 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.933 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.934 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.935 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.936 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.937 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "orchestrator", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "fail_fs": { 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "fail_fs", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T22:07:14.938 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.939 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.940 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.941 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.942 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.943 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "orchestrator": { 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "orchestrator", 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.944 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm", 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "rook", 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "test_orchestrator" 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Orchestrator backend", 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "osd_perf_query", 2024-04-26T22:07:14.945 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.946 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.947 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.948 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.949 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.950 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "osd_support", 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.951 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.952 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.953 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.954 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.955 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.956 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pg_autoscaler", 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.957 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.958 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.959 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.960 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.961 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.962 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.963 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.964 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.965 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.965 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:14.965 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.965 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.965 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.965 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.965 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.965 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "threshold": { 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "threshold", 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3.0", 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1.0", 2024-04-26T22:07:14.966 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "scaling threshold", 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "progress", 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.967 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_pg_recovery_event": { 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_pg_recovery_event", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "enabled": { 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enabled", 2024-04-26T22:07:14.968 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.969 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.970 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.971 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.972 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.973 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "max_completed_events": { 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_completed_events", 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "50", 2024-04-26T22:07:14.974 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "number of past completed events to remember", 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:14.975 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long the module is going to sleep", 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prometheus", 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.976 INFO:teuthology.orchestra.run.smithi055.stdout: "cache": { 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cache", 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "exclude_perf_counters": { 2024-04-26T22:07:14.977 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "exclude_perf_counters", 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.978 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.979 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.980 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.981 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.982 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_stats_pools": { 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_stats_pools", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.983 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "300", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.984 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "scrape_interval": { 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "scrape_interval", 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15.0", 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.985 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "::", 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T22:07:14.986 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "9283", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.987 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "stale_cache_strategy": { 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "stale_cache_strategy", 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "log", 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.988 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_behaviour": { 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_behaviour", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "default", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "default", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "error" 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.989 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_error_status_code": { 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_error_status_code", 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "500", 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "400", 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "599", 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.990 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_support", 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.991 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.992 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:14.993 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:14.994 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.995 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "max_concurrent_snap_create": { 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_concurrent_snap_create", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.996 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "mirror_snapshot_schedule": { 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.997 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "trash_purge_schedule": { 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "trash_purge_schedule", 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:14.998 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "restful", 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "enable_auth": { 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enable_auth", 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:14.999 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "key_file": { 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "key_file", 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.000 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.001 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.002 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.003 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.004 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.005 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.006 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.007 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rgw", 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.008 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.009 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.010 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.011 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.012 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.013 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rook", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.014 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.015 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.016 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.017 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.018 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "storage_class": { 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "storage_class", 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "local", 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.019 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "selftest", 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.020 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.021 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.022 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.023 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.024 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "roption1": { 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "roption1", 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.025 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "roption2": { 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "roption2", 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "xyz", 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.026 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption1": { 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption1", 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.027 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption2": { 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption2", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.028 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption3": { 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption3", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.029 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption4": { 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption4", 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.030 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption5": { 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption5", 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.031 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption6": { 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption6", 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.032 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption7": { 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption7", 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "42", 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.033 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "testkey": { 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testkey", 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.034 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "testlkey": { 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testlkey", 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.035 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "testnewline": { 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testnewline", 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.036 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snap_schedule", 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_m_granularity": { 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_m_granularity", 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.037 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "dump_on_update": { 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "dump_on_update", 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.038 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "dump database to debug log on update", 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.039 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.040 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.041 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.042 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.043 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "stats", 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.044 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.045 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.046 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.047 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.048 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "status", 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.049 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.050 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.051 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.053 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.054 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.055 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "telegraf", 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "address": { 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "address", 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.056 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.057 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.058 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.059 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.060 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.061 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.062 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "telemetry", 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_basic": { 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_basic", 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.063 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_crash": { 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_crash", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T22:07:15.064 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_device": { 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_device", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.065 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_ident": { 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_ident", 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.066 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_perf": { 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_perf", 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.067 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "contact": { 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "contact", 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.068 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "description": { 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "description", 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.069 INFO:teuthology.orchestra.run.smithi055.stdout: "device_url": { 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_url", 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.070 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "enabled": { 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enabled", 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.071 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "24", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "8", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "last_opt_revision": { 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "last_opt_revision", 2024-04-26T22:07:15.072 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "leaderboard": { 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "leaderboard", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.073 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "leaderboard_description": { 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "leaderboard_description", 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.074 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.075 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.076 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.077 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.078 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.079 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "organization": { 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "organization", 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.080 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "proxy": { 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "proxy", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.081 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "url": { 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "url", 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.082 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "test_orchestrator", 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.083 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.084 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.085 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.086 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.087 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.088 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "volumes", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.089 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.090 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.091 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.092 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.093 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "max_concurrent_clones": { 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_concurrent_clones", 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "4", 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T22:07:15.094 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "periodic_async_work": { 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "periodic_async_work", 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Periodically check for async work", 2024-04-26T22:07:15.095 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "snapshot_clone_delay": { 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snapshot_clone_delay", 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T22:07:15.096 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "snapshot_clone_no_wait": { 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T22:07:15.097 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix", 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "discovery_interval": { 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "discovery_interval", 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.098 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "100", 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "identifier": { 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "identifier", 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.099 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.100 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.101 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.102 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.103 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.104 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.105 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_host": { 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_host", 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.106 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_port": { 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_port", 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10051", 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.107 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_sender": { 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_sender", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.108 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "gid": 24116, 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "y", 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr_features": 4540701547738038271, 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "available_modules": [ 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "alerts", 2024-04-26T22:07:15.109 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T22:07:15.110 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.111 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.112 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.113 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.114 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.115 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_destination": { 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_destination", 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Email address to send alerts to", 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_from_name": { 2024-04-26T22:07:15.116 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_from_name", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "Ceph", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Email From: name", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_host": { 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_host", 2024-04-26T22:07:15.117 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP server", 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_password": { 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_password", 2024-04-26T22:07:15.118 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Password to authenticate with", 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_port": { 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_port", 2024-04-26T22:07:15.119 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "465", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP port", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_sender": { 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_sender", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.120 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP envelope sender", 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_ssl": { 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_ssl", 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.121 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_user": { 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_user", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.122 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "User to authenticate as", 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "balancer", 2024-04-26T22:07:15.123 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "active": { 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "active", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.124 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.125 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.125 INFO:teuthology.orchestra.run.smithi055.stdout: "begin_time": { 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "begin_time", 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0000", 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.126 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "begin_weekday": { 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "begin_weekday", 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0", 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "6", 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.127 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_max_iterations": { 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_max_iterations", 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "25", 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "250", 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.128 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_metrics": { 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_metrics", 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.129 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_step": { 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_step", 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0.5", 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0.001", 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "0.999", 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "aggressiveness of optimization", 2024-04-26T22:07:15.130 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "end_time": { 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "end_time", 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "2359", 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T22:07:15.131 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "end_weekday": { 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "end_weekday", 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0", 2024-04-26T22:07:15.132 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "6", 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.133 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.134 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.135 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.136 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.137 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "min_score": { 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "min_score", 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.138 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": { 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mode", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "upmap", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.139 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "crush-compat", 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "none", 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "read", 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap", 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap-read" 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Balancer mode", 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "pool_ids": { 2024-04-26T22:07:15.140 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pool_ids", 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.141 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap_max_deviation": { 2024-04-26T22:07:15.142 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "upmap_max_deviation", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap_max_optimizations": { 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "upmap_max_optimizations", 2024-04-26T22:07:15.143 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephadm", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_down_multiplier": { 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_down_multiplier", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3.0", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T22:07:15.145 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_refresh_rate": { 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_refresh_rate", 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "20", 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.146 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_starting_port": { 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_starting_port", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "4721", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T22:07:15.147 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_ptrace": { 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_ptrace", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T22:07:15.148 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "autotune_interval": { 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "autotune_interval", 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.149 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "autotune_memory_target_ratio": { 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0.7", 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.150 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm_log_destination": { 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephadm_log_destination", 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "file", 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "file,syslog", 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: "syslog" 2024-04-26T22:07:15.151 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "cgroups_split": { 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cgroups_split", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T22:07:15.152 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "config_checks_enabled": { 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_checks_enabled", 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T22:07:15.153 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "config_dashboard": { 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_dashboard", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.154 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_alertmanager": { 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_alertmanager", 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.155 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_base": { 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_base", 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Container image name, without the tag", 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.156 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_elasticsearch": { 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_elasticsearch", 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "elasticsearch container image", 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.157 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_grafana": { 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_grafana", 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.158 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_haproxy": { 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_haproxy", 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "HAproxy container image", 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.159 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_agent": { 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_agent", 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger agent container image", 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.160 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_collector": { 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_collector", 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger collector container image", 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.161 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_query": { 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_query", 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger query container image", 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_keepalived": { 2024-04-26T22:07:15.162 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_keepalived", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Keepalived container image", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_loki": { 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_loki", 2024-04-26T22:07:15.163 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Loki container image", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_node_exporter": { 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_node_exporter", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.164 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_nvmeof": { 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_nvmeof", 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.165 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Nvme-of container image", 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_prometheus": { 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_prometheus", 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.166 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_promtail": { 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_promtail", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.167 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Promtail container image", 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_samba": { 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_samba", 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.168 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Samba/SMB container image", 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_snmp_gateway": { 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_snmp_gateway", 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.169 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SNMP Gateway container image", 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "container_init": { 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_init", 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.170 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "daemon_cache_timeout": { 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "daemon_cache_timeout", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.171 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "default_cephadm_command_timeout": { 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "900", 2024-04-26T22:07:15.172 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "default_registry": { 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "default_registry", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.173 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "device_cache_timeout": { 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_cache_timeout", 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1800", 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.174 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache device inventory", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "device_enhanced_scan": { 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_enhanced_scan", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.175 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "facts_cache_timeout": { 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "facts_cache_timeout", 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.176 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache host facts data", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "grafana_dashboards_path": { 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "grafana_dashboards_path", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T22:07:15.177 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "host_check_interval": { 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "host_check_interval", 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to perform a host check", 2024-04-26T22:07:15.178 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "hw_monitoring": { 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "hw_monitoring", 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T22:07:15.179 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "inventory_list_all": { 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "inventory_list_all", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.180 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.181 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "log_refresh_metadata": { 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_refresh_metadata", 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.182 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.183 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.184 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.185 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T22:07:15.186 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T22:07:15.187 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "*", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "max_count_per_host": { 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_count_per_host", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.188 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "max number of daemons per service per host", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "max_osd_draining_count": { 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_osd_draining_count", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.189 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "migration_current": { 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "migration_current", 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.190 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "internal - do not modify", 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": { 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mode", 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.191 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "root", 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm-package", 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "root" 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "oob_default_addr": { 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "oob_default_addr", 2024-04-26T22:07:15.192 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "169.254.1.1", 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "prometheus_alerts_path": { 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prometheus_alerts_path", 2024-04-26T22:07:15.193 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_insecure": { 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_insecure", 2024-04-26T22:07:15.194 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_password": { 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_password", 2024-04-26T22:07:15.195 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_url": { 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_url", 2024-04-26T22:07:15.196 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_username": { 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_username", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.197 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "secure_monitoring_stack": { 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "secure_monitoring_stack", 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.198 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "service_discovery_port": { 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "service_discovery_port", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.199 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8765", 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "cephadm service discovery port", 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "ssh_config_file": { 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssh_config_file", 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.200 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "use_agent": { 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "use_agent", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.201 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "use_repo_digest": { 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "use_repo_digest", 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.202 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_failed_host_check": { 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_failed_host_check", 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.203 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_stray_daemons": { 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_stray_daemons", 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.204 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_stray_hosts": { 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_stray_hosts", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.205 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crash", 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.206 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.207 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.208 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.209 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.210 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "retain_interval": { 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "retain_interval", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.211 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "31536000", 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_recent_interval": { 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_recent_interval", 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.212 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1209600", 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "dashboard", 2024-04-26T22:07:15.213 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.214 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.215 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.216 INFO:teuthology.orchestra.run.smithi055.stdout: "AUDIT_API_ENABLED": { 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.217 INFO:teuthology.orchestra.run.smithi055.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.218 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.219 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.220 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T22:07:15.221 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.222 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T22:07:15.223 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.224 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T22:07:15.225 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T22:07:15.226 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T22:07:15.227 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.228 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_URL": { 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_URL", 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.229 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.230 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.231 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.232 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.233 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.234 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.235 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "[]", 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.236 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "{}", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.237 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.238 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.239 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.240 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.241 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.242 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.243 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.244 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.245 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.246 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.247 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.248 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8", 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.249 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "45", 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.250 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.252 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.253 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.254 INFO:teuthology.orchestra.run.smithi055.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.255 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.257 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T22:07:15.257 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T22:07:15.257 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T22:07:15.258 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T22:07:15.259 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "cross_origin_url": { 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cross_origin_url", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.260 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "crt_file": { 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crt_file", 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.261 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "debug": { 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "debug", 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.262 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable/disable debug options", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "jwt_token_ttl": { 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "jwt_token_ttl", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.263 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "28800", 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "key_file": { 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "key_file", 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.264 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.265 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.266 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.267 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.268 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.269 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "motd": { 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "motd", 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.270 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "The message of the day", 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "redirect_resolve_ip_addr": { 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.271 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "::", 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.272 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8080", 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.273 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl": { 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl", 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.274 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl_server_port": { 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl_server_port", 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8443", 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.275 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_behaviour": { 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_behaviour", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "redirect", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: "redirect" 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_error_status_code": { 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_error_status_code", 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "500", 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "400", 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "599", 2024-04-26T22:07:15.277 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "url_prefix": { 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "url_prefix", 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.278 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "devicehealth", 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "enable_monitoring": { 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enable_monitoring", 2024-04-26T22:07:15.279 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "monitor device health metrics", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.280 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.281 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.282 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.283 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.284 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "mark_out_threshold": { 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mark_out_threshold", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "2419200", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.285 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "pool_name": { 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pool_name", 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "device_health_metrics", 2024-04-26T22:07:15.286 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "retention_period": { 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "retention_period", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15552000", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.287 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long to retain device health metrics", 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "scrape_frequency": { 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "scrape_frequency", 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "86400", 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.288 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "self_heal": { 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "self_heal", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.289 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.290 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_threshold": { 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_threshold", 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "7257600", 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.291 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "diskprediction_local", 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.292 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.293 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.294 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.295 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.296 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "predict_interval": { 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "predict_interval", 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.297 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "86400", 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "predictor_model": { 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "predictor_model", 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.298 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "prophetstor", 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.299 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "influx", 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": false, 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "influxdb python module not found", 2024-04-26T22:07:15.300 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "batch_size": { 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "batch_size", 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5000", 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.301 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "database": { 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "database", 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "ceph", 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.302 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": { 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "hostname", 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB server hostname", 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.303 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "30", 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "5", 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.304 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.305 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.306 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.307 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.308 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "password": { 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "password", 2024-04-26T22:07:15.309 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "password of InfluxDB server user", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "port": { 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "port", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.310 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8086", 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB server port", 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl": { 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl", 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.311 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "false", 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "threads": { 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "threads", 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.312 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "32", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "username": { 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "username", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.313 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "username of InfluxDB server user", 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "verify_ssl": { 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "verify_ssl", 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.314 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "true", 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "insights", 2024-04-26T22:07:15.315 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.316 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.317 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.318 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.319 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.320 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "iostat", 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.321 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.322 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.323 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.324 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.325 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "k8sevents", 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_event_retention_days": { 2024-04-26T22:07:15.326 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ceph_event_retention_days", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "7", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "config_check_secs": { 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_check_secs", 2024-04-26T22:07:15.327 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "10", 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.328 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.329 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.330 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.331 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.332 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.333 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "localpool", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "failure_domain": { 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "failure_domain", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "host", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "failure domain for any created local pool", 2024-04-26T22:07:15.334 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.335 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.336 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.337 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.338 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.339 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "min_size": { 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "min_size", 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.340 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default min_size for any created local pool", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "num_rep": { 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "num_rep", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default replica count for any created local pool", 2024-04-26T22:07:15.341 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "pg_num": { 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pg_num", 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "128", 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T22:07:15.342 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "prefix": { 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prefix", 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "name prefix for any created local pool", 2024-04-26T22:07:15.343 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "subtree": { 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "subtree", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "rack", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T22:07:15.344 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mds_autoscaler", 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.345 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.346 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.347 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.348 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.349 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mirroring", 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.350 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.351 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.352 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.353 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.354 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.355 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "nfs", 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.356 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.357 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.358 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.359 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.360 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "orchestrator", 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.361 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "fail_fs": { 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "fail_fs", 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.362 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.363 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.364 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.365 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.366 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "orchestrator": { 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "orchestrator", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.367 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm", 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "rook", 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "test_orchestrator" 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Orchestrator backend", 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.368 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "osd_perf_query", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.369 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.370 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.371 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.372 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.373 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "osd_support", 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.374 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.375 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.376 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.377 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.378 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pg_autoscaler", 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.379 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.380 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.381 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.382 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.383 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.384 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "threshold": { 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "threshold", 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.385 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3.0", 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1.0", 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "scaling threshold", 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "progress", 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.386 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_pg_recovery_event": { 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_pg_recovery_event", 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.387 INFO:teuthology.orchestra.run.smithi055.stdout: "enabled": { 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enabled", 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.388 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.389 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.390 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.391 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.392 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "max_completed_events": { 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_completed_events", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "50", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.393 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "number of past completed events to remember", 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.394 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long the module is going to sleep", 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prometheus", 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "cache": { 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cache", 2024-04-26T22:07:15.395 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "exclude_perf_counters": { 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "exclude_perf_counters", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.396 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.397 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.398 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.399 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.400 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.401 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_stats_pools": { 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_stats_pools", 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.402 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "300", 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.403 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "scrape_interval": { 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "scrape_interval", 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15.0", 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.404 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "::", 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.405 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "9283", 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.406 INFO:teuthology.orchestra.run.smithi055.stdout: "stale_cache_strategy": { 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "stale_cache_strategy", 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "log", 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.407 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_behaviour": { 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_behaviour", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "default", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "default", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "error" 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.408 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_error_status_code": { 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_error_status_code", 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "500", 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "400", 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "599", 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.409 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_support", 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.410 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.411 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.412 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.413 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.414 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "max_concurrent_snap_create": { 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_concurrent_snap_create", 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.415 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "mirror_snapshot_schedule": { 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.416 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "trash_purge_schedule": { 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "trash_purge_schedule", 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.417 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "restful", 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "enable_auth": { 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enable_auth", 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.418 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "key_file": { 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "key_file", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.419 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.420 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.421 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.422 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.423 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.424 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.425 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.426 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rgw", 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.427 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.428 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.429 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.430 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.431 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rook", 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.432 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.433 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.435 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.436 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "storage_class": { 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "storage_class", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "local", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.437 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "selftest", 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.438 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.439 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.440 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.441 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.442 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "roption1": { 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "roption1", 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.443 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "roption2": { 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "roption2", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "xyz", 2024-04-26T22:07:15.444 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption1": { 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption1", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.445 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption2": { 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption2", 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.446 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption3": { 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption3", 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.447 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption4": { 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption4", 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.448 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption5": { 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption5", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.449 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption6": { 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption6", 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.450 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption7": { 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption7", 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "42", 2024-04-26T22:07:15.451 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "testkey": { 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testkey", 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.452 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "testlkey": { 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testlkey", 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.453 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "testnewline": { 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testnewline", 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.454 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snap_schedule", 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_m_granularity": { 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_m_granularity", 2024-04-26T22:07:15.455 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "dump_on_update": { 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "dump_on_update", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.456 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "dump database to debug log on update", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.457 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.458 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.459 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.460 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.461 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "stats", 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.462 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.463 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.464 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.465 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.466 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "status", 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.467 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.468 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.469 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.470 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.471 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "telegraf", 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "address": { 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "address", 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T22:07:15.473 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15", 2024-04-26T22:07:15.474 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.475 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.476 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.477 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.478 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.479 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "telemetry", 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_basic": { 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_basic", 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.480 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_crash": { 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_crash", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.481 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_device": { 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_device", 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.482 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_ident": { 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_ident", 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.483 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_perf": { 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_perf", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.484 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "contact": { 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "contact", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.485 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "description": { 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "description", 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.486 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "device_url": { 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_url", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.487 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "enabled": { 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enabled", 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.488 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "24", 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "8", 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.489 INFO:teuthology.orchestra.run.smithi055.stdout: "last_opt_revision": { 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "last_opt_revision", 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1", 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.490 INFO:teuthology.orchestra.run.smithi055.stdout: "leaderboard": { 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "leaderboard", 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "leaderboard_description": { 2024-04-26T22:07:15.491 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "leaderboard_description", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.492 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.493 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.494 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.495 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.496 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "organization": { 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "organization", 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.497 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "proxy": { 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "proxy", 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.498 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "url": { 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "url", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T22:07:15.499 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "test_orchestrator", 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.500 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.501 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.502 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.503 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.504 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.505 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "volumes", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.506 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.507 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.508 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.509 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.510 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "max_concurrent_clones": { 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_concurrent_clones", 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "4", 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.511 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "periodic_async_work": { 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "periodic_async_work", 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Periodically check for async work", 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.512 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "snapshot_clone_delay": { 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snapshot_clone_delay", 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.513 INFO:teuthology.orchestra.run.smithi055.stdout: "snapshot_clone_no_wait": { 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.514 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix", 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "discovery_interval": { 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "discovery_interval", 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "100", 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.515 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "identifier": { 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "identifier", 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.516 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.517 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.518 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.519 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.520 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.521 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.522 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_host": { 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_host", 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.523 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_port": { 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_port", 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10051", 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.524 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_sender": { 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_sender", 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.525 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: "modules": [ 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm", 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: "dashboard", 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: "iostat", 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: "nfs", 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: "restful" 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: "available_modules": [ 2024-04-26T22:07:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "alerts", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T22:07:15.527 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.528 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.529 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.530 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.531 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.532 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_destination": { 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_destination", 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Email address to send alerts to", 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_from_name": { 2024-04-26T22:07:15.533 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_from_name", 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "Ceph", 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Email From: name", 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.534 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_host": { 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_host", 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP server", 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_password": { 2024-04-26T22:07:15.535 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_password", 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Password to authenticate with", 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_port": { 2024-04-26T22:07:15.536 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_port", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "465", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP port", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_sender": { 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_sender", 2024-04-26T22:07:15.537 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SMTP envelope sender", 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_ssl": { 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_ssl", 2024-04-26T22:07:15.538 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "smtp_user": { 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "smtp_user", 2024-04-26T22:07:15.539 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "User to authenticate as", 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "balancer", 2024-04-26T22:07:15.540 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "active": { 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "active", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.541 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "begin_time": { 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "begin_time", 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0000", 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.542 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "begin_weekday": { 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "begin_weekday", 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0", 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "6", 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.543 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_max_iterations": { 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_max_iterations", 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "25", 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "250", 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.544 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_metrics": { 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_metrics", 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.545 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "crush_compat_step": { 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crush_compat_step", 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0.5", 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0.001", 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "0.999", 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "aggressiveness of optimization", 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.546 INFO:teuthology.orchestra.run.smithi055.stdout: "end_time": { 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "end_time", 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "2359", 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.547 INFO:teuthology.orchestra.run.smithi055.stdout: "end_weekday": { 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "end_weekday", 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "0", 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "6", 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.548 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.549 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.550 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.551 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.552 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "min_score": { 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "min_score", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.553 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": { 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mode", 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.554 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "upmap", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "crush-compat", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "none", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "read", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap-read" 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Balancer mode", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.555 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "pool_ids": { 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pool_ids", 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.556 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.557 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap_max_deviation": { 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "upmap_max_deviation", 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.558 INFO:teuthology.orchestra.run.smithi055.stdout: "upmap_max_optimizations": { 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "upmap_max_optimizations", 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.559 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephadm", 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_down_multiplier": { 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_down_multiplier", 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3.0", 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.560 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_refresh_rate": { 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_refresh_rate", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "20", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T22:07:15.561 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "agent_starting_port": { 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "agent_starting_port", 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "4721", 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T22:07:15.562 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_ptrace": { 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_ptrace", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T22:07:15.563 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "autotune_interval": { 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "autotune_interval", 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.564 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "autotune_memory_target_ratio": { 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0.7", 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.565 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm_log_destination": { 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephadm_log_destination", 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "file", 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "file,syslog", 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "syslog" 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.566 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "cgroups_split": { 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cgroups_split", 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.567 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "config_checks_enabled": { 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_checks_enabled", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T22:07:15.568 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "config_dashboard": { 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_dashboard", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.569 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_alertmanager": { 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_alertmanager", 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.570 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_base": { 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_base", 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Container image name, without the tag", 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.571 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_elasticsearch": { 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_elasticsearch", 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "elasticsearch container image", 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.572 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_grafana": { 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_grafana", 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.573 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_haproxy": { 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_haproxy", 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "HAproxy container image", 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.574 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_agent": { 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_agent", 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger agent container image", 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_collector": { 2024-04-26T22:07:15.575 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_collector", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger collector container image", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_jaeger_query": { 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_jaeger_query", 2024-04-26T22:07:15.576 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Jaeger query container image", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_keepalived": { 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_keepalived", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.577 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Keepalived container image", 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_loki": { 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_loki", 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.578 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Loki container image", 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_node_exporter": { 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_node_exporter", 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.579 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_nvmeof": { 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_nvmeof", 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.580 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Nvme-of container image", 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_prometheus": { 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_prometheus", 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.581 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Prometheus container image", 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_promtail": { 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_promtail", 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.582 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Promtail container image", 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_samba": { 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_samba", 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.583 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Samba/SMB container image", 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "container_image_snmp_gateway": { 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_image_snmp_gateway", 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.584 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "SNMP Gateway container image", 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "container_init": { 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "container_init", 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.585 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "daemon_cache_timeout": { 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "daemon_cache_timeout", 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.586 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "default_cephadm_command_timeout": { 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "900", 2024-04-26T22:07:15.587 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "default_registry": { 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "default_registry", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "docker.io", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.588 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "device_cache_timeout": { 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_cache_timeout", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1800", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.589 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache device inventory", 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "device_enhanced_scan": { 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_enhanced_scan", 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.590 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "facts_cache_timeout": { 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "facts_cache_timeout", 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.591 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "seconds to cache host facts data", 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "grafana_dashboards_path": { 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "grafana_dashboards_path", 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.592 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "host_check_interval": { 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "host_check_interval", 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.593 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to perform a host check", 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "hw_monitoring": { 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "hw_monitoring", 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.594 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "inventory_list_all": { 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "inventory_list_all", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T22:07:15.595 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.596 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "log_refresh_metadata": { 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_refresh_metadata", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.597 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.598 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.599 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.600 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.601 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.602 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.603 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.603 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.603 INFO:teuthology.orchestra.run.smithi055.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T22:07:15.603 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T22:07:15.603 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.603 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.603 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.604 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "*", 2024-04-26T22:07:15.604 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.604 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.604 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.604 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T22:07:15.604 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.604 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.604 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.605 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.605 INFO:teuthology.orchestra.run.smithi055.stdout: "max_count_per_host": { 2024-04-26T22:07:15.605 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_count_per_host", 2024-04-26T22:07:15.605 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.605 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.605 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.605 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.606 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.606 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.606 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.606 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "max number of daemons per service per host", 2024-04-26T22:07:15.606 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.606 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.606 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.607 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.607 INFO:teuthology.orchestra.run.smithi055.stdout: "max_osd_draining_count": { 2024-04-26T22:07:15.607 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_osd_draining_count", 2024-04-26T22:07:15.607 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.607 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.607 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.607 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.607 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.608 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.608 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.608 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T22:07:15.608 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.608 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.608 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.608 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.609 INFO:teuthology.orchestra.run.smithi055.stdout: "migration_current": { 2024-04-26T22:07:15.609 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "migration_current", 2024-04-26T22:07:15.609 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.609 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.609 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.609 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.609 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.610 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.610 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.610 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "internal - do not modify", 2024-04-26T22:07:15.610 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.610 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.610 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.610 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.610 INFO:teuthology.orchestra.run.smithi055.stdout: "mode": { 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mode", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "root", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm-package", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "root" 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.611 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "oob_default_addr": { 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "oob_default_addr", 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "169.254.1.1", 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.612 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "prometheus_alerts_path": { 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prometheus_alerts_path", 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.613 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_insecure": { 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_insecure", 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.614 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_password": { 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_password", 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.615 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_url": { 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_url", 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.616 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "registry_username": { 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "registry_username", 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.617 INFO:teuthology.orchestra.run.smithi055.stdout: "secure_monitoring_stack": { 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "secure_monitoring_stack", 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.618 INFO:teuthology.orchestra.run.smithi055.stdout: "service_discovery_port": { 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "service_discovery_port", 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8765", 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "cephadm service discovery port", 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "ssh_config_file": { 2024-04-26T22:07:15.619 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssh_config_file", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "use_agent": { 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "use_agent", 2024-04-26T22:07:15.620 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "use_repo_digest": { 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "use_repo_digest", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.621 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_failed_host_check": { 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_failed_host_check", 2024-04-26T22:07:15.622 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_stray_daemons": { 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_stray_daemons", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.623 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_on_stray_hosts": { 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_on_stray_hosts", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.624 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crash", 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.625 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.626 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.627 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.628 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.629 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.630 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "retain_interval": { 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "retain_interval", 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "31536000", 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.631 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_recent_interval": { 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_recent_interval", 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1209600", 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.632 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "dashboard", 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.633 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.634 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.635 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "AUDIT_API_ENABLED": { 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.636 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.637 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.638 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.639 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.640 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.641 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.642 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.643 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.644 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.645 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.646 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T22:07:15.647 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_URL": { 2024-04-26T22:07:15.648 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_URL", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T22:07:15.649 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.651 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.652 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.653 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "[]", 2024-04-26T22:07:15.654 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "{}", 2024-04-26T22:07:15.655 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.656 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.657 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.658 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.659 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.660 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.661 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.662 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.663 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.664 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.665 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.666 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.667 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8", 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.669 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "45", 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.670 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.671 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "admin", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.672 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.673 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.674 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.675 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.676 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.677 INFO:teuthology.orchestra.run.smithi055.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.678 INFO:teuthology.orchestra.run.smithi055.stdout: "cross_origin_url": { 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cross_origin_url", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "crt_file": { 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "crt_file", 2024-04-26T22:07:15.679 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "debug": { 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "debug", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.680 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Enable/disable debug options", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "jwt_token_ttl": { 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "jwt_token_ttl", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.681 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "28800", 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "key_file": { 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "key_file", 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.682 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.683 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.684 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.685 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.686 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.687 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "motd": { 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "motd", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "The message of the day", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.688 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "redirect_resolve_ip_addr": { 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.689 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "::", 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.690 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8080", 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.691 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl": { 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl", 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.692 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl_server_port": { 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl_server_port", 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8443", 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.693 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_behaviour": { 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_behaviour", 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "redirect", 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "redirect" 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_error_status_code": { 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_error_status_code", 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "500", 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "400", 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "599", 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.695 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "url_prefix": { 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "url_prefix", 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.696 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "devicehealth", 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "enable_monitoring": { 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enable_monitoring", 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.697 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "monitor device health metrics", 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.698 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.699 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.700 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.701 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.702 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "mark_out_threshold": { 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mark_out_threshold", 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "2419200", 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.703 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "pool_name": { 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pool_name", 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "device_health_metrics", 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.704 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "retention_period": { 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "retention_period", 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15552000", 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.705 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long to retain device health metrics", 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "scrape_frequency": { 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "scrape_frequency", 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "86400", 2024-04-26T22:07:15.706 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "self_heal": { 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "self_heal", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.707 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.708 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "warn_threshold": { 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "warn_threshold", 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "7257600", 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.709 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "diskprediction_local", 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.710 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.711 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.712 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.713 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.714 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "predict_interval": { 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "predict_interval", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "86400", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.715 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "predictor_model": { 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "predictor_model", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "prophetstor", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.716 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "600", 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.717 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "influx", 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": false, 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "influxdb python module not found", 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "batch_size": { 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "batch_size", 2024-04-26T22:07:15.718 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5000", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "database": { 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "database", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.719 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "ceph", 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "hostname": { 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "hostname", 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.720 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB server hostname", 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.721 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "30", 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "5", 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.722 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.723 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.724 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.725 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.726 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "password": { 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "password", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.727 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "password of InfluxDB server user", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "port": { 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "port", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "8086", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.728 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "InfluxDB server port", 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "ssl": { 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ssl", 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "false", 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.729 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "threads": { 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "threads", 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "32", 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.730 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "username": { 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "username", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "username of InfluxDB server user", 2024-04-26T22:07:15.731 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "verify_ssl": { 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "verify_ssl", 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "true", 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.732 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "insights", 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.733 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.734 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.735 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.736 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.737 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "iostat", 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.738 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.739 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.740 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.741 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.742 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.743 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "k8sevents", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_event_retention_days": { 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "ceph_event_retention_days", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "7", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.744 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "config_check_secs": { 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "config_check_secs", 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "10", 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.745 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.746 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.747 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.748 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.749 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.750 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "localpool", 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "failure_domain": { 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "failure_domain", 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "host", 2024-04-26T22:07:15.751 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "failure domain for any created local pool", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.752 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.753 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.754 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.755 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.756 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "min_size": { 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "min_size", 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.757 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default min_size for any created local pool", 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "num_rep": { 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "num_rep", 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3", 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.758 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default replica count for any created local pool", 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "pg_num": { 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pg_num", 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "128", 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.759 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "prefix": { 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prefix", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "name prefix for any created local pool", 2024-04-26T22:07:15.760 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "subtree": { 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "subtree", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "rack", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T22:07:15.761 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mds_autoscaler", 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.762 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.763 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.764 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.765 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.766 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mirroring", 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.767 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.768 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.769 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.770 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.771 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.772 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "nfs", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.773 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.774 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.775 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.776 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.777 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "orchestrator", 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "fail_fs": { 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "fail_fs", 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.778 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.779 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.780 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.781 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.782 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.783 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "orchestrator": { 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "orchestrator", 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "cephadm", 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "rook", 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: "test_orchestrator" 2024-04-26T22:07:15.784 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Orchestrator backend", 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "osd_perf_query", 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.785 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.786 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.787 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.788 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.789 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "osd_support", 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.790 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.791 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.792 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.793 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.794 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "pg_autoscaler", 2024-04-26T22:07:15.795 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.796 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.797 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.798 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.799 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.800 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "threshold": { 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "threshold", 2024-04-26T22:07:15.801 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "3.0", 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1.0", 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "scaling threshold", 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "progress", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_pg_recovery_event": { 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_pg_recovery_event", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T22:07:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "enabled": { 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enabled", 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "max_completed_events": { 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_completed_events", 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "50", 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "number of past completed events to remember", 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "sleep_interval": { 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "sleep_interval", 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "5", 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "how long the module is going to sleep", 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "prometheus", 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "cache": { 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cache", 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "exclude_perf_counters": { 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "exclude_perf_counters", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.818 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_stats_pools": { 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_stats_pools", 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.819 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "300", 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.820 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "scrape_interval": { 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "scrape_interval", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15.0", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.821 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "::", 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.822 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "9283", 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.823 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "stale_cache_strategy": { 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "stale_cache_strategy", 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "log", 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.824 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_behaviour": { 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_behaviour", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "default", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "default", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "error" 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.825 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "standby_error_status_code": { 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "standby_error_status_code", 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "500", 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "400", 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "599", 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.826 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_support", 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.827 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.828 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.829 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.830 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.831 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "max_concurrent_snap_create": { 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_concurrent_snap_create", 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10", 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "mirror_snapshot_schedule": { 2024-04-26T22:07:15.832 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "trash_purge_schedule": { 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "trash_purge_schedule", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.833 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "restful", 2024-04-26T22:07:15.834 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "enable_auth": { 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enable_auth", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.835 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "key_file": { 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "key_file", 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.836 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.837 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.838 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.839 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.840 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.841 INFO:teuthology.orchestra.run.smithi055.stdout: "server_addr": { 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_addr", 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "server_port": { 2024-04-26T22:07:15.842 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "server_port", 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.843 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rgw", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.844 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.845 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.846 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.847 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.848 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rook", 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.849 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.850 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.851 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.852 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.853 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "storage_class": { 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "storage_class", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "local", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.854 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "selftest", 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.855 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.856 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.857 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.858 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.859 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "roption1": { 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "roption1", 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.860 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "roption2": { 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "roption2", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "xyz", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.861 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption1": { 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption1", 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.862 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption2": { 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption2", 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.863 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption3": { 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption3", 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "float", 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.864 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption4": { 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption4", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.865 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption5": { 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption5", 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.866 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption6": { 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption6", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.867 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "rwoption7": { 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rwoption7", 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "1", 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "42", 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.868 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "testkey": { 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testkey", 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.869 INFO:teuthology.orchestra.run.smithi055.stdout: "testlkey": { 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testlkey", 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.870 INFO:teuthology.orchestra.run.smithi055.stdout: "testnewline": { 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "testnewline", 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.871 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snap_schedule", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "allow_m_granularity": { 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "allow_m_granularity", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.872 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "dump_on_update": { 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "dump_on_update", 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "dump database to debug log on update", 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.873 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.874 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.875 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.876 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.877 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "stats", 2024-04-26T22:07:15.878 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.879 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.880 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.881 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.882 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "status", 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.883 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.884 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.885 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.886 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.887 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "telegraf", 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "address": { 2024-04-26T22:07:15.888 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "address", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.889 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "15", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.890 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.891 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.892 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.893 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.894 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "telemetry", 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.895 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_basic": { 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_basic", 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.896 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_crash": { 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_crash", 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_device": { 2024-04-26T22:07:15.897 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_device", 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_ident": { 2024-04-26T22:07:15.898 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_ident", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "channel_perf": { 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "channel_perf", 2024-04-26T22:07:15.899 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "contact": { 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "contact", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.900 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "description": { 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "description", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.901 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "device_url": { 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "device_url", 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.902 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "enabled": { 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "enabled", 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.903 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "24", 2024-04-26T22:07:15.904 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "8", 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "last_opt_revision": { 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "last_opt_revision", 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "1", 2024-04-26T22:07:15.905 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "leaderboard": { 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "leaderboard", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.906 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "leaderboard_description": { 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "leaderboard_description", 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.907 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.908 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.909 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.910 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.911 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.912 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "organization": { 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "organization", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.913 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "proxy": { 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "proxy", 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.914 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "url": { 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "url", 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.915 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "test_orchestrator", 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.916 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.917 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.918 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.919 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.920 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "volumes", 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.921 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.922 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.923 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.924 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.925 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "max_concurrent_clones": { 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "max_concurrent_clones", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.926 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "4", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "periodic_async_work": { 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "periodic_async_work", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.927 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Periodically check for async work", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "snapshot_clone_delay": { 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snapshot_clone_delay", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "0", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.928 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "snapshot_clone_no_wait": { 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "True", 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.929 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix", 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "can_run": true, 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "error_string": "", 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "module_options": { 2024-04-26T22:07:15.930 INFO:teuthology.orchestra.run.smithi055.stdout: "discovery_interval": { 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "discovery_interval", 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "uint", 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "100", 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.931 INFO:teuthology.orchestra.run.smithi055.stdout: "identifier": { 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "identifier", 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "interval": { 2024-04-26T22:07:15.932 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "interval", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "secs", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "60", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "log_level": { 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_level", 2024-04-26T22:07:15.933 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.934 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster": { 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster", 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.935 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_cluster_level": { 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_cluster_level", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "info", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [ 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "", 2024-04-26T22:07:15.936 INFO:teuthology.orchestra.run.smithi055.stdout: "critical", 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "debug", 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "error", 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "info", 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "warning" 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "log_to_file": { 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "log_to_file", 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "bool", 2024-04-26T22:07:15.937 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 1, 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "False", 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_host": { 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_host", 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.938 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_port": { 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_port", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "int", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.939 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "10051", 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "zabbix_sender": { 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "zabbix_sender", 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "str", 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "level": "advanced", 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "flags": 0, 2024-04-26T22:07:15.940 INFO:teuthology.orchestra.run.smithi055.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "min": "", 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "max": "", 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "enum_allowed": [], 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "desc": "", 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "long_desc": "", 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "tags": [], 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "see_also": [] 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "services": { 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "dashboard": "https://172.21.15.55:8443/" 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.941 INFO:teuthology.orchestra.run.smithi055.stdout: "always_on_modules": { 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "octopus": [ 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "balancer", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "crash", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "devicehealth", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "orchestrator", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "pg_autoscaler", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "progress", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_support", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "status", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "telemetry", 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "volumes" 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "pacific": [ 2024-04-26T22:07:15.942 INFO:teuthology.orchestra.run.smithi055.stdout: "balancer", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "crash", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "devicehealth", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "orchestrator", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "pg_autoscaler", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "progress", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_support", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "status", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "telemetry", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "volumes" 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "quincy": [ 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "balancer", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "crash", 2024-04-26T22:07:15.943 INFO:teuthology.orchestra.run.smithi055.stdout: "devicehealth", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "orchestrator", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "pg_autoscaler", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "progress", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_support", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "status", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "telemetry", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "volumes" 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "reef": [ 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "balancer", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "crash", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "devicehealth", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "orchestrator", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "pg_autoscaler", 2024-04-26T22:07:15.944 INFO:teuthology.orchestra.run.smithi055.stdout: "progress", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_support", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "status", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "telemetry", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "volumes" 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: ], 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "squid": [ 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "balancer", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "crash", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "devicehealth", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "orchestrator", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "pg_autoscaler", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "progress", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "rbd_support", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "status", 2024-04-26T22:07:15.945 INFO:teuthology.orchestra.run.smithi055.stdout: "telemetry", 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "volumes" 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "last_failure_osd_epoch": 3, 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "active_clients": [ 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "devicehealth", 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "addrvec": [ 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v2", 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.55:0", 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 3125130596 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.946 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "libcephsqlite", 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "addrvec": [ 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v2", 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.55:0", 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 591233485 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "rbd_support", 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: "addrvec": [ 2024-04-26T22:07:15.947 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v2", 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.55:0", 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 1425243009 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "volumes", 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "addrvec": [ 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v2", 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.55:0", 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 892508231 2024-04-26T22:07:15.948 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "volumes", 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: "addrvec": [ 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: { 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: "type": "v2", 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: "addr": "172.21.15.55:0", 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: "nonce": 3206114798 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-04-26T22:07:15.949 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:07:15.951 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:14 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/676463425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:07:15.951 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:14 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2349458615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:07:15.951 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:14 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3794922982' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T22:07:15.951 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:14 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/676463425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:07:15.951 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:14 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2349458615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:07:15.951 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:14 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3794922982' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T22:07:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.648+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18740a3c40 msgr2=0x7f18740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.648+0000 7f188aac8640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18740a3c40 0x7f18740a6100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f188419a050 tx=0x7f1870005d90 comp rx=0 tx=0).stop 2024-04-26T22:07:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.648+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 msgr2=0x7f1884198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 0x7f1884198b40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1878006010 tx=0x7f18780042f0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 shutdown_connections 2024-04-26T22:07:15.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f18740a3c40 0x7f18740a6100 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1884104550 0x7f18841a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1884103b90 0x7f1884199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 --2- 172.21.15.55:0/3794922982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18841029a0 0x7f1884198b40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 >> 172.21.15.55:0/3794922982 conn(0x7f18840fdf30 msgr2=0x7f1884106e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 shutdown_connections 2024-04-26T22:07:15.953 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:14.649+0000 7f188aac8640 1 -- 172.21.15.55:0/3794922982 wait complete. 2024-04-26T22:07:15.958 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T22:07:15.959 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T22:07:15.959 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:14 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/676463425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:07:15.959 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:14 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2349458615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:07:15.959 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:14 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3794922982' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T22:07:15.984 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:15 smithi055 ceph-mon[29221]: pgmap v781: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.4 MiB/s wr, 213 op/s 2024-04-26T22:07:16.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:15 smithi064 ceph-mon[32765]: pgmap v781: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.4 MiB/s wr, 213 op/s 2024-04-26T22:07:16.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:15 smithi165 ceph-mon[33296]: pgmap v781: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.4 MiB/s wr, 213 op/s 2024-04-26T22:07:16.229 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:07:18.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:17 smithi064 ceph-mon[32765]: pgmap v782: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.3 KiB/s wr, 81 op/s 2024-04-26T22:07:18.053 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:17 smithi055 ceph-mon[29221]: pgmap v782: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.3 KiB/s wr, 81 op/s 2024-04-26T22:07:18.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:17 smithi165 ceph-mon[33296]: pgmap v782: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.3 KiB/s wr, 81 op/s 2024-04-26T22:07:18.227 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:07:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 266, 2024-04-26T22:07:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 264, 2024-04-26T22:07:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 355536, 2024-04-26T22:07:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 162960, 2024-04-26T22:07:18.227 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 0, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.064289838, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.023718739, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.017528594, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.018322125, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 263, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.057783031, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000219707 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:18.228 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 355536, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 162960 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 6, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 204, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 3264, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 193184, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 2, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.023914148, 2024-04-26T22:07:18.229 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.018283345, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.002197901, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 203, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.034696887, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000170920 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.230 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 193184 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 0, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 2, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 0, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 2, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 1, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.000359490, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.000279808, 2024-04-26T22:07:18.231 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.232 INFO:teuthology.orchestra.run.smithi055.stdout: "cct": { 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "total_workers": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "unhealthy_workers": 0 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "mempool": { 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:18.233 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:18.234 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_items": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:18.235 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_bytes": 148721, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_items": 30, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_bytes": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_items": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_bytes": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_items": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:18.236 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_items": 0, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_bytes": 928, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_items": 2, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_items": 0, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_items": 0 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 268, 2024-04-26T22:07:18.237 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 317946, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 268, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 268, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 317946, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:18.238 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:07:19.073 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-26T22:07:19.073 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-26T22:07:19.073 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T22:07:19.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T22:07:19.376 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:07:19.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:19 smithi055 ceph-mon[29221]: pgmap v783: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 62 op/s 2024-04-26T22:07:20.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:19 smithi064 ceph-mon[32765]: pgmap v783: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 62 op/s 2024-04-26T22:07:20.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:19 smithi165 ceph-mon[33296]: pgmap v783: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 62 op/s 2024-04-26T22:07:21.607 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:07:21.607 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 4, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 4, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 2944, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 448, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 0, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.001671221, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.000043267, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.000638397, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:21.608 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 3, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000861027, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000287009 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 273, 2024-04-26T22:07:21.609 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 271, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 357536, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 167104, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 2, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 1, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.054550440, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.022590709, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.018674766, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.010371466, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 270, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.053856765, 2024-04-26T22:07:21.610 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000199469 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 357536, 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 167104 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 2, 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 207, 2024-04-26T22:07:21.611 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 193616, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 2, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.023871670, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.018115868, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.001689798, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 205, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.024463962, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000119336 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:21.612 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 193616 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "cct": { 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "total_workers": 0, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "unhealthy_workers": 0 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "mempool": { 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:21.613 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:21.614 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:21.615 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_items": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_bytes": 148704, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_items": 29, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:21.616 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_bytes": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_items": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_bytes": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_items": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_items": 0, 2024-04-26T22:07:21.617 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_bytes": 928, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_items": 2, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_items": 0, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_items": 0 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 275, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 318959, 2024-04-26T22:07:21.618 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 275, 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 275, 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 318959, 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:21.619 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:07:21.902 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:21 smithi055 ceph-mon[29221]: pgmap v784: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 MiB/s wr, 73 op/s 2024-04-26T22:07:22.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:21 smithi064 ceph-mon[32765]: pgmap v784: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 MiB/s wr, 73 op/s 2024-04-26T22:07:22.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:21 smithi165 ceph-mon[33296]: pgmap v784: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 MiB/s wr, 73 op/s 2024-04-26T22:07:22.296 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T22:07:22.296 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T22:07:22.296 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T22:07:22.297 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T22:07:22.559 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:07:23.923 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:23 smithi055 ceph-mon[29221]: pgmap v785: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 21 MiB/s wr, 144 op/s 2024-04-26T22:07:24.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:23 smithi064 ceph-mon[32765]: pgmap v785: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 21 MiB/s wr, 144 op/s 2024-04-26T22:07:24.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:23 smithi165 ceph-mon[33296]: pgmap v785: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 21 MiB/s wr, 144 op/s 2024-04-26T22:07:24.747 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 0, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 2, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 0, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 0, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.001418657, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.000277487, 2024-04-26T22:07:24.808 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.809 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 273, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 269, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 359392, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 164992, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 2, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 1, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.055750756, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.022134049, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.018620809, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.011406653, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 269, 2024-04-26T22:07:24.810 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.050389671, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000187322 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 359392, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 164992 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 2, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 204, 2024-04-26T22:07:24.811 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 193088, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 2, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.019675589, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.014908005, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.001867004, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 202, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.028054706, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000138884 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:24.812 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 193088 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "cct": { 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "total_workers": 0, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "unhealthy_workers": 0 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "mempool": { 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:24.813 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:24.814 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:24.815 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:24.816 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:24.816 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:24.816 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:24.816 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:24.816 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:24.816 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:24.816 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_items": 0, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_bytes": 147960, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_items": 28, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:24.817 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_bytes": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_items": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_bytes": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_items": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_items": 0, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_bytes": 928, 2024-04-26T22:07:24.818 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_items": 2, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_items": 0, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_items": 0 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 271, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 318439, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 271, 2024-04-26T22:07:24.819 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 271, 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 318439, 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:24.820 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:07:25.521 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-26T22:07:25.521 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-26T22:07:25.522 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T22:07:25.522 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T22:07:25.784 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T22:07:26.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:25 smithi064 ceph-mon[32765]: pgmap v786: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 34 MiB/s wr, 223 op/s 2024-04-26T22:07:26.036 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:25 smithi055 ceph-mon[29221]: pgmap v786: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 34 MiB/s wr, 223 op/s 2024-04-26T22:07:26.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:25 smithi165 ceph-mon[33296]: pgmap v786: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 34 MiB/s wr, 223 op/s 2024-04-26T22:07:27.635 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-04-26T22:07:27.635 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:27.635 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 0, 2024-04-26T22:07:27.635 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 2, 2024-04-26T22:07:27.635 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 0, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 0, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.001619370, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.000498461, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:27.636 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 280, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 276, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 361360, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 168992, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 2, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 1, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.055795493, 2024-04-26T22:07:27.637 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.022184067, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.018363463, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.011602757, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 276, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.043391489, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000157215 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.638 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 361360, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 168992 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_messages": 2, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages": 209, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_bytes": 193968, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_active_connections": 2, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_total_time": 0.020197718, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_send_time": 0.015064609, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_recv_time": 0.001774138, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:27.639 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 207, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.041022101, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000198174 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "msgr_send_encrypted_bytes": 193968 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "cct": { 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "total_workers": 0, 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: "unhealthy_workers": 0 2024-04-26T22:07:27.640 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "mempool": { 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:27.641 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:27.642 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_items": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_bytes": 148704, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_anon_items": 29, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_bytes": 0, 2024-04-26T22:07:27.643 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_items": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_bytes": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_items": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "pgmap_items": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_bytes": 928, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "mds_co_items": 2, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_1_items": 0, 2024-04-26T22:07:27.644 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "unittest_2_items": 0 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "val": 0, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "max": 104857600, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "get_started": 0, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "get": 278, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "get_sum": 319413, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "get_or_fail_success": 278, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "take": 0, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "take_sum": 0, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "put": 278, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "put_sum": 319413, 2024-04-26T22:07:27.645 INFO:teuthology.orchestra.run.smithi055.stdout: "wait": { 2024-04-26T22:07:27.646 INFO:teuthology.orchestra.run.smithi055.stdout: "avgcount": 0, 2024-04-26T22:07:27.646 INFO:teuthology.orchestra.run.smithi055.stdout: "sum": 0.000000000, 2024-04-26T22:07:27.646 INFO:teuthology.orchestra.run.smithi055.stdout: "avgtime": 0.000000000 2024-04-26T22:07:27.646 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:27.646 INFO:teuthology.orchestra.run.smithi055.stdout: } 2024-04-26T22:07:27.646 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-04-26T22:07:27.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:27 smithi055 ceph-mon[29221]: pgmap v787: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 39 MiB/s wr, 256 op/s 2024-04-26T22:07:28.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:27 smithi064 ceph-mon[32765]: pgmap v787: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 39 MiB/s wr, 256 op/s 2024-04-26T22:07:28.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:27 smithi165 ceph-mon[33296]: pgmap v787: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 39 MiB/s wr, 256 op/s 2024-04-26T22:07:28.371 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T22:07:28.371 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T22:07:28.372 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T22:07:28.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T22:07:28.693 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:07:30.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:29 smithi064 ceph-mon[32765]: pgmap v788: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 59 MiB/s wr, 383 op/s 2024-04-26T22:07:30.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:29 smithi165 ceph-mon[33296]: pgmap v788: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 59 MiB/s wr, 383 op/s 2024-04-26T22:07:30.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:29 smithi055 ceph-mon[29221]: pgmap v788: 97 pgs: 97 active+clean; 4.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 59 MiB/s wr, 383 op/s 2024-04-26T22:07:31.014 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-26T22:07:31.014 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:31.014 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 6390, 2024-04-26T22:07:31.014 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 6699, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1849111, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 557691863, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 30, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 3.477944054, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 2.249707555, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.208938760, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.796591898, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6700, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.568129462, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000234049 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.015 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2232, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000783223, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000350 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 6091, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 6204, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1808468, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 882660987, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 172, 2024-04-26T22:07:31.016 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 5, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 3.875348216, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 2.489914492, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.258860376, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 1.078675734, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 6207, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.907437543, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000307304 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1582, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000506548, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000320 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.017 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448992 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 826542, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 828123, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 256660682, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 462141535, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 21, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 8, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 44.243963209, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 19.089695200, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 62.096813506, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.936168966, 2024-04-26T22:07:31.018 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 828115, 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 94.831464243, 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000114514 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 670785, 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.380890933, 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000567 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 898688, 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 235824 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-04-26T22:07:31.019 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2747, 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 737.281588668, 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.268395190 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-04-26T22:07:31.020 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1462, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.047635801, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000032582 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 95, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 95, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 95, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 61.164065840, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.643832272 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-04-26T22:07:31.021 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 1, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 31, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 1, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 24, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 2, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 9, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 13, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 20, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 4, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 5, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 5, 2024-04-26T22:07:31.022 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 3, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 140, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 129, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 25, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 1, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 3, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-04-26T22:07:31.023 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 693, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 4, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 234, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 693, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 237, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 24, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 213, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 456, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 1, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 24, 2024-04-26T22:07:31.024 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 3417, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 821833, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 821777, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 3, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 56 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 5, 2024-04-26T22:07:31.025 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 3, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 22, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 19, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 0, 2024-04-26T22:07:31.026 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 24, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 1, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 1, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 823602, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 712821, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 712821, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 110781, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 12290, 2024-04-26T22:07:31.027 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 967, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 839, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 839, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 128, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 10, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 14, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 1214048831, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 1416099890, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 0, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 8439, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3208.642558527, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.380215968 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.028 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 0 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 24, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 70, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 46, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 18, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 30, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 12, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 24, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 68, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 44, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 4, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 35, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 31, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 94860, 2024-04-26T22:07:31.029 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 231800 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 131, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 95, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 27, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 22, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.037425798, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001701172 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-04-26T22:07:31.030 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 15, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.002823411, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000188227 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2, 2024-04-26T22:07:31.031 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.001354956, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000677478 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000146147, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000146147 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.032 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-04-26T22:07:31.033 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.034 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.034 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.034 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-04-26T22:07:31.034 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 10, 2024-04-26T22:07:31.034 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.015407330, 2024-04-26T22:07:31.034 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001540733 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2024-04-26T22:07:31.035 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000377841, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000377841 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 60.426728253, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 5.035560687 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.002117075, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001058537 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.036 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.259768245, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.064942061 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.037 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.001263307, 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001263307 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-04-26T22:07:31.038 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-04-26T22:07:31.039 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 7, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.016509771, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002358538 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 18, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.400143706, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.022230205 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-04-26T22:07:31.040 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 2, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 0, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 2, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 34, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 17, 2024-04-26T22:07:31.041 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 928, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:31.042 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:31.043 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 288119, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 102, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:31.044 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7816, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 164, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 98843, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 908, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:31.045 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 12786, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 1671361406, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 37, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 12786, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12786, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 483.913523376, 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.037847139 2024-04-26T22:07:31.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 12786, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 12920 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 12786, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 743, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 12043, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 1, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 44, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 25, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 8440, 2024-04-26T22:07:31.047 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 80, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 1, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 2199, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 718, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 88, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-04-26T22:07:31.048 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-04-26T22:07:31.049 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 89, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 5, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 12, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-04-26T22:07:31.050 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 1309, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 13 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 1, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 2 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-04-26T22:07:31.051 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 521, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 9, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 1563, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 19, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 839019, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 197900252, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:31.052 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 839019, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 839019, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 197900252, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:31.053 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 12786, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 1671363317, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 12786, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1671363317, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.054 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 12786, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 12786, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 12786, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 12786, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.055 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 19, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1927, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 19, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 1, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1927, 2024-04-26T22:07:31.056 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x55a25040c500": { 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 2646, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 823602, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1411905586, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 823602, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:31.057 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 8439, 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1411902940, 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:31.058 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-26T22:07:32.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:31 smithi064 ceph-mon[32765]: pgmap v789: 97 pgs: 97 active+clean; 4.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 60 MiB/s wr, 388 op/s 2024-04-26T22:07:32.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:31 smithi165 ceph-mon[33296]: pgmap v789: 97 pgs: 97 active+clean; 4.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 60 MiB/s wr, 388 op/s 2024-04-26T22:07:32.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:31 smithi055 ceph-mon[29221]: pgmap v789: 97 pgs: 97 active+clean; 4.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 60 MiB/s wr, 388 op/s 2024-04-26T22:07:32.231 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-04-26T22:07:32.231 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-26T22:07:32.231 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T22:07:32.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T22:07:32.486 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:07:34.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:33 smithi165 ceph-mon[33296]: pgmap v790: 97 pgs: 97 active+clean; 4.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 50 MiB/s wr, 325 op/s 2024-04-26T22:07:34.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:33 smithi055 ceph-mon[29221]: pgmap v790: 97 pgs: 97 active+clean; 4.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 50 MiB/s wr, 325 op/s 2024-04-26T22:07:34.196 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:33 smithi064 ceph-mon[32765]: pgmap v790: 97 pgs: 97 active+clean; 4.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 50 MiB/s wr, 325 op/s 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1281, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1274, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 728160, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 485218, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.159943792, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.055819175, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.044863259, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.046944345, 2024-04-26T22:07:34.853 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1273, 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.083205624, 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000065361 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 395184, 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 170080 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.854 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1833, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2037, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 474560, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 1027384, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.147069804, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.038150680, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.036908280, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.061566106, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2036, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.095386357, 2024-04-26T22:07:34.855 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000046849 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 446336 2024-04-26T22:07:34.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 53, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 55, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 15055, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 16861, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.005274772, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.001243302, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002167125, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.001248672, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 53, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.003241796, 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000061165 2024-04-26T22:07:34.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-04-26T22:07:34.858 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4774, 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.144098368, 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000030183 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-04-26T22:07:34.859 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-04-26T22:07:34.860 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 6, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 2, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 4, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 0, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 4, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 0, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 3, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 0, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 0, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-04-26T22:07:34.861 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 0, 2024-04-26T22:07:34.862 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T22:07:34.863 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 0 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-04-26T22:07:34.864 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-04-26T22:07:34.865 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 6, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 2, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 2, 2024-04-26T22:07:34.866 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 4201344, 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 4200224, 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 6 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 8, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 8, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 0, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 6, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 6, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 0, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 6, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 6, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 0, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 0, 2024-04-26T22:07:34.868 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 49756, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 215416 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 0, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 0, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 0, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.869 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.871 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.872 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.873 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.874 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.874 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-04-26T22:07:34.874 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.874 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.874 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.876 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.877 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.879 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.880 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 2, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 0, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 2, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 0, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 0, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 1987, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:34.881 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:34.882 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:34.883 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 189401, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 51, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:34.884 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7816, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 164, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-04-26T22:07:34.885 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 30157, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 105, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 2871, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 21, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 3, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 2871, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2871, 2024-04-26T22:07:34.886 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3.297935275, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001148706 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2871, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2872 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 2871, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 2871, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-04-26T22:07:34.887 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 1870, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 999, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 1, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-04-26T22:07:34.888 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-04-26T22:07:34.889 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 89, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-04-26T22:07:34.890 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 4, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 5, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 6, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 1, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 0, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.891 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 0, 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.892 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 3163, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 915229, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 3163, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3163, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 915229, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.893 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:34.894 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 2871, 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 7045, 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 2871, 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 7045, 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 2871, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 2871, 2024-04-26T22:07:34.896 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 2871, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 2871, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:34.897 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x55934a6126e0": { 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:34.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.899 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:34.900 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-26T22:07:35.896 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-26T22:07:35.896 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-26T22:07:35.896 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T22:07:35.896 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T22:07:36.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:35 smithi064 ceph-mon[32765]: pgmap v791: 97 pgs: 97 active+clean; 3.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 38 MiB/s wr, 274 op/s 2024-04-26T22:07:36.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:35 smithi165 ceph-mon[33296]: pgmap v791: 97 pgs: 97 active+clean; 3.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 38 MiB/s wr, 274 op/s 2024-04-26T22:07:36.148 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:07:36.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:35 smithi055 ceph-mon[29221]: pgmap v791: 97 pgs: 97 active+clean; 3.1 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 38 MiB/s wr, 274 op/s 2024-04-26T22:07:37.922 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:37 smithi064 ceph-mon[32765]: pgmap v792: 97 pgs: 97 active+clean; 2.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 26 MiB/s wr, 203 op/s 2024-04-26T22:07:38.055 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-26T22:07:38.055 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:38.055 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1498, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1192, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 400414, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 537752, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 9, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.138616031, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.055333152, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.088800856, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.007641304, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1190, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.085292734, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000071674 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.056 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2451, 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000635127, 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000259 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 50, 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 261, 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 9262, 2024-04-26T22:07:38.057 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 712129, 2024-04-26T22:07:38.058 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 26, 2024-04-26T22:07:38.058 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-04-26T22:07:38.058 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.034269739, 2024-04-26T22:07:38.058 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.015896408, 2024-04-26T22:07:38.058 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.015046663, 2024-04-26T22:07:38.058 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000503855, 2024-04-26T22:07:38.058 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:38.058 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 259, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.033338812, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000128721 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 49, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000021371, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000436 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1888, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 538736 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 411, 2024-04-26T22:07:38.059 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 398, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 426362, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 652754, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 45, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 6, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.075034660, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.020849123, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.044189919, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.009053229, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 398, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.033353963, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000083803 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:38.060 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 38, 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000006872, 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000180 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 416784, 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 188864 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-04-26T22:07:38.061 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 86, 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.005888565, 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000068471 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2, 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000007407, 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000003703 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-04-26T22:07:38.062 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 2, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 1, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000943680, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000943680 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 1, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 11, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 18, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 0, 2024-04-26T22:07:38.063 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 18, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 0, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 16, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 0, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 2, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 2, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 3, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 4, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 1, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 2, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-04-26T22:07:38.064 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 1, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 3, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 36, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 4, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 234, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 0, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 36, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T22:07:38.065 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 18, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 18, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 18, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 2, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 3, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 2, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 1 2024-04-26T22:07:38.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 3, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-04-26T22:07:38.067 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-04-26T22:07:38.068 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 6, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 6, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 2, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 2, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 2, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-04-26T22:07:38.069 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 4201344, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 0, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 5, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.016123074, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003224614 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 0 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 20, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 20, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 0, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 17, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 18, 2024-04-26T22:07:38.070 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 1, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 18, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 18, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 0, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 2, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 2, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 51944, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 223608 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 3, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 2, 2024-04-26T22:07:38.071 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 20, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000943680, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000943680 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.072 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.073 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.074 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-04-26T22:07:38.075 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-04-26T22:07:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.076 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.077 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-04-26T22:07:38.078 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-04-26T22:07:38.079 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.080 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-04-26T22:07:38.081 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-04-26T22:07:38.082 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-04-26T22:07:38.083 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 2, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 0, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 2, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 1, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 1, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 881, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:38.084 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:38.085 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:38.086 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 287013, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 96, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 7816, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 164, 2024-04-26T22:07:38.087 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:38.088 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:38.088 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:38.088 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-04-26T22:07:38.088 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 80674, 2024-04-26T22:07:38.088 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 388, 2024-04-26T22:07:38.088 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:38.088 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 80, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 18130, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 28, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 80, 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-04-26T22:07:38.089 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 80, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.516989626, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.006462370 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 80, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 105 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 80, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 36, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 44, 2024-04-26T22:07:38.090 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 10, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 5, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 7, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 6, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-04-26T22:07:38.091 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 36, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 20, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-04-26T22:07:38.092 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-04-26T22:07:38.093 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 89, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 4, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 10, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 2, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 20, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 1 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-04-26T22:07:38.094 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 1, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 0, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2024-04-26T22:07:38.095 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 1955, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 639289, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 1955, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 1955, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 639289, 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:38.096 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 80, 2024-04-26T22:07:38.097 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 18130, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 80, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 18130, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:38.098 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 80, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 80, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 80, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 80, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-04-26T22:07:38.099 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.100 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x55e1f4c36500": { 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 6, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 7040, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 6, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 5, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 7040, 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:38.101 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:38.102 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:38.102 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:38.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.102 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:38.102 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-26T22:07:38.102 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:37 smithi165 ceph-mon[33296]: pgmap v792: 97 pgs: 97 active+clean; 2.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 26 MiB/s wr, 203 op/s 2024-04-26T22:07:38.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:37 smithi055 ceph-mon[29221]: pgmap v792: 97 pgs: 97 active+clean; 2.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 26 MiB/s wr, 203 op/s 2024-04-26T22:07:38.899 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T22:07:38.899 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-04-26T22:07:38.899 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T22:07:38.900 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T22:07:39.150 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T22:07:40.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:39 smithi064 ceph-mon[32765]: pgmap v793: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 21 MiB/s wr, 183 op/s 2024-04-26T22:07:40.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:39 smithi165 ceph-mon[33296]: pgmap v793: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 21 MiB/s wr, 183 op/s 2024-04-26T22:07:40.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:39 smithi055 ceph-mon[29221]: pgmap v793: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 21 MiB/s wr, 183 op/s 2024-04-26T22:07:40.972 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001164523, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000236629, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:40.973 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2, 2024-04-26T22:07:40.974 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 212, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 194496, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.012797189, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.009035576, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001530664, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 210, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.014216403, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000067697 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:40.975 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 194496 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 284, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 280, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 362448, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 171552, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-04-26T22:07:40.976 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.044137449, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.016422269, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.016099367, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.008975740, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 280, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.024344104, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000086943 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:40.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 362448, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 171552 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:40.978 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:40.979 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:40.980 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148761, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 31, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:40.981 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-04-26T22:07:40.982 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 282, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 319933, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 282, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 282, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 319933, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-04-26T22:07:40.983 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-04-26T22:07:41.684 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-26T22:07:41.684 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-26T22:07:41.684 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T22:07:41.684 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T22:07:42.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:41 smithi064 ceph-mon[32765]: pgmap v794: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 710 KiB/s wr, 57 op/s 2024-04-26T22:07:42.062 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:07:42.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:41 smithi165 ceph-mon[33296]: pgmap v794: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 710 KiB/s wr, 57 op/s 2024-04-26T22:07:42.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:41 smithi055 ceph-mon[29221]: pgmap v794: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 710 KiB/s wr, 57 op/s 2024-04-26T22:07:44.020 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:43 smithi165 ceph-mon[33296]: pgmap v795: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 182 KiB/s wr, 51 op/s 2024-04-26T22:07:44.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:43 smithi055 ceph-mon[29221]: pgmap v795: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 182 KiB/s wr, 51 op/s 2024-04-26T22:07:44.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:43 smithi064 ceph-mon[32765]: pgmap v795: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 182 KiB/s wr, 51 op/s 2024-04-26T22:07:44.603 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T22:07:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:44.603 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2535, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2537, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 550222878, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 806140, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.641444283, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.064252627, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.464900552, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.086166029, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2535, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.133445085, 2024-04-26T22:07:44.604 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000052641 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2026, 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2017, 2024-04-26T22:07:44.605 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 611694615, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 721534, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.563258927, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.054331018, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.428348326, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.062395449, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2017, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.084823882, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000042054 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.606 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 398560, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 171328 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 881, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1088, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 251906735, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 816725, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-04-26T22:07:44.607 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.191083047, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.033099387, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.122196954, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.026605670, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1086, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.040920671, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000037680 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.608 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 537520 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 7587, 2024-04-26T22:07:44.609 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.210834490, 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000027788 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-26T22:07:44.610 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-26T22:07:44.611 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 34, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 2, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 32, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 30, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 3, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-26T22:07:44.612 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-26T22:07:44.613 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T22:07:44.614 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-26T22:07:44.615 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T22:07:44.616 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-26T22:07:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-26T22:07:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-26T22:07:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-26T22:07:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-26T22:07:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.617 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-26T22:07:44.618 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-26T22:07:44.618 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-04-26T22:07:44.618 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-26T22:07:44.618 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-26T22:07:44.618 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 823603, 2024-04-26T22:07:44.618 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 142, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 69, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 1214048831, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 1416100489, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 1416099890, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.619 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 823603 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 14, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 36, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 22, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 9, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 21, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 12, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 34, 2024-04-26T22:07:44.620 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 57, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 23, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 61976, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-26T22:07:44.621 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.622 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.624 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.625 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-26T22:07:44.626 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.626 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.626 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.626 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-26T22:07:44.626 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.626 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-26T22:07:44.627 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-26T22:07:44.628 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.629 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.630 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.631 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-26T22:07:44.632 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 2, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-04-26T22:07:44.633 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 1982, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:44.635 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 238785, 2024-04-26T22:07:44.636 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 79, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7816, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 164, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:44.637 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 67476, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 488, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 5143, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 26, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 4, 2024-04-26T22:07:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 5143, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5143, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 33.088527123, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006433701 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5143, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5144 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 5143, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 5143, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-26T22:07:44.639 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 3222, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 1918, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-26T22:07:44.640 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 2, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-26T22:07:44.641 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 89, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-26T22:07:44.642 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 4, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 11, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 11, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-26T22:07:44.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:44.644 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 5438, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1413316791, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 5438, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5438, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1413316791, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.645 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5143, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1412031028, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5143, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1412031028, 2024-04-26T22:07:44.646 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:44.647 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 5143, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 5143, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 5143, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 5143, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:44.648 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:44.649 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x5624acf0a6e0": { 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:44.650 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T22:07:44.651 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T22:07:44.651 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:44.651 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:44.651 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:44.651 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:44.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:44.651 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T22:07:45.623 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-26T22:07:45.623 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-26T22:07:45.623 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T22:07:45.624 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T22:07:45.871 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:07:45.995 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:45 smithi165 ceph-mon[33296]: pgmap v796: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 81 KiB/s wr, 48 op/s 2024-04-26T22:07:46.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:45 smithi055 ceph-mon[29221]: pgmap v796: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 81 KiB/s wr, 48 op/s 2024-04-26T22:07:46.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:45 smithi064 ceph-mon[32765]: pgmap v796: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 81 KiB/s wr, 48 op/s 2024-04-26T22:07:47.800 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:47 smithi165 ceph-mon[33296]: pgmap v797: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 26 op/s 2024-04-26T22:07:47.820 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T22:07:47.820 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:47.820 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 979, 2024-04-26T22:07:47.820 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 981, 2024-04-26T22:07:47.820 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 258376, 2024-04-26T22:07:47.820 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 311386, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.089169543, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.025930461, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.024111085, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.030363936, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 979, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.035123971, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000035877 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.821 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1152, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1143, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 609296, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 444343, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-04-26T22:07:47.822 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.138019483, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.035311966, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.041674964, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.053123346, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1143, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.037329158, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000032658 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.823 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 398944, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 172448 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 974, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1183, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 327088, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 758886, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.130754504, 2024-04-26T22:07:47.824 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.042332561, 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.033404007, 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.043014164, 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1181, 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.059927896, 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000050743 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.825 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 449472 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4722, 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.155460845, 2024-04-26T22:07:47.826 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000032922 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-04-26T22:07:47.827 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 0, 2024-04-26T22:07:47.828 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-04-26T22:07:47.829 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-26T22:07:47.830 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-04-26T22:07:47.831 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-04-26T22:07:47.832 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.833 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 4, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-26T22:07:47.834 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 4195591, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 4195283, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 4 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.835 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 3, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 3, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 0, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 1, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 1, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 0, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 0, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 0, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 0, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 50344, 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215416 2024-04-26T22:07:47.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.837 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-26T22:07:47.838 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.839 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.840 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-26T22:07:47.841 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.841 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.841 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.841 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-26T22:07:47.841 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.842 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-26T22:07:47.843 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-26T22:07:47.844 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-26T22:07:47.845 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 2, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 1984, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-26T22:07:47.849 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:47.850 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:47.851 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:47.852 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:47.854 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:47.854 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:47.854 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-26T22:07:47.854 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 189440, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 52, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7816, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 164, 2024-04-26T22:07:47.855 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 8512, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 13, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-26T22:07:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 2807, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 2807, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2807, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 34.703504337, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.012363200 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2807, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2808 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.857 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 2807, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2807, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1826, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 979, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-26T22:07:47.858 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-26T22:07:47.859 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-26T22:07:47.860 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 89, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 4, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 5, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 5, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-26T22:07:47.861 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.862 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 3101, 2024-04-26T22:07:47.863 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 897638, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 3101, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 3101, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 897638, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-26T22:07:47.864 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2807, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1292, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2807, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1292, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2807, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2807, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2807, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2807, 2024-04-26T22:07:47.866 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:47.867 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x562c988a86e0": { 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:47.868 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:47.869 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T22:07:48.122 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:47 smithi064 ceph-mon[32765]: pgmap v797: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 26 op/s 2024-04-26T22:07:48.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:47 smithi055 ceph-mon[29221]: pgmap v797: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 26 op/s 2024-04-26T22:07:48.678 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-26T22:07:48.679 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-26T22:07:48.679 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T22:07:48.679 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T22:07:48.920 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:07:50.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:49 smithi165 ceph-mon[33296]: pgmap v798: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 18 op/s 2024-04-26T22:07:50.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:49 smithi055 ceph-mon[29221]: pgmap v798: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 18 op/s 2024-04-26T22:07:50.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:49 smithi064 ceph-mon[32765]: pgmap v798: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 18 op/s 2024-04-26T22:07:50.910 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T22:07:50.910 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:50.910 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 785, 2024-04-26T22:07:50.910 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 876, 2024-04-26T22:07:50.910 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 183843, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 1035040, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 57, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.112035285, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.030594493, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.075389758, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.006394276, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 874, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.047465435, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000054308 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:50.911 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1535, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000456358, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000297 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 106, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 312, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 50029, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 643605, 2024-04-26T22:07:50.912 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 14, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.023690662, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.012072999, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.010535095, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000340888, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 312, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.030875079, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000098958 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 166, 2024-04-26T22:07:50.913 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000047905, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000288 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 539136 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 359, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 349, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 414651, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 183219, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 10, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-04-26T22:07:50.914 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.054696064, 2024-04-26T22:07:50.918 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.019638365, 2024-04-26T22:07:50.918 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.023721308, 2024-04-26T22:07:50.918 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.008915295, 2024-04-26T22:07:50.918 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:50.918 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 347, 2024-04-26T22:07:50.918 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.018920195, 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000054525 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 47, 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000048214, 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000001025 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 411408, 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 178832 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-26T22:07:50.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 50, 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001593134, 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000031862 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000010905, 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000005452 2024-04-26T22:07:50.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 5, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 2, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000320493, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000160246 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 3, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-26T22:07:50.921 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 11, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 11, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 11, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 10, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 1, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 1, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 2, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 5, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 2, 2024-04-26T22:07:50.922 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 1, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-26T22:07:50.923 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 1, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T22:07:50.925 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 1, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 2 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.926 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 2, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-26T22:07:50.927 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 4, 2024-04-26T22:07:50.928 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 4, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 2, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 2, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 4195591, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-26T22:07:50.929 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.005353145, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001784381 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 14, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 14, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 0, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 12, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 13, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 1, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 11, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 11, 2024-04-26T22:07:50.930 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 0, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 1, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 2, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 1, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 52048, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 223608 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 6, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 5, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 25, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-26T22:07:50.931 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000154852, 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000154852 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.932 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.933 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000165641, 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000165641 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.934 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-26T22:07:50.935 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.936 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.936 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.936 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-26T22:07:50.936 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.936 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.937 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-26T22:07:50.938 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-26T22:07:50.939 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.940 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-26T22:07:50.942 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 2, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 1, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 946, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.943 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:50.944 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:50.945 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 239285, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 82, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-26T22:07:50.946 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7816, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 164, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 55440, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 214, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:50.947 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 38, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 11499, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 28, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 38, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 38, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.233996525, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006157803 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.948 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 38, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 60 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 38, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 0, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 38, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 0, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 3, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 6, 2024-04-26T22:07:50.949 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 6, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-26T22:07:50.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-26T22:07:50.951 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 89, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 4, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 8, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 13, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 5, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-26T22:07:50.952 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 14, 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 1 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-26T22:07:50.953 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 1246, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 516618, 2024-04-26T22:07:50.954 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 1246, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1246, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 516618, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T22:07:50.955 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 38, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 11499, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 38, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 11499, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-26T22:07:50.956 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 38, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 38, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 38, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 38, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.958 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55ad46ffa500": { 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 4, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1287, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 4, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 3, 2024-04-26T22:07:50.959 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1287, 2024-04-26T22:07:50.960 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:50.960 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:50.960 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:50.960 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:50.960 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.960 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:50.960 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T22:07:51.770 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-26T22:07:51.770 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-26T22:07:51.770 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T22:07:51.771 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid c84f1140-0415-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T22:07:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T22:07:52.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:51 smithi165 ceph-mon[33296]: pgmap v799: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T22:07:52.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:51 smithi055 ceph-mon[29221]: pgmap v799: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T22:07:52.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:51 smithi064 ceph-mon[32765]: pgmap v799: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T22:07:53.939 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:53 smithi165 ceph-mon[33296]: pgmap v800: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T22:07:53.965 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-26T22:07:53.965 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T22:07:53.965 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 4, 2024-04-26T22:07:53.965 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 4, 2024-04-26T22:07:53.965 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2944, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 448, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 0, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.001294619, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.000036543, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.000510341, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000703879, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000234626 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.966 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 284, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 282, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 360592, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 173664, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 2, 2024-04-26T22:07:53.967 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 1, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.042129299, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.016470716, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.014443686, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.008312023, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 281, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013554390, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000048236 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:53.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 360592, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 173664 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 215, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 320, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 195024, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.012781435, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.009636286, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.000903876, 2024-04-26T22:07:53.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 213, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.012831003, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000060239 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 195024 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.970 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T22:07:53.971 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T22:07:53.972 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 148722, 2024-04-26T22:07:53.973 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 30, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-26T22:07:53.974 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 928, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 2, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 286, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 320453, 2024-04-26T22:07:53.975 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 286, 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 286, 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 320453, 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-26T22:07:53.976 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-26T22:07:54.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:53 smithi055 ceph-mon[29221]: pgmap v800: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T22:07:54.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:53 smithi064 ceph-mon[32765]: pgmap v800: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T22:07:54.863 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-26T22:07:54.863 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-26T22:07:54.864 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T22:07:54.878 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T22:07:54.879 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T22:07:54.879 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:07:54.911 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-04-26T22:07:54.912 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:07:54.969 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:07:55.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.114+0000 7efca4de2640 1 -- 172.21.15.55:0/2324773829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0105d40 msgr2=0x7efca0108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.114+0000 7efca4de2640 1 --2- 172.21.15.55:0/2324773829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0105d40 0x7efca0108130 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efc88009900 tx=0x7efc8802a410 comp rx=0 tx=0).stop 2024-04-26T22:07:55.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.114+0000 7efca4de2640 1 -- 172.21.15.55:0/2324773829 shutdown_connections 2024-04-26T22:07:55.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.114+0000 7efca4de2640 1 --2- 172.21.15.55:0/2324773829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efca0108670 0x7efca010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.114+0000 7efca4de2640 1 --2- 172.21.15.55:0/2324773829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0105d40 0x7efca0108130 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.114+0000 7efca4de2640 1 --2- 172.21.15.55:0/2324773829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efca0103410 0x7efca0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.114+0000 7efca4de2640 1 -- 172.21.15.55:0/2324773829 >> 172.21.15.55:0/2324773829 conn(0x7efca00fce20 msgr2=0x7efca00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:55.117 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.114+0000 7efca4de2640 1 -- 172.21.15.55:0/2324773829 shutdown_connections 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 -- 172.21.15.55:0/2324773829 wait complete. 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 Processor -- start 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 -- start start 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0103410 0x7efca0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efca0105d40 0x7efca01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efca0108670 0x7efca01a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7efca010e4d0 con 0x7efca0103410 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efca010e350 con 0x7efca0105d40 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.115+0000 7efca4de2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efca010e650 con 0x7efca0108670 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0103410 0x7efca0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0103410 0x7efca0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36546/0 (socket says 172.21.15.55:36546) 2024-04-26T22:07:55.118 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 -- 172.21.15.55:0/3871674283 learned_addr learned my addr 172.21.15.55:0/3871674283 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9ed9d640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efca0108670 0x7efca01a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc97fff640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efca0105d40 0x7efca01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 -- 172.21.15.55:0/3871674283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efca0108670 msgr2=0x7efca01a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efca0108670 0x7efca01a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 -- 172.21.15.55:0/3871674283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efca0105d40 msgr2=0x7efca01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efca0105d40 0x7efca01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 -- 172.21.15.55:0/3871674283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca01a07d0 con 0x7efca0103410 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9ed9d640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efca0108670 0x7efca01a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc9e59c640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0103410 0x7efca0198bb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e9f0 tx=0x7efc9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc977fe640 1 -- 172.21.15.55:0/3871674283 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9000cd80 con 0x7efca0103410 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc977fe640 1 -- 172.21.15.55:0/3871674283 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900054d0 con 0x7efca0103410 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca01a0ac0 con 0x7efca0103410 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efc977fe640 1 -- 172.21.15.55:0/3871674283 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90010690 con 0x7efca0103410 2024-04-26T22:07:55.119 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.116+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca01a0f90 con 0x7efca0103410 2024-04-26T22:07:55.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.117+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca00fe910 con 0x7efca0103410 2024-04-26T22:07:55.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.118+0000 7efc977fe640 1 -- 172.21.15.55:0/3871674283 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efc90004e30 con 0x7efca0103410 2024-04-26T22:07:55.120 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.119+0000 7efc977fe640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efc8c0a3c40 0x7efc8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.119+0000 7efc977fe640 1 -- 172.21.15.55:0/3871674283 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7efc900d0c10 con 0x7efca0103410 2024-04-26T22:07:55.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.119+0000 7efc97fff640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efc8c0a3c40 0x7efc8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.121 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.120+0000 7efc97fff640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efc8c0a3c40 0x7efc8c0a6100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efc88009230 tx=0x7efc880314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:55.122 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.122+0000 7efc977fe640 1 -- 172.21.15.55:0/3871674283 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efc90054060 con 0x7efca0103410 2024-04-26T22:07:55.314 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.314+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efca002dd40 con 0x7efca0103410 2024-04-26T22:07:55.315 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.315+0000 7efc977fe640 1 -- 172.21.15.55:0/3871674283 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22760 (secure 0 0 0) 0x7efc9009d3f0 con 0x7efca0103410 2024-04-26T22:07:55.316 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:07:55.316 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:52:51.546766+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:07:55.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efc8c0a3c40 msgr2=0x7efc8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efc8c0a3c40 0x7efc8c0a6100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efc88009230 tx=0x7efc880314a0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0103410 msgr2=0x7efca0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0103410 0x7efca0198bb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e9f0 tx=0x7efc9000edf0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 shutdown_connections 2024-04-26T22:07:55.318 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7efc8c0a3c40 0x7efc8c0a6100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efca0108670 0x7efca01a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efca0105d40 0x7efca01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 --2- 172.21.15.55:0/3871674283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efca0103410 0x7efca0198bb0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 >> 172.21.15.55:0/3871674283 conn(0x7efca00fce20 msgr2=0x7efca0103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 shutdown_connections 2024-04-26T22:07:55.319 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.318+0000 7efca4de2640 1 -- 172.21.15.55:0/3871674283 wait complete. 2024-04-26T22:07:55.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T21:46:13.346319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T21:50:54.684639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T21:50:55.670730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:07:55.331 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:07:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/1639807798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b641086d0 msgr2=0x7f9b6410aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/1639807798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b641086d0 0x7f9b6410aac0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c009ca0 tx=0x7f9b5c02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/1639807798 shutdown_connections 2024-04-26T22:07:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/1639807798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b641086d0 0x7f9b6410aac0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/1639807798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b64105da0 0x7f9b64108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/1639807798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b64103470 0x7f9b64105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/1639807798 >> 172.21.15.55:0/1639807798 conn(0x7f9b640fce20 msgr2=0x7f9b640ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:55.439 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/1639807798 shutdown_connections 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.436+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/1639807798 wait complete. 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b6ac7b640 1 Processor -- start 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b6ac7b640 1 -- start start 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b6ac7b640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b64103470 0x7f9b64198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b6ac7b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b64105da0 0x7f9b641990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b6ac7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b641086d0 0x7f9b641a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b6ac7b640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410e530 con 0x7f9b64103470 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b6ac7b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410e3b0 con 0x7f9b641086d0 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b6ac7b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6410e6b0 con 0x7f9b64105da0 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b68a17640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b64103470 0x7f9b64198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b68a17640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b64103470 0x7f9b64198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36566/0 (socket says 172.21.15.55:36566) 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b68a17640 1 -- 172.21.15.55:0/550788126 learned_addr learned my addr 172.21.15.55:0/550788126 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b63fff640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b64105da0 0x7f9b641990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.440 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.437+0000 7f9b69218640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b641086d0 0x7f9b641a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b68a17640 1 -- 172.21.15.55:0/550788126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b64105da0 msgr2=0x7f9b641990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b68a17640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b64105da0 0x7f9b641990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b68a17640 1 -- 172.21.15.55:0/550788126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b641086d0 msgr2=0x7f9b641a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b68a17640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b641086d0 0x7f9b641a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b68a17640 1 -- 172.21.15.55:0/550788126 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b641a07d0 con 0x7f9b64103470 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b68a17640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b64103470 0x7f9b64198bb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400e9f0 tx=0x7f9b5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b61ffb640 1 -- 172.21.15.55:0/550788126 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400cd80 con 0x7f9b64103470 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b61ffb640 1 -- 172.21.15.55:0/550788126 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b540108e0 con 0x7f9b64103470 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b641a0ac0 con 0x7f9b64103470 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b61ffb640 1 -- 172.21.15.55:0/550788126 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54018980 con 0x7f9b64103470 2024-04-26T22:07:55.441 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.438+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b641a1010 con 0x7f9b64103470 2024-04-26T22:07:55.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.439+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b640fe910 con 0x7f9b64103470 2024-04-26T22:07:55.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.440+0000 7f9b61ffb640 1 -- 172.21.15.55:0/550788126 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b54010430 con 0x7f9b64103470 2024-04-26T22:07:55.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.441+0000 7f9b61ffb640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9b4c0a3c40 0x7f9b4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.441+0000 7f9b63fff640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9b4c0a3c40 0x7f9b4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.441+0000 7f9b61ffb640 1 -- 172.21.15.55:0/550788126 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f9b540d0900 con 0x7f9b64103470 2024-04-26T22:07:55.443 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.441+0000 7f9b63fff640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9b4c0a3c40 0x7f9b4c0a6100 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9b6419a0c0 tx=0x7f9b58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:55.445 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.445+0000 7f9b61ffb640 1 -- 172.21.15.55:0/550788126 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9b5409ca90 con 0x7f9b64103470 2024-04-26T22:07:55.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.630+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b6402dcc0 con 0x7f9b64103470 2024-04-26T22:07:55.630 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.630+0000 7f9b61ffb640 1 -- 172.21.15.55:0/550788126 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f9b540a0d90 con 0x7f9b64103470 2024-04-26T22:07:55.630 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-04-26T22:07:55.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9b4c0a3c40 msgr2=0x7f9b4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9b4c0a3c40 0x7f9b4c0a6100 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9b6419a0c0 tx=0x7f9b58005f90 comp rx=0 tx=0).stop 2024-04-26T22:07:55.632 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b64103470 msgr2=0x7f9b64198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b64103470 0x7f9b64198bb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400e9f0 tx=0x7f9b5400eeb0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 shutdown_connections 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f9b4c0a3c40 0x7f9b4c0a6100 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b641086d0 0x7f9b641a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b64105da0 0x7f9b641990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 --2- 172.21.15.55:0/550788126 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b64103470 0x7f9b64198bb0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 >> 172.21.15.55:0/550788126 conn(0x7f9b640fce20 msgr2=0x7f9b64103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 shutdown_connections 2024-04-26T22:07:55.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.632+0000 7f9b6ac7b640 1 -- 172.21.15.55:0/550788126 wait complete. 2024-04-26T22:07:55.644 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:07:55.750 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1027792074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998103b60 msgr2=0x7fb998103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1027792074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998103b60 0x7fb998103fe0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009900 tx=0x7fb98c02a410 comp rx=0 tx=0).stop 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1027792074 shutdown_connections 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1027792074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb998104520 0x7fb99810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1027792074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998103b60 0x7fb998103fe0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1027792074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb998102970 0x7fb998102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1027792074 >> 172.21.15.55:0/1027792074 conn(0x7fb9980fdee0 msgr2=0x7fb998100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1027792074 shutdown_connections 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.748+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1027792074 wait complete. 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb99e6c5640 1 Processor -- start 2024-04-26T22:07:55.751 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb99e6c5640 1 -- start start 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb99e6c5640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998102970 0x7fb998198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb99e6c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb998103b60 0x7fb998199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb99e6c5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb998104520 0x7fb9981a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb99e6c5640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fb99810e480 con 0x7fb998102970 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb99e6c5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb99810e300 con 0x7fb998103b60 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb99e6c5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb99810e600 con 0x7fb998104520 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb997fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998102970 0x7fb998198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb997fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998102970 0x7fb998198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36586/0 (socket says 172.21.15.55:36586) 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb997fff640 1 -- 172.21.15.55:0/1483137042 learned_addr learned my addr 172.21.15.55:0/1483137042 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.749+0000 7fb9977fe640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb998103b60 0x7fb998199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb997fff640 1 -- 172.21.15.55:0/1483137042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb998104520 msgr2=0x7fb9981a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb997fff640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb998104520 0x7fb9981a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.752 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb997fff640 1 -- 172.21.15.55:0/1483137042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb998103b60 msgr2=0x7fb998199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb997fff640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb998103b60 0x7fb998199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb997fff640 1 -- 172.21.15.55:0/1483137042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9981a0800 con 0x7fb998102970 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb9977fe640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb998103b60 0x7fb998199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb997fff640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998102970 0x7fb998198be0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e9c0 tx=0x7fb98800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb9957fa640 1 -- 172.21.15.55:0/1483137042 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98800cd80 con 0x7fb998102970 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb9957fa640 1 -- 172.21.15.55:0/1483137042 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9880035f0 con 0x7fb998102970 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9981a0af0 con 0x7fb998102970 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb9957fa640 1 -- 172.21.15.55:0/1483137042 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988010690 con 0x7fb998102970 2024-04-26T22:07:55.753 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.750+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9981a1090 con 0x7fb998102970 2024-04-26T22:07:55.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.751+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb998109db0 con 0x7fb998102970 2024-04-26T22:07:55.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.752+0000 7fb9957fa640 1 -- 172.21.15.55:0/1483137042 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb9880037a0 con 0x7fb998102970 2024-04-26T22:07:55.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.753+0000 7fb9957fa640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9800a3cc0 0x7fb9800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:55.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.753+0000 7fb9977fe640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9800a3cc0 0x7fb9800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:55.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.753+0000 7fb9957fa640 1 -- 172.21.15.55:0/1483137042 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fb9880d0a30 con 0x7fb998102970 2024-04-26T22:07:55.754 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.754+0000 7fb9977fe640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9800a3cc0 0x7fb9800a6180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb99819a0f0 tx=0x7fb98c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:55.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.758+0000 7fb9957fa640 1 -- 172.21.15.55:0/1483137042 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb98809d270 con 0x7fb998102970 2024-04-26T22:07:55.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:55 smithi055 ceph-mon[29221]: pgmap v801: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T22:07:55.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3871674283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:55.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:55 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/550788126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:07:55.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.948+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb99802dad0 con 0x7fb998102970 2024-04-26T22:07:55.949 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.949+0000 7fb9957fa640 1 -- 172.21.15.55:0/1483137042 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fb99802dad0 con 0x7fb998102970 2024-04-26T22:07:55.949 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2024-04-26T22:07:55.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9800a3cc0 msgr2=0x7fb9800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9800a3cc0 0x7fb9800a6180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb99819a0f0 tx=0x7fb98c0314a0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998102970 msgr2=0x7fb998198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:55.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998102970 0x7fb998198be0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e9c0 tx=0x7fb98800ee80 comp rx=0 tx=0).stop 2024-04-26T22:07:55.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 shutdown_connections 2024-04-26T22:07:55.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fb9800a3cc0 0x7fb9800a6180 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb998104520 0x7fb9981a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.951 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb998103b60 0x7fb998199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 --2- 172.21.15.55:0/1483137042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb998102970 0x7fb998198be0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 >> 172.21.15.55:0/1483137042 conn(0x7fb9980fdee0 msgr2=0x7fb998106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 shutdown_connections 2024-04-26T22:07:55.952 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:55.951+0000 7fb99e6c5640 1 -- 172.21.15.55:0/1483137042 wait complete. 2024-04-26T22:07:55.962 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:07:56.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 -- 172.21.15.55:0/2252886389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 msgr2=0x7f3b68105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 --2- 172.21.15.55:0/2252886389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 0x7f3b68105800 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009900 tx=0x7f3b5002a410 comp rx=0 tx=0).stop 2024-04-26T22:07:56.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 -- 172.21.15.55:0/2252886389 shutdown_connections 2024-04-26T22:07:56.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 --2- 172.21.15.55:0/2252886389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b68108670 0x7f3b6810aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 --2- 172.21.15.55:0/2252886389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b68105d40 0x7f3b68108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 --2- 172.21.15.55:0/2252886389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 0x7f3b68105800 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 -- 172.21.15.55:0/2252886389 >> 172.21.15.55:0/2252886389 conn(0x7f3b680fce20 msgr2=0x7f3b680ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:56.070 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 -- 172.21.15.55:0/2252886389 shutdown_connections 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.067+0000 7f3b6e620640 1 -- 172.21.15.55:0/2252886389 wait complete. 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b6e620640 1 Processor -- start 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b6e620640 1 -- start start 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b6e620640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 0x7f3b68198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b6e620640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b68105d40 0x7f3b68199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b6e620640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b68108670 0x7f3b681a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b6e620640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810e4c0 con 0x7f3b68103410 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b6e620640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810e340 con 0x7f3b68105d40 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b6e620640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6810e640 con 0x7f3b68108670 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 0x7f3b68198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 0x7f3b68198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36602/0 (socket says 172.21.15.55:36602) 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.068+0000 7f3b67fff640 1 -- 172.21.15.55:0/4169633624 learned_addr learned my addr 172.21.15.55:0/4169633624 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b677fe640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b68105d40 0x7f3b68199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b6cbbd640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b68108670 0x7f3b681a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b67fff640 1 -- 172.21.15.55:0/4169633624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b68108670 msgr2=0x7f3b681a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b67fff640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b68108670 0x7f3b681a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b67fff640 1 -- 172.21.15.55:0/4169633624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b68105d40 msgr2=0x7f3b68199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b67fff640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b68105d40 0x7f3b68199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b67fff640 1 -- 172.21.15.55:0/4169633624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b681a0830 con 0x7f3b68103410 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b67fff640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 0x7f3b68198c20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005940 tx=0x7f3b50004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b657fa640 1 -- 172.21.15.55:0/4169633624 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5003f030 con 0x7f3b68103410 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b681a0ac0 con 0x7f3b68103410 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b657fa640 1 -- 172.21.15.55:0/4169633624 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b50003cb0 con 0x7f3b68103410 2024-04-26T22:07:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b657fa640 1 -- 172.21.15.55:0/4169633624 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50035070 con 0x7f3b68103410 2024-04-26T22:07:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.069+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b681a1000 con 0x7f3b68103410 2024-04-26T22:07:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.070+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b680fe910 con 0x7f3b68103410 2024-04-26T22:07:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.071+0000 7f3b657fa640 1 -- 172.21.15.55:0/4169633624 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3b50042030 con 0x7f3b68103410 2024-04-26T22:07:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.072+0000 7f3b657fa640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3b540a3c40 0x7f3b540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.072+0000 7f3b677fe640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3b540a3c40 0x7f3b540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.072+0000 7f3b657fa640 1 -- 172.21.15.55:0/4169633624 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f3b500f0ff0 con 0x7f3b68103410 2024-04-26T22:07:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.073+0000 7f3b677fe640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3b540a3c40 0x7f3b540a6100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3b580058c0 tx=0x7f3b5800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:56.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:55 smithi165 ceph-mon[33296]: pgmap v801: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T22:07:56.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:55 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3871674283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:56.076 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:55 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/550788126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:07:56.077 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.077+0000 7f3b657fa640 1 -- 172.21.15.55:0/4169633624 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3b50003e60 con 0x7f3b68103410 2024-04-26T22:07:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:55 smithi064 ceph-mon[32765]: pgmap v801: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T22:07:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:55 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3871674283' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:56.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:55 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/550788126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T22:07:56.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.264+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3b6802dca0 con 0x7f3b68103410 2024-04-26T22:07:56.264 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.264+0000 7f3b657fa640 1 -- 172.21.15.55:0/4169633624 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f3b500bd7f0 con 0x7f3b68103410 2024-04-26T22:07:56.265 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.266+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3b540a3c40 msgr2=0x7f3b540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.266+0000 7f3b6e620640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3b540a3c40 0x7f3b540a6100 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3b580058c0 tx=0x7f3b5800a2d0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.266+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 msgr2=0x7f3b68198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.266+0000 7f3b6e620640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 0x7f3b68198c20 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3b50005940 tx=0x7f3b50004430 comp rx=0 tx=0).stop 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.267+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 shutdown_connections 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.267+0000 7f3b6e620640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f3b540a3c40 0x7f3b540a6100 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.267+0000 7f3b6e620640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b68108670 0x7f3b681a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.267+0000 7f3b6e620640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b68105d40 0x7f3b68199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.267+0000 7f3b6e620640 1 --2- 172.21.15.55:0/4169633624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b68103410 0x7f3b68198c20 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.267+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 >> 172.21.15.55:0/4169633624 conn(0x7f3b680fce20 msgr2=0x7f3b68105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:56.267 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.267+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 shutdown_connections 2024-04-26T22:07:56.268 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.267+0000 7f3b6e620640 1 -- 172.21.15.55:0/4169633624 wait complete. 2024-04-26T22:07:56.279 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3761233690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc104520 msgr2=0x7fe3bc10adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3761233690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc104520 0x7fe3bc10adb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009ca0 tx=0x7fe3b402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3761233690 shutdown_connections 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3761233690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc104520 0x7fe3bc10adb0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3761233690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3bc103b60 0x7fe3bc103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3761233690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc102970 0x7fe3bc102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3761233690 >> 172.21.15.55:0/3761233690 conn(0x7fe3bc0fdee0 msgr2=0x7fe3bc100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3761233690 shutdown_connections 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.381+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3761233690 wait complete. 2024-04-26T22:07:56.384 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3c1c64640 1 Processor -- start 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3c1c64640 1 -- start start 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3c1c64640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc102970 0x7fe3bc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3c1c64640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc103b60 0x7fe3bc199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3c1c64640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3bc104520 0x7fe3bc1a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3c1c64640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc10e480 con 0x7fe3bc102970 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3c1c64640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc10e300 con 0x7fe3bc104520 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3c1c64640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc10e600 con 0x7fe3bc103b60 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3bb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc102970 0x7fe3bc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3bb7fe640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc102970 0x7fe3bc198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36618/0 (socket says 172.21.15.55:36618) 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3bb7fe640 1 -- 172.21.15.55:0/3590706662 learned_addr learned my addr 172.21.15.55:0/3590706662 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.382+0000 7fe3baffd640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc103b60 0x7fe3bc199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3bbfff640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3bc104520 0x7fe3bc1a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:56.385 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3bb7fe640 1 -- 172.21.15.55:0/3590706662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc103b60 msgr2=0x7fe3bc199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3bb7fe640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc103b60 0x7fe3bc199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3bb7fe640 1 -- 172.21.15.55:0/3590706662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3bc104520 msgr2=0x7fe3bc1a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3bb7fe640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3bc104520 0x7fe3bc1a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3bb7fe640 1 -- 172.21.15.55:0/3590706662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc1a06f0 con 0x7fe3bc102970 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3bbfff640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3bc104520 0x7fe3bc1a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3bb7fe640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc102970 0x7fe3bc198ad0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c970 tx=0x7fe3ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3b8ff9640 1 -- 172.21.15.55:0/3590706662 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac007c10 con 0x7fe3bc102970 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3b8ff9640 1 -- 172.21.15.55:0/3590706662 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ac007db0 con 0x7fe3bc102970 2024-04-26T22:07:56.386 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc1a09e0 con 0x7fe3bc102970 2024-04-26T22:07:56.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3b8ff9640 1 -- 172.21.15.55:0/3590706662 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00f870 con 0x7fe3bc102970 2024-04-26T22:07:56.387 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.383+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc1a0f30 con 0x7fe3bc102970 2024-04-26T22:07:56.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.384+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3bc109c90 con 0x7fe3bc102970 2024-04-26T22:07:56.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.385+0000 7fe3b8ff9640 1 -- 172.21.15.55:0/3590706662 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe3ac00fa10 con 0x7fe3bc102970 2024-04-26T22:07:56.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.386+0000 7fe3b8ff9640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a40a3c40 0x7fe3a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:56.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.386+0000 7fe3baffd640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a40a3c40 0x7fe3a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:56.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.386+0000 7fe3b8ff9640 1 -- 172.21.15.55:0/3590706662 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fe3ac0d09c0 con 0x7fe3bc102970 2024-04-26T22:07:56.388 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.386+0000 7fe3baffd640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a40a3c40 0x7fe3a40a6100 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc199fe0 tx=0x7fe3b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:56.390 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.390+0000 7fe3b8ff9640 1 -- 172.21.15.55:0/3590706662 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe3ac09d230 con 0x7fe3bc102970 2024-04-26T22:07:56.590 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.590+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3bc02dad0 con 0x7fe3bc102970 2024-04-26T22:07:56.591 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.591+0000 7fe3b8ff9640 1 -- 172.21.15.55:0/3590706662 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+592 (secure 0 0 0) 0x7fe3bc02dad0 con 0x7fe3bc102970 2024-04-26T22:07:56.591 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:0/1387634878 2024-04-27T21:43:49.252173+0000 2024-04-26T22:07:56.591 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:0/2133918566 2024-04-27T21:43:49.252173+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6801/2562126894 2024-04-27T21:43:49.252173+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6800/3795671875 2024-04-27T21:43:21.996132+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:0/3938566585 2024-04-27T21:43:21.996132+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:0/4026035232 2024-04-27T21:43:21.996132+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:0/4038340190 2024-04-27T21:43:21.996132+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6801/3795671875 2024-04-27T21:43:21.996132+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:0/1492865350 2024-04-27T21:43:49.252173+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.55:6800/2562126894 2024-04-27T21:43:49.252173+0000 2024-04-26T22:07:56.592 INFO:teuthology.orchestra.run.smithi055.stderr:listed 10 entries 2024-04-26T22:07:56.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a40a3c40 msgr2=0x7fe3a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a40a3c40 0x7fe3a40a6100 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc199fe0 tx=0x7fe3b000b390 comp rx=0 tx=0).stop 2024-04-26T22:07:56.593 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc102970 msgr2=0x7fe3bc198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:56.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc102970 0x7fe3bc198ad0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c970 tx=0x7fe3ac00ce30 comp rx=0 tx=0).stop 2024-04-26T22:07:56.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 shutdown_connections 2024-04-26T22:07:56.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fe3a40a3c40 0x7fe3a40a6100 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe3bc104520 0x7fe3bc1a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc103b60 0x7fe3bc199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.594 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 --2- 172.21.15.55:0/3590706662 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3bc102970 0x7fe3bc198ad0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:56.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 >> 172.21.15.55:0/3590706662 conn(0x7fe3bc0fdee0 msgr2=0x7fe3bc106e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:56.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 shutdown_connections 2024-04-26T22:07:56.595 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:56.593+0000 7fe3c1c64640 1 -- 172.21.15.55:0/3590706662 wait complete. 2024-04-26T22:07:56.605 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T22:07:56.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:07:56.605 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T22:07:56.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:56 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1483137042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:07:56.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:56 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4169633624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:07:56.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:56 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3590706662' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:07:57.015 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2024-04-26T22:07:57.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:07:57.015 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T22:07:57.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:07:57.076 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2024-04-26T22:07:57.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:56 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1483137042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:07:57.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:56 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4169633624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:07:57.077 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:56 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3590706662' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:07:57.122 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T22:07:57.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:07:57.122 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:07:57.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:07:57.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2024-04-26T22:07:57.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:56 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1483137042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T22:07:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:56 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4169633624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T22:07:57.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:56 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3590706662' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T22:07:57.285 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T22:07:57.295 INFO:tasks.cephadm:Teardown begin 2024-04-26T22:07:57.296 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T22:07:57.296 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 -- 172.21.15.55:0/486363044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc104530 msgr2=0x7f64dc10adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 --2- 172.21.15.55:0/486363044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc104530 0x7f64dc10adc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009ca0 tx=0x7f64d402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 -- 172.21.15.55:0/486363044 shutdown_connections 2024-04-26T22:07:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 --2- 172.21.15.55:0/486363044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc104530 0x7f64dc10adc0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 --2- 172.21.15.55:0/486363044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64dc103b70 0x7f64dc103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 --2- 172.21.15.55:0/486363044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64dc102980 0x7f64dc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 -- 172.21.15.55:0/486363044 >> 172.21.15.55:0/486363044 conn(0x7f64dc0fdf10 msgr2=0x7f64dc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 -- 172.21.15.55:0/486363044 shutdown_connections 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 -- 172.21.15.55:0/486363044 wait complete. 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.399+0000 7f64e1893640 1 Processor -- start 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64e1893640 1 -- start start 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64e1893640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc102980 0x7f64dc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64e1893640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64dc103b70 0x7f64dc199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64e1893640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64dc104530 0x7f64dc1a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64e1893640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc10e490 con 0x7f64dc102980 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64e1893640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc10e310 con 0x7f64dc104530 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64e1893640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc10e610 con 0x7f64dc103b70 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc102980 0x7f64dc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc102980 0x7f64dc198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36630/0 (socket says 172.21.15.55:36630) 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 -- 172.21.15.55:0/20022937 learned_addr learned my addr 172.21.15.55:0/20022937 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64da7fc640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64dc103b70 0x7f64dc199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64db7fe640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64dc104530 0x7f64dc1a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 -- 172.21.15.55:0/20022937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64dc103b70 msgr2=0x7f64dc199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64dc103b70 0x7f64dc199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 -- 172.21.15.55:0/20022937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64dc104530 msgr2=0x7f64dc1a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64dc104530 0x7f64dc1a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 -- 172.21.15.55:0/20022937 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc1a0720 con 0x7f64dc102980 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64db7fe640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64dc104530 0x7f64dc1a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.400+0000 7f64daffd640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc102980 0x7f64dc198b00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00d950 tx=0x7f64cc00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.401+0000 7f64e0891640 1 -- 172.21.15.55:0/20022937 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc017020 con 0x7f64dc102980 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.401+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc1a0a10 con 0x7f64dc102980 2024-04-26T22:07:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.401+0000 7f64e0891640 1 -- 172.21.15.55:0/20022937 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64cc009c30 con 0x7f64dc102980 2024-04-26T22:07:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.401+0000 7f64e0891640 1 -- 172.21.15.55:0/20022937 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00f620 con 0x7f64dc102980 2024-04-26T22:07:57.404 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.401+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc1a0fb0 con 0x7f64dc102980 2024-04-26T22:07:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.402+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64dc109ca0 con 0x7f64dc102980 2024-04-26T22:07:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.403+0000 7f64e0891640 1 -- 172.21.15.55:0/20022937 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f64cc01a020 con 0x7f64dc102980 2024-04-26T22:07:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.403+0000 7f64e0891640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64c40a3c40 0x7f64c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.403+0000 7f64da7fc640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64c40a3c40 0x7f64c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.403+0000 7f64e0891640 1 -- 172.21.15.55:0/20022937 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f64cc0d0010 con 0x7f64dc102980 2024-04-26T22:07:57.405 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.404+0000 7f64da7fc640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64c40a3c40 0x7f64c40a6100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f64dc19a010 tx=0x7f64d000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:57.408 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.408+0000 7f64e0891640 1 -- 172.21.15.55:0/20022937 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f64cc021070 con 0x7f64dc102980 2024-04-26T22:07:57.633 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.633+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64dc02dad0 con 0x7f64dc102980 2024-04-26T22:07:57.634 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.633+0000 7f64e0891640 1 -- 172.21.15.55:0/20022937 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7f64dc02dad0 con 0x7f64dc102980 2024-04-26T22:07:57.634 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:07:57.634 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T22:07:01.422175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":244,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24509,24551,24502]},"id":1}]} 2024-04-26T22:07:57.634 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 25 2024-04-26T22:07:57.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64c40a3c40 msgr2=0x7f64c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:57.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64c40a3c40 0x7f64c40a6100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f64dc19a010 tx=0x7f64d000a2d0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc102980 msgr2=0x7f64dc198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:57.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc102980 0x7f64dc198b00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00d950 tx=0x7f64cc00dd50 comp rx=0 tx=0).stop 2024-04-26T22:07:57.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 shutdown_connections 2024-04-26T22:07:57.636 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f64c40a3c40 0x7f64c40a6100 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f64dc104530 0x7f64dc1a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64dc103b70 0x7f64dc199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 --2- 172.21.15.55:0/20022937 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f64dc102980 0x7f64dc198b00 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.635+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 >> 172.21.15.55:0/20022937 conn(0x7f64dc0fdf10 msgr2=0x7f64dc106e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:57.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.636+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 shutdown_connections 2024-04-26T22:07:57.637 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.636+0000 7f64e1893640 1 -- 172.21.15.55:0/20022937 wait complete. 2024-04-26T22:07:57.647 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- 172.21.15.55:0/384568740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40102990 msgr2=0x7f5a40102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 --2- 172.21.15.55:0/384568740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40102990 0x7f5a40102db0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009900 tx=0x7f5a3c02a410 comp rx=0 tx=0).stop 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- 172.21.15.55:0/384568740 shutdown_connections 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 --2- 172.21.15.55:0/384568740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a40104540 0x7f5a4010add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 --2- 172.21.15.55:0/384568740 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a40103b80 0x7f5a40104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 --2- 172.21.15.55:0/384568740 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40102990 0x7f5a40102db0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- 172.21.15.55:0/384568740 >> 172.21.15.55:0/384568740 conn(0x7f5a400fdee0 msgr2=0x7f5a40100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- 172.21.15.55:0/384568740 shutdown_connections 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- 172.21.15.55:0/384568740 wait complete. 2024-04-26T22:07:57.755 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 Processor -- start 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- start start 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a40102990 0x7f5a40198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a40103b80 0x7f5a40199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40104540 0x7f5a401a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010e4b0 con 0x7f5a40104540 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010e330 con 0x7f5a40102990 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.750+0000 7f5a46da4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010e630 con 0x7f5a40103b80 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40104540 0x7f5a401a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40104540 0x7f5a401a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36648/0 (socket says 172.21.15.55:36648) 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 -- 172.21.15.55:0/4066024213 learned_addr learned my addr 172.21.15.55:0/4066024213 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a44b40640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a40102990 0x7f5a40198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a37fff640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a40103b80 0x7f5a40199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:57.756 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 -- 172.21.15.55:0/4066024213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a40103b80 msgr2=0x7f5a40199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a40103b80 0x7f5a40199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 -- 172.21.15.55:0/4066024213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a40102990 msgr2=0x7f5a40198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a40102990 0x7f5a40198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 -- 172.21.15.55:0/4066024213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a401a0800 con 0x7f5a40104540 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a44b40640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a40102990 0x7f5a40198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a45341640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40104540 0x7f5a401a0190 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800eb10 tx=0x7f5a3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a35ffb640 1 -- 172.21.15.55:0/4066024213 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38004b20 con 0x7f5a40104540 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a401a0af0 con 0x7f5a40104540 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a35ffb640 1 -- 172.21.15.55:0/4066024213 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a38004cc0 con 0x7f5a40104540 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a35ffb640 1 -- 172.21.15.55:0/4066024213 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38010670 con 0x7f5a40104540 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.751+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401a0ff0 con 0x7f5a40104540 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.752+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a40109d80 con 0x7f5a40104540 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.753+0000 7f5a35ffb640 1 -- 172.21.15.55:0/4066024213 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5a38005020 con 0x7f5a40104540 2024-04-26T22:07:57.757 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.754+0000 7f5a35ffb640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5a280a3c40 0x7f5a280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:57.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.754+0000 7f5a35ffb640 1 -- 172.21.15.55:0/4066024213 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f5a380d0cb0 con 0x7f5a40104540 2024-04-26T22:07:57.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.754+0000 7f5a44b40640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5a280a3c40 0x7f5a280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:57.758 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.755+0000 7f5a44b40640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5a280a3c40 0x7f5a280a6100 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000c00 tx=0x7f5a3c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:57.759 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:57.759+0000 7f5a35ffb640 1 -- 172.21.15.55:0/4066024213 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5a3809d4f0 con 0x7f5a40104540 2024-04-26T22:07:57.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:57 smithi055 ceph-mon[29221]: pgmap v802: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-26T22:07:57.936 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:57 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/20022937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:58.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.137+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a4002dad0 con 0x7f5a40104540 2024-04-26T22:07:58.139 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.138+0000 7f5a35ffb640 1 -- 172.21.15.55:0/4066024213 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11865 (secure 0 0 0) 0x7f5a3809cdf0 con 0x7f5a40104540 2024-04-26T22:07:58.139 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:07:58.139 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1727764470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":1727764470},{"type":"v1","addr":"172.21.15.55:6835","nonce":1727764470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6837/3725383900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6836","nonce":3725383900},{"type":"v1","addr":"172.21.15.55:6837","nonce":3725383900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6839/2681477087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6838","nonce":2681477087},{"type":"v1","addr":"172.21.15.55:6839","nonce":2681477087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6841/1689426633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6840","nonce":1689426633},{"type":"v1","addr":"172.21.15.55:6841","nonce":1689426633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/2896142285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":2896142285},{"type":"v1","addr":"172.21.15.165:6833","nonce":2896142285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/689132199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":689132199},{"type":"v1","addr":"172.21.15.64:6833","nonce":689132199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T21:50:56.719940+0000","modified":"2024-04-26T22:07:01.422175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24551,"mds_1":24509,"mds_2":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/125466680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":125466680},{"type":"v1","addr":"172.21.15.165:6835","nonce":125466680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/2328208855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2328208855},{"type":"v1","addr":"172.21.15.64:6835","nonce":2328208855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6837/2954631237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2954631237},{"type":"v1","addr":"172.21.15.165:6837","nonce":2954631237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":18,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/1115440142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":1115440142},{"type":"v1","addr":"172.21.15.165:6839","nonce":1115440142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"h","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.64:6837/4014142220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":4014142220},{"type":"v1","addr":"172.21.15.64:6837","nonce":4014142220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":244,"addr":"172.21.15.64:6839/463662731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":463662731},{"type":"v1","addr":"172.21.15.64:6839","nonce":463662731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24509,24551,24502]},"id":1}]} 2024-04-26T22:07:58.140 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 25 2024-04-26T22:07:58.140 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5a280a3c40 msgr2=0x7f5a280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5a280a3c40 0x7f5a280a6100 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000c00 tx=0x7f5a3c0314a0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40104540 msgr2=0x7f5a401a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40104540 0x7f5a401a0190 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800eb10 tx=0x7f5a3800c2d0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 shutdown_connections 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f5a280a3c40 0x7f5a280a6100 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a40104540 0x7f5a401a0190 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a40103b80 0x7f5a40199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 --2- 172.21.15.55:0/4066024213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a40102990 0x7f5a40198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 >> 172.21.15.55:0/4066024213 conn(0x7f5a400fdee0 msgr2=0x7f5a40106f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 shutdown_connections 2024-04-26T22:07:58.141 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.140+0000 7f5a46da4640 1 -- 172.21.15.55:0/4066024213 wait complete. 2024-04-26T22:07:58.152 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:07:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 -- 172.21.15.55:0/4288347642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4103b90 msgr2=0x7f50a4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 --2- 172.21.15.55:0/4288347642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4103b90 0x7f50a4104010 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5094009900 tx=0x7f509402a3f0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 -- 172.21.15.55:0/4288347642 shutdown_connections 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 --2- 172.21.15.55:0/4288347642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a4104550 0x7f50a410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 --2- 172.21.15.55:0/4288347642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4103b90 0x7f50a4104010 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 --2- 172.21.15.55:0/4288347642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50a41029a0 0x7f50a4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 -- 172.21.15.55:0/4288347642 >> 172.21.15.55:0/4288347642 conn(0x7f50a40fdf30 msgr2=0x7f50a4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 -- 172.21.15.55:0/4288347642 shutdown_connections 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.254+0000 7f50ac704640 1 -- 172.21.15.55:0/4288347642 wait complete. 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50ac704640 1 Processor -- start 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50ac704640 1 -- start start 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50ac704640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50a41029a0 0x7f50a419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50ac704640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a4103b90 0x7f50a419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50ac704640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4104550 0x7f50a41a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.257 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50ac704640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410e520 con 0x7f50a4104550 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50ac704640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410e3a0 con 0x7f50a4103b90 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50ac704640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50a410e6a0 con 0x7f50a41029a0 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4104550 0x7f50a41a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4104550 0x7f50a41a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36658/0 (socket says 172.21.15.55:36658) 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 -- 172.21.15.55:0/2971346802 learned_addr learned my addr 172.21.15.55:0/2971346802 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50a9c9f640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a4103b90 0x7f50a419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aa4a0640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50a41029a0 0x7f50a419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 -- 172.21.15.55:0/2971346802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50a41029a0 msgr2=0x7f50a419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50a41029a0 0x7f50a419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 -- 172.21.15.55:0/2971346802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a4103b90 msgr2=0x7f50a419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a4103b90 0x7f50a419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.258 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 -- 172.21.15.55:0/2971346802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a41a2a00 con 0x7f50a4104550 2024-04-26T22:07:58.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aa4a0640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50a41029a0 0x7f50a419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:07:58.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.255+0000 7f50aaca1640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4104550 0x7f50a41a2390 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f509c00cce0 tx=0x7f509c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:58.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.256+0000 7f509b7fe640 1 -- 172.21.15.55:0/2971346802 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c00f450 con 0x7f50a4104550 2024-04-26T22:07:58.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.256+0000 7f509b7fe640 1 -- 172.21.15.55:0/2971346802 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f509c00faa0 con 0x7f50a4104550 2024-04-26T22:07:58.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.256+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a41a2cf0 con 0x7f50a4104550 2024-04-26T22:07:58.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.256+0000 7f509b7fe640 1 -- 172.21.15.55:0/2971346802 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c018b70 con 0x7f50a4104550 2024-04-26T22:07:58.259 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.256+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a41a31c0 con 0x7f50a4104550 2024-04-26T22:07:58.259 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:57 smithi064 ceph-mon[32765]: pgmap v802: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-26T22:07:58.260 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:57 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/20022937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:58.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.257+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a41a33f0 con 0x7f50a4104550 2024-04-26T22:07:58.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.258+0000 7f509b7fe640 1 -- 172.21.15.55:0/2971346802 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f509c018d10 con 0x7f50a4104550 2024-04-26T22:07:58.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.258+0000 7f509b7fe640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50900a3c40 0x7f50900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.258+0000 7f50aa4a0640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50900a3c40 0x7f50900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.258+0000 7f509b7fe640 1 -- 172.21.15.55:0/2971346802 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7f509c0d0a20 con 0x7f50a4104550 2024-04-26T22:07:58.260 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.259+0000 7f50aa4a0640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50900a3c40 0x7f50900a6100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f50a00058c0 tx=0x7f50a0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:58.263 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.263+0000 7f509b7fe640 1 -- 172.21.15.55:0/2971346802 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f509c014030 con 0x7f50a4104550 2024-04-26T22:07:58.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:57 smithi165 ceph-mon[33296]: pgmap v802: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.4 MiB/s wr, 22 op/s 2024-04-26T22:07:58.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:57 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/20022937' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:58.453 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.452+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50a41a3660 con 0x7f50a4104550 2024-04-26T22:07:58.454 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.454+0000 7f509b7fe640 1 -- 172.21.15.55:0/2971346802 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22760 (secure 0 0 0) 0x7f509c09d260 con 0x7f50a4104550 2024-04-26T22:07:58.455 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-04-26T22:07:58.455 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"c84f1140-0415-11ef-bc93-c7b262605968","created":"2024-04-26T21:42:52.792981+0000","modified":"2024-04-26T21:52:51.546766+0000","last_up_change":"2024-04-26T21:50:05.956082+0000","last_in_change":"2024-04-26T21:49:45.348596+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T21:46:13.346319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T21:50:54.684639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T21:50:55.670730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e004f5b1-6940-4e27-9d6a-7b4c81d08f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6803","nonce":2219207145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6805","nonce":2219207145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6809","nonce":2219207145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2219207145},{"type":"v1","addr":"172.21.15.55:6807","nonce":2219207145}]},"public_addr":"172.21.15.55:6803/2219207145","cluster_addr":"172.21.15.55:6805/2219207145","heartbeat_back_addr":"172.21.15.55:6809/2219207145","heartbeat_front_addr":"172.21.15.55:6807/2219207145","state":["exists","up"]},{"osd":1,"uuid":"a6715741-0a37-48df-8dd4-d07f539706f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6801","nonce":794827991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6803","nonce":794827991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6807","nonce":794827991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":794827991},{"type":"v1","addr":"172.21.15.64:6805","nonce":794827991}]},"public_addr":"172.21.15.64:6801/794827991","cluster_addr":"172.21.15.64:6803/794827991","heartbeat_back_addr":"172.21.15.64:6807/794827991","heartbeat_front_addr":"172.21.15.64:6805/794827991","state":["exists","up"]},{"osd":2,"uuid":"15bb5be1-55c9-4a43-8253-fcefc1d5041f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6801","nonce":2376036424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6803","nonce":2376036424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6807","nonce":2376036424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":2376036424},{"type":"v1","addr":"172.21.15.165:6805","nonce":2376036424}]},"public_addr":"172.21.15.165:6801/2376036424","cluster_addr":"172.21.15.165:6803/2376036424","heartbeat_back_addr":"172.21.15.165:6807/2376036424","heartbeat_front_addr":"172.21.15.165:6805/2376036424","state":["exists","up"]},{"osd":3,"uuid":"2fcc948f-ba34-488a-be7b-ec4579b3a4c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6811","nonce":3700452481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6813","nonce":3700452481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6817","nonce":3700452481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":3700452481},{"type":"v1","addr":"172.21.15.55:6815","nonce":3700452481}]},"public_addr":"172.21.15.55:6811/3700452481","cluster_addr":"172.21.15.55:6813/3700452481","heartbeat_back_addr":"172.21.15.55:6817/3700452481","heartbeat_front_addr":"172.21.15.55:6815/3700452481","state":["exists","up"]},{"osd":4,"uuid":"daf304fa-67ee-45a4-a3ca-8140292c27ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6809","nonce":526876100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6811","nonce":526876100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6815","nonce":526876100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":526876100},{"type":"v1","addr":"172.21.15.64:6813","nonce":526876100}]},"public_addr":"172.21.15.64:6809/526876100","cluster_addr":"172.21.15.64:6811/526876100","heartbeat_back_addr":"172.21.15.64:6815/526876100","heartbeat_front_addr":"172.21.15.64:6813/526876100","state":["exists","up"]},{"osd":5,"uuid":"1f42d9d6-e34c-4085-809b-fe73d7da4281","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6809","nonce":1409137199}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6811","nonce":1409137199}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6815","nonce":1409137199}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":1409137199},{"type":"v1","addr":"172.21.15.165:6813","nonce":1409137199}]},"public_addr":"172.21.15.165:6809/1409137199","cluster_addr":"172.21.15.165:6811/1409137199","heartbeat_back_addr":"172.21.15.165:6815/1409137199","heartbeat_front_addr":"172.21.15.165:6813/1409137199","state":["exists","up"]},{"osd":6,"uuid":"c68fd703-709c-4adf-8437-6cdafc737f7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6819","nonce":4025342485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6821","nonce":4025342485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6825","nonce":4025342485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":4025342485},{"type":"v1","addr":"172.21.15.55:6823","nonce":4025342485}]},"public_addr":"172.21.15.55:6819/4025342485","cluster_addr":"172.21.15.55:6821/4025342485","heartbeat_back_addr":"172.21.15.55:6825/4025342485","heartbeat_front_addr":"172.21.15.55:6823/4025342485","state":["exists","up"]},{"osd":7,"uuid":"d96f9d6b-186d-48ba-9ade-8ce41775d9d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6817","nonce":1454699976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6819","nonce":1454699976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6823","nonce":1454699976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1454699976},{"type":"v1","addr":"172.21.15.64:6821","nonce":1454699976}]},"public_addr":"172.21.15.64:6817/1454699976","cluster_addr":"172.21.15.64:6819/1454699976","heartbeat_back_addr":"172.21.15.64:6823/1454699976","heartbeat_front_addr":"172.21.15.64:6821/1454699976","state":["exists","up"]},{"osd":8,"uuid":"f954a5a8-dd31-40f3-88eb-34444e538d2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6817","nonce":2092227616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6819","nonce":2092227616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6823","nonce":2092227616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":2092227616},{"type":"v1","addr":"172.21.15.165:6821","nonce":2092227616}]},"public_addr":"172.21.15.165:6817/2092227616","cluster_addr":"172.21.15.165:6819/2092227616","heartbeat_back_addr":"172.21.15.165:6823/2092227616","heartbeat_front_addr":"172.21.15.165:6821/2092227616","state":["exists","up"]},{"osd":9,"uuid":"b9769f00-30fe-4ecd-9e58-b81cc6e94d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6827","nonce":3581040676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6829","nonce":3581040676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6833","nonce":3581040676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":3581040676},{"type":"v1","addr":"172.21.15.55:6831","nonce":3581040676}]},"public_addr":"172.21.15.55:6827/3581040676","cluster_addr":"172.21.15.55:6829/3581040676","heartbeat_back_addr":"172.21.15.55:6833/3581040676","heartbeat_front_addr":"172.21.15.55:6831/3581040676","state":["exists","up"]},{"osd":10,"uuid":"f3e0af7b-2c68-4ebe-9f85-dd15d958f0d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6825","nonce":3001316208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6827","nonce":3001316208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6831","nonce":3001316208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3001316208},{"type":"v1","addr":"172.21.15.64:6829","nonce":3001316208}]},"public_addr":"172.21.15.64:6825/3001316208","cluster_addr":"172.21.15.64:6827/3001316208","heartbeat_back_addr":"172.21.15.64:6831/3001316208","heartbeat_front_addr":"172.21.15.64:6829/3001316208","state":["exists","up"]},{"osd":11,"uuid":"09405356-ca7d-48a4-8cc2-caf08446e063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6825","nonce":1470164936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6827","nonce":1470164936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6831","nonce":1470164936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":1470164936},{"type":"v1","addr":"172.21.15.165:6829","nonce":1470164936}]},"public_addr":"172.21.15.165:6825/1470164936","cluster_addr":"172.21.15.165:6827/1470164936","heartbeat_back_addr":"172.21.15.165:6831/1470164936","heartbeat_front_addr":"172.21.15.165:6829/1470164936","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:23.208669+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:45:46.771093+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:10.804311+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:46:36.031196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:01.770633+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:27.410395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:47:53.516570+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:19.213922+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:48:44.809982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:11.412261+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:49:37.403630+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T21:50:04.154675+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/1387634878":"2024-04-27T21:43:49.252173+0000","172.21.15.55:0/2133918566":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6801/2562126894":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/3938566585":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4026035232":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/4038340190":"2024-04-27T21:43:21.996132+0000","172.21.15.55:6801/3795671875":"2024-04-27T21:43:21.996132+0000","172.21.15.55:0/1492865350":"2024-04-27T21:43:49.252173+0000","172.21.15.55:6800/2562126894":"2024-04-27T21:43:49.252173+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:07:58.458 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50900a3c40 msgr2=0x7f50900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50900a3c40 0x7f50900a6100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f50a00058c0 tx=0x7f50a0005850 comp rx=0 tx=0).stop 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4104550 msgr2=0x7f50a41a2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4104550 0x7f50a41a2390 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f509c00cce0 tx=0x7f509c00a620 comp rx=0 tx=0).stop 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 shutdown_connections 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f50900a3c40 0x7f50900a6100 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50a4104550 0x7f50a41a2390 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50a4103b90 0x7f50a419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 --2- 172.21.15.55:0/2971346802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50a41029a0 0x7f50a419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 >> 172.21.15.55:0/2971346802 conn(0x7f50a40fdf30 msgr2=0x7f50a4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 shutdown_connections 2024-04-26T22:07:58.459 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.458+0000 7f50ac704640 1 -- 172.21.15.55:0/2971346802 wait complete. 2024-04-26T22:07:58.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T22:07:58.580 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 -- 172.21.15.55:0/317177313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 msgr2=0x7fa59c10aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 --2- 172.21.15.55:0/317177313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 0x7fa59c10aaa0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa594009ca0 tx=0x7fa59402a8c0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 -- 172.21.15.55:0/317177313 shutdown_connections 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 --2- 172.21.15.55:0/317177313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 0x7fa59c10aaa0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 --2- 172.21.15.55:0/317177313 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa59c105d80 0x7fa59c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 --2- 172.21.15.55:0/317177313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59c103450 0x7fa59c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 -- 172.21.15.55:0/317177313 >> 172.21.15.55:0/317177313 conn(0x7fa59c0fcde0 msgr2=0x7fa59c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 -- 172.21.15.55:0/317177313 shutdown_connections 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.578+0000 7fa5a45af640 1 -- 172.21.15.55:0/317177313 wait complete. 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a45af640 1 Processor -- start 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a45af640 1 -- start start 2024-04-26T22:07:58.581 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a45af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59c103450 0x7fa59c102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a45af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa59c105d80 0x7fa59c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a45af640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 0x7fa59c101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a45af640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10e4f0 con 0x7fa59c1086b0 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a45af640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10e370 con 0x7fa59c103450 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a45af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10e670 con 0x7fa59c105d80 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a2b4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 0x7fa59c101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a2b4c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 0x7fa59c101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36684/0 (socket says 172.21.15.55:36684) 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a2b4c640 1 -- 172.21.15.55:0/582202706 learned_addr learned my addr 172.21.15.55:0/582202706 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a234b640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59c103450 0x7fa59c102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a1b4a640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa59c105d80 0x7fa59c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a2b4c640 1 -- 172.21.15.55:0/582202706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa59c105d80 msgr2=0x7fa59c101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a2b4c640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa59c105d80 0x7fa59c101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.582 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a2b4c640 1 -- 172.21.15.55:0/582202706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59c103450 msgr2=0x7fa59c102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a2b4c640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59c103450 0x7fa59c102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a2b4c640 1 -- 172.21.15.55:0/582202706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c102200 con 0x7fa59c1086b0 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.579+0000 7fa5a234b640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59c103450 0x7fa59c102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.580+0000 7fa5a2b4c640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 0x7fa59c101940 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa594033040 tx=0x7fa5940316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.580+0000 7fa58f7fe640 1 -- 172.21.15.55:0/582202706 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594003b70 con 0x7fa59c1086b0 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.580+0000 7fa58f7fe640 1 -- 172.21.15.55:0/582202706 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa594003d10 con 0x7fa59c1086b0 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.580+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c1a2cc0 con 0x7fa59c1086b0 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.580+0000 7fa58f7fe640 1 -- 172.21.15.55:0/582202706 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59402fb40 con 0x7fa59c1086b0 2024-04-26T22:07:58.583 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.580+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c1a3190 con 0x7fa59c1086b0 2024-04-26T22:07:58.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.581+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c0fe8d0 con 0x7fa59c1086b0 2024-04-26T22:07:58.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.582+0000 7fa58f7fe640 1 -- 172.21.15.55:0/582202706 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa594004610 con 0x7fa59c1086b0 2024-04-26T22:07:58.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.583+0000 7fa58f7fe640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa5880a3c40 0x7fa5880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.583+0000 7fa5a234b640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa5880a3c40 0x7fa5880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.583+0000 7fa58f7fe640 1 -- 172.21.15.55:0/582202706 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8298+0+0 (secure 0 0 0) 0x7fa5940f18e0 con 0x7fa59c1086b0 2024-04-26T22:07:58.584 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.583+0000 7fa5a234b640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa5880a3c40 0x7fa5880a6100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa598009fc0 tx=0x7fa598005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:58.587 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.587+0000 7fa58f7fe640 1 -- 172.21.15.55:0/582202706 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa5940be1d0 con 0x7fa59c1086b0 2024-04-26T22:07:58.814 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.814+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa59c02dcf0 con 0x7fa59c1086b0 2024-04-26T22:07:58.816 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.816+0000 7fa58f7fe640 1 -- 172.21.15.55:0/582202706 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7fa5940bdad0 con 0x7fa59c1086b0 2024-04-26T22:07:58.816 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T22:07:58.819 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa5880a3c40 msgr2=0x7fa5880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa5880a3c40 0x7fa5880a6100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa598009fc0 tx=0x7fa598005f90 comp rx=0 tx=0).stop 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 msgr2=0x7fa59c101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 0x7fa59c101940 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa594033040 tx=0x7fa5940316f0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 shutdown_connections 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fa5880a3c40 0x7fa5880a6100 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa59c1086b0 0x7fa59c101940 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa59c105d80 0x7fa59c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 --2- 172.21.15.55:0/582202706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa59c103450 0x7fa59c102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 >> 172.21.15.55:0/582202706 conn(0x7fa59c0fcde0 msgr2=0x7fa59c103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 shutdown_connections 2024-04-26T22:07:58.820 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.819+0000 7fa5a45af640 1 -- 172.21.15.55:0/582202706 wait complete. 2024-04-26T22:07:58.832 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T22:07:58.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 -- 172.21.15.55:0/2083356352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 msgr2=0x7f2108102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 --2- 172.21.15.55:0/2083356352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 0x7f2108102f00 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009900 tx=0x7f20f402a410 comp rx=0 tx=0).stop 2024-04-26T22:07:58.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 -- 172.21.15.55:0/2083356352 shutdown_connections 2024-04-26T22:07:58.944 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 --2- 172.21.15.55:0/2083356352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2108103440 0x7f2108109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 --2- 172.21.15.55:0/2083356352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 0x7f2108102f00 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 --2- 172.21.15.55:0/2083356352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2108101890 0x7f2108101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 -- 172.21.15.55:0/2083356352 >> 172.21.15.55:0/2083356352 conn(0x7f21080fce20 msgr2=0x7f21080ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 -- 172.21.15.55:0/2083356352 shutdown_connections 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.942+0000 7f210edc1640 1 -- 172.21.15.55:0/2083356352 wait complete. 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210edc1640 1 Processor -- start 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210edc1640 1 -- start start 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210edc1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2108101890 0x7f210819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.945 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210edc1640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 0x7f210819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210edc1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2108103440 0x7f21081a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210edc1640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f210810c2c0 con 0x7f2108102a80 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210edc1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f210810c140 con 0x7f2108101890 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210edc1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f210810c440 con 0x7f2108103440 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f2107fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 0x7f210819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f2107fff640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 0x7f210819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36712/0 (socket says 172.21.15.55:36712) 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f2107fff640 1 -- 172.21.15.55:0/2718605259 learned_addr learned my addr 172.21.15.55:0/2718605259 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210cb5d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2108101890 0x7f210819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.943+0000 7f210d35e640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2108103440 0x7f21081a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2107fff640 1 -- 172.21.15.55:0/2718605259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2108103440 msgr2=0x7f21081a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2107fff640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2108103440 0x7f21081a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2107fff640 1 -- 172.21.15.55:0/2718605259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2108101890 msgr2=0x7f210819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:58.946 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2107fff640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2108101890 0x7f210819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:58.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2107fff640 1 -- 172.21.15.55:0/2718605259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21081a2a60 con 0x7f2108102a80 2024-04-26T22:07:58.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f210cb5d640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2108101890 0x7f210819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:07:58.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2107fff640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 0x7f210819b380 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f20f4005940 tx=0x7f20f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:58.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2105ffb640 1 -- 172.21.15.55:0/2718605259 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f403f030 con 0x7f2108102a80 2024-04-26T22:07:58.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081a2cf0 con 0x7f2108102a80 2024-04-26T22:07:58.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2105ffb640 1 -- 172.21.15.55:0/2718605259 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f4003cb0 con 0x7f2108102a80 2024-04-26T22:07:58.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f2105ffb640 1 -- 172.21.15.55:0/2718605259 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4035070 con 0x7f2108102a80 2024-04-26T22:07:58.947 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.944+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a31d0 con 0x7f2108102a80 2024-04-26T22:07:58.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.945+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2108108cd0 con 0x7f2108102a80 2024-04-26T22:07:58.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.946+0000 7f2105ffb640 1 -- 172.21.15.55:0/2718605259 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f20f4042030 con 0x7f2108102a80 2024-04-26T22:07:58.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.947+0000 7f2105ffb640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20f80a3c40 0x7f20f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:58.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.947+0000 7f2105ffb640 1 -- 172.21.15.55:0/2718605259 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8556+0+0 (secure 0 0 0) 0x7f20f40f0ca0 con 0x7f2108102a80 2024-04-26T22:07:58.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.947+0000 7f210cb5d640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20f80a3c40 0x7f20f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:58.948 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.948+0000 7f210cb5d640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20f80a3c40 0x7f20f80a6100 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b3c0 tx=0x7f20fc00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:58.950 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:58.951+0000 7f2105ffb640 1 -- 172.21.15.55:0/2718605259 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f20f40bcd90 con 0x7f2108102a80 2024-04-26T22:07:59.077 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a[29217]: 2024-04-26T22:07:58.814+0000 7f8bd46e9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:07:59.077 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/4066024213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.077 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.077 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2971346802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/582202706' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T22:07:59.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T22:07:59.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:07:59.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/582202706' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T22:07:59.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T22:07:59.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:58 smithi055 ceph-mon[29221]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T22:07:59.193 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.193+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f210802dcd0 con 0x7f2108102a80 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/4066024213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2971346802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/582202706' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/582202706' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T22:07:59.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mon[32765]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T22:07:59.258 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e[62497]: ignoring --setuser ceph since I am not root 2024-04-26T22:07:59.258 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e[62497]: ignoring --setgroup ceph since I am not root 2024-04-26T22:07:59.258 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mds[62501]: -- 172.21.15.64:0/565094108 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56379de67a40 con 0x56379de3a800 2024-04-26T22:07:59.258 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e[62497]: starting mds.e at 2024-04-26T22:07:59.258 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h[62805]: ignoring --setuser ceph since I am not root 2024-04-26T22:07:59.258 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h[62805]: ignoring --setgroup ceph since I am not root 2024-04-26T22:07:59.258 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mds[62809]: -- 172.21.15.64:0/2210938104 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562daf591a40 con 0x562daf564c00 2024-04-26T22:07:59.258 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h[62805]: starting mds.h at 2024-04-26T22:07:59.259 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: ignoring --setuser ceph since I am not root 2024-04-26T22:07:59.259 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: ignoring --setgroup ceph since I am not root 2024-04-26T22:07:59.259 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-mds[63119]: -- 172.21.15.64:0/3255116538 <== mon.2 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558a00a5a40 con 0x5558a0078c00 2024-04-26T22:07:59.259 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:07:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: starting mds.b at 2024-04-26T22:07:59.326 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c[63597]: ignoring --setuser ceph since I am not root 2024-04-26T22:07:59.327 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c[63597]: ignoring --setgroup ceph since I am not root 2024-04-26T22:07:59.327 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mds[63601]: -- 172.21.15.165:0/1846431714 <== mon.0 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563578cfba40 con 0x563578ccf000 2024-04-26T22:07:59.327 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c[63597]: starting mds.c at 2024-04-26T22:07:59.327 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f[63287]: ignoring --setuser ceph since I am not root 2024-04-26T22:07:59.327 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f[63287]: ignoring --setgroup ceph since I am not root 2024-04-26T22:07:59.327 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mds[63291]: -- 172.21.15.165:0/1331437329 <== mon.0 v2:172.21.15.55:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b4cca0ba40 con 0x55b4cc9de800 2024-04-26T22:07:59.327 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f[63287]: starting mds.f at 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/4066024213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2971346802' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/582202706' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/582202706' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mon[33296]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-26T22:07:59.328 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i[63903]: ignoring --setuser ceph since I am not root 2024-04-26T22:07:59.329 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i[63903]: ignoring --setgroup ceph since I am not root 2024-04-26T22:07:59.329 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-mds[63907]: -- 172.21.15.165:0/3733042009 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558afeb83a40 con 0x558afeb56800 2024-04-26T22:07:59.329 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:07:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i[63903]: starting mds.i at 2024-04-26T22:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.842+0000 7f2105ffb640 1 -- 172.21.15.55:0/2718605259 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f20f40c1090 con 0x7f2108102a80 2024-04-26T22:07:59.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20f80a3c40 msgr2=0x7f20f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:59.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20f80a3c40 0x7f20f80a6100 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b3c0 tx=0x7f20fc00d040 comp rx=0 tx=0).stop 2024-04-26T22:07:59.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 msgr2=0x7f210819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:59.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 0x7f210819b380 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f20f4005940 tx=0x7f20f40042f0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 shutdown_connections 2024-04-26T22:07:59.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f20f80a3c40 0x7f20f80a6100 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.845 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2108103440 0x7f21081a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2108102a80 0x7f210819b380 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 --2- 172.21.15.55:0/2718605259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2108101890 0x7f210819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 >> 172.21.15.55:0/2718605259 conn(0x7f21080fce20 msgr2=0x7f2108105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:59.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 shutdown_connections 2024-04-26T22:07:59.846 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.845+0000 7f210edc1640 1 -- 172.21.15.55:0/2718605259 wait complete. 2024-04-26T22:07:59.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T22:07:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3749713653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04102a60 msgr2=0x7f0b04102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3749713653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04102a60 0x7f0b04102ee0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009900 tx=0x7f0af402a410 comp rx=0 tx=0).stop 2024-04-26T22:07:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3749713653 shutdown_connections 2024-04-26T22:07:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3749713653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b04103420 0x7f0b04109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3749713653 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04102a60 0x7f0b04102ee0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3749713653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b04101870 0x7f0b04101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3749713653 >> 172.21.15.55:0/3749713653 conn(0x7f0b040fcde0 msgr2=0x7f0b040ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3749713653 shutdown_connections 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3749713653 wait complete. 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 Processor -- start 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 -- start start 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b04101870 0x7f0b0406a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b04102a60 0x7f0b0419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.968+0000 7f0b0b90a640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04103420 0x7f0b0419ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b0b90a640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410d3a0 con 0x7f0b04103420 2024-04-26T22:07:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b0b90a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410d220 con 0x7f0b04101870 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b0b90a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0410d520 con 0x7f0b04102a60 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04103420 0x7f0b0419ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04103420 0x7f0b0419ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36730/0 (socket says 172.21.15.55:36730) 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 -- 172.21.15.55:0/3489538019 learned_addr learned my addr 172.21.15.55:0/3489538019 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b096a6640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b04101870 0x7f0b0406a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b08ea5640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b04102a60 0x7f0b0419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 -- 172.21.15.55:0/3489538019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b04102a60 msgr2=0x7f0b0419d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b04102a60 0x7f0b0419d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 -- 172.21.15.55:0/3489538019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b04101870 msgr2=0x7f0b0406a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b04101870 0x7f0b0406a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 -- 172.21.15.55:0/3489538019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b041a0830 con 0x7f0b04103420 2024-04-26T22:07:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b09ea7640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04103420 0x7f0b0419ff70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000d750 tx=0x7f0b0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0afa7fc640 1 -- 172.21.15.55:0/3489538019 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00017070 con 0x7f0b04103420 2024-04-26T22:07:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b041a0b20 con 0x7f0b04103420 2024-04-26T22:07:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b041a10c0 con 0x7f0b04103420 2024-04-26T22:07:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0afa7fc640 1 -- 172.21.15.55:0/3489538019 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000054d0 con 0x7f0b04103420 2024-04-26T22:07:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.969+0000 7f0afa7fc640 1 -- 172.21.15.55:0/3489538019 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00020060 con 0x7f0b04103420 2024-04-26T22:07:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.970+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae80052c0 con 0x7f0b04103420 2024-04-26T22:07:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.972+0000 7f0afa7fc640 1 -- 172.21.15.55:0/3489538019 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0b00009bd0 con 0x7f0b04103420 2024-04-26T22:07:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.972+0000 7f0afa7fc640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0af00a3c40 0x7f0af00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:07:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.972+0000 7f0afa7fc640 1 -- 172.21.15.55:0/3489538019 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8556+0+0 (secure 0 0 0) 0x7f0b000d0370 con 0x7f0b04103420 2024-04-26T22:07:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.972+0000 7f0b096a6640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0af00a3c40 0x7f0af00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:07:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.973+0000 7f0b096a6640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0af00a3c40 0x7f0af00a6100 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0b041028c0 tx=0x7f0afc009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:07:59.976 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:07:59.976+0000 7f0afa7fc640 1 -- 172.21.15.55:0/3489538019 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0b0009cab0 con 0x7f0b04103420 2024-04-26T22:08:00.163 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.163+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ae80055b0 con 0x7f0b04103420 2024-04-26T22:08:00.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:59 smithi055 ceph-mon[29221]: pgmap v803: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 28 MiB/s wr, 196 op/s 2024-04-26T22:08:00.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:59 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:00.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:59 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:00.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:59 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:08:00.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:07:59 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2718605259' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:08:00.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:59 smithi064 ceph-mon[32765]: pgmap v803: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 28 MiB/s wr, 196 op/s 2024-04-26T22:08:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:59 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:59 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:59 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:08:00.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:07:59 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2718605259' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:08:00.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:59 smithi165 ceph-mon[33296]: pgmap v803: 97 pgs: 97 active+clean; 2.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 28 MiB/s wr, 196 op/s 2024-04-26T22:08:00.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:59 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:00.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:59 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:00.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:59 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' 2024-04-26T22:08:00.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:07:59 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2718605259' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T22:08:00.856 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.855+0000 7f0afa7fc640 1 -- 172.21.15.55:0/3489538019 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f0b0009c3b0 con 0x7f0b04103420 2024-04-26T22:08:00.856 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2024-04-26T22:08:00.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0af00a3c40 msgr2=0x7f0af00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:00.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0af00a3c40 0x7f0af00a6100 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0b041028c0 tx=0x7f0afc009670 comp rx=0 tx=0).stop 2024-04-26T22:08:00.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04103420 msgr2=0x7f0b0419ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:00.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04103420 0x7f0b0419ff70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000d750 tx=0x7f0b0000dc10 comp rx=0 tx=0).stop 2024-04-26T22:08:00.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 shutdown_connections 2024-04-26T22:08:00.859 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7f0af00a3c40 0x7f0af00a6100 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b04103420 0x7f0b0419ff70 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b04102a60 0x7f0b0419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 --2- 172.21.15.55:0/3489538019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b04101870 0x7f0b0406a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 >> 172.21.15.55:0/3489538019 conn(0x7f0b040fcde0 msgr2=0x7f0b04105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:00.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.859+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 shutdown_connections 2024-04-26T22:08:00.860 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.860+0000 7f0b0b90a640 1 -- 172.21.15.55:0/3489538019 wait complete. 2024-04-26T22:08:00.878 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T22:08:00.994 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 -- 172.21.15.55:0/3983675609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054103400 msgr2=0x7fd054109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/3983675609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054103400 0x7fd054109c90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009ca0 tx=0x7fd04c02a8c0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 -- 172.21.15.55:0/3983675609 shutdown_connections 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/3983675609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054103400 0x7fd054109c90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/3983675609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd054102a40 0x7fd054102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/3983675609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd054101850 0x7fd054101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 -- 172.21.15.55:0/3983675609 >> 172.21.15.55:0/3983675609 conn(0x7fd0540fcdc0 msgr2=0x7fd0540ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 -- 172.21.15.55:0/3983675609 shutdown_connections 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 -- 172.21.15.55:0/3983675609 wait complete. 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 Processor -- start 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 -- start start 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.992+0000 7fd05c7f0640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054101850 0x7fd054198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:00.995 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05c7f0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd054102a40 0x7fd0541990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05c7f0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd054103400 0x7fd0541a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05c7f0640 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410d360 con 0x7fd054101850 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05c7f0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410d1e0 con 0x7fd054102a40 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05c7f0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410d4e0 con 0x7fd054103400 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054101850 0x7fd054198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054101850 0x7fd054198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41756/0 (socket says 172.21.15.55:41756) 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 -- 172.21.15.55:0/1604775154 learned_addr learned my addr 172.21.15.55:0/1604775154 (peer_addr_for_me v2:172.21.15.55:0/0) 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd059d8b640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd054102a40 0x7fd0541990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05ad8d640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd054103400 0x7fd0541a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 -- 172.21.15.55:0/1604775154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd054103400 msgr2=0x7fd0541a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd054103400 0x7fd0541a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 -- 172.21.15.55:0/1604775154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd054102a40 msgr2=0x7fd0541990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd054102a40 0x7fd0541990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:00.996 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 -- 172.21.15.55:0/1604775154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0541a0780 con 0x7fd054101850 2024-04-26T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05ad8d640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd054103400 0x7fd0541a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05a58c640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054101850 0x7fd054198b60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd05000d950 tx=0x7fd05000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd04b7fe640 1 -- 172.21.15.55:0/1604775154 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050017020 con 0x7fd054101850 2024-04-26T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd04b7fe640 1 -- 172.21.15.55:0/1604775154 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd050009c30 con 0x7fd054101850 2024-04-26T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0541a0a70 con 0x7fd054101850 2024-04-26T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd04b7fe640 1 -- 172.21.15.55:0/1604775154 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000f6b0 con 0x7fd054101850 2024-04-26T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.993+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0541a1010 con 0x7fd054101850 2024-04-26T22:08:00.997 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.994+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0541a11e0 con 0x7fd054101850 2024-04-26T22:08:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.995+0000 7fd04b7fe640 1 -- 172.21.15.55:0/1604775154 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd05001a020 con 0x7fd054101850 2024-04-26T22:08:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.996+0000 7fd04b7fe640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd0400a3cc0 0x7fd0400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:08:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.996+0000 7fd04b7fe640 1 -- 172.21.15.55:0/1604775154 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7973+0+0 (secure 0 0 0) 0x7fd0500d0010 con 0x7fd054101850 2024-04-26T22:08:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.996+0000 7fd059d8b640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd0400a3cc0 0x7fd0400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:08:00.998 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:00.997+0000 7fd059d8b640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd0400a3cc0 0x7fd0400a6180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd05419a070 tx=0x7fd04400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:08:01.000 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.001+0000 7fd04b7fe640 1 -- 172.21.15.55:0/1604775154 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd050009de0 con 0x7fd054101850 2024-04-26T22:08:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T22:08:01.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: Cluster is now healthy 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/2718605259' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.165:6834/1479190764,v1:172.21.15.165:6835/1479190764] up:boot 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.165:6837/822625710,v1:172.21.15.165:6839/822625710] up:boot 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.165:6836/3509987963,v1:172.21.15.165:6838/3509987963] up:boot 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6834/4016100783,v1:172.21.15.64:6835/4016100783] up:boot 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6837/3816728588,v1:172.21.15.64:6838/3816728588] up:boot 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: mds.? [v2:172.21.15.64:6836/356938788,v1:172.21.15.64:6839/356938788] up:boot 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: fsmap 12 up:standby 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:08:01.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:08:01.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:08:01.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:08:01.189 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:00 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3489538019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:08:01.189 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.187+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0541a13b0 con 0x7fd054101850 2024-04-26T22:08:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T22:08:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T22:08:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: Cluster is now healthy 2024-04-26T22:08:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/2718605259' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:08:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.165:6834/1479190764,v1:172.21.15.165:6835/1479190764] up:boot 2024-04-26T22:08:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.165:6837/822625710,v1:172.21.15.165:6839/822625710] up:boot 2024-04-26T22:08:01.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.165:6836/3509987963,v1:172.21.15.165:6838/3509987963] up:boot 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6834/4016100783,v1:172.21.15.64:6835/4016100783] up:boot 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6837/3816728588,v1:172.21.15.64:6838/3816728588] up:boot 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: mds.? [v2:172.21.15.64:6836/356938788,v1:172.21.15.64:6839/356938788] up:boot 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: fsmap 12 up:standby 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:08:01.258 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:00 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3489538019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:08:01.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: Cluster is now healthy 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/2718605259' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.165:6834/1479190764,v1:172.21.15.165:6835/1479190764] up:boot 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.165:6837/822625710,v1:172.21.15.165:6839/822625710] up:boot 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.165:6836/3509987963,v1:172.21.15.165:6838/3509987963] up:boot 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6834/4016100783,v1:172.21.15.64:6835/4016100783] up:boot 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6837/3816728588,v1:172.21.15.64:6838/3816728588] up:boot 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: mds.? [v2:172.21.15.64:6836/356938788,v1:172.21.15.64:6839/356938788] up:boot 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: fsmap 12 up:standby 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T22:08:01.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T22:08:01.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T22:08:01.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T22:08:01.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: from='mgr.14152 172.21.15.55:0/197229763' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T22:08:01.328 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:00 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3489538019' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:08:01.871 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.871+0000 7fd04b7fe640 1 -- 172.21.15.55:0/1604775154 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fd05009ca40 con 0x7fd054101850 2024-04-26T22:08:01.871 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2024-04-26T22:08:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd0400a3cc0 msgr2=0x7fd0400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd0400a3cc0 0x7fd0400a6180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd05419a070 tx=0x7fd04400b390 comp rx=0 tx=0).stop 2024-04-26T22:08:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054101850 msgr2=0x7fd054198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:08:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054101850 0x7fd054198b60 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd05000d950 tx=0x7fd05000de10 comp rx=0 tx=0).stop 2024-04-26T22:08:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 shutdown_connections 2024-04-26T22:08:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:6800/3284422558,v1:172.21.15.55:6801/3284422558] conn(0x7fd0400a3cc0 0x7fd0400a6180 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd054103400 0x7fd0541a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:01.875 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd054102a40 0x7fd0541990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:01.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 --2- 172.21.15.55:0/1604775154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd054101850 0x7fd054198b60 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:08:01.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 >> 172.21.15.55:0/1604775154 conn(0x7fd0540fcdc0 msgr2=0x7fd0540fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:08:01.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 shutdown_connections 2024-04-26T22:08:01.876 INFO:teuthology.orchestra.run.smithi055.stderr:2024-04-26T22:08:01.875+0000 7fd05c7f0640 1 -- 172.21.15.55:0/1604775154 wait complete. 2024-04-26T22:08:01.894 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:08:01.925 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:08:01.955 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:08:01.991 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T22:08:01.991 DEBUG:teuthology.orchestra.run.smithi055:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T22:08:02.008 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T22:08:02.025 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T22:08:02.048 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T22:08:02.048 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T22:08:02.048 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.a 2024-04-26T22:08:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:01 smithi055 ceph-mon[29221]: pgmap v805: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 48 MiB/s wr, 342 op/s 2024-04-26T22:08:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:01 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/3489538019' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:08:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:01 smithi055 ceph-mon[29221]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:08:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:01 smithi055 ceph-mon[29221]: from='client.? 172.21.15.55:0/1604775154' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:08:02.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:02 smithi055 systemd[1]: Stopping Ceph mon.a for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:02.256 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:01 smithi064 ceph-mon[32765]: pgmap v805: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 48 MiB/s wr, 342 op/s 2024-04-26T22:08:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:01 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/3489538019' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:08:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:01 smithi064 ceph-mon[32765]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:08:02.257 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:01 smithi064 ceph-mon[32765]: from='client.? 172.21.15.55:0/1604775154' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:08:02.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:01 smithi165 ceph-mon[33296]: pgmap v805: 97 pgs: 97 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 48 MiB/s wr, 342 op/s 2024-04-26T22:08:02.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:01 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/3489538019' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T22:08:02.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:01 smithi165 ceph-mon[33296]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T22:08:02.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:01 smithi165 ceph-mon[33296]: from='client.? 172.21.15.55:0/1604775154' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T22:08:02.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a[29217]: 2024-04-26T22:08:02.409+0000 7f8bd9ef4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:02.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a[29217]: 2024-04-26T22:08:02.409+0000 7f8bd9ef4700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T22:08:02.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:02 smithi055 podman[94241]: 2024-04-26 22:08:02.533063287 +0000 UTC m=+0.272890841 container died 8c4a4ee318a8d27bdd6b29be04c79001b7f625ffb9b2800b282376e26bf68907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T22:08:03.186 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:02 smithi055 podman[94241]: 2024-04-26 22:08:02.941848561 +0000 UTC m=+0.681676156 container cleanup 8c4a4ee318a8d27bdd6b29be04c79001b7f625ffb9b2800b282376e26bf68907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T22:08:03.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Apr 26 22:08:02 smithi055 bash[94241]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-a 2024-04-26T22:08:03.325 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T22:08:03.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:03.364 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T22:08:03.364 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T22:08:03.364 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.b 2024-04-26T22:08:03.743 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:03 smithi064 systemd[1]: Stopping Ceph mon.b for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:04.006 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-b[32761]: 2024-04-26T22:08:03.742+0000 7ff39d3b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:04.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-b[32761]: 2024-04-26T22:08:03.742+0000 7ff39d3b8700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T22:08:04.007 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:03 smithi064 podman[67947]: 2024-04-26 22:08:03.793910148 +0000 UTC m=+0.179393745 container died b8ed5b06c3bf864fa36b6f2c21201de1f1ee6165148bac8d2b921099559fc414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:08:04.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:04 smithi064 podman[67947]: 2024-04-26 22:08:04.17785132 +0000 UTC m=+0.563334913 container cleanup b8ed5b06c3bf864fa36b6f2c21201de1f1ee6165148bac8d2b921099559fc414 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-04-26T22:08:04.463 INFO:journalctl@ceph.mon.b.smithi064.stdout:Apr 26 22:08:04 smithi064 bash[67947]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-b 2024-04-26T22:08:04.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T22:08:04.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:04.655 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T22:08:04.655 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T22:08:04.655 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.c 2024-04-26T22:08:05.016 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:04 smithi165 systemd[1]: Stopping Ceph mon.c for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:05.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-c[33278]: 2024-04-26T22:08:05.015+0000 7f9ba3a3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:05.326 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-c[33278]: 2024-04-26T22:08:05.015+0000 7f9ba3a3b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T22:08:05.327 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:05 smithi165 podman[69424]: 2024-04-26 22:08:05.097799078 +0000 UTC m=+0.206458609 container died 2db75f7cb6093c3d3c696b34530d53df5f45535a01349296dd2c80649ddb9c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:08:05.759 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:05 smithi165 podman[69424]: 2024-04-26 22:08:05.542220365 +0000 UTC m=+0.650879961 container cleanup 2db75f7cb6093c3d3c696b34530d53df5f45535a01349296dd2c80649ddb9c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T22:08:05.760 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:05 smithi165 bash[69424]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-c 2024-04-26T22:08:06.010 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T22:08:06.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:05 smithi165 podman[69438]: 2024-04-26 22:08:05.760424749 +0000 UTC m=+0.652110554 container remove 2db75f7cb6093c3d3c696b34530d53df5f45535a01349296dd2c80649ddb9c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:08:06.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:06 smithi165 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-26T22:08:06.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:06 smithi165 systemd[1]: Stopped Ceph mon.c for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T22:08:06.031 INFO:journalctl@ceph.mon.c.smithi165.stdout:Apr 26 22:08:06 smithi165 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mon.c.service: Consumed 14.793s CPU time. 2024-04-26T22:08:06.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:06.053 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T22:08:06.053 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T22:08:06.053 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.x 2024-04-26T22:08:06.381 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 22:08:06 smithi055 systemd[1]: Stopping Ceph mgr.x for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:06.686 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 22:08:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T22:08:06.378+0000 7f8163383700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:06.686 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 22:08:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x[29451]: 2024-04-26T22:08:06.378+0000 7f8163383700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-26T22:08:06.686 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 22:08:06 smithi055 podman[94361]: 2024-04-26 22:08:06.41144986 +0000 UTC m=+0.157094779 container died 60ea280c07805f4512b85e66ee4c46ef7cec8d36ed20e7e237d9dba51314ddae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:08:07.186 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 22:08:06 smithi055 podman[94361]: 2024-04-26 22:08:06.823777604 +0000 UTC m=+0.569422522 container cleanup 60ea280c07805f4512b85e66ee4c46ef7cec8d36ed20e7e237d9dba51314ddae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:08:07.186 INFO:journalctl@ceph.mgr.x.smithi055.stdout:Apr 26 22:08:06 smithi055 bash[94361]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-x 2024-04-26T22:08:07.439 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T22:08:07.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:07.477 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T22:08:07.477 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T22:08:07.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.y 2024-04-26T22:08:07.757 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 22:08:07 smithi064 systemd[1]: Stopping Ceph mgr.y for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:08.101 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 22:08:07 smithi064 podman[68070]: 2024-04-26 22:08:07.813295953 +0000 UTC m=+0.131250209 container died 1e53c41971930f7f67a3383fe4e07b0b7d3aabeb1ea86a9dc70b17b307a7f419 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T22:08:08.452 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 22:08:08 smithi064 podman[68070]: 2024-04-26 22:08:08.192403155 +0000 UTC m=+0.510357404 container cleanup 1e53c41971930f7f67a3383fe4e07b0b7d3aabeb1ea86a9dc70b17b307a7f419 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T22:08:08.452 INFO:journalctl@ceph.mgr.y.smithi064.stdout:Apr 26 22:08:08 smithi064 bash[68070]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-y 2024-04-26T22:08:08.586 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T22:08:08.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:08.623 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T22:08:08.623 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T22:08:08.623 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.z 2024-04-26T22:08:09.035 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:08 smithi165 systemd[1]: Stopping Ceph mgr.z for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:09.310 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:09 smithi165 podman[69547]: 2024-04-26 22:08:09.03552668 +0000 UTC m=+0.220526481 container died fa46b4a04d462701b54d8d6d84c3636cd1ab874ae58c31364b83558d16709be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T22:08:09.576 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:09 smithi165 podman[69547]: 2024-04-26 22:08:09.499079153 +0000 UTC m=+0.684078955 container cleanup fa46b4a04d462701b54d8d6d84c3636cd1ab874ae58c31364b83558d16709be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T22:08:09.576 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:09 smithi165 bash[69547]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z 2024-04-26T22:08:09.992 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T22:08:10.012 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:09 smithi165 podman[69561]: 2024-04-26 22:08:09.759516965 +0000 UTC m=+0.716432356 container remove fa46b4a04d462701b54d8d6d84c3636cd1ab874ae58c31364b83558d16709be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T22:08:10.013 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:09 smithi165 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-26T22:08:10.013 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:09 smithi165 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-04-26T22:08:10.013 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:09 smithi165 systemd[1]: Stopped Ceph mgr.z for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T22:08:10.013 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Apr 26 22:08:09 smithi165 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mgr.z.service: Consumed 15.280s CPU time. 2024-04-26T22:08:10.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:10.027 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T22:08:10.028 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T22:08:10.028 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.0 2024-04-26T22:08:10.345 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:10 smithi055 systemd[1]: Stopping Ceph osd.0 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:10.686 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:10 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0[41522]: 2024-04-26T22:08:10.344+0000 7f5b1dbd3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:10.686 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:10 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0[41522]: 2024-04-26T22:08:10.344+0000 7f5b1dbd3700 -1 osd.0 92 *** Got signal Terminated *** 2024-04-26T22:08:10.687 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:10 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0[41522]: 2024-04-26T22:08:10.344+0000 7f5b1dbd3700 -1 osd.0 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:08:16.093 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:15 smithi055 podman[94482]: 2024-04-26 22:08:15.722644639 +0000 UTC m=+5.492401482 container died 40b044c2c7d68d69f05ff089f89429c3dd36f31ec1810fc4a447d9bc5d366e71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T22:08:16.436 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:16 smithi055 podman[94482]: 2024-04-26 22:08:16.263553383 +0000 UTC m=+6.033310216 container cleanup 40b044c2c7d68d69f05ff089f89429c3dd36f31ec1810fc4a447d9bc5d366e71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T22:08:16.437 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:16 smithi055 bash[94482]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0 2024-04-26T22:08:16.935 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:16 smithi055 podman[94500]: 2024-04-26 22:08:16.524136054 +0000 UTC m=+0.793569926 container remove 40b044c2c7d68d69f05ff089f89429c3dd36f31ec1810fc4a447d9bc5d366e71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T22:08:17.186 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:16 smithi055 podman[94593]: 2024-04-26 22:08:16.838388702 +0000 UTC m=+0.019174221 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:08:17.186 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:17 smithi055 podman[94593]: 2024-04-26 22:08:17.092136909 +0000 UTC m=+0.272922411 container create a6b07b8400843069d51f83aacedc2818c9ef53681a73bd98ef8156f660f97f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T22:08:17.492 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:17 smithi055 podman[94593]: 2024-04-26 22:08:17.345286274 +0000 UTC m=+0.526071774 container init a6b07b8400843069d51f83aacedc2818c9ef53681a73bd98ef8156f660f97f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T22:08:17.492 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:17 smithi055 podman[94593]: 2024-04-26 22:08:17.348943096 +0000 UTC m=+0.529728603 container start a6b07b8400843069d51f83aacedc2818c9ef53681a73bd98ef8156f660f97f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T22:08:17.492 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:17 smithi055 podman[94593]: 2024-04-26 22:08:17.470417747 +0000 UTC m=+0.651203264 container attach a6b07b8400843069d51f83aacedc2818c9ef53681a73bd98ef8156f660f97f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:08:18.373 INFO:journalctl@ceph.osd.0.smithi055.stdout:Apr 26 22:08:18 smithi055 podman[94593]: 2024-04-26 22:08:18.008406973 +0000 UTC m=+1.189192467 container died a6b07b8400843069d51f83aacedc2818c9ef53681a73bd98ef8156f660f97f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-26T22:08:18.698 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T22:08:18.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:18.738 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T22:08:18.738 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T22:08:18.738 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.1 2024-04-26T22:08:19.073 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:18 smithi064 systemd[1]: Stopping Ceph osd.1 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:19.506 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:19 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1[37384]: 2024-04-26T22:08:19.072+0000 7faf18ca8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:19.507 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:19 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1[37384]: 2024-04-26T22:08:19.072+0000 7faf18ca8700 -1 osd.1 92 *** Got signal Terminated *** 2024-04-26T22:08:19.507 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:19 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1[37384]: 2024-04-26T22:08:19.072+0000 7faf18ca8700 -1 osd.1 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:08:24.957 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:24 smithi064 podman[68205]: 2024-04-26 22:08:24.613805629 +0000 UTC m=+5.674886994 container died 893270bd87562af12db8982ab0022480f8046d47fa7c276308d800a2ee96ec1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T22:08:25.249 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:25 smithi064 podman[68205]: 2024-04-26 22:08:25.022917918 +0000 UTC m=+6.083999283 container cleanup 893270bd87562af12db8982ab0022480f8046d47fa7c276308d800a2ee96ec1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:08:25.249 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:25 smithi064 bash[68205]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1 2024-04-26T22:08:25.506 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:25 smithi064 podman[68222]: 2024-04-26 22:08:25.250188574 +0000 UTC m=+0.638977958 container remove 893270bd87562af12db8982ab0022480f8046d47fa7c276308d800a2ee96ec1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T22:08:25.986 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:25 smithi064 podman[68315]: 2024-04-26 22:08:25.610119505 +0000 UTC m=+0.018409179 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:08:25.987 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:25 smithi064 podman[68315]: 2024-04-26 22:08:25.751154967 +0000 UTC m=+0.159444632 container create 43092558a9cabc97ee7c2d890f2a38791a933ab3f14af630ec61ece66e945a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T22:08:26.257 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:25 smithi064 podman[68315]: 2024-04-26 22:08:25.987451481 +0000 UTC m=+0.395741158 container init 43092558a9cabc97ee7c2d890f2a38791a933ab3f14af630ec61ece66e945a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T22:08:26.257 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:25 smithi064 podman[68315]: 2024-04-26 22:08:25.990959353 +0000 UTC m=+0.399249026 container start 43092558a9cabc97ee7c2d890f2a38791a933ab3f14af630ec61ece66e945a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:08:26.257 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:26 smithi064 podman[68315]: 2024-04-26 22:08:26.104584824 +0000 UTC m=+0.512874488 container attach 43092558a9cabc97ee7c2d890f2a38791a933ab3f14af630ec61ece66e945a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T22:08:26.846 INFO:journalctl@ceph.osd.1.smithi064.stdout:Apr 26 22:08:26 smithi064 podman[68315]: 2024-04-26 22:08:26.556914191 +0000 UTC m=+0.965203865 container died 43092558a9cabc97ee7c2d890f2a38791a933ab3f14af630ec61ece66e945a51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T22:08:27.167 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T22:08:27.206 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:27.206 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T22:08:27.207 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T22:08:27.207 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.2 2024-04-26T22:08:27.529 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:27 smithi165 systemd[1]: Stopping Ceph osd.2 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:27.826 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2[38444]: 2024-04-26T22:08:27.528+0000 7f864e484700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:27.826 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2[38444]: 2024-04-26T22:08:27.528+0000 7f864e484700 -1 osd.2 92 *** Got signal Terminated *** 2024-04-26T22:08:27.826 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2[38444]: 2024-04-26T22:08:27.528+0000 7f864e484700 -1 osd.2 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:08:33.246 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:32 smithi165 podman[69683]: 2024-04-26 22:08:32.930015426 +0000 UTC m=+5.528220307 container died bb50a1830a012e62e70b3e5c76b2b7160b09ae02f5a38157a94e2841190c731f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T22:08:33.576 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:33 smithi165 podman[69683]: 2024-04-26 22:08:33.346187562 +0000 UTC m=+5.944392431 container cleanup bb50a1830a012e62e70b3e5c76b2b7160b09ae02f5a38157a94e2841190c731f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T22:08:33.576 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:33 smithi165 bash[69683]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2 2024-04-26T22:08:34.013 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:33 smithi165 podman[69702]: 2024-04-26 22:08:33.597795351 +0000 UTC m=+0.670451282 container remove bb50a1830a012e62e70b3e5c76b2b7160b09ae02f5a38157a94e2841190c731f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T22:08:34.326 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:34 smithi165 podman[69790]: 2024-04-26 22:08:33.916650664 +0000 UTC m=+0.018084217 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:08:34.326 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:34 smithi165 podman[69790]: 2024-04-26 22:08:34.065438856 +0000 UTC m=+0.166872391 container create a77fa0744f27287a452098b11b77fcb94028ff4a64aa7302f9753047e8ee79e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:08:34.826 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:34 smithi165 podman[69790]: 2024-04-26 22:08:34.377595501 +0000 UTC m=+0.479029041 container init a77fa0744f27287a452098b11b77fcb94028ff4a64aa7302f9753047e8ee79e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T22:08:34.827 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:34 smithi165 podman[69790]: 2024-04-26 22:08:34.380988739 +0000 UTC m=+0.482422274 container start a77fa0744f27287a452098b11b77fcb94028ff4a64aa7302f9753047e8ee79e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T22:08:34.827 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:34 smithi165 podman[69790]: 2024-04-26 22:08:34.527631754 +0000 UTC m=+0.629065292 container attach a77fa0744f27287a452098b11b77fcb94028ff4a64aa7302f9753047e8ee79e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:08:35.247 INFO:journalctl@ceph.osd.2.smithi165.stdout:Apr 26 22:08:34 smithi165 podman[69790]: 2024-04-26 22:08:34.95459065 +0000 UTC m=+1.056024195 container died a77fa0744f27287a452098b11b77fcb94028ff4a64aa7302f9753047e8ee79e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T22:08:35.595 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T22:08:35.635 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:35.635 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T22:08:35.635 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T22:08:35.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.3 2024-04-26T22:08:35.936 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:35 smithi055 systemd[1]: Stopping Ceph osd.3 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:36.364 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:35 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3[48042]: 2024-04-26T22:08:35.957+0000 7fe7e842d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:36.365 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:35 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3[48042]: 2024-04-26T22:08:35.957+0000 7fe7e842d700 -1 osd.3 92 *** Got signal Terminated *** 2024-04-26T22:08:36.365 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:35 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3[48042]: 2024-04-26T22:08:35.957+0000 7fe7e842d700 -1 osd.3 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:08:36.686 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:36 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:36.364+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:37.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:36.637+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:37.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:36 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:36.877+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:37.686 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:37 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:37.412+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:38.028 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:37 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:37.667+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:38.256 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:37 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:37.913+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:38.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:38.028+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:38.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:38 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:38.102+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:38.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:38 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:38.450+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:38.936 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:38 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3[48042]: 2024-04-26T22:08:38.447+0000 7fe7e4a21700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.421047+0000 front 2024-04-26T22:08:14.421280+0000 (oldest deadline 2024-04-26T22:08:37.921117+0000) 2024-04-26T22:08:38.980 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:38.660+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:39.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:38 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:38.963+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:39.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:38.980+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:39.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:39 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:39.071+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:39.936 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:39 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3[48042]: 2024-04-26T22:08:39.484+0000 7fe7e4a21700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.421047+0000 front 2024-04-26T22:08:14.421280+0000 (oldest deadline 2024-04-26T22:08:37.921117+0000) 2024-04-26T22:08:39.937 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:39 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:39.476+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:40.000 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:39 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:39.638+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:40.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:39 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:39.967+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:40.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:40.000+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:40.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:40 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:40.081+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:40.936 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:40 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3[48042]: 2024-04-26T22:08:40.512+0000 7fe7e4a21700 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.421047+0000 front 2024-04-26T22:08:14.421280+0000 (oldest deadline 2024-04-26T22:08:37.921117+0000) 2024-04-26T22:08:40.937 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:40 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:40.517+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:41.040 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:40.634+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:41.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:40 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:40.952+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:41.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:41.041+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:41.436 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:41 smithi055 podman[94711]: 2024-04-26 22:08:41.306321282 +0000 UTC m=+5.467733797 container died 5d6246f64f800d4044965c9199c48f1ab77cf2f7fefb3e758daa375442be1316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:08:41.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:41 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:41.103+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:41.812 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:41 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:41.514+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:42.002 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:41.626+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:42.080 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:41 smithi055 podman[94711]: 2024-04-26 22:08:41.812983729 +0000 UTC m=+5.974396253 container cleanup 5d6246f64f800d4044965c9199c48f1ab77cf2f7fefb3e758daa375442be1316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T22:08:42.081 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:41 smithi055 bash[94711]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3 2024-04-26T22:08:42.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:41 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:41.989+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:42.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:42.002+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:42.393 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:42.113+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:42.393 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:42 smithi055 podman[94729]: 2024-04-26 22:08:42.08127362 +0000 UTC m=+0.768156040 container remove 5d6246f64f800d4044965c9199c48f1ab77cf2f7fefb3e758daa375442be1316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-04-26T22:08:42.686 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:42 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:42.494+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:42.686 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:42 smithi055 podman[94821]: 2024-04-26 22:08:42.29634071 +0000 UTC m=+0.018431520 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:08:42.687 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:42 smithi055 podman[94821]: 2024-04-26 22:08:42.424058936 +0000 UTC m=+0.146149718 container create 28e3b519735897b5857f1126ad82e0086c261cbdc61bf19e5525567606bec269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T22:08:42.962 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:42.591+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:43.017 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:42 smithi055 podman[94821]: 2024-04-26 22:08:42.710472719 +0000 UTC m=+0.432563503 container init 28e3b519735897b5857f1126ad82e0086c261cbdc61bf19e5525567606bec269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T22:08:43.018 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:42 smithi055 podman[94821]: 2024-04-26 22:08:42.714406911 +0000 UTC m=+0.436497695 container start 28e3b519735897b5857f1126ad82e0086c261cbdc61bf19e5525567606bec269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T22:08:43.018 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:42 smithi055 podman[94821]: 2024-04-26 22:08:42.810803446 +0000 UTC m=+0.532894244 container attach 28e3b519735897b5857f1126ad82e0086c261cbdc61bf19e5525567606bec269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0) 2024-04-26T22:08:43.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:42 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:42.960+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:43.304 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:43 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:43.154+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:43.304 INFO:journalctl@ceph.osd.3.smithi055.stdout:Apr 26 22:08:43 smithi055 podman[94821]: 2024-04-26 22:08:43.018067243 +0000 UTC m=+0.740158032 container died 28e3b519735897b5857f1126ad82e0086c261cbdc61bf19e5525567606bec269 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:08:43.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:42.961+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:43.686 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:43 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:43.527+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:43.730 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T22:08:43.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:43.769 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T22:08:43.770 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T22:08:43.770 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.4 2024-04-26T22:08:43.999 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:43.585+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:44.095 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:43 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:43.938+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:44.095 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:43 smithi064 systemd[1]: Stopping Ceph osd.4 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:44.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:43.998+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:44.482 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:44 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:44.198+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:44.506 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:44 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4[43877]: 2024-04-26T22:08:44.094+0000 7ff0070bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:44.507 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:44 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4[43877]: 2024-04-26T22:08:44.094+0000 7ff0070bc700 -1 osd.4 92 *** Got signal Terminated *** 2024-04-26T22:08:44.507 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:44 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4[43877]: 2024-04-26T22:08:44.094+0000 7ff0070bc700 -1 osd.4 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:08:44.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:44 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:44.482+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:44.982 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:44.606+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:45.256 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:44 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:44.951+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:45.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:44.982+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:45.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:44.982+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:45.523 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:45 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:45.192+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:45.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:45 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:45.523+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:45.962 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:45.577+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:46.256 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:45 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:45.942+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:46.287 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:45.962+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:46.288 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:45.988+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:46.288 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:45.988+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:46.523 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:46 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:46.239+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:46.857 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:46 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:46.587+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:46.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:46 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:46.523+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:46.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:46 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:46.523+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:47.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:46 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:46.925+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:47.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:46 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:46.954+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:47.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:47 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:47.001+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:47.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:47 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:47.001+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:47.538 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:47.206+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:47.538 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:47.206+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:47.757 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:47 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4[43877]: 2024-04-26T22:08:47.571+0000 7ff0036b0700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.718421+0000 front 2024-04-26T22:08:23.718395+0000 (oldest deadline 2024-04-26T22:08:46.618497+0000) 2024-04-26T22:08:47.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:47 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:47.423+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:47.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:47.538+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:47.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:47 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:47.538+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:47.992 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:47 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:47.626+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:48.198 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:47 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:47.927+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:48.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:47 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:47.993+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:48.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:48 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:48.040+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:48.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:48 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:48.040+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:48.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:48.168+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:48.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:48.168+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:48.506 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:48 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:48.401+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:48.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:48.524+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:48.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:48.524+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:48.941 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:48 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4[43877]: 2024-04-26T22:08:48.524+0000 7ff0036b0700 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.718421+0000 front 2024-04-26T22:08:23.718395+0000 (oldest deadline 2024-04-26T22:08:46.618497+0000) 2024-04-26T22:08:48.960 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:48 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:48.580+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:48.960 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:48 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:48.580+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:49.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:48 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:48.941+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:49.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:48 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:48.941+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:49.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:48 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:48.960+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:49.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:49 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:49.001+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:49.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:49 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:49.001+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:49.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:49 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:49.129+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:49.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:49 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:49.129+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:49.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:49 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:49.436+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:49.757 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:49 smithi064 podman[68435]: 2024-04-26 22:08:49.525844415 +0000 UTC m=+5.548977535 container died b95b8e659f2efbd9ef9affe78e8ba0d7e4d9b3176a873febeab3c560150293e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T22:08:49.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:49 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:49.558+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:49.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:49 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:49.558+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:49.966 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:49 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:49.578+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:49.966 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:49 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:49.578+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:50.172 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:49 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:49.948+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:50.172 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:49 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:49.948+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:50.172 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:49 smithi064 podman[68435]: 2024-04-26 22:08:49.928991447 +0000 UTC m=+5.952124556 container cleanup b95b8e659f2efbd9ef9affe78e8ba0d7e4d9b3176a873febeab3c560150293e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:08:50.172 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:49 smithi064 bash[68435]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4 2024-04-26T22:08:50.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:49 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:49.966+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:50.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:50 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:50.022+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:50.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:50 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:50.022+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:50.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:50 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:50.178+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:50.437 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:50 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:50.178+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:50.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:50 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:50.415+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:50.507 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:50 smithi064 podman[68454]: 2024-04-26 22:08:50.172664472 +0000 UTC m=+0.642885075 container remove b95b8e659f2efbd9ef9affe78e8ba0d7e4d9b3176a873febeab3c560150293e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:08:50.800 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:50 smithi064 podman[68545]: 2024-04-26 22:08:50.418078182 +0000 UTC m=+0.018390861 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:08:50.800 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:50 smithi064 podman[68545]: 2024-04-26 22:08:50.531921339 +0000 UTC m=+0.132234048 container create 710f34e34477cee9d5937cd4cb4ef4fb71cd641d9a1e1621513b25a51e1d6442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T22:08:50.801 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:50 smithi064 podman[68545]: 2024-04-26 22:08:50.743406252 +0000 UTC m=+0.343718918 container init 710f34e34477cee9d5937cd4cb4ef4fb71cd641d9a1e1621513b25a51e1d6442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T22:08:50.801 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:50 smithi064 podman[68545]: 2024-04-26 22:08:50.747073539 +0000 UTC m=+0.347386206 container start 710f34e34477cee9d5937cd4cb4ef4fb71cd641d9a1e1621513b25a51e1d6442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T22:08:50.801 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:50 smithi064 podman[68545]: 2024-04-26 22:08:50.801273863 +0000 UTC m=+0.401586547 container attach 710f34e34477cee9d5937cd4cb4ef4fb71cd641d9a1e1621513b25a51e1d6442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T22:08:50.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:50 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:50.589+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:50.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:50 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:50.589+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:50.939 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:50 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:50.612+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:50.939 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:50 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:50.612+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:51.056 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:50 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:50.956+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:51.056 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:50 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:50.956+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:51.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:50 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:50.940+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:51.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:50 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:50.999+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:51.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:50 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:50.999+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:51.403 INFO:journalctl@ceph.osd.4.smithi064.stdout:Apr 26 22:08:51 smithi064 podman[68545]: 2024-04-26 22:08:51.057691235 +0000 UTC m=+0.658003899 container died 710f34e34477cee9d5937cd4cb4ef4fb71cd641d9a1e1621513b25a51e1d6442 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T22:08:51.543 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:51 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:51.209+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:51.543 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:51 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:51.209+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:51.680 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:51 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:51.407+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:51.710 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T22:08:51.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:51.751 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T22:08:51.751 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T22:08:51.751 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.5 2024-04-26T22:08:51.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:51 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:51.543+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:51.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:51 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:51.543+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:51.983 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:51 smithi165 systemd[1]: Stopping Ceph osd.5 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:08:51.984 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:51 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:51.578+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:51.984 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:51 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:51.578+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:52.007 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:51 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:51.911+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:52.007 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:51 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:51.911+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:52.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:51 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:51.984+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:52.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:52 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:52.044+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:52.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:52 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:52.044+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:52.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:52 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:52.068+0000 7f503800f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:08:52.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:52 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:52.068+0000 7f503800f700 -1 osd.5 92 *** Got signal Terminated *** 2024-04-26T22:08:52.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:52 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:52.068+0000 7f503800f700 -1 osd.5 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:08:52.539 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:52 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:52.201+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:52.539 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:52 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:52.201+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:52.756 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:52 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:52.372+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:52.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:52 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:52.564+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:52.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:52 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:52.564+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:52.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:52 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:52.539+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:52.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:52 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:52.539+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:53.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:52 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:52.863+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:53.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:52 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:52.863+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:53.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:52 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:52.979+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:53.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:53 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:53.021+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:53.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:53 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:53.021+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:53.492 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:53 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:53.208+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:53.492 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:53 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:53.208+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:53.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:53 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:53.332+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:53.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:53 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:53.492+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:53.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:53 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:53.492+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:54.019 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:53 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:53.577+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:54.019 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:53 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:53.577+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:54.020 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:53 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:53.577+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:08:54.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:53 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:53.900+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:54.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:53 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:53.900+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:54.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:54 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:54.020+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:54.326 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:54 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:54.038+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:54.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:54 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:54.038+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:54.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:54 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:54.172+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:54.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:54 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:54.172+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:54.436 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:54 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:54.172+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:08:54.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:54 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:54.318+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:54.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:54 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:54.318+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:08:54.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:54 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:54.510+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:54.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:54 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:54.510+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:55.018 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:54 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:54.616+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:55.018 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:54 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:54.616+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:55.018 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:54 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:54.616+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:08:55.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:54 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:54.930+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:55.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:54 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:54.930+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:55.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:55.018+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:55.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:55.084+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:55.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:55.084+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:55.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:55.084+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:28.346249+0000 front 2024-04-26T22:08:28.346283+0000 (oldest deadline 2024-04-26T22:08:54.246417+0000) 2024-04-26T22:08:55.545 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:55 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:55.208+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:55.545 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:55 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:55.208+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:55.545 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:55 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:55.208+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:08:55.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:55 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:55.317+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:55.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:55 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:55.317+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:08:55.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:55.573+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:55.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:55.573+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:55.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:55.573+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:08:55.937 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:55 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:55.546+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:55.937 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:55 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:55.546+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:56.256 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:55 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:55.907+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:56.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:55 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:55.907+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:56.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:55.987+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:56.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:55 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:55.987+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:08:56.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:56 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:56.113+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:13.144503+0000 front 2024-04-26T22:08:13.144545+0000 (oldest deadline 2024-04-26T22:08:37.244333+0000) 2024-04-26T22:08:56.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:56 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:56.113+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.145728+0000 front 2024-04-26T22:08:23.145742+0000 (oldest deadline 2024-04-26T22:08:44.845626+0000) 2024-04-26T22:08:56.327 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:56 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5[45087]: 2024-04-26T22:08:56.113+0000 7f5034603700 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:28.346249+0000 front 2024-04-26T22:08:28.346283+0000 (oldest deadline 2024-04-26T22:08:54.246417+0000) 2024-04-26T22:08:56.565 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:56 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:56.216+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:56.565 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:56 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:56.216+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:56.565 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:56 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:56.216+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:08:56.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:56 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:56.280+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:56.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:56 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:56.280+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:08:56.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:56 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:56.568+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:56.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:56 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:56.568+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:56.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:56 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:56.568+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:08:56.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:56 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:56.564+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:56.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:56 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:56.565+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:56.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:56 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:56.565+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:08:57.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:56 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:56.905+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:57.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:56 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:56.905+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:57.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:56 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:56.905+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:08:57.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:56 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:56.957+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:57.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:56 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:56.957+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:08:57.611 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:57.250+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:57.611 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:57.250+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:57.611 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:57.250+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:08:57.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:57 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:57.265+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:57.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:57 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:57.265+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:08:57.826 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:57 smithi165 podman[69906]: 2024-04-26 22:08:57.606622454 +0000 UTC m=+5.659965482 container died 3cc43595ade5b05ec74a58fb39ab5e2d5f9684533cd841c9e1052ebf64ac05b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T22:08:57.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:57 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:57.559+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:57.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:57 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:57.559+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:57.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:57 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:57.559+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:08:57.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:57.610+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:57.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:57.610+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:57.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:57.610+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:08:58.225 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:57 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:57.919+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:58.225 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:57 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:57.919+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:58.225 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:57 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:57.919+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:08:58.254 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:57 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:57.974+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:58.254 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:57 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:57.974+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:08:58.254 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:58 smithi165 podman[69906]: 2024-04-26 22:08:58.026882577 +0000 UTC m=+6.080225611 container cleanup 3cc43595ade5b05ec74a58fb39ab5e2d5f9684533cd841c9e1052ebf64ac05b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:08:58.254 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:58 smithi165 bash[69906]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5 2024-04-26T22:08:58.506 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:58.224+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:58.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:58.224+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:08:58.543 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:58 smithi165 podman[69923]: 2024-04-26 22:08:58.254218268 +0000 UTC m=+0.647075739 container remove 3cc43595ade5b05ec74a58fb39ab5e2d5f9684533cd841c9e1052ebf64ac05b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T22:08:58.565 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:58 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:58.299+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:58.565 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:58 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:58.299+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:58.565 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:58 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:58.299+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:08:58.821 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:58 smithi165 podman[70014]: 2024-04-26 22:08:58.603180612 +0000 UTC m=+0.018073915 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:08:58.821 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:58 smithi165 podman[70014]: 2024-04-26 22:08:58.721848186 +0000 UTC m=+0.136741472 container create da70f96db702d3d781458579b3acc4d9a1d08ba2cc97c65ee27bd16adfccd6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-26T22:08:58.821 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:58.543+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:58.821 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:58.543+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:58.821 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:58.543+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:08:58.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:58 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:58.565+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:58.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:58 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:58.565+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:58.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:58 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:58.565+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:08:59.076 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:58.988+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:08:59.076 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:58 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:58.988+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:08:59.077 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:58 smithi165 podman[70014]: 2024-04-26 22:08:58.958818112 +0000 UTC m=+0.373711404 container init da70f96db702d3d781458579b3acc4d9a1d08ba2cc97c65ee27bd16adfccd6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T22:08:59.077 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:58 smithi165 podman[70014]: 2024-04-26 22:08:58.962147716 +0000 UTC m=+0.377041000 container start da70f96db702d3d781458579b3acc4d9a1d08ba2cc97c65ee27bd16adfccd6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T22:08:59.077 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:59 smithi165 podman[70014]: 2024-04-26 22:08:59.058549111 +0000 UTC m=+0.473442402 container attach da70f96db702d3d781458579b3acc4d9a1d08ba2cc97c65ee27bd16adfccd6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T22:08:59.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:58.956+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:08:59.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:58.956+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:08:59.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:58 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:58.956+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:08:59.521 INFO:journalctl@ceph.osd.5.smithi165.stdout:Apr 26 22:08:59 smithi165 podman[70014]: 2024-04-26 22:08:59.25546987 +0000 UTC m=+0.670363143 container died da70f96db702d3d781458579b3acc4d9a1d08ba2cc97c65ee27bd16adfccd6e1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T22:08:59.546 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:59 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:59.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:08:59.546 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:59 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:59.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:08:59.546 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:08:59 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:08:59.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:08:59.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:59 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:59.256+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:08:59.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:08:59 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:08:59.256+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:08:59.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:59 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:59.521+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:08:59.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:59 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:59.521+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:08:59.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:08:59 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:08:59.521+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:08:59.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:59 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:59.546+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:08:59.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:59 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:59.546+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:08:59.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:08:59 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:08:59.546+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:08:59.948 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T22:08:59.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:08:59.988 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T22:08:59.988 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T22:08:59.988 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.6 2024-04-26T22:09:00.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:59 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:59.922+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:00.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:59 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:59.922+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:00.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:08:59 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:08:59.922+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:00.271 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:00 smithi055 systemd[1]: Stopping Ceph osd.6 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:00.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:59 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:59.992+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:00.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:08:59 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:08:59.992+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:00.592 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:00.327+0000 7fd67f554700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:00.592 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:00.327+0000 7fd67f554700 -1 osd.6 92 *** Got signal Terminated *** 2024-04-26T22:09:00.592 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:00.327+0000 7fd67f554700 -1 osd.6 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:09:00.593 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:00.270+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:00.593 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:00.270+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:00.593 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:00.270+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:00.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:00 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:00.276+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:00.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:00 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:00.276+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:00.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:00 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:00.553+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:00.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:00 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:00.553+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:00.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:00 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:00.553+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:00.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:00.592+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:09:00.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:00.592+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:09:00.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:00 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:00.592+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:09:01.237 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:00 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:00.941+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:01.237 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:00 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:00.941+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:01.237 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:00 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:00.941+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:01.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:01 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:01.035+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:01.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:01 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:01.035+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:01.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:01 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:01.237+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:01.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:01 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:01.237+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:01.615 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:01.301+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:01.615 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:01.301+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:01.615 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:01.301+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:01.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:01 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:01.520+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:01.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:01 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:01.520+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:01.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:01 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:01.520+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:01.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:01.615+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:09:01.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:01.615+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:09:01.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:01 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:01.615+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:09:02.207 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:01 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:01.949+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:02.207 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:01 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:01.949+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:02.207 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:01 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:01.949+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:02.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:01.999+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:02.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:01.999+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:02.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:02.207+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:02.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:02.207+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:02.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:02.207+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:02.604 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:02.345+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:02.604 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:02.345+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:02.604 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:02.345+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:02.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:02.533+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:02.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:02.533+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:02.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:02 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:02.533+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:02.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:02.603+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:09:02.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:02.603+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:09:02.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:02.603+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:09:03.257 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:03.174+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:03.257 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:03.174+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:03.257 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:03.174+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:03.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:02.966+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:03.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:02.966+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:03.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:02 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:02.966+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:03.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:03.010+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:03.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:03.010+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:03.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:03.010+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:03.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:03.375+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:03.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:03.375+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:03.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:03.375+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:03.687 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:03.611+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:09:03.687 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:03.611+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:09:03.687 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:03.611+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:09:03.687 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:03 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:03.611+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.774979+0000 front 2024-04-26T22:08:40.775064+0000 (oldest deadline 2024-04-26T22:09:03.075040+0000) 2024-04-26T22:09:03.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:03.545+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:03.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:03.545+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:03.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:03.545+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:04.257 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:04.219+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:04.257 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:04.219+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:04.257 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:04.219+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:04.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:03.996+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:04.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:03.996+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:04.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:03 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:03.996+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:04.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:03.986+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:04.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:03.986+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:04.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:03 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:03.986+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:04.639 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:04.381+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:04.639 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:04.381+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:04.639 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:04.381+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:04.639 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:04.381+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:04.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:04.638+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:12.468478+0000 front 2024-04-26T22:08:12.468616+0000 (oldest deadline 2024-04-26T22:08:35.869199+0000) 2024-04-26T22:09:04.936 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:04.638+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:19.970366+0000 front 2024-04-26T22:08:19.970393+0000 (oldest deadline 2024-04-26T22:08:45.870425+0000) 2024-04-26T22:09:04.937 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:04.638+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.073390+0000 front 2024-04-26T22:08:32.073352+0000 (oldest deadline 2024-04-26T22:08:55.573339+0000) 2024-04-26T22:09:04.937 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:04 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6[54739]: 2024-04-26T22:09:04.638+0000 7fd67bb48700 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.774979+0000 front 2024-04-26T22:08:40.775064+0000 (oldest deadline 2024-04-26T22:09:03.075040+0000) 2024-04-26T22:09:04.968 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:04.582+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:04.968 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:04.582+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:04.968 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:04.582+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:05.236 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:04.986+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:05.236 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:04.986+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:05.237 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:04.986+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:05.237 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:04 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:04.986+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:05.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:04.967+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:05.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:04.967+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:05.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:04 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:04.967+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:05.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:05.236+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:05.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:05.236+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:05.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:05.236+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:05.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:05 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:05.365+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:05.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:05 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:05.365+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:05.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:05 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:05.365+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:05.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:05 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:05.365+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:06.008 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:05.588+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:06.008 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:05.588+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:06.009 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:05 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:05.588+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:06.087 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:05 smithi055 podman[94937]: 2024-04-26 22:09:05.803469174 +0000 UTC m=+5.611597532 container died fdf9272fffea9a4dfeab11731fb3ddf391e88d1081807db67fb5bcff4dfbd63b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T22:09:06.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:05.938+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:06.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:05.938+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:06.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:05.938+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:06.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:05 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:05.938+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:06.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:06 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:06.008+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:06.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:06 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:06.008+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:06.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:06 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:06.008+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:06.338 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:06 smithi055 podman[94937]: 2024-04-26 22:09:06.17832426 +0000 UTC m=+5.986452613 container cleanup fdf9272fffea9a4dfeab11731fb3ddf391e88d1081807db67fb5bcff4dfbd63b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:09:06.338 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:06 smithi055 bash[94937]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6 2024-04-26T22:09:06.686 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:06 smithi055 podman[94957]: 2024-04-26 22:09:06.446404584 +0000 UTC m=+0.635645159 container remove fdf9272fffea9a4dfeab11731fb3ddf391e88d1081807db67fb5bcff4dfbd63b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-26T22:09:06.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:06.337+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:06.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:06.337+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:06.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:06.337+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:06.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:06 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:06.337+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:06.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:06.272+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:06.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:06.272+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:06.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:06.272+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:07.016 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:06 smithi055 podman[95049]: 2024-04-26 22:09:06.654777943 +0000 UTC m=+0.018316559 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:09:07.016 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:06 smithi055 podman[95049]: 2024-04-26 22:09:06.772398438 +0000 UTC m=+0.135937038 container create 8f928a6f0214c1863c5973207615d5b18be8cd9d58b6088b27fce857f05c9128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:09:07.016 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:07 smithi055 podman[95049]: 2024-04-26 22:09:07.017008898 +0000 UTC m=+0.380547498 container init 8f928a6f0214c1863c5973207615d5b18be8cd9d58b6088b27fce857f05c9128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:09:07.035 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:06 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:06.636+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:07.035 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:06 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:06.636+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:07.035 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:06 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:06.636+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:07.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:06.893+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:07.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:06.893+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:07.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:06.893+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:07.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:06.894+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:07.299 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:07 smithi055 podman[95049]: 2024-04-26 22:09:07.020427658 +0000 UTC m=+0.383966258 container start 8f928a6f0214c1863c5973207615d5b18be8cd9d58b6088b27fce857f05c9128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T22:09:07.299 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:07 smithi055 podman[95049]: 2024-04-26 22:09:07.125537933 +0000 UTC m=+0.489076550 container attach 8f928a6f0214c1863c5973207615d5b18be8cd9d58b6088b27fce857f05c9128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T22:09:07.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:07 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:07.035+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:07.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:07 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:07.035+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:07.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:07 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:07.035+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:07.544 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:07.286+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:07.544 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:07.286+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:07.544 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:07.286+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:07.628 INFO:journalctl@ceph.osd.6.smithi055.stdout:Apr 26 22:09:07 smithi055 podman[95049]: 2024-04-26 22:09:07.327684293 +0000 UTC m=+0.691222896 container died 8f928a6f0214c1863c5973207615d5b18be8cd9d58b6088b27fce857f05c9128 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:09:07.628 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:07.298+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:07.628 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:07.298+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:07.628 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:07.298+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:07.628 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:07 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:07.298+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:07.947 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T22:09:07.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:09:07.987 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T22:09:07.987 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T22:09:07.987 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.7 2024-04-26T22:09:08.005 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:07 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:07.685+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:08.005 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:07 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:07.685+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:08.005 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:07 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:07.685+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:08.256 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:08 smithi064 systemd[1]: Stopping Ceph osd.7 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:08.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:07.873+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:08.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:07.873+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:08.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:07.873+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:08.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:07 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:07.873+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:08.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:08.004+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:08.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:08.004+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:08.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:08.004+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:08.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:08 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:08.326+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:08.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:08 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:08.326+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:08.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:08 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:08.326+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:08.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:08 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:08.326+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:08.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:08.285+0000 7fb868e6c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:08.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:08.285+0000 7fb868e6c700 -1 osd.7 92 *** Got signal Terminated *** 2024-04-26T22:09:08.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:08.285+0000 7fb868e6c700 -1 osd.7 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:09:08.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:08.312+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:08.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:08.312+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:08.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:08.312+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:08.985 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:08.708+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:08.985 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:08.708+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:08.985 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:08.708+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:09.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:08.904+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:09.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:08.904+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:09.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:08.904+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:09.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:08 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:08.904+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:09.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:08.984+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:09.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:08.984+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:09.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:08 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:08.984+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:09.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:09 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:09.317+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:09.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:09 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:09.317+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:09.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:09 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:09.317+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:09.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:09 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:09.317+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:09.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:09.352+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:09.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:09.352+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:09.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:09.352+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:09.986 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:09 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:09.712+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:09.986 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:09 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:09.712+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:09.986 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:09 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:09.712+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:10.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:09.934+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:10.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:09.934+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:10.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:09.934+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:10.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:09 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:09.934+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:10.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:09 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:09.986+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:10.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:09 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:09.986+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:10.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:09 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:09.986+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:10.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:10 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:10.268+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:10.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:10 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:10.268+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:10.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:10 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:10.268+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:10.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:10 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:10.269+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:10.756 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:10 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:10.328+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:10.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:10 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:10.328+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:10.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:10 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:10.328+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:10.969 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:10 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:10.682+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:10.969 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:10 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:10.682+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:10.970 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:10 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:10.682+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:11.242 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:10 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:10.969+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:11.242 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:10 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:10.969+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:11.242 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:10 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:10.969+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:11.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:10 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:10.983+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:11.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:10 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:10.983+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:11.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:10 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:10.983+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:11.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:10 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:10.983+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:11.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:11 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:11.260+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:11.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:11 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:11.260+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:11.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:11 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:11.260+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:11.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:11 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:11.260+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:11.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:11 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:11.359+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:11.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:11 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:11.359+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:11.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:11 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:11.359+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:11.949 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:11.638+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:11.949 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:11.638+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:11.949 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:11.638+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:11.949 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:11.638+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:12.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:11.948+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:12.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:11.948+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:12.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:11 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:11.948+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:12.357 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:12 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:12.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:12.357 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:12 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:12.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:12.357 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:12 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:12.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:12.358 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:12 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:12.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:12.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:12 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:12.218+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:12.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:12 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:12.218+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:12.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:12 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:12.218+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:12.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:12 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:12.218+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:12.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:12 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:12.218+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:12.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:12 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:12.357+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:21.737596+0000 front 2024-04-26T22:08:21.737710+0000 (oldest deadline 2024-04-26T22:08:46.437742+0000) 2024-04-26T22:09:12.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:12 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:12.357+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.239273+0000 front 2024-04-26T22:08:32.239224+0000 (oldest deadline 2024-04-26T22:08:53.339279+0000) 2024-04-26T22:09:12.757 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:12 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7[50711]: 2024-04-26T22:09:12.357+0000 7fb865460700 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.941320+0000 front 2024-04-26T22:08:40.941389+0000 (oldest deadline 2024-04-26T22:09:02.041220+0000) 2024-04-26T22:09:12.908 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:12 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:12.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:12.908 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:12 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:12.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:12.909 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:12 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:12.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:12.909 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:12 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:12.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:13.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:12 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:12.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:13.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:12 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:12.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:13.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:12 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:12.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:13.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:12 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:12.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:13.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:13 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:13.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:13.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:13 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:13.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:13.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:13 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:13.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:13.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:13 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:13.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:13.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:13 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:13.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:13.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:13 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:13.232+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:13.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:13 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:13.232+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:13.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:13 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:13.232+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:13.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:13 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:13.232+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:13.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:13 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:13.232+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:13.945 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:13 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:13.615+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:13.945 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:13 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:13.615+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:13.945 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:13 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:13.615+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:13.946 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:13 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:13.615+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:14.150 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:13 smithi064 podman[68661]: 2024-04-26 22:09:13.886919131 +0000 UTC m=+5.712716237 container died 025ddc51c965f1596d182c3b4a7d35217061c8de4d6492f7043c0c75561faf5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T22:09:14.151 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:14 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:14.107+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:14.151 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:14 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:14.107+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:14.151 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:14 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:14.107+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:14.151 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:14 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:14.107+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:14.151 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:14 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:14.107+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:14.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:13 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:13.945+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:14.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:13 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:13.945+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:14.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:13 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:13.945+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:14.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:13 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:13.945+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:14.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:14 smithi064 podman[68661]: 2024-04-26 22:09:14.258891149 +0000 UTC m=+6.084688264 container cleanup 025ddc51c965f1596d182c3b4a7d35217061c8de4d6492f7043c0c75561faf5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T22:09:14.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:14 smithi064 bash[68661]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7 2024-04-26T22:09:14.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:14 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:14.269+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:14.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:14 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:14.269+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:14.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:14 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:14.269+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:14.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:14 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:14.269+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:14.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:14 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:14.269+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:14.860 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:14 smithi064 podman[68681]: 2024-04-26 22:09:14.55175496 +0000 UTC m=+0.660441114 container remove 025ddc51c965f1596d182c3b4a7d35217061c8de4d6492f7043c0c75561faf5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T22:09:14.926 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:14 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:14.616+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:14.926 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:14 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:14.616+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:14.926 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:14 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:14.616+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:14.926 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:14 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:14.616+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:15.146 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:14 smithi064 podman[68769]: 2024-04-26 22:09:14.763600154 +0000 UTC m=+0.018402719 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:09:15.146 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:14 smithi064 podman[68769]: 2024-04-26 22:09:14.869347224 +0000 UTC m=+0.124149789 container create ab8f67b45c4c0bdfdbe870b8a88a33c9e4d268c8eeb48c7c2a4e45f36b7f7856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T22:09:15.146 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:15 smithi064 podman[68769]: 2024-04-26 22:09:15.072100517 +0000 UTC m=+0.326903088 container init ab8f67b45c4c0bdfdbe870b8a88a33c9e4d268c8eeb48c7c2a4e45f36b7f7856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T22:09:15.146 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:15 smithi064 podman[68769]: 2024-04-26 22:09:15.075556356 +0000 UTC m=+0.330358920 container start ab8f67b45c4c0bdfdbe870b8a88a33c9e4d268c8eeb48c7c2a4e45f36b7f7856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T22:09:15.146 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:15 smithi064 podman[68769]: 2024-04-26 22:09:15.146744659 +0000 UTC m=+0.401547237 container attach ab8f67b45c4c0bdfdbe870b8a88a33c9e4d268c8eeb48c7c2a4e45f36b7f7856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:09:15.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:14 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:14.926+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:15.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:14 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:14.926+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:15.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:14 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:14.926+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:15.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:14 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:14.926+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:15.507 INFO:journalctl@ceph.osd.7.smithi064.stdout:Apr 26 22:09:15 smithi064 podman[68769]: 2024-04-26 22:09:15.386061486 +0000 UTC m=+0.640864053 container died ab8f67b45c4c0bdfdbe870b8a88a33c9e4d268c8eeb48c7c2a4e45f36b7f7856 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:09:15.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:15 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:15.148+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:15.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:15 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:15.148+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:15.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:15 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:15.148+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:15.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:15 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:15.148+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:15.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:15 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:15.148+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:15.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:15.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:15.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:15.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:15.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:15.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:15.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:15.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:15.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:15 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:15.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:15.956 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:15 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:15.603+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:15.957 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:15 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:15.603+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:15.957 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:15 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:15.603+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:15.957 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:15 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:15.603+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:16.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T22:09:16.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:09:16.091 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T22:09:16.091 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T22:09:16.091 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.8 2024-04-26T22:09:16.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:15 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:15.956+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:16.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:15 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:15.956+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:16.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:15 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:15.956+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:16.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:15 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:15.956+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:16.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:16 smithi165 systemd[1]: Stopping Ceph osd.8 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:16.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:16 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:16.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:16.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:16 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:16.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:16.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:16 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:16.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:16.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:16 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:16.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:16.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:16 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:16.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:16.588 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:16 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:16.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:16.588 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:16 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:16.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:16.588 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:16 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:16.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:16.588 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:16 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:16.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:16.588 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:16 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:16.264+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:16.826 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:16.408+0000 7f15d22e1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:16.826 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:16.408+0000 7f15d22e1700 -1 osd.8 92 *** Got signal Terminated *** 2024-04-26T22:09:16.826 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:16.408+0000 7f15d22e1700 -1 osd.8 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:09:16.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:16.634+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:16.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:16.634+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:16.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:16.634+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:16.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:16.634+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:17.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:16.912+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:17.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:16.912+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:17.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:16.912+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:17.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:16 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:16.912+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:17.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:17 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:17.092+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:17.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:17 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:17.092+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:17.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:17 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:17.092+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:17.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:17 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:17.092+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:17.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:17 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:17.092+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:17.604 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:17.296+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:17.604 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:17.296+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:17.604 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:17.296+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:17.604 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:17.296+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:17.604 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:17 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:17.296+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:18.076 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:17 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:17.896+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:18.077 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:17 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:17.896+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:18.077 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:17 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:17.896+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:18.077 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:17 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:17.896+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:18.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:17 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:17.677+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:18.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:17 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:17.677+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:18.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:17 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:17.677+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:18.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:17 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:17.677+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:18.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:18 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:18.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:18.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:18 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:18.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:18.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:18 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:18.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:18.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:18 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:18.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:18.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:18 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:18.122+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:18.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:18 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:18.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:18.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:18 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:18.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:18.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:18 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:18.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:18.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:18 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:18.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:18.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:18 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:18.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:18.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:18 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:18.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:18.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:18.636+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:18.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:18.636+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:18.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:18.636+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:18.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:18.636+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:18.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:18.636+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:19.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:18.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:19.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:18.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:19.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:18.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:19.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:18.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:19.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:18 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:18.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:52.672644+0000 front 2024-04-26T22:08:52.672682+0000 (oldest deadline 2024-04-26T22:09:18.572717+0000) 2024-04-26T22:09:19.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:19 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:19.163+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:19.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:19 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:19.163+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:19.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:19 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:19.163+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:19.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:19 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:19.163+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:19.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:19 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:19.163+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:19.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:19.277+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:19.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:19.277+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:19.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:19.277+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:19.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:19.277+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:19.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:19.277+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:19.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:19 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:19.277+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:20.076 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:19.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:20.076 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:19.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:20.076 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:19.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:20.076 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:19.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:20.077 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:19.907+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:52.672644+0000 front 2024-04-26T22:08:52.672682+0000 (oldest deadline 2024-04-26T22:09:18.572717+0000) 2024-04-26T22:09:20.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:19.659+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:20.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:19.659+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:20.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:19.659+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:20.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:19.659+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:20.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:19 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:19.659+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:20.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:20.186+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:20.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:20.186+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:20.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:20.186+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:20.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:20.186+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:20.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:20.186+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:20.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:20.263+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:20.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:20.263+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:20.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:20.263+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:20.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:20.263+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:20.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:20.263+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:20.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:20 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:20.263+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:20.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:20.641+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:20.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:20.641+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:20.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:20.641+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:20.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:20.641+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:20.943 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:20.641+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:21.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:20.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.564323+0000 front 2024-04-26T22:08:23.564339+0000 (oldest deadline 2024-04-26T22:08:45.264311+0000) 2024-04-26T22:09:21.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:20.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:32.166525+0000 front 2024-04-26T22:08:32.166559+0000 (oldest deadline 2024-04-26T22:08:55.666670+0000) 2024-04-26T22:09:21.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:20.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:40.069179+0000 front 2024-04-26T22:08:40.069317+0000 (oldest deadline 2024-04-26T22:09:02.869728+0000) 2024-04-26T22:09:21.326 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:20.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.572287+0000 front 2024-04-26T22:08:48.572400+0000 (oldest deadline 2024-04-26T22:09:12.672204+0000) 2024-04-26T22:09:21.327 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:20 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8[51788]: 2024-04-26T22:09:20.942+0000 7f15ca8cd700 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:52.672644+0000 front 2024-04-26T22:08:52.672682+0000 (oldest deadline 2024-04-26T22:09:18.572717+0000) 2024-04-26T22:09:21.506 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:21 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:21.144+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:21.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:21 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:21.144+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:21.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:21 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:21.144+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:21.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:21 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:21.144+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:21.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:21 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:21.144+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:21.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:21.214+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:21.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:21.214+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:21.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:21.214+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:21.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:21.214+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:21.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:21.214+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:21.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:21 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:21.215+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:21.941 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:21 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:21.610+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:21.941 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:21 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:21.610+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:21.941 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:21 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:21.610+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:21.941 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:21 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:21.610+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:21.941 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:21 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:21.610+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:22.227 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:21 smithi165 podman[70130]: 2024-04-26 22:09:21.942071675 +0000 UTC m=+5.658782455 container died 612a6165f1d8ac103afda739b08358bd434d45548af261d898c0ce91f54876c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T22:09:22.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:22 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:22.105+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:22.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:22 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:22.105+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:22.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:22 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:22.105+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:22.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:22 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:22.105+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:22.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:22 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:22.105+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:22.576 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:22 smithi165 podman[70130]: 2024-04-26 22:09:22.352090323 +0000 UTC m=+6.068801103 container cleanup 612a6165f1d8ac103afda739b08358bd434d45548af261d898c0ce91f54876c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T22:09:22.576 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:22 smithi165 bash[70130]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8 2024-04-26T22:09:22.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:22 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:22.254+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:22.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:22 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:22.254+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:22.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:22 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:22.254+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:22.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:22 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:22.254+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:22.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:22 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:22.254+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:22.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:22 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:22.254+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:22.953 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:22 smithi165 podman[70148]: 2024-04-26 22:09:22.662005405 +0000 UTC m=+0.700956955 container remove 612a6165f1d8ac103afda739b08358bd434d45548af261d898c0ce91f54876c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T22:09:22.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:22 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:22.625+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:22.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:22 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:22.625+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:22.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:22 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:22.625+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:22.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:22 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:22.625+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:22.954 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:22 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:22.625+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:23.249 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:22 smithi165 podman[70239]: 2024-04-26 22:09:22.857199574 +0000 UTC m=+0.018334989 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:09:23.249 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:23 smithi165 podman[70239]: 2024-04-26 22:09:23.004871265 +0000 UTC m=+0.166006671 container create 22395de4f2253766443f0f3353fed0e53f8cf42a77695342041bbe57a11abd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T22:09:23.249 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:23 smithi165 podman[70239]: 2024-04-26 22:09:23.249650674 +0000 UTC m=+0.410786089 container init 22395de4f2253766443f0f3353fed0e53f8cf42a77695342041bbe57a11abd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T22:09:23.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:23 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:23.140+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:23.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:23 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:23.140+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:23.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:23 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:23.140+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:23.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:23 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:23.140+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:23.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:23 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:23.140+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:23.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:23 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:23.140+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:23.546 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:23 smithi165 podman[70239]: 2024-04-26 22:09:23.253361667 +0000 UTC m=+0.414497074 container start 22395de4f2253766443f0f3353fed0e53f8cf42a77695342041bbe57a11abd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T22:09:23.546 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:23 smithi165 podman[70239]: 2024-04-26 22:09:23.341032723 +0000 UTC m=+0.502168136 container attach 22395de4f2253766443f0f3353fed0e53f8cf42a77695342041bbe57a11abd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:09:23.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:23 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:23.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:23.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:23 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:23.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:23.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:23 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:23.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:23.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:23 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:23.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:23.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:23 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:23.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:23.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:23 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:23.256+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:23.826 INFO:journalctl@ceph.osd.8.smithi165.stdout:Apr 26 22:09:23 smithi165 podman[70239]: 2024-04-26 22:09:23.547397011 +0000 UTC m=+0.708532427 container died 22395de4f2253766443f0f3353fed0e53f8cf42a77695342041bbe57a11abd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T22:09:23.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:23 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:23.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:23.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:23 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:23.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:23.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:23 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:23.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:23.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:23 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:23.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:23.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:23 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:23.609+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:24.182 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T22:09:24.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:09:24.221 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T22:09:24.221 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T22:09:24.221 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.9 2024-04-26T22:09:24.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:24 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:24.090+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:24.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:24 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:24.090+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:24.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:24 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:24.090+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:24.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:24 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:24.090+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:24.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:24 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:24.090+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:24.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:24 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:24.090+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 systemd[1]: Stopping Ceph osd.9 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.283+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.283+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.283+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.283+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.283+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.283+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.549+0000 7f35fad12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:24.550 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.549+0000 7f35fad12700 -1 osd.9 92 *** Got signal Terminated *** 2024-04-26T22:09:24.551 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:24 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:24.549+0000 7f35fad12700 -1 osd.9 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:09:24.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:24 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:24.620+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:24.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:24 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:24.620+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:24.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:24 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:24.620+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:24.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:24 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:24.620+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:24.827 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:24 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:24.620+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:25.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:25 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:25.132+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:25.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:25 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:25.132+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:25.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:25 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:25.132+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:25.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:25 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:25.132+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:25.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:25 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:25.132+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:25.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:25 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:25.132+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:25.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:25 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:25.280+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:25.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:25 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:25.280+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:25.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:25 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:25.280+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:25.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:25 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:25.280+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:25.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:25 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:25.280+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:25.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:25 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:25.280+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:26.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:25 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:25.633+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:26.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:25 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:25.633+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:26.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:25 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:25.633+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:26.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:25 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:25.633+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:26.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:25 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:25.633+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:26.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:26 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:26.085+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:26.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:26 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:26.085+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:26.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:26 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:26.085+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:26.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:26 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:26.085+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:26.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:26 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:26.085+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:26.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:26 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:26.085+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:26.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:26 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:26.251+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:26.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:26 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:26.251+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:26.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:26 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:26.251+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:26.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:26 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:26.251+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:26.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:26 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:26.251+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:26.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:26 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:26.251+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:27.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:26 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:26.622+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:27.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:26 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:26.622+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:27.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:26 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:26.622+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:27.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:26 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:26.622+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:27.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:26 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:26.622+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:27.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:26 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:26.622+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:27.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:27 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:27.084+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:27.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:27 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:27.084+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:27.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:27 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:27.084+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:27.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:27 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:27.084+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:27.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:27 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:27.084+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:27.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:27 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:27.084+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:27.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:27 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:27.084+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:27.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:27.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:27.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:27.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:27.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:27.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:27.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:27.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:27.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:27.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:27.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:27 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:27.286+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:28.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:27.652+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:28.078 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:27.652+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:28.078 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:27.652+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:28.078 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:27.652+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:28.078 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:27.652+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:28.078 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:27 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:27.652+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:28.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:28 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:28.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:28.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:28 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:28.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:28.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:28 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:28.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:28.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:28 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:28.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:28.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:28 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:28.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:28.508 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:28 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:28.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:28.508 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:28 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:28.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:28.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:28 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:28.273+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:28.689 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:28 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:28.273+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:28.689 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:28 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:28.273+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:28.689 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:28 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:28.273+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:28.689 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:28 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:28.273+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:28.690 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:28 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:28.273+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:29.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:28 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:28.623+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:29.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:28 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:28.623+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:29.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:28 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:28.623+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:29.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:28 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:28.623+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:29.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:28 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:28.623+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:29.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:28 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:28.623+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:29.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:29 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:29.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:29.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:29 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:29.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:29.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:29 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:29.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:29.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:29 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:29.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:29.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:29 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:29.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:29.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:29 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:29.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:29.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:29 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:29.060+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:29.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:29 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:29.295+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.127020+0000 front 2024-04-26T22:08:10.127135+0000 (oldest deadline 2024-04-26T22:08:37.727670+0000) 2024-04-26T22:09:29.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:29 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:29.295+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.628640+0000 front 2024-04-26T22:08:23.628739+0000 (oldest deadline 2024-04-26T22:08:46.528860+0000) 2024-04-26T22:09:29.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:29 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:29.295+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.130639+0000 front 2024-04-26T22:08:31.130543+0000 (oldest deadline 2024-04-26T22:08:53.330994+0000) 2024-04-26T22:09:29.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:29 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:29.295+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:39.231877+0000 front 2024-04-26T22:08:39.231930+0000 (oldest deadline 2024-04-26T22:09:03.931963+0000) 2024-04-26T22:09:29.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:29 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:29.295+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:43.932271+0000 front 2024-04-26T22:08:43.932338+0000 (oldest deadline 2024-04-26T22:09:12.132918+0000) 2024-04-26T22:09:29.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:29 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9[61615]: 2024-04-26T22:09:29.295+0000 7f35f32fe700 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.234141+0000 front 2024-04-26T22:08:56.234281+0000 (oldest deadline 2024-04-26T22:09:17.933935+0000) 2024-04-26T22:09:30.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:29 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:29.629+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:30.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:29 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:29.629+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:30.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:29 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:29.629+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:30.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:29 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:29.629+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:30.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:29 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:29.629+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:30.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:29 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:29.629+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:30.309 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:30 smithi055 podman[95165]: 2024-04-26 22:09:30.048099591 +0000 UTC m=+5.624431620 container died 15314487a0547a7a60c121e1429889220a07243c684817fa3a8cefeea31a7053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T22:09:30.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:30 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:30.039+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:30.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:30 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:30.039+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:30.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:30 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:30.039+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:30.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:30 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:30.039+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:30.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:30 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:30.039+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:30.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:30 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:30.039+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:30.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:30 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:30.039+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:30.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:30 smithi055 podman[95165]: 2024-04-26 22:09:30.400076571 +0000 UTC m=+5.976408592 container cleanup 15314487a0547a7a60c121e1429889220a07243c684817fa3a8cefeea31a7053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T22:09:30.687 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:30 smithi055 bash[95165]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9 2024-04-26T22:09:31.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:30 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:30.640+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:31.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:30 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:30.640+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:31.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:30 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:30.640+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:31.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:30 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:30.640+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:31.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:30 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:30.640+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:31.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:30 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:30.640+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:31.129 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:30 smithi055 podman[95182]: 2024-04-26 22:09:30.801667592 +0000 UTC m=+0.747507355 container remove 15314487a0547a7a60c121e1429889220a07243c684817fa3a8cefeea31a7053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T22:09:31.421 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:31 smithi055 podman[95272]: 2024-04-26 22:09:31.032901366 +0000 UTC m=+0.018145276 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:09:31.421 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:31 smithi055 podman[95272]: 2024-04-26 22:09:31.135760589 +0000 UTC m=+0.121004499 container create b190bbdec9d7f229f464f9383252cf326e69e21927e3789daf704c1252f5c724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T22:09:31.421 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:31 smithi055 podman[95272]: 2024-04-26 22:09:31.347251804 +0000 UTC m=+0.332495726 container init b190bbdec9d7f229f464f9383252cf326e69e21927e3789daf704c1252f5c724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T22:09:31.422 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:31 smithi055 podman[95272]: 2024-04-26 22:09:31.35063468 +0000 UTC m=+0.335878590 container start b190bbdec9d7f229f464f9383252cf326e69e21927e3789daf704c1252f5c724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T22:09:31.422 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:31 smithi055 podman[95272]: 2024-04-26 22:09:31.421687508 +0000 UTC m=+0.406931430 container attach b190bbdec9d7f229f464f9383252cf326e69e21927e3789daf704c1252f5c724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:09:31.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:31 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:31.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:31.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:31 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:31.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:31.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:31 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:31.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:31.508 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:31 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:31.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:31.508 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:31 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:31.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:31.508 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:31 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:31.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:31.508 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:31 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:31.050+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:31.686 INFO:journalctl@ceph.osd.9.smithi055.stdout:Apr 26 22:09:31 smithi055 podman[95272]: 2024-04-26 22:09:31.651858534 +0000 UTC m=+0.637102455 container died b190bbdec9d7f229f464f9383252cf326e69e21927e3789daf704c1252f5c724 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:09:32.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:31 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:31.680+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:32.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:31 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:31.680+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:32.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:31 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:31.680+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:32.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:31 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:31.680+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:32.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:31 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:31.680+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:32.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:31 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:31.680+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:32.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T22:09:32.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:09:32.423 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T22:09:32.423 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T22:09:32.423 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.10 2024-04-26T22:09:32.450 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.081+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:32.450 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.081+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:32.450 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.081+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:32.450 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.081+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:32.450 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.081+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:32.451 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.081+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:32.451 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.081+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:32.719 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 systemd[1]: Stopping Ceph osd.10 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:33.007 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.719+0000 7fa4768bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:33.007 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.719+0000 7fa4768bd700 -1 osd.10 92 *** Got signal Terminated *** 2024-04-26T22:09:33.007 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:32 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:32.719+0000 7fa4768bd700 -1 osd.10 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:09:33.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:32 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:32.721+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:33.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:32 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:32.721+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:33.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:32 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:32.721+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:33.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:32 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:32.721+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:33.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:32 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:32.721+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:33.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:32 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:32.721+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:33.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:33 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:33.065+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:33.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:33 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:33.065+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:33.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:33 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:33.065+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:33.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:33 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:33.065+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:33.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:33 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:33.065+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:33.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:33 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:33.065+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:33.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:33 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:33.065+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:34.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:33 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:33.722+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:34.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:33 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:33.722+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:34.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:33 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:33.722+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:34.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:33 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:33.722+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:34.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:33 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:33.722+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:34.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:33 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:33.722+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:34.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:34 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:34.018+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:34.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:34 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:34.018+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:34.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:34 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:34.018+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:34.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:34 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:34.018+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:34.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:34 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:34.018+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:34.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:34 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:34.018+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:34.508 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:34 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:34.018+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:35.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:34 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:34.715+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:35.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:34 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:34.715+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:35.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:34 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:34.715+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:35.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:34 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:34.715+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:35.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:34 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:34.715+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:35.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:34 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:34.715+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:35.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:35.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:35.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:35.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:35.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:35.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:35.507 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.021+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:36.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:35 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:35.749+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:36.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:35 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:35.749+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:36.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:35 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:35.749+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:36.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:35 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:35.749+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:36.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:35 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:35.749+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:36.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:35 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:35.749+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:36.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.988+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:36.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.988+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:36.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.988+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:36.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.988+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:36.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.988+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:36.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.988+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:36.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:35 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:35.988+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:37.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:36.731+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:37.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:36.731+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:37.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:36.731+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:37.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:36.731+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:37.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:36.731+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:37.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:36.731+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:37.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:36.731+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:37.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:36 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:36.963+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:10.055207+0000 front 2024-04-26T22:08:10.055162+0000 (oldest deadline 2024-04-26T22:08:35.955266+0000) 2024-04-26T22:09:37.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:36 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:36.963+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:23.556849+0000 front 2024-04-26T22:08:23.556781+0000 (oldest deadline 2024-04-26T22:08:48.256913+0000) 2024-04-26T22:09:37.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:36 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:36.963+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:30.557862+0000 front 2024-04-26T22:08:30.558004+0000 (oldest deadline 2024-04-26T22:08:56.457721+0000) 2024-04-26T22:09:37.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:36 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:36.963+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6814 osd.3 since back 2024-04-26T22:08:38.758697+0000 front 2024-04-26T22:08:38.758813+0000 (oldest deadline 2024-04-26T22:09:04.559045+0000) 2024-04-26T22:09:37.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:36 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:36.963+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.660120+0000 front 2024-04-26T22:08:48.660079+0000 (oldest deadline 2024-04-26T22:09:12.760219+0000) 2024-04-26T22:09:37.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:36 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:36.963+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:56.861666+0000 front 2024-04-26T22:08:56.861638+0000 (oldest deadline 2024-04-26T22:09:22.161445+0000) 2024-04-26T22:09:37.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:36 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10[57733]: 2024-04-26T22:09:36.963+0000 7fa46eea9700 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.161943+0000 front 2024-04-26T22:09:02.162075+0000 (oldest deadline 2024-04-26T22:09:26.762267+0000) 2024-04-26T22:09:38.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:37 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:37.723+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:38.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:37 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:37.723+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:38.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:37 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:37.723+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:38.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:37 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:37.723+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:38.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:37 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:37.723+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:38.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:37 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:37.723+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:38.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:37 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:37.723+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:38.546 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:38 smithi064 podman[68875]: 2024-04-26 22:09:38.280229062 +0000 UTC m=+5.661319724 container died 0b0de950e92f9d2e2a637a52f45a18d9d57521098ef2dca5a3eeba50e1eec966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T22:09:38.928 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:38 smithi064 podman[68875]: 2024-04-26 22:09:38.644490725 +0000 UTC m=+6.025581387 container cleanup 0b0de950e92f9d2e2a637a52f45a18d9d57521098ef2dca5a3eeba50e1eec966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T22:09:38.928 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:38 smithi064 bash[68875]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10 2024-04-26T22:09:39.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:38.725+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:39.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:38.725+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:39.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:38.725+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:39.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:38.725+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:39.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:38.725+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:39.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:38.725+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:39.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:38 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:38.725+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:39.257 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:38 smithi064 podman[68893]: 2024-04-26 22:09:38.929049325 +0000 UTC m=+0.647935280 container remove 0b0de950e92f9d2e2a637a52f45a18d9d57521098ef2dca5a3eeba50e1eec966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T22:09:39.557 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:39 smithi064 podman[68985]: 2024-04-26 22:09:39.186178428 +0000 UTC m=+0.018607324 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:09:39.557 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:39 smithi064 podman[68985]: 2024-04-26 22:09:39.288251396 +0000 UTC m=+0.120680291 container create d45174caf0993601e69f4e48598a76d481f27de93f63c0a3a5a443a158149748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:09:39.557 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:39 smithi064 podman[68985]: 2024-04-26 22:09:39.48268192 +0000 UTC m=+0.315110830 container init d45174caf0993601e69f4e48598a76d481f27de93f63c0a3a5a443a158149748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T22:09:39.557 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:39 smithi064 podman[68985]: 2024-04-26 22:09:39.48629392 +0000 UTC m=+0.318722824 container start d45174caf0993601e69f4e48598a76d481f27de93f63c0a3a5a443a158149748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:09:39.557 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:39 smithi064 podman[68985]: 2024-04-26 22:09:39.557104648 +0000 UTC m=+0.389533564 container attach d45174caf0993601e69f4e48598a76d481f27de93f63c0a3a5a443a158149748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T22:09:40.007 INFO:journalctl@ceph.osd.10.smithi064.stdout:Apr 26 22:09:39 smithi064 podman[68985]: 2024-04-26 22:09:39.785872531 +0000 UTC m=+0.618301427 container died d45174caf0993601e69f4e48598a76d481f27de93f63c0a3a5a443a158149748 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T22:09:40.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:39 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:39.724+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:40.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:39 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:39.724+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:40.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:39 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:39.724+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:40.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:39 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:39.724+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:40.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:39 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:39.724+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:40.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:39 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:39.724+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:40.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:39 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:39.724+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:40.377 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T22:09:40.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:09:40.458 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T22:09:40.458 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T22:09:40.458 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.11 2024-04-26T22:09:40.735 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 systemd[1]: Stopping Ceph osd.11 for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:40.993 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.735+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.735+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.735+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.735+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.735+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.735+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.735+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.779+0000 7f8d86705700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.779+0000 7f8d86705700 -1 osd.11 92 *** Got signal Terminated *** 2024-04-26T22:09:40.994 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:40 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:40.779+0000 7f8d86705700 -1 osd.11 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T22:09:42.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:41.770+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:42.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:41.770+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:42.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:41.770+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:42.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:41.770+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:42.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:41.770+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:42.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:41.770+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:42.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:41 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:41.770+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:43.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:42.800+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:43.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:42.800+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:43.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:42.800+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:43.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:42.800+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:43.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:42.800+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:43.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:42.800+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:43.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:42.800+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:43.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:42 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:42.800+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-04-26T22:09:17.001544+0000 front 2024-04-26T22:09:17.001598+0000 (oldest deadline 2024-04-26T22:09:42.301672+0000) 2024-04-26T22:09:44.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:43.816+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:44.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:43.816+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:44.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:43.816+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:44.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:43.816+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:44.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:43.816+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:44.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:43.816+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:44.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:43.816+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:44.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:43 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:43.816+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-04-26T22:09:17.001544+0000 front 2024-04-26T22:09:17.001598+0000 (oldest deadline 2024-04-26T22:09:42.301672+0000) 2024-04-26T22:09:45.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:44.805+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:45.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:44.805+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:45.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:44.805+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:45.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:44.805+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:45.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:44.805+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:45.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:44.805+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:45.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:44.805+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:45.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:44 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:44.805+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-04-26T22:09:17.001544+0000 front 2024-04-26T22:09:17.001598+0000 (oldest deadline 2024-04-26T22:09:42.301672+0000) 2024-04-26T22:09:46.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:45.775+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6806 osd.0 since back 2024-04-26T22:08:14.290720+0000 front 2024-04-26T22:08:14.290832+0000 (oldest deadline 2024-04-26T22:08:36.590819+0000) 2024-04-26T22:09:46.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:45.775+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6804 osd.1 since back 2024-04-26T22:08:22.392595+0000 front 2024-04-26T22:08:22.392405+0000 (oldest deadline 2024-04-26T22:08:47.692439+0000) 2024-04-26T22:09:46.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:45.775+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-04-26T22:08:31.793519+0000 front 2024-04-26T22:08:31.793688+0000 (oldest deadline 2024-04-26T22:08:53.493685+0000) 2024-04-26T22:09:46.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:45.775+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6812 osd.4 since back 2024-04-26T22:08:48.596952+0000 front 2024-04-26T22:08:48.597002+0000 (oldest deadline 2024-04-26T22:09:10.896849+0000) 2024-04-26T22:09:46.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:45.775+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-04-26T22:08:55.597911+0000 front 2024-04-26T22:08:55.598003+0000 (oldest deadline 2024-04-26T22:09:18.497998+0000) 2024-04-26T22:09:46.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:45.775+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.55:6822 osd.6 since back 2024-04-26T22:09:02.499606+0000 front 2024-04-26T22:09:02.499634+0000 (oldest deadline 2024-04-26T22:09:25.999391+0000) 2024-04-26T22:09:46.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:45.775+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.64:6820 osd.7 since back 2024-04-26T22:09:10.001032+0000 front 2024-04-26T22:09:10.000990+0000 (oldest deadline 2024-04-26T22:09:35.900941+0000) 2024-04-26T22:09:46.077 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:45 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11[58545]: 2024-04-26T22:09:45.775+0000 7f8d7ecf1700 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-04-26T22:09:17.001544+0000 front 2024-04-26T22:09:17.001598+0000 (oldest deadline 2024-04-26T22:09:42.301672+0000) 2024-04-26T22:09:46.514 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:46 smithi165 podman[70359]: 2024-04-26 22:09:46.24840584 +0000 UTC m=+5.591555097 container died 12bd386a26db665cb2b680ecc466f625d86bd509eb39b00e9972787988523526 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-26T22:09:46.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:46 smithi165 podman[70359]: 2024-04-26 22:09:46.605040955 +0000 UTC m=+5.948190228 container cleanup 12bd386a26db665cb2b680ecc466f625d86bd509eb39b00e9972787988523526 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T22:09:46.826 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:46 smithi165 bash[70359]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11 2024-04-26T22:09:47.232 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:46 smithi165 podman[70392]: 2024-04-26 22:09:46.931346884 +0000 UTC m=+0.677246528 container remove 12bd386a26db665cb2b680ecc466f625d86bd509eb39b00e9972787988523526 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T22:09:47.492 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:47 smithi165 podman[70483]: 2024-04-26 22:09:47.136783239 +0000 UTC m=+0.017832891 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-26T22:09:47.493 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:47 smithi165 podman[70483]: 2024-04-26 22:09:47.274060632 +0000 UTC m=+0.155110284 container create d6c1f263259c99c8bb39fc6f5e75326b7545342cb382fedfd84eaaa0dc69f929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T22:09:47.493 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:47 smithi165 podman[70483]: 2024-04-26 22:09:47.49355433 +0000 UTC m=+0.374603982 container init d6c1f263259c99c8bb39fc6f5e75326b7545342cb382fedfd84eaaa0dc69f929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-26T22:09:47.793 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:47 smithi165 podman[70483]: 2024-04-26 22:09:47.496636814 +0000 UTC m=+0.377686473 container start d6c1f263259c99c8bb39fc6f5e75326b7545342cb382fedfd84eaaa0dc69f929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T22:09:47.793 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:47 smithi165 podman[70483]: 2024-04-26 22:09:47.576757784 +0000 UTC m=+0.457807456 container attach d6c1f263259c99c8bb39fc6f5e75326b7545342cb382fedfd84eaaa0dc69f929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T22:09:48.076 INFO:journalctl@ceph.osd.11.smithi165.stdout:Apr 26 22:09:47 smithi165 podman[70483]: 2024-04-26 22:09:47.793737005 +0000 UTC m=+0.674786656 container died d6c1f263259c99c8bb39fc6f5e75326b7545342cb382fedfd84eaaa0dc69f929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T22:09:48.373 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T22:09:48.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:09:48.410 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T22:09:48.410 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T22:09:48.411 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.a 2024-04-26T22:09:48.692 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 22:09:48 smithi055 systemd[1]: Stopping Ceph mds.a for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:48.977 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 22:09:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a[68296]: 2024-04-26T22:09:48.728+0000 7fa85e8bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:48.977 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 22:09:48 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a[68296]: 2024-04-26T22:09:48.728+0000 7fa85e8bd700 -1 mds.a *** got signal Terminated *** 2024-04-26T22:09:52.746 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 22:09:52 smithi055 podman[95388]: 2024-04-26 22:09:52.482864907 +0000 UTC m=+3.866685611 container died b8a6de870765e91c067df02f0dc18bff31ce1e9890fc803d6f774e1cd74232cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:09:53.186 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 22:09:52 smithi055 podman[95388]: 2024-04-26 22:09:52.845746637 +0000 UTC m=+4.229567337 container cleanup b8a6de870765e91c067df02f0dc18bff31ce1e9890fc803d6f774e1cd74232cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T22:09:53.187 INFO:journalctl@ceph.mds.a.smithi055.stdout:Apr 26 22:09:52 smithi055 bash[95388]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-a 2024-04-26T22:09:53.336 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T22:09:53.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:09:53.373 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T22:09:53.373 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T22:09:53.373 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.d 2024-04-26T22:09:53.936 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 22:09:53 smithi055 systemd[1]: Stopping Ceph mds.d for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:53.936 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 22:09:53 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d[67328]: 2024-04-26T22:09:53.691+0000 7f5f27ac8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:53.936 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 22:09:53 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d[67328]: 2024-04-26T22:09:53.691+0000 7f5f27ac8700 -1 mds.d *** got signal Terminated *** 2024-04-26T22:09:56.686 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 22:09:56 smithi055 podman[95519]: 2024-04-26 22:09:56.338782575 +0000 UTC m=+2.746163781 container died 757b72f2bc210bb52074e88f5efa2f264e99b7b2223c6d956122b4f1863ede68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:09:57.067 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 22:09:56 smithi055 podman[95519]: 2024-04-26 22:09:56.800086554 +0000 UTC m=+3.207467760 container cleanup 757b72f2bc210bb52074e88f5efa2f264e99b7b2223c6d956122b4f1863ede68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:09:57.067 INFO:journalctl@ceph.mds.d.smithi055.stdout:Apr 26 22:09:56 smithi055 bash[95519]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-d 2024-04-26T22:09:57.240 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T22:09:57.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:09:57.274 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T22:09:57.274 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T22:09:57.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.g 2024-04-26T22:09:57.603 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:09:57 smithi055 systemd[1]: Stopping Ceph mds.g for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:09:57.936 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:09:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-g[67935]: 2024-04-26T22:09:57.603+0000 7fe3c204a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:09:57.937 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:09:57 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-g[67935]: 2024-04-26T22:09:57.603+0000 7fe3c204a700 -1 mds.g *** got signal Terminated *** 2024-04-26T22:10:01.937 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:10:01 smithi055 podman[95642]: 2024-04-26 22:10:01.616214222 +0000 UTC m=+4.118498168 container died ad1578d168e479088d382fce186e1601f23768e13abd072120d2914bc97c613c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:10:02.338 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:10:02 smithi055 podman[95642]: 2024-04-26 22:10:02.121461275 +0000 UTC m=+4.623745261 container cleanup ad1578d168e479088d382fce186e1601f23768e13abd072120d2914bc97c613c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-g, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-26T22:10:02.339 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:10:02 smithi055 bash[95642]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-g 2024-04-26T22:10:02.578 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T22:10:02.600 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:10:02 smithi055 podman[95657]: 2024-04-26 22:10:02.339415864 +0000 UTC m=+0.721066101 container remove ad1578d168e479088d382fce186e1601f23768e13abd072120d2914bc97c613c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:10:02.600 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:10:02 smithi055 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.g.service: Deactivated successfully. 2024-04-26T22:10:02.600 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:10:02 smithi055 systemd[1]: Stopped Ceph mds.g for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T22:10:02.601 INFO:journalctl@ceph.mds.g.smithi055.stdout:Apr 26 22:10:02 smithi055 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.g.service: Consumed 1.164s CPU time. 2024-04-26T22:10:02.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:02.613 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T22:10:02.613 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T22:10:02.614 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.j 2024-04-26T22:10:02.936 INFO:journalctl@ceph.mds.j.smithi055.stdout:Apr 26 22:10:02 smithi055 systemd[1]: Stopping Ceph mds.j for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:03.436 INFO:journalctl@ceph.mds.j.smithi055.stdout:Apr 26 22:10:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-j[66980]: 2024-04-26T22:10:02.941+0000 7f1b42ab5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:03.436 INFO:journalctl@ceph.mds.j.smithi055.stdout:Apr 26 22:10:02 smithi055 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-j[66980]: 2024-04-26T22:10:02.941+0000 7f1b42ab5700 -1 mds.j *** got signal Terminated *** 2024-04-26T22:10:05.658 INFO:journalctl@ceph.mds.j.smithi055.stdout:Apr 26 22:10:05 smithi055 podman[95764]: 2024-04-26 22:10:05.363709886 +0000 UTC m=+2.528616544 container died 6442bc02a4d48a394d7cbb1cceeaa38e610563daafaa2c8ce486a5811af5e2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T22:10:05.936 INFO:journalctl@ceph.mds.j.smithi055.stdout:Apr 26 22:10:05 smithi055 podman[95764]: 2024-04-26 22:10:05.658601111 +0000 UTC m=+2.823507778 container cleanup 6442bc02a4d48a394d7cbb1cceeaa38e610563daafaa2c8ce486a5811af5e2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True) 2024-04-26T22:10:05.936 INFO:journalctl@ceph.mds.j.smithi055.stdout:Apr 26 22:10:05 smithi055 bash[95764]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-j 2024-04-26T22:10:05.937 INFO:journalctl@ceph.mds.j.smithi055.stdout:Apr 26 22:10:05 smithi055 podman[95779]: 2024-04-26 22:10:05.884731626 +0000 UTC m=+0.520333914 container remove 6442bc02a4d48a394d7cbb1cceeaa38e610563daafaa2c8ce486a5811af5e2f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T22:10:06.106 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T22:10:06.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:06.138 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T22:10:06.138 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T22:10:06.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.b 2024-04-26T22:10:06.444 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:10:06 smithi064 systemd[1]: Stopping Ceph mds.b for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:06.756 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:10:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: 2024-04-26T22:10:06.444+0000 7fd5c3136700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:06.756 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:10:06 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b[63115]: 2024-04-26T22:10:06.444+0000 7fd5c3136700 -1 mds.b *** got signal Terminated *** 2024-04-26T22:10:10.711 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:10:10 smithi064 podman[69092]: 2024-04-26 22:10:10.456476696 +0000 UTC m=+4.121089009 container died 5568b9c16c3dc924c4bc38ffa578fcadc4641c0777cf387a00860b44084760e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T22:10:10.977 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:10:10 smithi064 podman[69092]: 2024-04-26 22:10:10.802414913 +0000 UTC m=+4.467027265 container cleanup 5568b9c16c3dc924c4bc38ffa578fcadc4641c0777cf387a00860b44084760e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T22:10:10.978 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:10:10 smithi064 bash[69092]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b 2024-04-26T22:10:11.234 INFO:journalctl@ceph.mds.b.smithi064.stdout:Apr 26 22:10:10 smithi064 podman[69106]: 2024-04-26 22:10:10.978527917 +0000 UTC m=+0.518391962 container remove 5568b9c16c3dc924c4bc38ffa578fcadc4641c0777cf387a00860b44084760e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T22:10:11.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T22:10:11.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:11.273 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T22:10:11.274 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T22:10:11.274 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.e 2024-04-26T22:10:11.506 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:11 smithi064 systemd[1]: Stopping Ceph mds.e for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:12.007 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:11 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e[62497]: 2024-04-26T22:10:11.598+0000 7ffae2395700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:12.007 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:11 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e[62497]: 2024-04-26T22:10:11.598+0000 7ffae2395700 -1 mds.e *** got signal Terminated *** 2024-04-26T22:10:15.257 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:14 smithi064 podman[69215]: 2024-04-26 22:10:14.938991778 +0000 UTC m=+3.426010727 container died 318e305602cae9061ab5cc5d96c2af622953769b2f0c88d5dfd727d0a8f9b072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T22:10:15.573 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:15 smithi064 podman[69215]: 2024-04-26 22:10:15.38140081 +0000 UTC m=+3.868419800 container cleanup 318e305602cae9061ab5cc5d96c2af622953769b2f0c88d5dfd727d0a8f9b072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T22:10:15.573 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:15 smithi064 bash[69215]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e 2024-04-26T22:10:15.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T22:10:15.845 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:15 smithi064 podman[69241]: 2024-04-26 22:10:15.574144858 +0000 UTC m=+0.628829232 container remove 318e305602cae9061ab5cc5d96c2af622953769b2f0c88d5dfd727d0a8f9b072 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T22:10:15.845 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:15 smithi064 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-26T22:10:15.845 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:15 smithi064 systemd[1]: Stopped Ceph mds.e for c84f1140-0415-11ef-bc93-c7b262605968. 2024-04-26T22:10:15.845 INFO:journalctl@ceph.mds.e.smithi064.stdout:Apr 26 22:10:15 smithi064 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.e.service: Consumed 2.438s CPU time. 2024-04-26T22:10:15.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:15.859 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T22:10:15.859 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T22:10:15.859 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.h 2024-04-26T22:10:16.184 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:10:15 smithi064 systemd[1]: Stopping Ceph mds.h for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:16.506 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:10:16 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h[62805]: 2024-04-26T22:10:16.184+0000 7fda02100700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:16.507 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:10:16 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h[62805]: 2024-04-26T22:10:16.184+0000 7fda02100700 -1 mds.h *** got signal Terminated *** 2024-04-26T22:10:19.218 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:10:18 smithi064 podman[69347]: 2024-04-26 22:10:18.936195735 +0000 UTC m=+2.854322265 container died d2f9a90bf3b0821a478dd9d2aabcf0543709cc5a072ca8d733858e6ff5af7bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T22:10:19.506 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:10:19 smithi064 podman[69347]: 2024-04-26 22:10:19.309642476 +0000 UTC m=+3.227768998 container cleanup d2f9a90bf3b0821a478dd9d2aabcf0543709cc5a072ca8d733858e6ff5af7bcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T22:10:19.507 INFO:journalctl@ceph.mds.h.smithi064.stdout:Apr 26 22:10:19 smithi064 bash[69347]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-h 2024-04-26T22:10:19.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T22:10:19.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:19.707 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T22:10:19.707 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T22:10:19.707 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.k 2024-04-26T22:10:20.007 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 22:10:19 smithi064 systemd[1]: Stopping Ceph mds.k for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:20.506 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 22:10:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k[62175]: 2024-04-26T22:10:20.025+0000 7f313d502700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:20.507 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 22:10:20 smithi064 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k[62175]: 2024-04-26T22:10:20.025+0000 7f313d502700 -1 mds.k *** got signal Terminated *** 2024-04-26T22:10:22.697 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 22:10:22 smithi064 podman[69468]: 2024-04-26 22:10:22.343253943 +0000 UTC m=+2.408474449 container died a6f81ec610861b06f5eba4d3e61f262f14779fbdd009bbf2d7eb4a954250e583 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True) 2024-04-26T22:10:22.997 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 22:10:22 smithi064 podman[69468]: 2024-04-26 22:10:22.788483732 +0000 UTC m=+2.853704242 container cleanup a6f81ec610861b06f5eba4d3e61f262f14779fbdd009bbf2d7eb4a954250e583 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:10:22.997 INFO:journalctl@ceph.mds.k.smithi064.stdout:Apr 26 22:10:22 smithi064 bash[69468]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-k 2024-04-26T22:10:23.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T22:10:23.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:23.154 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T22:10:23.154 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T22:10:23.154 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.c 2024-04-26T22:10:23.445 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:10:23 smithi165 systemd[1]: Stopping Ceph mds.c for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:23.707 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:10:23 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c[63597]: 2024-04-26T22:10:23.444+0000 7fefad9ac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:23.707 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:10:23 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c[63597]: 2024-04-26T22:10:23.444+0000 7fefad9ac700 -1 mds.c *** got signal Terminated *** 2024-04-26T22:10:27.296 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:10:26 smithi165 podman[70589]: 2024-04-26 22:10:26.910781157 +0000 UTC m=+3.568037455 container died deb82a91f5eb408ae91782593706760101dad61f4e8c78c8e969200595ce7ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T22:10:27.576 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:10:27 smithi165 podman[70589]: 2024-04-26 22:10:27.394752987 +0000 UTC m=+4.052009291 container cleanup deb82a91f5eb408ae91782593706760101dad61f4e8c78c8e969200595ce7ca6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T22:10:27.576 INFO:journalctl@ceph.mds.c.smithi165.stdout:Apr 26 22:10:27 smithi165 bash[70589]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-c 2024-04-26T22:10:27.820 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T22:10:27.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:27.856 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T22:10:27.856 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T22:10:27.856 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.f 2024-04-26T22:10:28.326 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:10:27 smithi165 systemd[1]: Stopping Ceph mds.f for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:28.326 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:10:28 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f[63287]: 2024-04-26T22:10:28.173+0000 7f4f30a11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:28.326 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:10:28 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f[63287]: 2024-04-26T22:10:28.173+0000 7f4f30a11700 -1 mds.f *** got signal Terminated *** 2024-04-26T22:10:31.227 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:10:30 smithi165 podman[70724]: 2024-04-26 22:10:30.917520583 +0000 UTC m=+2.847998342 container died 5fe755a3e5039f885a46bc3cd3893a25336e9cb538f7c3b942b4b72296c62e23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:10:31.542 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:10:31 smithi165 podman[70724]: 2024-04-26 22:10:31.315837395 +0000 UTC m=+3.246315153 container cleanup 5fe755a3e5039f885a46bc3cd3893a25336e9cb538f7c3b942b4b72296c62e23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T22:10:31.542 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:10:31 smithi165 bash[70724]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f 2024-04-26T22:10:31.792 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:10:31 smithi165 podman[70738]: 2024-04-26 22:10:31.542196254 +0000 UTC m=+0.622657072 container remove 5fe755a3e5039f885a46bc3cd3893a25336e9cb538f7c3b942b4b72296c62e23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T22:10:31.792 INFO:journalctl@ceph.mds.f.smithi165.stdout:Apr 26 22:10:31 smithi165 systemd[1]: ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-26T22:10:31.794 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T22:10:31.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:31.827 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T22:10:31.827 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T22:10:31.827 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.i 2024-04-26T22:10:32.076 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:10:31 smithi165 systemd[1]: Stopping Ceph mds.i for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:32.576 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:10:32 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i[63903]: 2024-04-26T22:10:32.177+0000 7f46bc56f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:32.576 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:10:32 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i[63903]: 2024-04-26T22:10:32.177+0000 7f46bc56f700 -1 mds.i *** got signal Terminated *** 2024-04-26T22:10:35.077 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:10:34 smithi165 podman[70844]: 2024-04-26 22:10:34.901019679 +0000 UTC m=+2.861477272 container died c86513383e0e7ba9a4c43d6a44e3a2ef867dbbf8df53ae487391aabf856d3faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T22:10:35.528 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:10:35 smithi165 podman[70844]: 2024-04-26 22:10:35.269658442 +0000 UTC m=+3.230116038 container cleanup c86513383e0e7ba9a4c43d6a44e3a2ef867dbbf8df53ae487391aabf856d3faf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T22:10:35.528 INFO:journalctl@ceph.mds.i.smithi165.stdout:Apr 26 22:10:35 smithi165 bash[70844]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-i 2024-04-26T22:10:35.642 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T22:10:35.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:35.674 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T22:10:35.674 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T22:10:35.674 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.l 2024-04-26T22:10:35.826 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 22:10:35 smithi165 systemd[1]: Stopping Ceph mds.l for c84f1140-0415-11ef-bc93-c7b262605968... 2024-04-26T22:10:36.326 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 22:10:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l[62965]: 2024-04-26T22:10:36.006+0000 7f8a77770700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T22:10:36.326 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 22:10:36 smithi165 ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l[62965]: 2024-04-26T22:10:36.006+0000 7f8a77770700 -1 mds.l *** got signal Terminated *** 2024-04-26T22:10:39.648 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 22:10:39 smithi165 podman[70969]: 2024-04-26 22:10:39.347642707 +0000 UTC m=+3.447122848 container died 1ab915fa5ffd25a47428a2565eb150a9a9585d1cd39e6525632a2c2b0e53a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T22:10:39.930 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 22:10:39 smithi165 podman[70969]: 2024-04-26 22:10:39.740130508 +0000 UTC m=+3.839610649 container cleanup 1ab915fa5ffd25a47428a2565eb150a9a9585d1cd39e6525632a2c2b0e53a787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T22:10:39.930 INFO:journalctl@ceph.mds.l.smithi165.stdout:Apr 26 22:10:39 smithi165 bash[70969]: ceph-c84f1140-0415-11ef-bc93-c7b262605968-mds-l 2024-04-26T22:10:40.153 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c84f1140-0415-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T22:10:40.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T22:10:40.184 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T22:10:40.184 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c84f1140-0415-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:10:40.386 INFO:teuthology.orchestra.run.smithi055.stdout:Deleting cluster with fsid: c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T22:10:42.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c84f1140-0415-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:10:43.123 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T22:10:45.736 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c84f1140-0415-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T22:10:45.925 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T22:10:48.334 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:10:48.367 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:10:48.399 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T22:10:48.424 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T22:10:48.426 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi055/crash 2024-04-26T22:10:48.427 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:10:48.460 INFO:teuthology.orchestra.run.smithi055.stderr:tar: /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:10:48.461 INFO:teuthology.orchestra.run.smithi055.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:10:48.463 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi064/crash 2024-04-26T22:10:48.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:10:48.537 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:10:48.537 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:10:48.539 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi165/crash 2024-04-26T22:10:48.540 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash -- . 2024-04-26T22:10:48.586 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/c84f1140-0415-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T22:10:48.587 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-04-26T22:10:48.588 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T22:10:48.588 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T22:10:48.672 INFO:teuthology.orchestra.run.smithi055.stdout:2024-04-26T22:06:45.387392+0000 mds.b (mds.0) 119 : cluster [WRN] client.24916 does not advance its oldest_flush_tid (700245), 100000 completed flushes recorded in session 2024-04-26T22:10:48.673 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-26T22:10:48.673 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[SEC\]' /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T22:10:48.709 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]' /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T22:10:48.784 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[WRN\]' /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-26T22:10:48.859 INFO:teuthology.orchestra.run.smithi055.stdout:2024-04-26T22:06:45.387392+0000 mds.b (mds.0) 119 : cluster [WRN] client.24916 does not advance its oldest_flush_tid (700245), 100000 completed flushes recorded in session 2024-04-26T22:10:48.859 INFO:tasks.cephadm:Compressing logs... 2024-04-26T22:10:48.859 DEBUG:teuthology.orchestra.run.smithi055:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:10:48.901 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:10:48.904 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T22:10:48.927 INFO:teuthology.orchestra.run.smithi055.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:: No such file or directory 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:10:48.928 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T22:10:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:10:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T22:10:48.929 INFO:teuthology.orchestra.run.smithi055.stderr: 87.1% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:10:48.929 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T22:10:48.936 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T22:10:48.938 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T22:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T22:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T22:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T22:10:48.940 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T22:10:48.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T22:10:48.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T22:10:48.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:10:48.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T22:10:48.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:10:48.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T22:10:48.943 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T22:10:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-26T22:10:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T22:10:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log 2024-04-26T22:10:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T22:10:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T22:10:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T22:10:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T22:10:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T22:10:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.5.log: /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T22:10:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T22:10:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T22:10:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T22:10:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.z.log: /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T22:10:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T22:10:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T22:10:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T22:10:48.954 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T22:10:49.002 INFO:teuthology.orchestra.run.smithi165.stderr: 82.2% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:10:49.024 INFO:teuthology.orchestra.run.smithi165.stderr: 93.2% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T22:10:49.092 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.9.log: 82.6% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:10:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.k.log: 93.0% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T22:10:49.095 INFO:teuthology.orchestra.run.smithi064.stderr: 91.1% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:10:49.115 INFO:teuthology.orchestra.run.smithi055.stderr: 91.1% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:10:49.151 INFO:teuthology.orchestra.run.smithi165.stderr: 90.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:10:49.167 INFO:teuthology.orchestra.run.smithi165.stderr: 87.1% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:10:49.173 INFO:teuthology.orchestra.run.smithi165.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:10:49.180 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T22:10:49.210 INFO:teuthology.orchestra.run.smithi055.stderr: 90.2% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:10:49.276 INFO:teuthology.orchestra.run.smithi055.stderr: 94.5% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T22:10:49.277 INFO:teuthology.orchestra.run.smithi055.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr: 82.3% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T22:10:49.301 INFO:teuthology.orchestra.run.smithi064.stderr: 87.1% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T22:10:49.335 INFO:teuthology.orchestra.run.smithi165.stderr: 94.5% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T22:10:49.375 INFO:teuthology.orchestra.run.smithi055.stderr: 94.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T22:10:49.376 INFO:teuthology.orchestra.run.smithi055.stderr: 94.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T22:10:49.393 INFO:teuthology.orchestra.run.smithi055.stderr: 94.6% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T22:10:49.408 INFO:teuthology.orchestra.run.smithi165.stderr: 92.7% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T22:10:49.994 INFO:teuthology.orchestra.run.smithi064.stderr: 94.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T22:10:50.232 INFO:teuthology.orchestra.run.smithi064.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T22:10:50.232 INFO:teuthology.orchestra.run.smithi064.stderr: 90.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T22:10:50.432 INFO:teuthology.orchestra.run.smithi055.stderr: 89.9% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T22:10:50.525 INFO:teuthology.orchestra.run.smithi165.stderr: 92.0% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T22:10:51.246 INFO:teuthology.orchestra.run.smithi064.stderr: 94.6% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T22:10:51.258 INFO:teuthology.orchestra.run.smithi064.stderr: 92.3% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T22:10:56.507 INFO:teuthology.orchestra.run.smithi064.stderr: 91.9% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T22:11:00.140 INFO:teuthology.orchestra.run.smithi055.stderr: 90.2% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T22:13:39.472 INFO:teuthology.orchestra.run.smithi165.stderr: 93.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T22:13:40.485 INFO:teuthology.orchestra.run.smithi165.stderr: 93.5% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T22:14:23.481 INFO:teuthology.orchestra.run.smithi055.stderr: 93.3% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T22:15:39.820 INFO:teuthology.orchestra.run.smithi165.stderr: 93.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T22:15:41.447 INFO:teuthology.orchestra.run.smithi055.stderr: 93.3% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T22:16:17.017 INFO:teuthology.orchestra.run.smithi165.stderr: 97.5% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T22:17:04.008 INFO:teuthology.orchestra.run.smithi165.stderr: 93.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T22:17:04.009 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-26T22:17:04.009 INFO:teuthology.orchestra.run.smithi165.stderr:real 6m15.091s 2024-04-26T22:17:04.009 INFO:teuthology.orchestra.run.smithi165.stderr:user 4m53.682s 2024-04-26T22:17:04.010 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m13.460s 2024-04-26T22:17:06.935 INFO:teuthology.orchestra.run.smithi055.stderr: 93.3% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T22:17:29.390 INFO:teuthology.orchestra.run.smithi055.stderr: 93.3% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T22:17:29.392 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-04-26T22:17:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:real 6m40.478s 2024-04-26T22:17:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:user 4m59.620s 2024-04-26T22:17:29.392 INFO:teuthology.orchestra.run.smithi055.stderr:sys 0m12.526s 2024-04-26T22:24:56.172 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T22:25:38.025 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T22:28:59.379 INFO:teuthology.orchestra.run.smithi064.stderr: 93.5% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T22:30:06.886 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T22:33:44.646 INFO:teuthology.orchestra.run.smithi064.stderr: 94.4% -- replaced with /var/log/ceph/c84f1140-0415-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T22:33:44.745 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-04-26T22:33:44.745 INFO:teuthology.orchestra.run.smithi064.stderr:real 22m55.825s 2024-04-26T22:33:44.745 INFO:teuthology.orchestra.run.smithi064.stderr:user 9m48.721s 2024-04-26T22:33:44.745 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m27.843s 2024-04-26T22:33:44.745 INFO:tasks.cephadm:Archiving logs... 2024-04-26T22:33:44.746 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi055/log 2024-04-26T22:33:44.748 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:34:30.352 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi064/log 2024-04-26T22:34:30.353 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:35:51.585 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi165/log 2024-04-26T22:35:51.586 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T22:36:38.260 INFO:tasks.cephadm:Removing cluster... 2024-04-26T22:36:38.260 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c84f1140-0415-11ef-bc93-c7b262605968 --force 2024-04-26T22:36:38.461 INFO:teuthology.orchestra.run.smithi055.stdout:Deleting cluster with fsid: c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T22:36:39.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c84f1140-0415-11ef-bc93-c7b262605968 --force 2024-04-26T22:36:39.742 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T22:36:40.730 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c84f1140-0415-11ef-bc93-c7b262605968 --force 2024-04-26T22:36:40.925 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: c84f1140-0415-11ef-bc93-c7b262605968 2024-04-26T22:36:41.541 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T22:36:41.541 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:36:41.561 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:36:41.590 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T22:36:41.613 INFO:tasks.cephadm:Teardown complete 2024-04-26T22:36:41.613 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T22:36:41.624 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T22:36:41.624 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:36:41.627 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:36:41.634 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T22:36:41.759 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T22:36:41.769 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T22:36:41.769 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:36:41.772 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:36:41.774 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:36:41.787 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:36:41.788 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:36:41.789 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:36:41.791 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:36:41.791 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-04-26T22:36:41.791 INFO:teuthology.orchestra.run.smithi055.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +1412us[+1403us] +/- 45ms 2024-04-26T22:36:41.791 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 31 +335us[ +335us] +/- 66ms 2024-04-26T22:36:41.791 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -3314us[-3314us] +/- 91ms 2024-04-26T22:36:41.791 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:36:41.815 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:36:41.815 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-26T22:36:41.815 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 +1350us[+1366us] +/- 45ms 2024-04-26T22:36:41.815 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 +245us[ +245us] +/- 66ms 2024-04-26T22:36:41.815 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -3403us[-3403us] +/- 91ms 2024-04-26T22:36:41.815 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:36:41.867 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:36:41.867 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-04-26T22:36:41.867 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +1426us[+1415us] +/- 45ms 2024-04-26T22:36:41.867 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 +327us[ +327us] +/- 66ms 2024-04-26T22:36:41.868 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -3346us[-3357us] +/- 91ms 2024-04-26T22:36:41.868 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:36:41.876 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T22:36:41.886 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T22:36:41.887 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T22:36:41.935 DEBUG:teuthology.orchestra.run.smithi055:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:36:42.060 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:36:42.219 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T22:36:42.329 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:36:42.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:36:42.362 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:36:42.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:36:42.393 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:36:42.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:36:42.420 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T22:36:42.431 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T22:36:42.441 INFO:teuthology.task.internal:Duration was 3641.318638 seconds 2024-04-26T22:36:42.442 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T22:36:42.460 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T22:36:42.460 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:36:42.463 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:36:42.465 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T22:36:42.517 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:36:42.521 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:36:42.529 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:36:42.867 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T22:36:42.867 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2024-04-26T22:36:42.868 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:36:42.891 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-04-26T22:36:42.891 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:36:42.921 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-04-26T22:36:42.921 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T22:36:42.954 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T22:36:42.955 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:36:42.957 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:36:42.963 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T22:36:43.021 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T22:36:43.021 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:36:43.025 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:36:43.028 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T22:36:43.415 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T22:36:43.427 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T22:36:43.427 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:36:43.451 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:36:43.491 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T22:36:43.527 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T22:36:43.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:36:43.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:36:43.542 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:36:43.567 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2024-04-26T22:36:43.577 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-04-26T22:36:43.589 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-04-26T22:36:43.605 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:36:43.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:36:43.636 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:36:43.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:36:43.659 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T22:36:43.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:36:43.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T22:36:43.684 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T22:36:43.685 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi055 2024-04-26T22:36:43.686 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:36:44.036 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi064 2024-04-26T22:36:44.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:36:44.501 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231/remote/smithi165 2024-04-26T22:36:44.501 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T22:36:44.542 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T22:36:44.543 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:36:44.545 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:36:44.548 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T22:36:44.593 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T22:36:44.608 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T22:36:44.609 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T22:36:44.656 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T22:36:44.656 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:36:44.659 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:36:44.661 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T22:36:44.674 INFO:teuthology.orchestra.run.smithi055.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:36 /home/ubuntu/cephtest 2024-04-26T22:36:44.676 INFO:teuthology.orchestra.run.smithi064.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:36 /home/ubuntu/cephtest 2024-04-26T22:36:44.678 INFO:teuthology.orchestra.run.smithi165.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 22:36 /home/ubuntu/cephtest 2024-04-26T22:36:44.689 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T22:36:44.700 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T22:36:44.732 INFO:teuthology.nuke:Checking targets against current locks 2024-04-26T22:36:44.759 DEBUG:teuthology.nuke:shortname: smithi055 2024-04-26T22:36:44.759 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T22:36:44.781 DEBUG:teuthology.nuke:shortname: smithi064 2024-04-26T22:36:44.782 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T22:36:44.786 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:30:15.400739', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNHfiHQS3I6dJ/WKWqtAsSAT9nRcTILNLOnD6K1apilLYOYmbpnRRXf4UeH6aatsiRUTfrpI1fJyqC9OCd2TYzs='} 2024-04-26T22:36:44.803 DEBUG:teuthology.nuke:shortname: smithi165 2024-04-26T22:36:44.804 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T22:36:44.810 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:30:15.398736', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPR52kOYXVRc++48vWLWaJdMwOVhhuFoCYGtUuvxmig3vlX4qnZkFipYunP9pQsGGHdhA6tD5UtcmTKTgvBtspo='} 2024-04-26T22:36:44.821 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 21:30:15.402510', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJG7YMoZoGS5PR8OAdy72bO3isLjhgdYKa8FoLF16DUpxZCAvyw8Ruq6QrvaWsksvxxeUQMmpflsmDDyp5Wyx7M='} 2024-04-26T22:36:44.836 INFO:teuthology.orchestra.console.smithi055:Power off 2024-04-26T22:36:44.836 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T22:36:44.861 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2024-04-26T22:36:44.861 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:44.881 INFO:teuthology.orchestra.console.smithi064:Power off 2024-04-26T22:36:44.882 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T22:36:44.895 INFO:teuthology.orchestra.console.smithi165:Power off 2024-04-26T22:36:44.895 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-26T22:36:44.905 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2024-04-26T22:36:44.905 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:44.913 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-04-26T22:36:44.918 DEBUG:teuthology.orchestra.console.smithi165:power off output: Chassis Power Control: Down/Off 2024-04-26T22:36:44.918 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:44.928 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-04-26T22:36:44.942 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-04-26T22:36:48.914 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:48.929 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:48.943 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:49.025 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-04-26T22:36:49.040 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-04-26T22:36:49.053 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-04-26T22:36:53.027 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:53.041 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:53.054 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:53.138 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-04-26T22:36:53.151 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2024-04-26T22:36:53.164 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off 2024-04-26T22:36:53.252 INFO:teuthology.orchestra.console.smithi064:Power off completed 2024-04-26T22:36:53.265 INFO:teuthology.orchestra.console.smithi165:Power off completed 2024-04-26T22:36:57.142 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-26T22:36:57.252 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-04-26T22:36:57.352 INFO:teuthology.orchestra.console.smithi055:Power off completed 2024-04-26T22:36:57.459 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} duration: 3641.318638086319 failure_reason: '"2024-04-26T22:06:45.387392+0000 mds.b (mds.0) 119 : cluster [WRN] client.24916 does not advance its oldest_flush_tid (700245), 100000 completed flushes recorded in session" in cluster log' flavor: default owner: scheduled_rishabh@teuthology success: false 2024-04-26T22:36:57.459 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T22:36:57.559 INFO:teuthology.run:FAIL